Archives for May 2011

Article written by Andy Finney

Andy Finney

Producer on the original BBC Domesday Project

The discussion pages on Wikipedia are sometimes more interesting than the articles.

The BBC Domesday Project has its page and, until recently, it included a note about the kinds of online applications we know and love, such as mapping and Google Street View, that had their equivalents in the Domesday Project.

Any mention of the BBC being a red rag to certain Wikibulls, this note was expunged by an anonymous editor who was concerned about "weasel words and no citations to any reference that this project in any way 'led the way' or 'inspired' any present mapping software or service."

This got me thinking. What key 'new' bits of technology were there in, and were we first, and is it possible that the likes of Google were influenced by this? My colleagues will probably come up with a few more (and that's what comments are for) but here's my set.

Mapping

Some Domesday features just grew organically. Once we knew that the UK mapping had to be based on 4 by 3 blocks - the aspect ratio of 1980s TV screens - and there were existing paper maps that allowed us to show 4 by 3 km or 40 by 30 km in that space it seemed logical to allow horizontal and vertical jumps across those maps and to be able to 'zoom' from one scale to another.

It was a simple concept once an algorithm had been developed (my memory says by Dr Martin Porter) and worked by jumping between frames on the disc. Unlike current systems such as Google Maps and MultiMap it was not possible to dynamically render the maps on demand so paper maps for the whole of the UK had to be cut up, joined together where necessary and put beneath a video rostrum camera at the BBC Open University Production Centre in Milton Keynes.

The National Disc allowed users to select sets of data (such as population) and display these overlaid on maps on demand. This also was highly unusual in the mid 1980s, when computers managing statistics still usually occupied a whole air conditioned room and required white-coated acolytes.

Searching

Every photograph and every piece of text was allocated keywords based on a pre-determined vocabulary. This concept was well-established long before the 1980s (and today keywords sit in the invisible headers of every well-structured web page).

Where Domesday pushed the envelope a little was with the way the queries could be entered. We didn't feel that members of the public could (and should be asked to) use boolean logic with this and that but not the other. Dr Martin Porter came to the rescue.

Martin had produced a search system called Muscat early in the decade and had also developed what is now known as the Porter Stemming Algorithm. Stemming is the linguistic method by which your computer system knows that the words 'search', 'searching' and 'searched' actually refer to the same concept.

For Domesday this allowed the text searching to work out what the key words were when a user entered a sentence in the search box. In the original system you can watch as the system extracts the key words from your sentence and counts through the hits it finds on one or more of them.

Stemming in free text searching is used by today's web search engines. You can try it for yourself by putting in different versions of a word and seeing what the search engine returns.

The virtual machine

Whereas my original interactive video experiments had been written in BBC Basic, the Domesday Project required more sophisticated programming, notably to overcome the memory limitations of the BBC Micro and its 8 bit environment.

Fortunately Acorn had published a BBC Micro chip for a language called BCPL, which is often referred to as a forerunner of C, arguably the most influential computer language.

BCPL, which was developed In Cambridge by Dr Martin Richards, was a two-stage system whereby the program would be compiled into what was called CINTCODE (Compiled INTermediate CODE) and a second piece of software, called a virtual machine (VM) ran that CINTCODE on the computer.

In theory you could 'easily' change the VM and run the same code on another computer. Today the virtual machine concept is probably best known in Java. It should be possible to take Domesday's BCPL code and run it on a VM under Windows or Linux or whatever on a modern computer.

This process was used back in the 1980s for the Research Machines version of the Domesday System and to demonstrate Domesday running on an Acorn Archimedes computer.

There are four key reasons why the BBC couldn't just update the BCPL VM to run Domesday now: no source code (the BBC were not given it), translating disc commands such as frame numbers (for images) into file names, working around a few bits of the code that relied on the word size (number of bits) in the original system ... and the cost of doing the first three. Besides, even if this were achieved it would produce a stand-alone Domesday system, rather than the web-based one that the Reloaded team have made.

Virtual Reality

The Domesday team were aware of an interactive video project made around 1978-80 at the American University MIT which enabled a user to take a virtual trip around Aspen Colorado. It was called the Aspen Moviemap.

The MIT team used a specially-rigged car (not unlike the Google Street View car) to take thousands of still images of the streets and could then navigate around them from a touch-screen and an array of videodisc players.

We wondered whether we could use a simplified version of this idea in the UK and after some experiments decided we could use this to show the main types of housing in the UK and a few typical examples of 'the outside world'.

The Domesday walk model looks clunky compared to Street View and QuickTime VR since we took eight photographs at each point and the user can only turn in 45 degree increments, rather than smoothly. We also extended the idea to produce a way of accessing some of the contents of the National Disc, whereby the user could move around a computer-generated art gallery.

I had also hoped we might use the walk around a newly-built show house as a way of accessing information on sales of domestic products. Click on the TV and see the market for televisions. But we didn't have time to add that feature.

Crowd sourcing and Wikis

The internet provides an instant way of gathering information from people. Domesday aimed to do the same but using the Royal Mail as it's network. There were elements of a Wiki in the text pages of the Community Disc but, until now, there was no way for those pages to be rewritten and developed.

So it's a snapshot of a Wiki rather than a true Wiki but in the media landscape of the 1980s the notion of people writing whatever they wanted and it being published (and by the BBC at that) was very, very rare. And mentioning Wikis brings us back to where we started.

Like all innovative projects ... indeed like all projects of any kind ... Domesday was a mixture of existing ideas, ideas that developed from existing ideas, and new ideas. And it is important to remember that Domesday was a consumer product and was built with this in mind.

Could Domesday have influenced companies like Google? I did demonstrate it at America's main interactive video conference in 1988 and the project is often cited as part of what is known as the prior art when patents for interactive media are being considered in the USA.

Actually whether Domesday influenced 21st century interactive media is irrelevant. We tried out ideas that, in most cases, came around again and worked better because the technology was more suited to them.

Sunny Side School, Middlesbrough

Over the past few weeks the BBC's Outreach team have been taking Domesday Reloaded into communities and schools around the country.

In Northampton The Ravensthorpe WI have been busy describing how life has changed in their village since 1986. Meanwhile, students from Harry Gosling Primary School met with a former student and original Domesday Project contributor and produced an audio-slideshow about the experience.

Members of the Sheffield 50+ network have also been getting involved with photographing their local area, as have The Priory Day Centre in Bodmin, and Sunnyside School Middlesbrough.

A big thank you to all the local history societies, schools and groups for their involvement and support.

Here are pictures from some of the workshops:

Ravensthorpe WI 90th anniversary celebrations - 2011

Harry Gosling Workshop

More to follow as we visit more schools and communities to get Reloaded...

Travel back to the 1980s with the Domesday Reloaded Experience roadshow, touring the country this week.

The roadshow features a 1980s retro bedroom area complete with games, toys and informative BBC Domesday reloaded film. There's also a chance to present an 80s-style edition of Top of the Pops in our interactive greenscreen area.

You can also explore the website with one of the team and ask questions about Domesday Reloaded.

We've already stopped off at Coventry and Warrington but the roadshow will also be calling at:

Wednesday 18th May = St Georges, Preston

Thursday 19th May = Arndale, Manchester

Friday 20th May = The Potteries, Stoke-on-Trent

If you visit the roadshow do let us know what you think.

Andy Finney

Article written by Andy Finney

Producer on the original BBC Domesday Project

When the Domesday Project was devised back in 1984, one key question we had to resolve was how to deliver it.

There were two strong reasons for using the BBC micro: it was widely available in schools and its video output was compatible with 'real' television. Similarly, the Philips LaserVision system was suitable because it was a tried and tested tool for interactive video in the UK.

 In fact, there was no other real option. (In the USA the Apple II occupied this niche and the laser videodisc players came from Magnavox.)

BBC Video had already demonstrated that a videodisc with a data channel could be produced commercially. They had published the Videobook of British Garden Birds (presented by David Attenborough) in 1982.

This disc was designed to be a little more interactive than most consumer videodiscs by being 'active play', which meant you could still-frame the video on the disc, carefully indexing the sections on each bird, having two soundtracks (with and without commentary) and adding a teletext magazine. It was apparent from the technical specification of LaserVision that it had enough bandwidth and was stable enough to carry broadcast standard teletext. This was something that was not possible on video cassette.

The BBC Natural History Unit, who produced the disc, also provided information on the birds and the Ceefax department produced the teletext data. The oddest part of the whole process was that, in order to insert the teletext data into the TV signal BBC2 had to be taken off-air while its network equipment was used to make the videodisc master. As far as we know, this disc was the first consumer product anywhere in the world to use an optical disc to store data, rather than digital audio. I was later told by Philips that this BBC disc helped them decide that optical data discs could be viable. If only we had patented it.

Originally, the Domesday team thought it might be possible to store the data for the project as television frames full of teletext. The BBC Micro teletext adapter, another piece of readily available kit at the time, could gather teletext from any line of the TV picture. I had written a demonstration interactive application (in BBC Basic) which provided a user interface to the Birdbook while also taking background data from the teletext on the disc and blending it into a single application.

A team at the University of London had produced a demonstration disc which included some full-frame teletext. The problem was one of error correction in a mass-produced disc, since teletext refreshes regularly when transmitted and so relies partly on this repetition to handle occasional errors. This would not work with a videodisc.

As it happened, the question of teletext was sidestepped by Philips Research who told us that they could add a rugged data channel to the videodisc format by replacing the two analogue sound channels with a channel carrying the data. This was LV-ROM. In the final system, the player also included a system to lock the disc video to the computer video and combine them in various ways under control of the software. So the computer was plugged into the player and the player was plugged into the monitor.

The computer was almost a standard BBC Master computer, complete with a second processor (yes, Domesday used two cores) and a special filing system to control the player. The only new piece of hardware was an interface between the computer and player which transferred the data and controlled the player. This was the then brand new SCSI interface, beloved of high performance hard discs to this day, and it slotted into the micro in a new slim circuit board.

The firming up of our delivery platform and the production of the content took place side-by-side and we were a long way into the project when Philips were able to produce a real LV-ROM disc for us. But it all became a little more real when we were able to hold that first disc in our hands.

Read the rest of this entry

Many of our BBC colleagues have been looking into the Domesday Reloaded content over the past months and preparing stories for broadcast this week. There'll be more and more over the summer as we try to get the country re-surveyed and the project fit for the future.

Radio 4's Making History are giving us a lot of attention. This afternoon you can hear an interview with Sally Pearce, lecturer and 80s Domesday-er, whose late husband Adrian did so much pioneering work on getting Domesday online.

I can't recommend highly enough the Archive on 4 from this week. Catch it while you can.

There are articles, films and galleries all over the BBC news site, but this one got me going...

To all the many many who have submitted content to our new Domesday Reloaded website yesterday and today, thank you. It's been a marvellous response and we're dealing with adding your text and pictures as fast as we can - please bear with us if your block hasn't been updated yet... it will be.

Today's pic of the day was clearly about to be developed. Comparing the maps from then and now shows a lot of change... does anybody recognise it?

There's quite a lot of programming around Domesday Reloaded over the coming weeks. I thought some might like to have a run down of what's planned...

This morning BBC Breakfast and the Today Programme both gave Domesday Reloaded a mention, News/technology wrote a useful piece for us, and later this morning Rory C-J did another quick chat on the BBC News channel.

Radio 4's considerable range of coverage gets going this afternoon at 4.30pm on Material World when Tim Gollins, Head of Digital Preservation at The National Archives and George Auckland, formerly of this parish, will be talking about... digital preservation.

Saturday Live this week will be featuring the first of their pieces about changing spaces on Saturday morning, and then at 8pm Archive on 4 tells a lot more about the history of this project. Michael Wood presents, and from the 1986 team, Andy Finney, Ruth Rosenthal, Mike Tibbets and Peter Armstong contribute.

But if it's stories from your area you're looking for then listen out on Monday to local TV and Radio from your area. Some magnificent accounts of 25 years of change near you, wherever you are, have been surreptitiously researched and recorded over the last few weeks, ready for your enjoyment from Monday onwards...

Twenty five years ago the BBC published the Domesday Project, now after many years we've made the community disc archive available to all and created the Domeday Reloaded website - and as of today it's now live!

If you've never seen your contribution or just want to explore pictures from the past, then this is the place to do it. There are over 150,000 text entries and around 23,000 images from all over the UK - contributed by school children, community groups and the general public. It's a real snapshot of life and a nostalgic look back at the 80s.

Now the archive is available online we also need your help. We want to bring this project into the present day, compare how life in Britain has changed - and how some things have stayed the same. We're asking you to send us updated pictures and stories from your area via the Domesday Reloaded website.

In November 2011 the website will be handed over to The National Archives. With their help and expertise in web archiving and digital preservation, this valuable resource will be available to the public for generations to come.