Places features

Listen to Professor Kevin Warwick from the University of Reading explain what happens in the Turing Test and in the contest for the Loebner Prize for Artificial Intelligence:

The Turing Test is something I've been interested in since my university days - the whole question about whether computers can one day be programmed to think and interact conversationally like a human being. And if so can a computer then be said to think of its own accord?

Sculpture of a Cyborg head on display

So when the opportunity arose to judge in such an event at Reading university, which hosted the 18th Loebner Prize for Artificial Intelligence, I went along to see how far computers have come in replicating human conversational maxims and speech patterns.

For those who aren't sure what the Turing Test is, Alan Turing is considered to be the founder of computer science. During the Second World War Turing worked at Bletchley Park, the UK's code-breaking centre.

In 1950 he proposed a test of a machine's ability to demonstrate intelligence in an article entitled Computing, Machinery and Intelligence.

If a computer acts indistinguishably from a human in a speech-based chat environment, then we can argue the machine is thinking.

On the day, Sunday 12 October 2008, I was asked to sit behind a computer terminal with on the screen two chat windows.

Judges taking part in the test

I'd be having two conversations at the same time - like a live online chat - but one with a human being in another room and the other with a computer. I would carry out the test four times on four different computer terminals.

See my on-screen conversations here. Can you work out which one is the human and which one is the computer?:

My approach to it was to just have a normal conversation with both parties to see if they picked up on certain colloquialisms or understood the content of my sentences.

On my first go it seemed immediately apparent which window displayed the computer speak. For starters, I began by simply typing "hi there", and the immediate response was "Do you believe in God?". Hardly a normal human opening gambit.

While in the other window the response to my "hi there" was "heya" - certainly more convincing.

It became more obvious that the right-hand panel was an automaton when I wrote, with mild exasperation, "are you a computer?", and the response immediately was "yes I am".

See more screen shots in the gallery

A rather half-hearted attempt on the robot's part.

It did momentarily cross my mind that this conversation could be with a human trying to outfox me by pretending to be a nonsensical robot, but I decided that a person couldn't possibly deliver such an illogical conversation with apparent ease.

I was shuffled to the next computer, where the conversation windows conducted by the previous judge was still open. He had produced a set of carefully constructed questions that meant the computer would have to understand the logic in what he's asking, not just recognise words and formulate a set response from that.

His questioning included: The four capitals of the UK are three, Manchester and Liverpool. What's wrong with this sentence?

Perhaps this judge needn't have tried so hard considering the befuddling responses I was receiving from just basic chat.

But the conversations I had in my second test I had did amaze me - to the extent that I was convinced I was writing to two humans.

Both conversations I was having at the second terminal turned to my job. I wrote that I worked for BBC Berkshire. And both responses were reliant on understanding what the BBC is and both came up with convincing follow-ups to that.

The left window response was "a man at our church is a production executive at the BBC", and the right-hand window response showed: "Cool my olks work(ed) for CBS and ABC, London beuro" (sic).

Professor Kevin Warwick

It was only that the right-hand window then immediately corrected olks to folks, using '*' - a known symbol used in an online chat environment to show you're correcting a spelling error. That made me realise that the conversation in the other window must be from a robot.

My third test was equally as baffling.

The left-hand window chat attempted a double bluff by saying from the outset that it's not human and came up with jovial bits of chat relating to its robot state such as "I talked for hours with these black and white spotted creatures you call cows. You humans could learn so much from them".

While the right-hand window came up with nonsensical chat supposedly from a 13-year-old Ukrainian lad who wrote: "I like to play language cassettes for Guinean to my guinea pig".

The fourth and final round of the test featured two very curt conversations - one was nonsensical but it proclaimed it was from a human (could it be another pesky human posing as a computer?), and the other followed more logical lines of enquiry but using short general sentences.

Dr Hugh Loebner

In short it was an interesting experiment but ultimately there's no contest - in my view computers simply can't compete with natural flowing human conversation that follows certain indices and maxims, that use subtle nuances of language, and can be written in a detailed logical manner.

Knightrider's KITT car is a long way off from being a reality, but it definitely is a case of: more's the pity!

Listen to what Cybernetics Professor Kevin Warwick has to say about the philosophical questions thrown up by this test and how thinking computers can have a place in our everyday lives:

Listen to what Dr Hugh Loebner talking about why he's sponsoring the Turing Test competition and on Alan Turing's 1950 article Computing and Machine Intelligence:

Listen to some of the judges' responses to the test: