Social media's sentimental journey

It's the Holy Grail for advertisers, marketers, companies, economists and politicians - how to know what people are thinking as soon as they think it.

Leaving aside the possibility of discovering a way of telepathically reading minds en masse, sentiment analysis has been creating a bit of a buzz for the past year or so. But it's the UK general election that has brought it more into mainstream awareness.

Real-time chatter on social media sites like Twitter and Facebook - which are now increasingly searchable at deeper levels - means that it's easy to see what is being talked about, or, using Twitter lingo, what is 'trending'. Sentiment analysis takes it one step further, giving a weighting to the emotion, or sentiment, behind the key word or phrase being used.

It's all based on clever algorithms that attach a positive or negative rating (or context) to the subject matter. The BBC's Digital Correspondent, Rory Cellan-Jones, blogged about how these tools were being used to measure the performance of the three party leaders in the first prime ministerial TV debate of the campaign.

Tweetminster, the site which tracks political Tweets, is investigating how sentiment analysis measures up to traditional polling data by seeing if it can predict the outcome of the general election more accurately.

Although the idea has been ridiculed by some, it will be interesting to see the results. Tweetminster founder Alberto Nardelli has pointed out that a similar exercise predicting the Japanese general election results last year proved to be pretty accurate. That experiment, by Tokyo-based internet venture firm Hatena, showed predictive ability comparable with polls published in major newspapers.

But can sentiment analysis do better than that? The financial industry has been ahead of the game with this and there are already companies offering pretty sophisticated tools to help traders judge market sentiment through the way companies are being reported in the media in real time.

As far as I can tell, all of these sorts of tools are currently automated - and therein lies the problem. As good as mathematical formulae are, can they beat the analytical ability of a regular human being? Will human intervention still be needed to get high degrees of accuracy?

There was an interesting conference in New York which looked at this issue a couple of weeks ago. It's been shown that automated tools can give an accuracy prediction of between 70 and 90%. But as Greg Radner of Thomson Reuters said: "Beyond 80%, the law of diminishing returns sets in as it becomes more costly." And, I'd guess, quite possibly to the extent that it doesn't make financial sense.

Another interesting fact at the conference came from Seth Grimes: did you know that the accuracy level for two humans to agree on an expressed sentiment is 82%? That means that, in nearly one in five instances, two people won't agree on what is meant by a statement. 

So if computer programmes can indeed predict accuracy of sentiment beyond this figure, it could well prove financially viable in the long term for certain sectors. Magnify that model so that it can assess the meaning behind what hundreds of millions of people are saying on social media in real time, then the Holy Grail may not be a myth for much longer.

Trushar Barot is Election Editorial Lead, for User Generated Content and Social Media, BBC Newsroom.