Crunching election stats for SEO

This is a guest blog by Martin Asser, BBC Search Engine Optimisation specialist.

Nearly two weeks ago I made my first foray into BBC News' political nerve-centre at Millbank. I was there to do some SEO (search engine optimisation) coaching with the Politics team, in the hope of capitalising on increased public interest in their content during the election campaign. 

I was a bit concerned that referrals to the BBC's politics pages from search engines have been quite irregular over the past months - and pretty low compared to the rest of BBC News. Perhaps the writers and subs were missing some tricks. 

I didn't want to burden them with too much extra work at this busy time, but the great thing about SEO is that when the chips are down there is one thing and one thing only that journalists need to do to make a difference: fill headlines with keywords and - if at all possible - put the 'key' keywords at the front. 

A keyword, for the SEO uninitiated, is a word you would expect search engine users to employ when looking for your story. A 'key' keyword or phrase is the one that best sums up what the story is about: 'Election', 'Conservative Manifesto', 'TV debate' etc. Do not say 'contest, race, battle' in the headline when you mean 'election'; or 'televisual expostulation' when you mean 'TV debate'. 

So that was the message I left them with and since then I have been closely monitoring their pages and not hesitating to jump in if I spot somebody calling a spade a 'long-handled digging implement', or failing to front-focus the most important terms. 

And the headline efforts have for the most part been good: 

Election: IMF clash between Darling, Osborne and Cable

Election: Nick Clegg attacks 'desperate' Gordon Brown

BNP's Griffin urges restrictions on Chinese imports 

But the message hasn't got through everywhere ...

Candidates separated by 77 years 

(Where? Skipton and Ripon, as it turns out)

Izzard urges young people to vote 

(Eddie Izzard, and he's in Scotland campaigning for Labour!)

Row over valleys candidate's home 

(Maria Caulfield, Conservative, in Caerphilly)

Despite these blips, there's been a clear improvement in the referral statistics. Below is my daily referral analytics chart (source: SageAnalyst). Since I started working at Millbank at the end of the first week (9 April), the two weeks in the table represent, roughly, a 'before and after' comparison.

So how does it work? 

The second column (Searches total) tracks the number of searches driving traffic to the BBC's UK Politics pages for each date, and the third (Visits total) is the number of visits that accrue from search on that date. 

The difference between a search and a visit is that a search is registered every time someone clicks to a page on the site, but if they come in several times during the same session (using the site continuously without a half-hour break) then it only counts as a single visit. 

The fourth column (No Keywords visits) is where no keyword is recorded on the referring search URL, which I believe is mostly from aggregated news pages like Google News UK. 

The next column (Top 20 aggregate) is the number of visits accruing from the 20 most popular search terms used to get to the politics pages (excluding branded search - e.g. "bbc+news"). The point for SEO is to pick keywords that produce bigger figures for all the top 20 terms, which will make the aggregate go up. 

Finally, the 'SE referral %' is the benchmark calculation of how many of the total visits for that day have come from external search engines. Most referrals to the pages come from other sources, such as BBC pages or indexes, bookmarking or social media.

(For the sake of comparison, I have put the weekends in a different colour.) 

The figures are as good as they look: searches (second column) have gone from a weekday average of 66,141 in the first week to 84,977 in the second.

Visits (third column) have gone from 52,273 to 66,317. 

Weekend search traffic has also clearly increased. 

'No keyword' visits have yet to show such a dramatic improvement. But you can see the improvement is even more remarkable at the heavy traffic end of the spectrum with the 'top 20 aggregate' up from 5,536 to 7,888 in the week, and 2,574 to 5,356 in the weekend - more than double. 

To show that this is not all just a result of increased interest in the election, we can see that the search engine referral rate has gone up from 4.5% in the first week to 5.4% in the second. 

Another set of daily stats I have been gathering (below) also sheds some light on the improvement. In addition to looking at the 20 most popular non-branded search terms, as above, I have recorded all terms which get more than 500 visits and totted up the total. This way you can track the performance of keywords like 'election', 'hung parliament', 'bnp' and 'ukip' on any given day. Again, weekends are a different colour.

You can clearly see how the numbers are getting bigger at the top end of the list for each day (most popular search terms), thanks to better SEO. But, as I always try to impress upon web content producers, it is of utmost importance to optimise every story so that all of them can contribute to the growth. Now, who wrote that Caerphilly story ...