BBC audience research in 1962
Anyone in the BBC who’s thinking of commissioning some research should remember that it becomes ‘BBC research’ - so you have to be very careful that it is robust. If it’s any kind of statistical work, there’s plenty of guidance from BBC Editorial Policy.
To save time I should say straight away that, if you’re thinking about a poll to find out which political party people support or who they might vote for, you should probably forget it. The BBC rarely commissions such polling, except for exit polls, which talk to people after they have voted.
If you think you have a good reason to conduct such a political poll you’ll have to get really senior approval. If there’s an election campaign underway it’s definitely not allowed. Almost all plans for polling, especially on controversial subjects, need to be cleared by the chief adviser, politics.
A good starting point is the five questions I have written about before on this blog, which you would ask about somebody else’s research. (Well, actually only four of them, because you will know the answer to question one: ‘where does it come from?’)
The most important point to remember is that if you aren’t asking the right people then it doesn’t matter how many people you ask, your research will be meaningless. There is no substitute for robust methodology.
Who will do the research?
You can pay a polling company to do it for you or you can try to do it yourself. The BBC does not have a preferred pollster. Find yourself a selection of members of the British Polling Council.
If you are trying to say something about a really big population such as the whole of the UK, you will really need to pay a company to do it for you. Random sampling in populations of millions is really hard to do. If you are trying to say something about a smaller group such as all MPs or schools in Barnet, which would allow you to ask everyone, you may consider doing it yourself.
The two main ways of conducting polls are random phone polls (as I discussed in my previous blog) or online panels. If you want to commission an online poll, again, you must talk to the chief adviser, politics.
What you should think about before you call a pollster:
What are you trying to find out?
It is really important to get your questionnaire designed properly, because if you commission research that you cannot use you will have wasted a considerable amount of money and left the BBC open to accusations of bias for conducting a poll and not running it.
For the same reason, you must be prepared to run your results whether or not they fit the story you were planning to tell.
One of the greatest dangers is that you ask questions that are ambiguous. So a good first step is to ask your questions of friends and colleagues. You want clear answers, not ‘it depends’. Are your questions pushing respondents to give a particular answer? If you rephrased the question, do you think you would be likely to get different responses?
Have a think about what it is you want to ask and then discuss it with some polling companies, which will also be able to help with question design.
Who do you want to be asked?
It could be people in the UK or women in Wales, or doctors in Scotland. It is important that if you want to say something about people aged between 18 and 24 in the East of England you do not commission a survey of everyone in the country. That is because a random sample of people across the country will only include a handful of young people in the East of England, certainly not enough to allow you to say anything robust about them.
If you want to say something about the whole of the population, for example, and then something about men and something about women, you need to survey more people so that the breakdown is also robust.
How soon do you need it?
Polling companies would like you to give them at least two weeks, but that’s not always possible with news. Try to talk to a few companies early in the process so you know whether your timescale is realistic. A simple question for a random sample of the UK population should be possible pretty quickly.
How much can you spend?
In general, the more precise you need to be about your population, the more you will be charged for the work. A survey of 1,000 doctors in Scotland will be much more expensive than a survey of 1,000 randomly selected people in the UK. Call around the pollsters and get a few quotes for what you’re trying to do.
Numbers in the news get a fresh pair of eyes
Data journalism: What’s new, what’s not and work in progress
