How hapless journalists binged on a chocolate story they should have resisted

This post was co-written with Fiona Fox of the Science Media Centre.

Scientific experts: Sugar intake 'should be halved'. 

Another day, another nutrition story. This morning’s was based on a report from the Scientific Advisory Committee on Nutrition (SACN), suggesting that we should reduce our sugar intake.

While nutrition headlines come thick and fast, they’re not always based on solid foundations.

So how should journalists, and their audiences, distinguish the scientifically-sound from the decidedly dodgy? This is our recipe (sorry) for credible reporting of this tricky field.

The science world was up in arms last month as we learned that a journalist posing as a researcher fooled millions of people into thinking that chocolate helps weight loss. The hoax was based on a bogus study carried out by science writer John Bohannon purely to make a statement about the superficiality of nutritional science reporting, and to highlight how easy it is for claims that completely lack substance or scientific merit to hit the headlines around the world.

“It was terrible science. The results are meaningless, and the health claims that the media blasted out to millions of people around the world are utterly unfounded,” Bohannon wrote of his study in an article for io9.com.

Some on Twitter argued that general news journalists could be forgiven for reporting research findings published in the grandly named International Archives of Medicine. Others argued that it raises more questions about the scientific publishing process and dodgy press releases. But, whichever way you look at it, the fact that so many newspapers (including the Daily Express and Daily Star in the UK) ran the story uncritically leaves journalists with questions to answer about how nutrition stories are reported.

For us at the Science Media Centre, which was set up to improve the quality and accuracy of science reporting, the ‘slim by chocolate’ story was hardly more shocking than many of the apparently legitimate nutrition stories that irk us every day. Media headlines declaring “Sugar is the new tobacco”, “Butter ISN'T bad for you after all”, “Sugar blamed for obesity epidemic” and “FAT is the key to living longer” are standard fare these days, and it’s hard to imagine how the public can be anything other than bemused.

These should be good times for food and nutrition scientists. The huge media appetite (sorry again) for food stories reflects an obsessive love/hate relationship with food. If we can either prolong or shorten our lives through our food and drink choices, then we want to know about it. Add a global obesity ‘epidemic’ and you have a level of media interest that other scientists can only dream about. But, far from enjoying reading their newspapers over their porridge (“Porridge for breakfast cuts the risk of diabetes”), leading food scientists are spluttering into their oats.

Catherine Collins, principal dietician at St George's Hospital London, said: “For dieticians involved in interpreting nutrition research for the public, the ‘devil in the detail’ is often ignored. Demonising one nutrient or food for column inches can adversely influence dietary quality.

“‘Breakfast cereals are as salty as seawater’ was at the time accurate. But consumers swapping cereal for an alternative breakfast often chose higher calorie, higher fat and similar salt content foods such as croissants. The benefits of blood pressure reduction from reducing intake of salty cereal was counteracted by inappropriate substitution of non-demonised food. Medical advances are carefully quantified in current media reporting, and nutrition should be afforded similar respect.”

Even with the best science, the public would face a bewildering array of claims and counter-claims about diet and health. That is due to the way science works: researchers test hundreds of different hypotheses in different types of experiment, asking a slightly different question each time.

Researchers might test individual nutrients when in real life we all eat a mix of foods. They might study them in cells in a laboratory Petri dish or in a mouse, rather than in people, and they have to design studies carefully to rule out other factors that might affect the outcome. No study is ever the last word and many are very tiny developments on previous studies. These might be significant or interesting for other researchers, but shouldn’t necessarily hit the headlines.

They don’t all reflect the real world of breakfast, lunch and dinner, and they don’t all imply that we can or should make changes to our complex diets.

This is particularly relevant when a single nutrient or ingredient is being studied in the absence of what it comes wrapped in - for example, singling out sugar, fat or protein rather than muesli, fruit juice or milk. Or when one individual study and its incremental findings are being reported without discussion or acknowledgement of the wealth of evidence that came before it, some of which might paint a rather more complicated picture.

Added to the demonising of single nutrients, we often hear from campaigners who suggest that all our health woes can be solved if only we ate ‘natural’ or ‘real’ foods - but what does that even mean? Not many nutrition experts or dieticians would suggest we should all eat butter, honey or peanuts three times a day. It’s complicated: needing a varied and balanced diet is the reason for this complexity, and we shouldn’t pretend it isn’t.

The physical basics of obesity at least come down to calories in and calories out - in a way that is as simple as it gets. We can all keep an eye on that, depending on body weight and our levels of activity. But when it comes to living longer or staying healthier into old age, if you read claims suggesting the answer is simple they’re probably wrong!

So how can journalists navigate this minefield? If the processes of good dietary science are so complicated, how can we expect our newspapers to get it right every time? And on top of such challenges is the fact that some of the claims being made are not even good science. So identifying who to trust and whose opinion is sufficiently evidence-based to create headlines is a minefield.

But that doesn’t mean that journalists shouldn’t try!

Of course we are all entitled to our opinions. But when it comes to informing public understanding on an issue that can literally change and save lives, we surely ought to give more weight to those who have been carrying out relevant research for decades rather than just those with the loudest voices and most strident opinions.

With the chocolate and weight-loss story, for example, had there been more fact-checking and consultation with third-party experts a bogus story may have never seen the light of day.

Extraordinary claims need extraordinary evidence, especially if a finding goes against the general consensus - and particularly if the top line seems too good to be true. Robust evidence for health and diet comes from a combination of three types of research: epidemiological studies, which can identify trends or associations in sections of the population but can’t prove that one thing causes another; mechanistic studies, which look for plausible reasons for trends but are often done in cells or animals so can’t always be extrapolated to people; and double blind placebo controlled trials in humans - the most robust kind of evidence but also difficult and expensive because they involve people accepting strict dietary interventions as well as measuring non-diet factors for prolonged periods.

It doesn’t necessarily mean the findings are wrong if one of these three is missing. But putting all of these studies together builds a strong and reliable body of evidence for a claim, shows where the general consensus is, and the direction in which research is moving.

The problem for mainstream food scientists of course is that the main public health message tends not to change much as the evidence base grows: the need for a balanced diet does not a sexy headline make. And that leaves plenty of room for hyped claims or unreliable but sexier nutrition stories to fill the pages of our hungry (sorry yet again) newspapers.

The growing problem of obesity will keep nutrition research well-funded for years to come, and journalists are rightly keen to cover the topic.

Our best advice to responsible journalists is to build up a list of credible experts and subject their hypotheses to rigorous questioning. Even then there will be lots of great headlines and different views, but at least they will be evidence-based.

In a follow-up blog we’ll offer our top 10 tips for successful reporting of food science.