Science Reporting Needs Less Hype, More Analysis

Fancy Comma, LLC

Mar 31

Not all studies with flashy claims are ready for the spotlight.

Tweet

An uncomfortable reality about science

Science is a process of discovery in which results and hypotheses change. Scientists' understanding of the natural world – their model of the truth – changes along with those things. Despite this, media outlets often write about emerging research studies in terms that lead non-experts to interpret each study that comes out as fact.

It's not realistic to expect every member of the public to take mainstream science reporting (often also called "pop science") with a grain of salt. As scientists and science communicators, we can make the interpretation of emerging research findings easier. Stories we share as scientists should be rigorously evaluated before sharing with the public in the media.

The unfortunate catch is that this approach doesn't sell stories! Keep reading to learn about one instance in which the results of a study were magnified incorrectly by the media, and ways we can improve science reporting as scientists and science communicators.

When science reporting undermines science

When scientists make a discovery, journalists are tasked with communicating it to the general public. In the biomedical research world, messing up the message can spread health misinformation and fake news that can persist for decades.

In the 2010s, people perpetuated the myth that lengthening one's DNA can improve life expectancy; they built a multi-million dollar industry based on weak data. Even strong evidence to the contrary – that lengthening telomeres can increase cancer risk – failed to fully dismantle it. DNA lengthening products are still sold today.

To guide healthcare and lifestyle, biomedical science must satisfy a sufficient burden of proof. Said another way, hypotheses must be tested and supported by multiple experiments and in different contexts.

We saw science communication break down in the pandemic as breaking news journalists without a science background or even science literacy training had to repeatedly analyze, contextualize, and report emerging science – in the pandemic, the science was often the story.

When journalists report a single experiment as absolute truth, this misleads the public to draw premature conclusions. Sadly, that happens a lot. Next, I discuss one study about coffee with results that were interpreted in a misleading fashion by the media.

The coffee study that gained overblown media attention

In June 2020, scientists at the University of Bath, led by physiology professor James Betts, published a study that warned against drinking coffee before breakfast. They based their conclusions on their preliminary results, which did not reflect real-world conditions. In their research report, the researchers speculated about the health implications of their study, which the media overzealously translated as public health advice.

For this article, I spoke to James Betts and Harry Smith, two of the authors of this paper, via Zoom. Both agreed that the study's results were blown out of proportion in media coverage.

A preliminary study

Among caffeinated beverages, coffee seems like a good choice. Researchers have linked drinking a cup or two of coffee a day to many health benefits. Betts believes that this correlation is misleading. For one thing, in the United States, coffee drinkers are likely to be more socioeconomically well-off. A Chinese study showed that coffee drinkers tend to lead healthier lifestyles. "People erroneously assume there's some protective effect of coffee," Betts told me in a Zoom interview. He was the lead scientist on this project and also co-directs the Centre for Nutrition, Exercise and Metabolism at the University of Bath in England.

Betts decided to investigate the metabolic impact of drinking coffee after waking from a bad night's sleep. His paper showed that after disrupted sleep, drinking coffee back-to-back with a sugary beverage led to a bigger spike in blood sugar.

What was the actual research study?

Betts designed a study to test the effects of poor sleep on the body's ability to tolerate sugar. After a night of disrupted sleep, the researchers gave study participants either coffee or hot water, followed by breakfast. They found that the coffee drinkers had higher blood sugar, indicating an affected metabolic response.

High blood sugar after a meal can indicate a problem with carbohydrate tolerance. However, in this experiment, the volunteers consumed neither a normal cup of coffee nor a typical breakfast. The researchers used 300 milligrams of caffeine: the equivalent of five shots of espresso and over double the average daily intake in the United States. "Breakfast" was 65 grams of a sugar substitute, maltodextrin, that makes blood sugar spike higher and faster than normal sugar. In one gulp, the study participants consumed nearly three times the maximum daily amount of sugar recommended by the American Heart Association.

Proof-of-principle studies like these are important as a first pass. To establish real-world conclusions, though, they must be bolstered by further investigation.

The blitz

After the University of Bath issued a press release about the work, Betts and his co-author, Harry Smith, were met with a media storm. They were flooded with interview requests, and dozens of articles were published either overblowing or completely misstating their findings.

"Drink coffee after breakfast, not before, for better metabolic control," stated the University of Bath press release.

"A study has found that drinking coffee first thing can have a negative effect on blood sugar control – a risk factor for diabetes and heart disease," CNN reported.

A headline on SciTechDaily read, "Scientists Find It Is Better to Drink Coffee After Breakfast, Not Before – Here's Why." Remember, Betts and his team only tested what happened when coffee was given before breakfast; they never measured the blood sugar of people who drank coffee after.

Betts said that in his experience, accuracy was not the journalists' priority. He was misquoted in multiple articles and many of the headlines were misleading. "It was a bad experience," Betts told me. "It was sad that we spent a lot of time and money on learning these things and then it gets completely contorted by the media."

Smith says the recommendation that people should avoid pre-breakfast coffee was overblown. "That's not what we said, but it's unavoidable, really," Smith said. "It ends up being just [a game of telephone]."

The consequences

Unless I start swapping my morning oatmeal for Yoo-Hoo, I won't be skipping coffee. Smith still starts his day with espresso. If someone publishes a follow-up study that tests a normal breakfast with proper experimental controls, I'm sure we'll both reconsider.

But while Smith and I are trained to interpret raw data, non-experts rely on journalists to explain the science in plain language. When journalists overblow data, they misrepresent preliminary findings as definitive conclusions.

Journalists' responsibility

"There is a temptation to think that leaders in science have the answers to everything including health, and I think that makes things confusing," says Dr. Mary Armanios, Professor of Oncology at Johns Hopkins University School of Medicine.

In the lab, experiments are frequently artifactual and unintentionally misleading. A scientific paper is like one dot on a graph; you can't tell if it's an outlier until more dots fill in to form a pattern. How can the reader distinguish patterns from single dots?

"Everybody should be taught more critical thinking skills in school," Betts says. "That would be the biggest jump for society, if people could analyze information for themselves."

It's unrealistic to expect everyone to interpret data like a biomedical scientist. Scientific papers are notoriously dense with jargon. Even with rigorous training, interpreting data is hard work, though Science has published an explainer on the subject. Further, despite a movement towards open access publishing, much of this literature is still guarded behind paywalls that charge, on average, $32 for a single paper.

Journalists exist to bridge this gap and report scientists' findings both accurately and in context, which also means being discerning. Not all studies with flashy claims are ready for the spotlight.

Audrey Goldfarb is a biomedical Ph.D. student in the de Lange lab at The Rockefeller University in NYC. As a scientist, she is passionate about effective communication, collectivism, and mentorship. In 2020, she co-founded Rockefeller's peer advising program to create equitable opportunities for new graduate students. Audrey works to support women in science and mentors undergraduate and high school students. Outside of science, Audrey enjoys writing fiction, biking adventures, playing any and all team sports, and parenting her cat, Edith. Read more of her work on Rockefeller's outreach blog and newsletter or connect with her on Twitter or LinkedIn.