Maybe you have seen stories like this floating around:
Or perhaps you noticed this one that made the rounds a few months ago:
For the record, farts do NOT cure cancer. Drinking a glass of red wine is NOT the same as getting actual exercise. And scientists definitely did not say those things or lots of others you've maybe read about.
The BMJ recently published a study about exaggeration in health-related science news; they found a strong association between hype in health reporting and exaggerated press releases. The study authors suggest that if university press offices and researchers were more careful about their press releases being accurate, the news articles written about their research would potentially also be more accurate. (Others pointed out that if health journalists did their jobs better, this wouldn't be an issue either. Point well taken.)
Until that happens, though... how can you as a reader understand how to read health journalism critically, so that you know how to call B.S. on fantastical headlines or sloppy reporting? BuzzFeed Life reached out to four experts who work in medicine and/or health journalism to help answer this question.
Here are some basic pointers:
1. If the research wasn't conducted on humans, the findings can't necessarily be extrapolated to humans.
2. Correlation is not causation!
This is a big one. Lots of studies can find connections between two things — but that doesn't mean that one of those things caused the other thing. It just means that there is an interesting association, which could be good reason to do further research to learn more.
As far as what's behind that association — there could be any number of answers. Maybe one thing did cause the other. Or maybe something else caused both. Or maybe two unrelated things were responsible, and they only appear related but are in fact not related. Or maybe some other reason!
Here are some funny examples of how confusing correlation and causation can go very, very wrong, courtesy of Tyler Vigan, the genius behind the blog Spurious Correlations:
3. Does the story address both positives and negatives?
4. The most trustworthy health articles will have multiple sources, and at least one of them shouldn't be connected to the research in any direct way.
5. Also, those sources should be the most qualified people to talk about what they're talking about.
6. Good health reporting should discuss costs.
7. The headline should not overstate the findings of the research.
8. If the study was done on humans, pay attention to who those humans were: The study is stronger if they're randomly selected, nationally representative, and if there are a LOT of them.
9. The article should discuss the findings within a greater context.
11. Maybe you could do a little more reading about this, if you want.
And one last thing!
This post has me thinking a lot about glass houses and stone-throwing. In my years of reporting on health-related topics, I have definitely been guilty of writing over-hyped headlines and stories with varying degrees of inaccuracies or incompleteness. There are plenty of reasons for this — quick turnaround deadlines, inability to access sufficient sources in a short amount of time, an actual business need to drive clicks all come to mind. That's one of the reasons I wanted to write this post. In my new role as health editor at BuzzFeed, my aim is to be as accurate in my reporting as possible. The BMJ study, and reporting out this story, have both made me think a lot about how I can do better in the future.