Theresa May has given a big speech about mental health care. She's promised extra training for schoolteachers and a review of mental health services for young people.
It's part of a big push by the government on mental health issues – Jeremy Hunt, the health secretary, was on the Today programme on Monday morning describing young people's mental health care as a "black spot".
In the speech, May said "1 in 4" of us suffer from mental health issues at any given time.
She has previously used that statistic in a blog post for the Huffington Post, writing: "I want to see a focus on prevention as well as treatment, especially since so many adult mental health problems – which one in four of us will suffer from at any one time – begin in childhood."
But it's not at all clear where that statistic comes from.
That's not exactly true. In a sense, it's extremely clear where the "1 in 4" bit comes from. It comes from everywhere, as the British neuroscientist and pseudonymous blogger Neuroskeptic pointed out in a 2009 post. Several mental health charities have referred to it, and there was even a mental wellbeing magazine called One in Four for a while (it closed in 2014). But it's…complicated.
For a start, all the people saying "1 in 4" seem to disagree with each other on exactly what that means.
The prime minister says it's 1 in 4 at any given time. But the mental health charity Mind says it's 1 in 4 in any given year, apparently basing that statistic on a 2007 NHS survey that said "nearly one person in four (23.0 per cent) in England had at least one psychiatric disorder".
That doesn't sound like too terrible a difference. But a more recent NHS survey, carried out in 2014, said 26% of us report having been diagnosed with a mental health illness in our lifetimes.
And weirdly, the "1 in 4" trope seems to have come into existence well before any of these surveys had even been carried out.
Neuroskeptic said newspapers started referring to "1 in 4 Americans" suffering from mental illness in the early-to-mid 1990s. At first it tended to mean "in any given year" – now, he says, it has largely morphed to mean "at some point in life". But both co-exist, and the original source is still not clear.
The US National Comorbidity Survey came out in 1994, which matches the time-frame, said that lifetime incidence was 50%, while, just to confuse things, the seven-yearly British Adult Psychiatric Morbidity Survey began in 1993 and said 1 in 6 per week. (The 2014 version of that survey found the same figure.)
And in the 1980s the Epidemiological Catchment Area (ECA) project found that 1 in 3 Americans suffered it in their lifetimes, while 1 in 5 do in any given six months.
A 2001 World Health Organization report said 1 in 4 people in the world "will be affected by mental or neurological disorders at some point in their lives" – but that was based on the ECA.
"Where did '1 in 4' come from?" wrote Neuroskeptic. "When I set out to write this post, I thought it would be fairly easy to find out, but having done a lot of digging, I genuinely don't know."
Depending on which definition you use, "1 in 4" could be a fairly major overestimate or a fairly major underestimate.
"As a measure of lifetime problems, it's almost certainly too low," Neuroskeptic tells BuzzFeed News. "The [NHS survey quoted above] shows 1 in 4 people are diagnosed with a psychiatric disorder. But it says that another 18% on top of that suffered a problem without being diagnosed.
"So the number of people with a disorder, whether diagnosed or not, is actually closer to 1 in 2. If you're talking about the lifetime prevalence, it's going to be very high. That raises questions about whether it's going to be meaningful – whether we're talking about severe conditions that need treatment, or whether they're being overdiagnosed."
Likewise, he says, if we're talking about "at any one time" or "in any given week", it's a probable overestimate, although there was a 2005 meta-analysis that found that 27% of Europeans suffered in any given year.
And there are inherent problems with these kinds of surveys – and these kinds of statistics. One major issue is what they include.
That 2005 meta-analysis is a case in point. Neuroskeptic notes that in 2005 it said 27%, but a 2010 update said 38%. That's not because people had any more mental health issues in 2010, but because the survey included more disorders – the report itself says "this higher estimate is entirely due to the inclusion of 14 new disorders also covering childhood/adolescence as well as the elderly".
"It's a bit arbitrary, what they include," says Neuroskeptic. There are hundreds of psychological disorders listed in the Diagnostic and Statistical Manual of Mental Disorders (DSM 5). The 2010 version of the study mentioned above included things like ADHD, which it hadn't before, so the incidence shot up.
"I don't think any survey has ever looked at all the disorders," says Neuroskeptic. "For instance, if they did, it would include nicotine dependence, which is officially a mental disorder but which no one treats as one. If they included it, the prevalence of mental health problems would shoot up."
Researchers don't tend to include it and similar things because smoking doesn't, intuitively, feel like a mental health issue. But other things aren't so clear-cut, such as specific phobias like arachnophobia, which are sometimes included and sometimes not. "They're fairly common," says Neuroskeptic, "so they increase the estimates quite a lot if they're included."
Another is how the surveys are carried out.
Essentially, they're usually done by asking people a long list of questions over the phone (although the APMS is different, a 90-minute face-to-face interview with an expert).
"You ask them things like, 'Have you ever suffered from low mood for more than two weeks?'" says Neuroskeptic. "If they say yes then you ask follow-up questions, and if they answer yes to enough questions then they meet the criteria of some disorder and they are said to have suffered from it at some point."
But that's only a small part of how GPs and psychiatrists would diagnose someone, he says. They'd go through the checklist, but they'd also establish the impact it has on the person's life. "Some surveys have questions like that," he says, "But it's very difficult for a patient to answer. A lot of people who have mental health disorders won't be aware of it.
"That's especially true for more severe disorders like bipolar or schizophrenia. Not everyone has the level of insight required."
So they're relying on self-reported data, without any outside view on the person's life, such as you'd get from a GP or a psychiatrist.
So the "1 in 4" number needs to be treated with severe scepticism. Depending on what you mean by it, it might be too high, it might be too low, and it might just not be very meaningful at all.
But does that matter? Possibly. To the extent "1 in 4" is just a rallying cry, maybe it doesn't. "If it’s just a convenient way of saying that mental illness is very common, which it is, and under-resourced, which it is," says Neuroskeptic, "then maybe it doesn't matter that it's not technically accurate, because I don't think anyone's basing any important decisions on it.
"But if it's a symptom of a lack of respect for the complexities of the evidence – because, at best, it's a massively simplified number – if it's a symptom of a dumbed-down, selective approach rather than the breadth of the evidence, then it's a problem."
BuzzFeed News has approached 10 Downing Street to ask where the prime minister got her statistic from.
CORRECTION
The British Adult Psychiatric Morbidity Survey is carried out every seven years and involves face-to-face interviews. An earlier version of this piece misstated those facts.