There's a study out in a journal called Acupuncture in Medicine (AIM) that claims to have found evidence that acupuncture helps relieve colic in babies.
Colic is a catch-all term for when babies cry for unknown reasons. The usual medical definition is that if a baby cries for more than three hours a day, they have got colic.
The new study was carried out by scientists at Lund University, Sweden, and looked at 426 babies whose parents were worried about colic. Of those, 157 were found to actually have the condition.
AIM is published by the British Medical Journal (BMJ), one of the most respected medical journals in the world.
But other scientists are extremely sceptical about the results – to the point that they suggest that not only should the journal not have run it, but the BMJ shouldn't even publish the journal in the first place.
They say that the study is full of statistical errors and bad practice, and that journals like this one, which exclusively study particular forms of alternative medicine, rarely if ever publish studies that suggest that the alternative medicine doesn't work.
Here's what the study did. It took its 426 infants and divided them up at random into three groups. One group had a very minimal acupuncture session, one had a more intensive one, and one had no acupuncture at all.
Dr Kajsa Landgren of Lund University, one of the authors of the study, told BuzzFeed News that all three groups underwent the same procedure. The parents brought their baby in to a child health centre, they met a nurse who spoke to them for 20 minutes, and then the baby was taken through to another room where there was an acupuncturist.
Depending on which group they'd been put in, the acupuncturist either performed a single jab, lasting no more than five seconds; up to five jabs, lasting up to 30 seconds each; or no acupuncture at all, simply keeping the baby in the room for five minutes. Neither the parents nor the nurses knew what treatment the babies received.
The headline result of the study is that the babies who underwent acupuncture suffered less from colic than the babies who didn't.
"Those who had acupuncture cried significantly less" than those who didn't, said Landgren. She said that almost double the number of babies dropped below "the magic three hours per day" in the treatment groups that had acupuncture, compared with those in the control group that didn't.
But two scientists told BuzzFeed News that in fact the study showed no such thing.
Edzard Ernst, a retired professor of complementary medicine at the University of Exeter, told BuzzFeed News: "I’m more than sceptical. I think the study says the opposite of what the authors conclude."
David Colquhoun, a professor of pharmacology at University College London and author of the sceptical blog Improbable Science, called the paper "ghastly".
There are various problems with it. One is the lack of a proper control group, critics say.
Ernst points out that the control group – the babies who didn't have acupuncture – didn't have any sort of treatment, such as sham acupuncture. "The sceptical view is that acupuncture is a theatrical placebo – there's a ritual involved, and it's the ritual that works, not the acupuncture itself." It may be that shaking a magic stick over the baby would work just as well, he said, if it was associated with a similar ritual.
Colquhoun agreed, telling BuzzFeed News that an earlier study into chiropractic found that simply cuddling a baby was just as effective as giving them chiropractic treatment for colic. So there's a chance, he argued, that the babies cried less simply because the acupuncturist did something stimulating, rather than letting them lie there unattended for five minutes.
Another problem is that the two forms of acupuncture treatment – a single jab or many – seem to be equally effective, which isn't what you'd expect if they actually worked.
Landgren said that it was "extraordinary" that such a small intervention could have this effect. But Ernst said that this was a problem with the study, not a strength – if acupuncture is effective, you'd expect a larger dose to have more effect than a tiny one.
"I think it's basically shooting the acupuncture hypothesis in the foot," he said. "This elaborate TCM [traditional Chinese medicine] acupuncture [the five jab, 30-second-per-jab version] did no better than what they call minimal acupuncture.
"To me, and to most sceptical minds, that suggests that acupuncture is a placebo."
And, most damningly in the eyes of the scientists, there are some basic statistical practices that the study has not followed, meaning that it is far more likely to find an apparent connection.
These failures are quite technical, but the most noteworthy is that the study's authors looked at 24 different possible outcomes and reported on the only three that achieved "statistical significance". Doing that after the results are in is called "outcome switching" or "p-hacking", and allows you to – in the words of another scientist who spoke to BuzzFeed News for an earlier piece – "choose your lottery numbers after seeing the draw".
If you measure 24 different things, "you'd expect one or two [statistically significant] results just by chance," said Colquhoun. But, he said, the study hasn't accounted for that.
Also, of the 426 babies examined, only 157 actually were found to have colic, which, he said, reduces the sample size and therefore the power of the study still further. "The statistical analysis is appalling," he said.
"There are plenty of problems with the statistical analysis," agreed Ernst.
What the scientists feel is particularly bad is that the BMJ publishes the journal in question.
"It's horrifying that the BMJ should publish AIM at all," said Colquhoun. "It's reviewed by other acupuncturists who don't understand statistics. The BMJ itself is usually pretty good on statistics and p-hacking, but this wasn't seen by the BMJ."
Ernst said that he had studied journals such as AIM before – journals that solely looked at one kind of alternative medicine – and found that they rarely, if ever, publish studies that do not find a positive result for the treatment they specialise in: "We found that in three five-year intervals, so a total of 15 years of publishing, the journals we looked at did not publish a single negative result."
AIM makes a "tidy profit", said Ernst, because it is sent to all members of the British Medical Acupuncture Society, and was acquired by the BMJ in 2008.
Overall, they say, the study doesn't give us any more reason to think that acupuncture works, but it does tell "a sad story about medical publishing today", according to Ernst.
Colquhoun agreed: "It's likely that there aren't any effects at all [of the acupuncture treatment], and this study muddies the water more than it clarifies them."
The BMJ said in a statement: "BMJ publishes Acupuncture in Medicine on behalf of the British Medical Acupuncture Society, and this is one of several titles BMJ publishes on behalf of academic and scientific bodies.
"The journal aims to gather good-quality evidence to further the understanding of medical acupuncture in healthcare, to promote safety and to create a reliable evidence base to help people decide which types of acupuncture, if any, work in which conditions.
"This is also part of BMJ's wider mission to advance healthcare worldwide by sharing evidence-based knowledge and expertise."
CORRECTION
The BMJ publishes AIM on behalf of the British Medical Acupuncture Society. An earlier version of this piece incorrectly stated that it owned the journal.