A couple years ago, Kit Huckvale, a researcher at Imperial College London, noticed an increasing number of apps that purported to help diabetics track and manage their insulin levels. Which made sense to Huckvale: Though insulin dose calculators already exist as stand-alone devices, smartphones are portable, convenient, and ever-present. "There's a really good a priori case as to why … a mobile device would be good for health," he told BuzzFeed News. "They're always carried, they're always on. If something happens to you, it could be something you turn to. That's particularly relevant for insulin calculators — it's something you have to do at mealtimes."
But Huckvale, a clinician by training, was curious about these apps' accuracy and efficacy — especially because they, like the vast majority of health-related smartphone apps, are not regulated by the Food and Drug Administration. So in 2013 and 2014, he and his colleagues downloaded 46 iOS and Android apps that claimed to help users calculate how much insulin to take before a meal to cover the expected blood-glucose increase. The apps let users enter the amount of carbohydrates they planned to eat and then calculated the dosage needed to offset the subsequent spike in blood sugar.
In results published in May, Huckvale's team discovered that most apps yielded a surprising amount of errors — some small, some significant. While most of the apps came with a disclaimer, less than one-third detailed how their calculations worked, which made it hard to know if they were correct. More than 90% of the apps failed to block clear numerical errors — like extremely high insulin doses or blood sugar levels — and more than half generated a calculation even when they weren't given enough data to work off of. Two-thirds gave potentially inappropriate dose recommendations.
In fact, only one of the 46 apps was issue-free by the study's criteria. (Huckvale declined to name the app, saying the point wasn't to single out apps but to examine the category as a whole.) Most insulin dose calculator apps, the researchers concluded in no uncertain terms, "provide no protection against, and may actively contribute to, incorrect or inappropriate dose recommendations that put current users at risk of both catastrophic overdose and more subtle harms resulting from suboptimal glucose control."
To be fair, Huckvale told BuzzFeed News, "a lot of individuals will have lots of experience managing insulin, and they'll probably reject a really silly insulin dose." But, he said, it's possible that "some individuals could experience immediate harm because an app gives a false recommendation." And the stakes of that harm is high: An insulin overdose can lead to dizziness, headaches, and nausea at best — and at worst, coma and seizures.
Though the FDA may soon re-evaluate how it polices insulin apps, these are far from the only health apps that go unregulated by the FDA. Some of these apps, which number in the thousands, have been found to give misinformation for conditions as wide-ranging as cancer, skin lesions, colorectal disease, and reproductive health, putting unsuspecting consumers at risk of harm.
In 2013, researchers tested four skin cancer diagnosis apps, which analyzed pictures of users' moles, and found that three missed at least one in three melanomas. In February, the Federal Trade Commission fined two companies behind apps that purported, without scientific evidence, to calculate users' melanoma risk based on a picture of their skin. (A third such company settled with the agency today.) In 2011, the pharmaceutical giant Pfizer withdrew a rheumatology calculator app after discovering that its swollen-joint measurements — based on user-reported data — were faulty. In addition, the many apps that track menstrual cycles and fertility are virtually un-vetted by experts, and only one-third of colorectoral disease apps involve medical professionals, a 2012 study found.
As smartphones become increasingly entwined with all our activities, from banking to shopping to socializing, and as the quantified-self movement marches on, more and more people are downloading health apps: In 2013, the total number of health apps downloaded from the Google Play store alone topped 660 million times. "If you have an app that's inexpensive, that somebody believes will lower of the cost of health care, there's going to be a certain percentage of the population that's going to be swayed by that and say, 'You know what, I'm going to give that a try,'" said David Rosen, a health law attorney at Foley & Lardner LLP.
But there are trade-offs that come with, say, using an app instead of a dermatologist to check out that weird mole. "You've got to take the claims made with a grain of salt, and you can't rely on that the FDA is watching," Marc Scheineson, an attorney who specializes in food and drug law at the law firm Alston & Bird, told BuzzFeed News. "You need to be your own medical consumer and your own medical advocate, i.e., don't rely exclusively on an app ... If you have questions, talk to your doctor."
In the case of insulin-calculator apps, perhaps fortunately, the number of people who actually use them seems to be low: Many of the apps in that category lack any user ratings in Apple's App Store, and Matt Petersen, managing director of medical information at the American Diabetes Association, said he had not heard of patients or physicians using any such apps. One of the few exceptions, however, is BlueStar, a type 2 diabetes management app that requires a doctor's prescription to download.
BlueStar is one of the select apps regulated by the FDA. According to guidelines updated in February, the FDA regulates only apps that it considers to be highly risky. These are apps that act as accessories to a medical device — like an app that remotely controls an infusion pump — or turn a smartphone into a medical device. Only a small subset of all health apps meet this definition: The FDA reviews approximately 20 new apps a year, said Bakul Patel, associate director for digital health at the FDA's Center for Devices and Radiological Health.
"It's a very complicated area," Patel said. "If [an insulin dosage recommendation] was wrong and people did take the wrong dose, you could actually have a very serious situation on hand. And calculator-app formats run the gamut from really simple — adding two numbers — to extremely complex … That's what we're trying to figure out: what's the right and appropriate way to think about calculators in a very sensitive disease area."
Makers of insulin-calculator apps defended their product when contacted by BuzzFeed News. Simon Carter, who runs the software company DataMystic, created one such app as a way to help himself and his daughter, both diagnosed with type 1 diabetes. In an attempt to prevent the user from overdosing, he said, his app recalculates the estimated amount of insulin remaining in a user's body every minute. Other apps, he says, update their estimates much less frequently, or ask the user to guess how much insulin remains, which leaves more room for error.
"I certainly know of other dose calculators that are plain wrong, confusing, (and) have no validation of inputs," Carter wrote in an email. "The issue is: doctors and people with few computer skills try to do something that requires a LOT of skill. My background is electrical engineering and computer science, and I have been running a software company for 15 years. I have had diabetes for 26 years, so my business is ideally placed to develop these tools."
Maria Yu was inspired by a friend with type 1 diabetes to make two apps that perform insulin calculations. She found a "commonly used" formula by searching the internet and attending diabetes events, she said, and checking it with her friend. "Many users contacted me and told me that my two apps were very easy for them to use," she wrote in an email. She noted that the app warns users that "it must only be used as a guide, and upon advice and consultation with your physician."
But Huckvale, the researcher, said that kind of disclaimer isn't good enough, nor are well-meaning intentions, when it comes to people's health. Developers "seemed to be acting in good faith, of those responded to us. One-third said, 'I have diabetes or a family member has diabetes, I wanted to help them out,'" he said. "That, I guess, is very honorable and useful to them. But there's just a bit of a gap between that hacking attitude and producing a clinically safe robust tool."
Stephanie M. Lee is a science reporter for BuzzFeed News and is based in San Francisco.
Contact Stephanie M. Lee at email@example.com.
Got a confidential tip? Submit it here.