Consumers beware: High user 'star ratings' don't mean a mobile medical app works

June 8, 2018, Johns Hopkins University School of Medicine
Credit: CC0 Public Domain

By screening 250 user reviews and comments for a once popular—but proven inaccurate—mobile app claiming to change your iPhone into a blood pressure monitor, Johns Hopkins researchers have added to evidence that a high "star rating" doesn't necessarily reflect medical accuracy or value.

Their analysis found that even the presence of app disclaimers warning users not to rely on the app for medical monitoring, diagnosis or treatment effectiveness failed to uniformly dissuade people from doing so, or stop health care professionals from recommending the app. That app, called Instant Blood Pressure and made by AuraLife, was withdrawn from the market in July 2015.

"People tend to trust user reviews when shopping online and use them to decide which products to purchase, but that doesn't cut it for medical apps," says study author Timothy Plante, M.D., M.H.S., a former fellow at the Johns Hopkins University School of Medicine and current assistant professor of medicine in the Larner College of Medicine at the University of Vermont. "There are certain thresholds of accuracy that need to be maintained, and a five-star rating doesn't replace clinical validation studies and FDA ."

The researchers say that unregulated mobile health app use could give people a false sense of security, which could lead to dire health consequences.

In a report on the study, published on June 7 in the Journal of the American Medical Informatics Association, the physicians call for medical professionals to be more skeptical of apps that don't have scientific studies backing them, and warn members of the public that just because an app has been endorsed by a health care professional doesn't mean it works.

Their new findings are a follow-up to the authors' March 2016 JAMA Internal Medicine report that analyzed the commercially available Instant Blood Pressure app, showing that four out of five times someone with used the app, they received falsely reassuring, nonelevated results.

For the new study, the researchers analyzed 261 user ratings and reviews downloaded from the Apple iTunes store before the app was taken off the market for reasons that were not made public.

The average star rating of the latest version of the app was four out of five stars, and 59 percent of the reviews assigned the app five stars. Commentary praising the accuracy of the app based on anecdotal experience comprised 42 percent of the reviews, and 10 percent of the reviews mention inaccuracy.

The Instant Blood Pressure app included a disclaimer—as do many mobile health app companies hoping to avoid FDA oversight—that the app shouldn't be used as a medical device and is for "recreational" purposes only. Yet, 24 reviews —10 percent—claimed to use the app for medical purposes, with 11 people using it to manage their high blood pressure treatments, one person using it to manage kidney disease and another person using it to monitor blood pressure after a heart transplant.

"The data showed that disclaimers aren't a complete solution. Consumers will continue to use these devices to manage their health care, which could be dangerous if they are substituting the app for medical care with a professional," says Seth Martin, M.D., M.H.S., assistant professor of medicine at the Johns Hopkins University School of Medicine and one of the study's investigators.

Six reviews came from people who claimed to be including four nurses and one physician. These people gave an average rating of 4.2 stars out of five. Eleven reviews came from people who said their health care provider—a total of four physicians and seven nurses—approved of the app. Two people said that a nurse and an emergency room physician disapproved of the app, and as a result gave a one-star rating.

"Physicians need to be careful; if you're saying you personally use an app, people will trust it," says Plante. "Our take-home message to health care providers is to make sure you hold apps to the highest standards, and not just give out recommendations willy-nilly for apps that you think are neat. And consumers should be skeptical of reviews from self-proclaimed health professionals in product reviews."

The Instant Blood Pressure app sold for $4.99 on iTunes and was downloaded 140,000 times before it was withdrawn. It ranked among the best-selling apps on all of iTunes for much of its stretch on the . After the app was withdrawn and following the authors' last study, the Federal Trade Commission settled with the company for $600,000, with payment suspended for lack of funds in the company. Many medical apps are still available for consumers, some downloading as many as 1,000,000 times each.

"I'm glad for the high demand from consumers to be able to monitor their from day to day," says Martin. "Now we need to actually make sure mobile apps work as intended, in properly designed and conducted clinical studies, before they are relied on for ."

The authors recommend using home devices proven to be accurate by independent reviewers.

Explore further: Data from online physician review sites may be skewed and misleading to consumers, new study finds

Related Stories

Data from online physician review sites may be skewed and misleading to consumers, new study finds

May 29, 2018
Physician satisfaction scores on online third-party review sites tend to be skewed and can easily mislead patients, according to a new study by Cedars-Sinai investigators.

Searching for a nursing home and don't know where to turn?

April 26, 2018
Yelp reviews of nursing homes tend to focus more on staff attitudes, staff responsiveness and the physical facility itself than government reviews, a new USC study finds.

How well did an instant blood pressure app work?

March 2, 2016
A blood pressure (BP) smartphone app delivered inaccurate results in a small study, which suggests more than three-quarters of individuals with hypertensive BP levels may be falsely reassured that their BP is in the nonhypertensive ...

Study finds significant limitations of physician-rating websites

February 21, 2017
An analysis of 28 commercial physician-rating websites finds that search mechanisms are cumbersome, and reviews scarce, according to a study appearing in the February 21 issue of JAMA.

Be wary of cosmetic surgeons' online reviews

April 26, 2018
Consumers consider online reviews important for choosing physicians, but they should be wary of using those ratings to choose plastic surgeons. The reviews tend to be polarized, and some are written by people whom consulted ...

Don't rely on smartphone apps to treat back pain

June 7, 2017
University of Sydney researchers have found that smartphone apps for treating back pain have questionable value as they are generally of poor quality, and have not been rigorously evaluated.

Recommended for you

E-cigarette vapor disables key immune cells in the lung and boosts inflammation

August 14, 2018
E-cigarette vapour boosts the production of inflammatory chemicals and disables key protective cells in the lung that keep the air spaces clear of potentially harmful particles, reveals a small experimental study, published ...

New study finds fake, low-quality medicines prevalent in the developing world

August 10, 2018
A new study from the University of North Carolina at Chapel Hill found that substandard and falsified medicines, including medicines to treat malaria, are a serious problem in much of the world. In low- and middle-income ...

Insurance status tied to higher self-perceived poor/fair health

August 9, 2018
(HealthDay)—Underinsured and never insured adults are more likely than adequately insured adults to report poor/fair health and frequent mental distress (FMD), according to a study published online July 19 in the U.S. Centers ...

Giving kids plates with segments and pictures caused them to eat more vegetables

August 8, 2018
A pair of researchers at the University of Colorado has found that preschool kids ate more vegetables when presented with segmented plates with pictures of fruits and vegetables on them. In their paper published in JAMA Pediatrics, ...

Carbon dioxide levels on flight deck affect airline pilot performance

August 8, 2018
Commercial airline pilots were significantly better at performing advanced maneuvers in a flight simulator when carbon dioxide (CO2) levels on the flight deck (cockpit) were 700 parts per million (ppm) and 1500 ppm than when ...

Boxers or briefs? Loose-fitting underwear may benefit sperm production

August 8, 2018
Men who most frequently wore boxers had significantly higher sperm concentrations and total sperm counts when compared with men who did not usually wear boxers, according to new research led by Harvard T.H. Chan School of ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.