Consumers beware: High user 'star ratings' don't mean a mobile medical app works

June 8, 2018, Johns Hopkins University School of Medicine
Credit: CC0 Public Domain

By screening 250 user reviews and comments for a once popular—but proven inaccurate—mobile app claiming to change your iPhone into a blood pressure monitor, Johns Hopkins researchers have added to evidence that a high "star rating" doesn't necessarily reflect medical accuracy or value.

Their analysis found that even the presence of app disclaimers warning users not to rely on the app for medical monitoring, diagnosis or treatment effectiveness failed to uniformly dissuade people from doing so, or stop health care professionals from recommending the app. That app, called Instant Blood Pressure and made by AuraLife, was withdrawn from the market in July 2015.

"People tend to trust user reviews when shopping online and use them to decide which products to purchase, but that doesn't cut it for medical apps," says study author Timothy Plante, M.D., M.H.S., a former fellow at the Johns Hopkins University School of Medicine and current assistant professor of medicine in the Larner College of Medicine at the University of Vermont. "There are certain thresholds of accuracy that need to be maintained, and a five-star rating doesn't replace clinical validation studies and FDA ."

The researchers say that unregulated mobile health app use could give people a false sense of security, which could lead to dire health consequences.

In a report on the study, published on June 7 in the Journal of the American Medical Informatics Association, the physicians call for medical professionals to be more skeptical of apps that don't have scientific studies backing them, and warn members of the public that just because an app has been endorsed by a health care professional doesn't mean it works.

Their new findings are a follow-up to the authors' March 2016 JAMA Internal Medicine report that analyzed the commercially available Instant Blood Pressure app, showing that four out of five times someone with used the app, they received falsely reassuring, nonelevated results.

For the new study, the researchers analyzed 261 user ratings and reviews downloaded from the Apple iTunes store before the app was taken off the market for reasons that were not made public.

The average star rating of the latest version of the app was four out of five stars, and 59 percent of the reviews assigned the app five stars. Commentary praising the accuracy of the app based on anecdotal experience comprised 42 percent of the reviews, and 10 percent of the reviews mention inaccuracy.

The Instant Blood Pressure app included a disclaimer—as do many mobile health app companies hoping to avoid FDA oversight—that the app shouldn't be used as a medical device and is for "recreational" purposes only. Yet, 24 reviews —10 percent—claimed to use the app for medical purposes, with 11 people using it to manage their high blood pressure treatments, one person using it to manage kidney disease and another person using it to monitor blood pressure after a heart transplant.

"The data showed that disclaimers aren't a complete solution. Consumers will continue to use these devices to manage their health care, which could be dangerous if they are substituting the app for medical care with a professional," says Seth Martin, M.D., M.H.S., assistant professor of medicine at the Johns Hopkins University School of Medicine and one of the study's investigators.

Six reviews came from people who claimed to be including four nurses and one physician. These people gave an average rating of 4.2 stars out of five. Eleven reviews came from people who said their health care provider—a total of four physicians and seven nurses—approved of the app. Two people said that a nurse and an emergency room physician disapproved of the app, and as a result gave a one-star rating.

"Physicians need to be careful; if you're saying you personally use an app, people will trust it," says Plante. "Our take-home message to health care providers is to make sure you hold apps to the highest standards, and not just give out recommendations willy-nilly for apps that you think are neat. And consumers should be skeptical of reviews from self-proclaimed health professionals in product reviews."

The Instant Blood Pressure app sold for $4.99 on iTunes and was downloaded 140,000 times before it was withdrawn. It ranked among the best-selling apps on all of iTunes for much of its stretch on the . After the app was withdrawn and following the authors' last study, the Federal Trade Commission settled with the company for $600,000, with payment suspended for lack of funds in the company. Many medical apps are still available for consumers, some downloading as many as 1,000,000 times each.

"I'm glad for the high demand from consumers to be able to monitor their from day to day," says Martin. "Now we need to actually make sure mobile apps work as intended, in properly designed and conducted clinical studies, before they are relied on for ."

The authors recommend using home devices proven to be accurate by independent reviewers.

Explore further: Data from online physician review sites may be skewed and misleading to consumers, new study finds

Related Stories

Data from online physician review sites may be skewed and misleading to consumers, new study finds

May 29, 2018
Physician satisfaction scores on online third-party review sites tend to be skewed and can easily mislead patients, according to a new study by Cedars-Sinai investigators.

Searching for a nursing home and don't know where to turn?

April 26, 2018
Yelp reviews of nursing homes tend to focus more on staff attitudes, staff responsiveness and the physical facility itself than government reviews, a new USC study finds.

How well did an instant blood pressure app work?

March 2, 2016
A blood pressure (BP) smartphone app delivered inaccurate results in a small study, which suggests more than three-quarters of individuals with hypertensive BP levels may be falsely reassured that their BP is in the nonhypertensive ...

Study finds significant limitations of physician-rating websites

February 21, 2017
An analysis of 28 commercial physician-rating websites finds that search mechanisms are cumbersome, and reviews scarce, according to a study appearing in the February 21 issue of JAMA.

Be wary of cosmetic surgeons' online reviews

April 26, 2018
Consumers consider online reviews important for choosing physicians, but they should be wary of using those ratings to choose plastic surgeons. The reviews tend to be polarized, and some are written by people whom consulted ...

Don't rely on smartphone apps to treat back pain

June 7, 2017
University of Sydney researchers have found that smartphone apps for treating back pain have questionable value as they are generally of poor quality, and have not been rigorously evaluated.

Recommended for you

Self-lubricating latex could boost condom use: study

October 17, 2018
A perpetually unctuous, self-lubricating latex developed by a team of scientists in Boston could boost the use of condoms, they reported Wednesday in the journal Royal Society Open Science.

How healthy will we be in 2040?

October 17, 2018
A new scientific study of forecasts and alternative scenarios for life expectancy and major causes of death in 2040 shows all countries are likely to experience at least a slight increase in lifespans. In contrast, one scenario ...

Study finds evidence of intergenerational transmission of trauma among ex-POWs from the Civil War

October 16, 2018
A trio of researchers affiliated with the National Bureau of Economic Research has found evidence that suggests men who were traumatized while POWs during the U.S. Civil War transmitted that trauma to their offspring—many ...

Father's nicotine use can cause cognitive problems in children and grandchildren

October 16, 2018
A father's exposure to nicotine may cause cognitive deficits in his children and even grandchildren, according to a study in mice publishing on October 16 in the open-access journal PLOS Biology by Pradeep Bhide of Florida ...

Many supplements contain unapproved, dangerous ingredients: study

October 13, 2018
(HealthDay)—U.S. health officials have issued more than 700 warnings during the last decade about the sale of dietary supplements that contain unapproved and potentially dangerous drug ingredients, new research reveals.

Age at which women experience their first period is linked to their sons' age at puberty

October 12, 2018
The age at which young women experience their first menstrual bleeding is linked to the age at which their sons start puberty, according to the largest study to investigate this association in both sons and daughters.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.