Consumers beware: High user 'star ratings' don't mean a mobile medical app works

June 8, 2018, Johns Hopkins University School of Medicine
Credit: CC0 Public Domain

By screening 250 user reviews and comments for a once popular—but proven inaccurate—mobile app claiming to change your iPhone into a blood pressure monitor, Johns Hopkins researchers have added to evidence that a high "star rating" doesn't necessarily reflect medical accuracy or value.

Their analysis found that even the presence of app disclaimers warning users not to rely on the app for medical monitoring, diagnosis or treatment effectiveness failed to uniformly dissuade people from doing so, or stop health care professionals from recommending the app. That app, called Instant Blood Pressure and made by AuraLife, was withdrawn from the market in July 2015.

"People tend to trust user reviews when shopping online and use them to decide which products to purchase, but that doesn't cut it for medical apps," says study author Timothy Plante, M.D., M.H.S., a former fellow at the Johns Hopkins University School of Medicine and current assistant professor of medicine in the Larner College of Medicine at the University of Vermont. "There are certain thresholds of accuracy that need to be maintained, and a five-star rating doesn't replace clinical validation studies and FDA ."

The researchers say that unregulated mobile health app use could give people a false sense of security, which could lead to dire health consequences.

In a report on the study, published on June 7 in the Journal of the American Medical Informatics Association, the physicians call for medical professionals to be more skeptical of apps that don't have scientific studies backing them, and warn members of the public that just because an app has been endorsed by a health care professional doesn't mean it works.

Their new findings are a follow-up to the authors' March 2016 JAMA Internal Medicine report that analyzed the commercially available Instant Blood Pressure app, showing that four out of five times someone with used the app, they received falsely reassuring, nonelevated results.

For the new study, the researchers analyzed 261 user ratings and reviews downloaded from the Apple iTunes store before the app was taken off the market for reasons that were not made public.

The average star rating of the latest version of the app was four out of five stars, and 59 percent of the reviews assigned the app five stars. Commentary praising the accuracy of the app based on anecdotal experience comprised 42 percent of the reviews, and 10 percent of the reviews mention inaccuracy.

The Instant Blood Pressure app included a disclaimer—as do many mobile health app companies hoping to avoid FDA oversight—that the app shouldn't be used as a medical device and is for "recreational" purposes only. Yet, 24 reviews —10 percent—claimed to use the app for medical purposes, with 11 people using it to manage their high blood pressure treatments, one person using it to manage kidney disease and another person using it to monitor blood pressure after a heart transplant.

"The data showed that disclaimers aren't a complete solution. Consumers will continue to use these devices to manage their health care, which could be dangerous if they are substituting the app for medical care with a professional," says Seth Martin, M.D., M.H.S., assistant professor of medicine at the Johns Hopkins University School of Medicine and one of the study's investigators.

Six reviews came from people who claimed to be including four nurses and one physician. These people gave an average rating of 4.2 stars out of five. Eleven reviews came from people who said their health care provider—a total of four physicians and seven nurses—approved of the app. Two people said that a nurse and an emergency room physician disapproved of the app, and as a result gave a one-star rating.

"Physicians need to be careful; if you're saying you personally use an app, people will trust it," says Plante. "Our take-home message to health care providers is to make sure you hold apps to the highest standards, and not just give out recommendations willy-nilly for apps that you think are neat. And consumers should be skeptical of reviews from self-proclaimed health professionals in product reviews."

The Instant Blood Pressure app sold for $4.99 on iTunes and was downloaded 140,000 times before it was withdrawn. It ranked among the best-selling apps on all of iTunes for much of its stretch on the . After the app was withdrawn and following the authors' last study, the Federal Trade Commission settled with the company for $600,000, with payment suspended for lack of funds in the company. Many medical apps are still available for consumers, some downloading as many as 1,000,000 times each.

"I'm glad for the high demand from consumers to be able to monitor their from day to day," says Martin. "Now we need to actually make sure mobile apps work as intended, in properly designed and conducted clinical studies, before they are relied on for ."

The authors recommend using home devices proven to be accurate by independent reviewers.

Explore further: Data from online physician review sites may be skewed and misleading to consumers, new study finds

Related Stories

Data from online physician review sites may be skewed and misleading to consumers, new study finds

May 29, 2018
Physician satisfaction scores on online third-party review sites tend to be skewed and can easily mislead patients, according to a new study by Cedars-Sinai investigators.

Searching for a nursing home and don't know where to turn?

April 26, 2018
Yelp reviews of nursing homes tend to focus more on staff attitudes, staff responsiveness and the physical facility itself than government reviews, a new USC study finds.

How well did an instant blood pressure app work?

March 2, 2016
A blood pressure (BP) smartphone app delivered inaccurate results in a small study, which suggests more than three-quarters of individuals with hypertensive BP levels may be falsely reassured that their BP is in the nonhypertensive ...

Study finds significant limitations of physician-rating websites

February 21, 2017
An analysis of 28 commercial physician-rating websites finds that search mechanisms are cumbersome, and reviews scarce, according to a study appearing in the February 21 issue of JAMA.

Be wary of cosmetic surgeons' online reviews

April 26, 2018
Consumers consider online reviews important for choosing physicians, but they should be wary of using those ratings to choose plastic surgeons. The reviews tend to be polarized, and some are written by people whom consulted ...

Don't rely on smartphone apps to treat back pain

June 7, 2017
University of Sydney researchers have found that smartphone apps for treating back pain have questionable value as they are generally of poor quality, and have not been rigorously evaluated.

Recommended for you

Living the high life: How altitude influences bone growth

June 19, 2018
High altitude is a particularly challenging environment—the terrain is physically challenging and the land has a relatively poor crop yield, so food can be sparse. Most importantly, oxygen levels are lower meaning that ...

Risks of cancer and mortality by average lifetime alcohol intake

June 19, 2018
The risk of mortality, and of developing a number of cancers, is lowest in light drinkers consuming an average of less than one drink per day across their lifetime, and the risk of some cancers increases with each additional ...

Bad habits that lead to cancer, chronic disease corrected by simple lifestyle intervention

June 19, 2018
Does this sound like someone you know? He or she spends too much time in front of screens, gets little exercise and eats a diet high in fat and low in fruits and vegetables.

Introduction of alcohol found to adversely impact fertility rates in hunter-gatherer community

June 19, 2018
Fernando Ramirez Rozzi, a research director with the French National Centre for Scientific Research has found that the introduction of alcohol to a Baka pygmy hunter-gatherer society caused fertility rates to fall. In his ...

Graphic warning labels linked to reduced sugary drink purchases

June 18, 2018
Warning labels that include photos linking sugary drink consumption with obesity, type 2 diabetes, and tooth decay, may reduce purchases of the drinks, according to a new study by researchers from Harvard T.H. Chan School ...

Greater levels of vitamin D associated with decreasing risk of breast cancer

June 15, 2018
Researchers at University of California San Diego School of Medicine suggest higher levels of vitamin D are associated with decreasing risk of breast cancer. Their epidemiological study is published in the June 15 online ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.