Physician-rating websites are biased, study says

Patients posting their opinions about doctors on online ratings websites are much less likely to discuss physicians with low perceived quality and are more prone than offline populations to exaggerate their opinions, according to a paper being presented at a healthcare conference sponsored by the Institute for Operations Research and the Management Sciences (INFORMS).

"Patients need high quality information about the most consequential service that they consume: healthcare," said Ritu Agarwal, professor of information systems and director of the Center for Health Information and Decision Systems at the University of Maryland's Robert H. Smith School of Business. "They want and need to be able to make informed choices."

"The Information Value of Online Physician Ratings" is co-authored by Agarwal, Guodong "Gordon" Gao, assistant professor, and PhD candidate Brad Greenwood, both of the Smith School, along with Jeffrey McCullough of the School of Public Health at the University of Minnesota. The researchers will deliver their findings at INFORMS Healthcare 2011 http://meetings2.informs.org/healthcare2011/ on June 20 in Montreal, Quebec. The conference takes place June 20-22 at the Hilton Montreal Bonaventure. Agarwal is editor-in-chief of the INFORMS journal Information Systems Research.

The authors investigated potential among Internet users rating on websites such as Angie's List, healthgrades.com, RateMds.com, Vitals.com, and .com. They looked specifically at the way patients determine which to rate and the intensity of opinions they express.

Citing a 2010 Pew Internet and Life Project survey, the authors write, "Online physician ratings are gaining popularity among patients. The public's demand for this information is striking: 59% of U.S. adults have looked online for health information, and among them 16% have consulted rankings or reviews online of doctors or other providers. Angie's list also found that 76% of its users welcome physician rating information."

Doctors have expressed a contrasting view. The American Medical Association and many physicians have criticized online physician rating services, charging that consumers lack detailed clinical information and that those who input ratings represent a disgruntled minority rather than average patients. They contend that consumer ratings provide negligible or even misleading information.

The authors addressed these concerns in their research.

On the selection of physicians rated, they asked if patients using the online sites review the full range of doctors (including those viewed positively, neutrally, and negatively); if they are biased toward giving negative reviews to doctors (which the authors characterize as "bad mouthing"); or if they are biased toward giving mostly positive reviews (providing what the authors call "sounds of silence" about poor caregivers).

On the selection of opinions to express, the authors investigated if respondents tend toward hyperbole and largely report their most extreme experiences with doctors.

Data for the study was taken from four data sets: online data from RateMDs.com, one of the largest American physician rating services; an offline patient survey conducted by the consumer advocacy group Consumers' Checkbook for the cities of Denver, Memphis, and Kansas City; the U.S. Census 2007 Economic Census, which was a source of information about population and median income in the three cities examined; and state medical board websites.

The authors came to two major conclusions that give credence to concerns about bias among respondents to online physicians rating services.

First, they find that physicians with low ratings in offline surveys are less likely to be rated online, therefore supporting the "Sound of Silence" effect in selecting what physicians to rate.

Second, although authors find a strong correlation between the online ratings and offline population opinion, the association is strongest in the lowest quartile of opinions. These results suggest that online ratings are more informative when identifying low-quality physicians, but not as effective in discerning high quality physicians from median ones. The authors also confirm that patients are most likely to provide ratings for their most flagrant or negative experiences with physicians.

add to favorites email to friend print save as pdf

Related Stories

People more likely to overestimate their credit quality

Jun 02, 2008

A new study published in the Journal of Consumer Affairs examined consumers' self-assessments of their credit rating and found that respondents were more likely to believe they had average or above average credit and those ...

Recommended for you

Exploring 3-D printing to make organs for transplants

23 hours ago

Printing whole new organs for transplants sounds like something out of a sci-fi movie, but the real-life budding technology could one day make actual kidneys, livers, hearts and other organs for patients ...

High frequency of potential entrapment gaps in hospital beds

Jul 30, 2014

A survey of beds within a large teaching hospital in Ireland has shown than many of them did not comply with dimensional standards put in place to minimise the risk of entrapment. The report, published online in the journal ...

Key element of CPR missing from guidelines

Jul 29, 2014

Removing the head tilt/chin lift component of rescue breaths from the latest cardiopulmonary resuscitation (CPR) guidelines could be a mistake, according to Queen's University professor Anthony Ho.

Burnout impacts transplant surgeons (w/ Video)

Jul 28, 2014

Despite saving thousands of lives yearly, nearly half of organ transplant surgeons report a low sense of personal accomplishment and 40% feel emotionally exhausted, according to a new national study on transplant surgeon ...

User comments