Study suggests product reviews posted on shopping sites do not accurately reflect product benefits

Research from the University of Aberdeen suggests that product reviews posted on shopping sites like Amazon do not provide an accurate reflection of the actual benefit of the product.

The first study to compare clinical trial data with user-generated online reviews is published in Social Science and Medicine this week.

Psychologist Dr Mícheál de Barra examined more than sixteen hundred online reviews of weight loss pills and high cholesterol treatments on Amazon.com. He found the reviews portrayed the products in a far more positive light than the clinical trial data would suggest.

Dr de Barra found that the average drop in cholesterol using 'Benecol' was more than 3 times larger in Amazon reviews than was found in carefully controlled trials. Similarly, reviewers on weight loss pill Orlistat lost about twice as much weight (14kg) as those in clinical trials (7kg).

Dr de Barra who has an interest in historical and contemporary inaccurate medical beliefs said: "These treatments are not entirely ineffective. However, what we show is that the reputation as described in these reviews is much more positive than the clinical trial data show. "

Dr de Barra clarified that these reviews are not a deliberate attempt to mislead, he explains that the reviews appear exaggerated but are perhaps a result of a bias towards sharing positive outcomes rather than negative ones: "Only some people who try a treatment will then go on to tell other people about their experience, however, this subset of people are usually only those who have good outcomes. So, you hear a friend of yours had a good result using a treatment of some kind, and you think 'well maybe this works'. Your friend is probably not lying – but the problem is that people with average or poor outcomes don't tend to share their experiences. This means you get a positively skewed view of the treatment."

So - should we step away from review sites and user-generated content altogether? Dr de Barra warns: "We should be cautious about using reviews like these when deciding about health choices. These narratives have a powerful influence on our own future health behaviour because they provide simple and clear anecdotes, but this study shows that they can be very misleading."

"These results also shine a new light on medical overuse, the use of treatments that are unnecessary and ineffective. Medical overuse is estimated to cost the $226bn in the US alone, and patient demand for medicines with limited value is one important cause. This study shows how a demand for ineffective medicines can easily develop when people rely on hearsay and narratives alone.

"Realistically, however, it is ridiculous to think that every health decision we make will be informed by systematic reviews – people have lives to lead! This study shows though that it's important to be aware of the biases that can make informally acquired information unreliable."