For a less biased study, try randomization

By Randy Dotinga

A new review of existing research confirms that the so-called “gold standard” of medical research — the randomized controlled study — provides a safeguard against bias.

The researchers found that certain kinds of studies — those whose recruiters were more likely to know in which comparison group the next participant would land — had larger estimates of effects than those that whose “blinding” was considered adequate. Yet the finding does not mean such studies are inaccurate.

The findings could give researchers more insight into the pros and cons of the strategies they embrace to figure out whether one drug or treatment is better than another. While it might seem obvious that a randomized study is the most reliable option, scientists do not universally believe it is, said review lead author Jan Odgaard-Jensen.

“Although there is wide agreement about the value of , some people still argue that randomized trials may not be more reliable than non-randomized studies,” said Odgaard-Jensen, a statistician with the Norwegian Knowledge Center for the Health Service.

Even if researchers do acknowledge the superiority of randomized studies, many turn to a non-randomized approach that is cheaper and less complicated. For example, they could try to allocate patients in a study in a way that is not entirely random, such as assigning them to treatments based on ZIP code or gender, Odgaard-Jensen said.

So how much of a difference does randomization make in the results of a study? The review authors sought to find out by updating the results of a previous review.

The researchers deemed 18 papers — all reviews or meta-analyses — to fit their criteria to include in the review, which appears in the latest issue of The Cochrane Library.

The journal is a publication of The Cochrane Collaboration, an international organization that evaluates medical research. Systematic reviews draw evidence-based conclusions about medical practice after considering both the content and quality of existing medical trials on a topic.

The review authors found that randomized and non-randomized studies did not consistently yield a certain kind of result. Sometimes the estimates of effect — a measurement of, say, whether a medication works — were larger in randomized studies. However, they were larger in some non-randomized studies too.

The review authors write that, in general, it is not possible to figure out how biases in the kinds of studies could throw off the results. In other words, one cannot assume that a randomized controlled trial that yields a certain estimates of effect is more or less accurate than one that does not randomly assign participants, which yields a different estimate.

Dr. Gordon Guyatt, a professor of medicine at McMaster University, in Ontario, Canada, said the review findings confirm that we do not live in a “simpler and more easy to handle world” in which “biases would always go in the same direction.”

“It turns out that the world is not a simple place,” he said. “Sometimes the bias goes in one direction and sometimes in another.”

However, the unpredictability of randomization in a randomized study — when it’s impossible to know beforehand whether a patient will end up in one group or another — adds to the value of that specific approach, review lead author Odgaard-Jensen said. It is “the best protection against the unpredictability of the extent and direction of bias in clinical trials that are not properly randomized.”

Still, randomized trials are not guaranteed routes to the truth regarding a medical question, said Dr. Arthur Hartz, director of health service research at Huntsman Cancer Institute in Salt Lake City. “I don’t think any single study is conclusive. You often get randomized controlled trials that disagree with each other.”

Randomized studies are often a nuisance too, he said. “They take incredible amounts of time and money, and they’re boring, and by the time you get the results, medical technology has moved on.”

One alternative is an observational study, in which researchers do not control which subjects get a particular medication or treatment, he said. (The new review did not examine the value of observational studies.)

“We have some indications of where observational studies are giving bad results, but we don’t understand exactly why they give bad results and how it could be fixed,” Hartz said. “That’s where the emphasis should be.”

For his part, Guyatt said randomization is one of several safeguards against bias in studies, such as preventing various players from knowing who is getting which treatment and following up with as many subjects as possible to see what happened to them. “If all these bias-reducing strategies are in place, randomized trials will always get you closer to the truth than studies that don’t have all the safeguards,” he said.

Ultimately, said Tufts University professor of medicine Dr. Joseph Lau, the new review “has value to those developing tools to critically appraise studies in systematic reviews. It also has value to those proposing guidelines on improving the conduct and reporting of primary studies.”

More information: Odgaard-Jensen J, et al. Randomisation to protect against selection bias in healthcare trials. Cochrane Database of Systematic Reviews 2011, Issue 4.

add to favorites email to friend print save as pdf

Related Stories

Mechanical versus manual CPR may be too close to call

Jan 19, 2011

Pushing on the chest to simulate the heart’s rhythmic pumping action is an essential part of cardiopulmonary resuscitation after cardiac arrest. In recent decades, manufacturers have developed several mechanical devices ...

Evidence of ginseng-boosted brain power is weak

Dec 08, 2010

Many people believe that the popular herb ginseng can improve thinking ability and prevent or even treat dementia. However, a comprehensive review of research failed to find convincing evidence of these benefits.

For back pain, spinal manipulation holds its own

Feb 16, 2011

If you're suffering from chronic lower back pain, a new review of existing research finds that spinal manipulation − the kind of hands-on regimen that a chiropractor might perform on you − is as helpful as other ...

Recommended for you

The hunt for botanicals

Dec 19, 2014

Herbal medicine can be a double-edged sword and should be more rigorously investigated for both its beneficial and harmful effects, say researchers writing in a special supplement of Science.

Mozambique decriminalises abortion to stem maternal deaths

Dec 19, 2014

Mozambique has passed a law permitting women to terminate unwanted pregnancies under specified conditions, a move hailed by activists in a country where clandestine abortions account for a large number of maternal deaths.

User comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.