For a less biased study, try randomization

April 13, 2011 By Randy Dotinga, Health Behavior News Service

A new review of existing research confirms that the so-called “gold standard” of medical research — the randomized controlled study — provides a safeguard against bias.

The researchers found that certain kinds of studies — those whose recruiters were more likely to know in which comparison group the next participant would land — had larger estimates of effects than those that whose “blinding” was considered adequate. Yet the finding does not mean such studies are inaccurate.

The findings could give researchers more insight into the pros and cons of the strategies they embrace to figure out whether one drug or treatment is better than another. While it might seem obvious that a randomized study is the most reliable option, scientists do not universally believe it is, said review lead author Jan Odgaard-Jensen.

“Although there is wide agreement about the value of , some people still argue that randomized trials may not be more reliable than non-randomized studies,” said Odgaard-Jensen, a statistician with the Norwegian Knowledge Center for the Health Service.

Even if researchers do acknowledge the superiority of randomized studies, many turn to a non-randomized approach that is cheaper and less complicated. For example, they could try to allocate patients in a study in a way that is not entirely random, such as assigning them to treatments based on ZIP code or gender, Odgaard-Jensen said.

So how much of a difference does randomization make in the results of a study? The review authors sought to find out by updating the results of a previous review.

The researchers deemed 18 papers — all reviews or meta-analyses — to fit their criteria to include in the review, which appears in the latest issue of The Cochrane Library.

The journal is a publication of The Cochrane Collaboration, an international organization that evaluates medical research. Systematic reviews draw evidence-based conclusions about medical practice after considering both the content and quality of existing medical trials on a topic.

The review authors found that randomized and non-randomized studies did not consistently yield a certain kind of result. Sometimes the estimates of effect — a measurement of, say, whether a medication works — were larger in randomized studies. However, they were larger in some non-randomized studies too.

The review authors write that, in general, it is not possible to figure out how biases in the kinds of studies could throw off the results. In other words, one cannot assume that a randomized controlled trial that yields a certain estimates of effect is more or less accurate than one that does not randomly assign participants, which yields a different estimate.

Dr. Gordon Guyatt, a professor of medicine at McMaster University, in Ontario, Canada, said the review findings confirm that we do not live in a “simpler and more easy to handle world” in which “biases would always go in the same direction.”

“It turns out that the world is not a simple place,” he said. “Sometimes the bias goes in one direction and sometimes in another.”

However, the unpredictability of randomization in a randomized study — when it’s impossible to know beforehand whether a patient will end up in one group or another — adds to the value of that specific approach, review lead author Odgaard-Jensen said. It is “the best protection against the unpredictability of the extent and direction of bias in clinical trials that are not properly randomized.”

Still, randomized trials are not guaranteed routes to the truth regarding a medical question, said Dr. Arthur Hartz, director of health service research at Huntsman Cancer Institute in Salt Lake City. “I don’t think any single study is conclusive. You often get randomized controlled trials that disagree with each other.”

Randomized studies are often a nuisance too, he said. “They take incredible amounts of time and money, and they’re boring, and by the time you get the results, medical technology has moved on.”

One alternative is an observational study, in which researchers do not control which subjects get a particular medication or treatment, he said. (The new review did not examine the value of observational studies.)

“We have some indications of where observational studies are giving bad results, but we don’t understand exactly why they give bad results and how it could be fixed,” Hartz said. “That’s where the emphasis should be.”

For his part, Guyatt said randomization is one of several safeguards against bias in studies, such as preventing various players from knowing who is getting which treatment and following up with as many subjects as possible to see what happened to them. “If all these bias-reducing strategies are in place, randomized trials will always get you closer to the truth than studies that don’t have all the safeguards,” he said.

Ultimately, said Tufts University professor of medicine Dr. Joseph Lau, the new review “has value to those developing tools to critically appraise studies in systematic reviews. It also has value to those proposing guidelines on improving the conduct and reporting of primary studies.”

More information: Odgaard-Jensen J, et al. Randomisation to protect against selection bias in healthcare trials. Cochrane Database of Systematic Reviews 2011, Issue 4.

Related Stories

Recommended for you

Group suggests pushing age of adolescence to 24

January 22, 2018
A small group of researchers with the Royal Children's Hospital in Australia is suggesting that it might be time to change the span of years that define adolescence—from the current 10 to 19 to a proposed 10 to 24 years ...

Placental accumulation of flame retardant chemical alters serotonin production in rats

January 22, 2018
A North Carolina State University-led research team has shown a connection between exposure to a widely used flame retardant chemical mixture and disruption of normal placental function in rats, leading to altered production ...

Marijuana use does not lower chances of getting pregnant

January 22, 2018
Marijuana use—by either men or women—does not appear to lower a couple's chances of getting pregnant, according to a new study led by Boston University School of Public Health (BUSPH) researchers.

Women run faster after taking newly developed supplement, study finds

January 19, 2018
A new study found that women who took a specially prepared blend of minerals and nutrients for a month saw their 3-mile run times drop by almost a minute.

Americans are getting more sleep

January 19, 2018
Although more than one in three Americans still don't get enough sleep, a new analysis shows first signs of success in the fight for more shut eye. According to data from 181,335 respondents aged 15 and older who participated ...

Wine is good for you—to a point

January 18, 2018
The Mediterranean diet has become synonymous with healthy eating, but there's one thing in it that stands out: It's cool to drink wine.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.