A new approach to understanding research relevance

March 5, 2013

(Medical Xpress)—"Science is broken; let's fix it," says the University of Sydney's Associate Professor Alex Holcombe, who is part of a major new effort to improve the reliability of psychological research.

The new approach encourages to publish replication studies of previously published results, helping make this a part of ongoing scientific practice.

"Reproducing results using the same methods as the original experiment but with different participants is critical to science, but we are currently facing a crisis," says Associate Professor Holcombe.

"If someone makes an important research discovery we need to replicate the results and publish the findings to have confidence in the original research.

"Addressing this issue will provide benefits to the scientific community and the general public as it will make research more transparent and accountable."

The 'replicability crisis' exists because few scientists are interested in publishing their attempts to reproduce , despite its importance.

Professor Holcombe, from the University's School of Psychology, has helped devise a new initiative to solve the problem. He will be one of two editors of registered replication reports to be coordinated by the journal , published by peak professional society, the Association for Psychological Science.

"The crisis in published replications of psychological and other scientific research is a result of scientists receiving greater encouragement, both from their institutions and from journals, to produce new findings, instead of attempting to reproduce others' results."

"So there is typically an uphill battle to get a replication study published as well as the fear you may antagonise other researchers if your results contradict theirs."

There are several innovative aspects to the journal's approach. One is that, where possible, the original research authors will be involved so that the replicating authors can get their support and ensure their method is as close as possible to the original study.

"Another difference is that researchers will commit in advance to how they will analyse their results instead of being free to analyse their findings until they find a significant but possibly spurious result," Professor Holcombe said.

"The researchers will be assured that their results will be published even if they do not obtain a statistically significant result."

Before publication the method will be made public and laboratories will be invited to join the replication attempt, then all the results will be posted.

"Most importantly, the results will not be presented as a 'successful replication' or 'failed replication'. It is very rare that a set of results is definitive by itself - that is the whole point of what we are undertaking."

"Instead we will analyse and summarise to what extent all the replications, considered together, reproduce the original findings or differ from them. This is essential information for understanding those original findings and their applications."

An example of the reproduction problem is American research done in the 1990s that described 'verbal overshadowing', where immediately verbally describing an object appears to make recall harder, not easier. The finding continues to be highly influential but the original researcher, Jonathan Schooler, has reported on the difficulty of reproducing his results, undermining his own faith in the original findings.

Professor Holcombe's editorial role is the culmination of 10 years of ongoing effort to improve the transparency and accountability of .

"It started with an unsuccessful attempt to persuade journals to post comments on published papers to allow flaws and missed connections to come to light earlier."

"So I am excited, 10 years on, to be part of this initiative because I think it can overcome the main obstacles to replication studies being undertaken and published."

Explore further: Taking on the challenges of replication in psychological science

Related Stories

Taking on the challenges of replication in psychological science

November 8, 2012
Psychological science has come of age. But the rights of a mature discipline carry with them responsibilities, among them the responsibility to maximize confidence in our findings through good data practices and replication.

The perils of 'bite-size' science

December 28, 2011
Short, fast, and frequent: Those 21st-century demands on publication have radically changed the news, politics, and culture—for the worse, many say. Now an article in January's Perspectives on Psychological Science, ...

Controversial study promoting psychic ability debunked

March 14, 2012
In response to a 2011 study suggesting the existence of precognition, or the ability to predict future events using psychic powers, a new group of researchers report that attempts to replicate the previous results were unsuccessful. ...

Improvements are needed for accuracy in gene-by-environment interaction studies

September 8, 2011
A new study from McLean Hospital/Harvard Medical School and the University of Colorado concludes that genetic research drawing correlations between specific genes, environmental variables and the combined impact they have ...

Chemists set up web site to crowd source chemical reaction validation

January 23, 2013
(Phys.org)—A group of chemists, some willing to give their real names, some not, have set up a web site called Blog Syn, for the purpose of inviting chemists to recreate the chemical reactions as presented in professional ...

Recommended for you

Many kinds of happiness promote better health, study finds

July 21, 2017
A new study links the capacity to feel a variety of upbeat emotions to better health.

Study examines effects of stopping psychiatric medication

July 20, 2017
Despite numerous obstacles and severe withdrawal effects, long-term users of psychiatric drugs can stop taking them if they choose, and mental health care professionals could be more helpful to such individuals, according ...

Study finds gene variant increases risk for depression

July 20, 2017
A University of Central Florida study has found that a gene variant, thought to be carried by nearly 25 percent of the population, increases the odds of developing depression.

In making decisions, are you an ant or a grasshopper?

July 20, 2017
In one of Aesop's famous fables, we are introduced to the grasshopper and the ant, whose decisions about how to spend their time affect their lives and future. The jovial grasshopper has a blast all summer singing and playing, ...

Perceiving oneself as less physically active than peers is linked to a shorter lifespan

July 20, 2017
Would you say that you are physically more active, less active, or about equally active as other people your age?

Old antibiotic could form new depression treatment

July 19, 2017
An antibiotic used mostly to treat acne has been found to improve the quality of life for people with major depression, in a world-first clinical trial conducted at Deakin University.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.