A new approach to understanding research relevance

March 5, 2013, University of Sydney

(Medical Xpress)—"Science is broken; let's fix it," says the University of Sydney's Associate Professor Alex Holcombe, who is part of a major new effort to improve the reliability of psychological research.

The new approach encourages to publish replication studies of previously published results, helping make this a part of ongoing scientific practice.

"Reproducing results using the same methods as the original experiment but with different participants is critical to science, but we are currently facing a crisis," says Associate Professor Holcombe.

"If someone makes an important research discovery we need to replicate the results and publish the findings to have confidence in the original research.

"Addressing this issue will provide benefits to the scientific community and the general public as it will make research more transparent and accountable."

The 'replicability crisis' exists because few scientists are interested in publishing their attempts to reproduce , despite its importance.

Professor Holcombe, from the University's School of Psychology, has helped devise a new initiative to solve the problem. He will be one of two editors of registered replication reports to be coordinated by the journal , published by peak professional society, the Association for Psychological Science.

"The crisis in published replications of psychological and other scientific research is a result of scientists receiving greater encouragement, both from their institutions and from journals, to produce new findings, instead of attempting to reproduce others' results."

"So there is typically an uphill battle to get a replication study published as well as the fear you may antagonise other researchers if your results contradict theirs."

There are several innovative aspects to the journal's approach. One is that, where possible, the original research authors will be involved so that the replicating authors can get their support and ensure their method is as close as possible to the original study.

"Another difference is that researchers will commit in advance to how they will analyse their results instead of being free to analyse their findings until they find a significant but possibly spurious result," Professor Holcombe said.

"The researchers will be assured that their results will be published even if they do not obtain a statistically significant result."

Before publication the method will be made public and laboratories will be invited to join the replication attempt, then all the results will be posted.

"Most importantly, the results will not be presented as a 'successful replication' or 'failed replication'. It is very rare that a set of results is definitive by itself - that is the whole point of what we are undertaking."

"Instead we will analyse and summarise to what extent all the replications, considered together, reproduce the original findings or differ from them. This is essential information for understanding those original findings and their applications."

An example of the reproduction problem is American research done in the 1990s that described 'verbal overshadowing', where immediately verbally describing an object appears to make recall harder, not easier. The finding continues to be highly influential but the original researcher, Jonathan Schooler, has reported on the difficulty of reproducing his results, undermining his own faith in the original findings.

Professor Holcombe's editorial role is the culmination of 10 years of ongoing effort to improve the transparency and accountability of .

"It started with an unsuccessful attempt to persuade journals to post comments on published papers to allow flaws and missed connections to come to light earlier."

"So I am excited, 10 years on, to be part of this initiative because I think it can overcome the main obstacles to replication studies being undertaken and published."

Explore further: Taking on the challenges of replication in psychological science

Related Stories

Taking on the challenges of replication in psychological science

November 8, 2012
Psychological science has come of age. But the rights of a mature discipline carry with them responsibilities, among them the responsibility to maximize confidence in our findings through good data practices and replication.

The perils of 'bite-size' science

December 28, 2011
Short, fast, and frequent: Those 21st-century demands on publication have radically changed the news, politics, and culture—for the worse, many say. Now an article in January's Perspectives on Psychological Science, ...

Controversial study promoting psychic ability debunked

March 14, 2012
In response to a 2011 study suggesting the existence of precognition, or the ability to predict future events using psychic powers, a new group of researchers report that attempts to replicate the previous results were unsuccessful. ...

Improvements are needed for accuracy in gene-by-environment interaction studies

September 8, 2011
A new study from McLean Hospital/Harvard Medical School and the University of Colorado concludes that genetic research drawing correlations between specific genes, environmental variables and the combined impact they have ...

Chemists set up web site to crowd source chemical reaction validation

January 23, 2013
(Phys.org)—A group of chemists, some willing to give their real names, some not, have set up a web site called Blog Syn, for the purpose of inviting chemists to recreate the chemical reactions as presented in professional ...

Recommended for you

Study of learning and memory problems in OCD helps young people unlock potential at school

January 22, 2018
Adolescents with obsessive-compulsive disorder (OCD) have widespread learning and memory problems, according to research published today. The findings have already been used to assist adolescents with OCD obtain the help ...

Intensive behavior therapy no better than conventional support in treating teenagers with antisocial behavior

January 19, 2018
Research led by UCL has found that intensive and costly multisystemic therapy is no better than conventional therapy in treating teenagers with moderate to severe antisocial behaviour.

Babies' babbling betters brains, language

January 18, 2018
Babies are adept at getting what they need - including an education. New research shows that babies organize mothers' verbal responses, which promotes more effective language instruction, and infant babbling is the key.

College branding makes beer more salient to underage students

January 18, 2018
In recent years, major beer companies have tried to capitalize on the salience of students' university affiliations, unveiling marketing campaigns and products—such as "fan cans," store displays, and billboard ads—that ...

Inherited IQ can increase in early childhood

January 18, 2018
When it comes to intelligence, environment and education matter – more than we think.

Modulating molecules: Study shows oxytocin helps the brain to modulate social signals

January 17, 2018
Between sights, sounds, smells and other senses, the brain is flooded with stimuli on a moment-to-moment basis. How can it sort through the flood of information to decide what is important and what can be relegated to the ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.