Psychologists collaborate to reproduce experimental results

by Marcia Malory report

(Medical Xpress)—Psychological studies have a bad reputation for delivering results that researchers never reproduce. Repeatedly, psychologists have been unable to replicate the effects of classic studies. To test the reproducibility of psychological experiments, researchers from around the world worked together to repeat 13 studies, using thousands of subjects in different locations. Their collaborative work, the Many Labs Replication Project, reproduced the effects of 10 of these 13 studies. The research can be found online and will appear in a special issue of Social Psychology in Spring 2014.

In scientific research, reproducibility is essential. An irreproducible experiment might have a , or its results might be applicable only to a small population or within a particular setting. While direct replications of experiments are standard in other scientific disciplines, they are uncommon in psychology. The failure of psychologists to reproduce the results of classic studies and the recent discovery of instances of fraud have tarnished psychology's reputation as a legitimate science.

To see just how reliable psychological studies are, a team of more than 150 researchers in 36 laboratories, 25 in the United States and 11 in other countries, repeated 13 classic and contemporary , combined into one interactive experiment, on 6,344 different subjects. Some of the subjects completed the studies in the lab, while others completed them remotely over the Internet. Such a large collaborative project, covering a diverse group of subjects in a wide range of settings, is unusual in psychology, where a small group of in one lab will often be the only people conducting an experiment.

The scientists were able to reproduce the results of 10 of the 13 studies. In five of these, including a classic study on anchoring by Nobel laureate Daniel Kahneman, the statistical effect was stronger than in the original experiment. The anchoring study showed that the first piece of information you receive about a topic biases your future decisions.

They could not replicate the effects of two recent studies on social priming. Originally, one of these showed that exposing Americans to an American flag makes them more likely to express conservative opinions. The other showed that exposing people to money makes them more likely to endorse their country's current social system.

There was only weak support for the 13th study, which showed that imagining someone from another ethnic group makes you less likely to be prejudiced toward members of the group.

The nationality of the participants, and whether they completed the study online or in the lab, had no effect on the results. This means that reproducibility depended more on the perceived effect itself than on the setting or subject sample.

More information: openscienceframework.org/project/WX7Ck/

Related Stories

Washing your hands makes you optimistic

Oct 24, 2013

Washing our hands influences how we think, judge and decide. This is what researchers were able to confirm through experiments over the last few years.

A new approach to understanding research relevance

Mar 05, 2013

(Medical Xpress)—"Science is broken; let's fix it," says the University of Sydney's Associate Professor Alex Holcombe, who is part of a major new effort to improve the reliability of psychological research.

Recommended for you

Could summer camp be the key to world peace?

4 hours ago

According to findings from a new study by University of Chicago Booth School of Business Professor Jane Risen, and Chicago Booth doctoral student Juliana Schroeder, it may at least be a start.

Gender disparities in cognition will not diminish

Jul 28, 2014

The study, published in the Proceedings of the National Academy of Science, investigated the extent to which improvements in living conditions and educational opportunities over a person's life affect cognitive abilities and th ...

Facial features are the key to first impressions

Jul 28, 2014

A new study by researchers in the Department of Psychology at the University of York shows that it is possible to accurately predict first impressions using measurements of physical features in everyday images of faces, such ...

User comments

Adjust slider to filter visible comments by rank

Display comments: newest first

JVK
1 / 5 (5) Nov 28, 2013
When replication is done using students who have already learned about the results of prior experiments, as most students obviously have learned, their responses should produce a statistical effect that is stronger than in the original experiment. Thus, what these new results exemplify is the ability of psychologists to continue to shoot themselves in the foot at the same time they have set their sights on scientific progress akin to the hard sciences like physic, and even biology, which are the basis for psychology but rarely considered in study design that shows behavioral affect sans the thermodynamics of biolophysical constraints or conserved molecular mechanisms of epigenetically effect behavior in species from microbes to man.
julianpenrod
1 / 5 (6) Nov 28, 2013
More demonstration that "science" is rampant fraud.
How many people shelling out a hundred dollars an hour for for "treatment" or who see their tax dollars go to "social programs" know that the basis behind them is all hogwash?
And how "science" dotes on that garbage, excusing away irreproducibility as "experiment design error" or being performed in the wrong setting. When unconventional studies even get reproducible results it's not beyond "science" shills to say that proves it's a lie!
And how encouraging that "experiments" that produced irreproducible results repeatedly before suddenly are being reproducible. When using the method of anonymous "results" sent through a computer from anywhere! How soon before this is revealed to be all a swindle?
JVK
1.5 / 5 (6) Nov 28, 2013
The benefits of treatment by psychologists have been reported by individuals. Whether or not those benefits can be confirmed by study design that incorporates questionnaires is something different. So are the side-effect of pharmaceuticals designed to help but do no harm. Yet, what we've just seen is a visceral response that attacks nearly all experimental evidence as if there were no such thing as scientific pursuits.