Psychologists collaborate to reproduce experimental results

November 28, 2013 by Marcia Malory report

(Medical Xpress)—Psychological studies have a bad reputation for delivering results that researchers never reproduce. Repeatedly, psychologists have been unable to replicate the effects of classic studies. To test the reproducibility of psychological experiments, researchers from around the world worked together to repeat 13 studies, using thousands of subjects in different locations. Their collaborative work, the Many Labs Replication Project, reproduced the effects of 10 of these 13 studies. The research can be found online and will appear in a special issue of Social Psychology in Spring 2014.

In scientific research, reproducibility is essential. An irreproducible experiment might have a , or its results might be applicable only to a small population or within a particular setting. While direct replications of experiments are standard in other scientific disciplines, they are uncommon in psychology. The failure of psychologists to reproduce the results of classic studies and the recent discovery of instances of fraud have tarnished psychology's reputation as a legitimate science.

To see just how reliable psychological studies are, a team of more than 150 researchers in 36 laboratories, 25 in the United States and 11 in other countries, repeated 13 classic and contemporary , combined into one interactive experiment, on 6,344 different subjects. Some of the subjects completed the studies in the lab, while others completed them remotely over the Internet. Such a large collaborative project, covering a diverse group of subjects in a wide range of settings, is unusual in psychology, where a small group of in one lab will often be the only people conducting an experiment.

The scientists were able to reproduce the results of 10 of the 13 studies. In five of these, including a classic study on anchoring by Nobel laureate Daniel Kahneman, the statistical effect was stronger than in the original experiment. The anchoring study showed that the first piece of information you receive about a topic biases your future decisions.

They could not replicate the effects of two recent studies on social priming. Originally, one of these showed that exposing Americans to an American flag makes them more likely to express conservative opinions. The other showed that exposing people to money makes them more likely to endorse their country's current social system.

There was only weak support for the 13th study, which showed that imagining someone from another ethnic group makes you less likely to be prejudiced toward members of the group.

The nationality of the participants, and whether they completed the study online or in the lab, had no effect on the results. This means that reproducibility depended more on the perceived effect itself than on the setting or subject sample.

Explore further: Washing your hands makes you optimistic

More information:

Related Stories

Washing your hands makes you optimistic

October 24, 2013
Washing our hands influences how we think, judge and decide. This is what researchers were able to confirm through experiments over the last few years.

Repeatedly exposing yourself to a negative event may prevent it from affecting you

November 25, 2013
Psychology shows that it doesn't take much to put you in a bad mood. Just reading the morning news can do it. And being in a bad mood slows your reaction time, and affects your basic cognitive abilities like speech, writing, ...

A new approach to understanding research relevance

March 5, 2013
(Medical Xpress)—"Science is broken; let's fix it," says the University of Sydney's Associate Professor Alex Holcombe, who is part of a major new effort to improve the reliability of psychological research.

Great results in the psych lab -- but do they hold up in the field?

March 26, 2012
(Medical Xpress) -- How well do findings in the psychology lab generalize to real life? This criterion—“external validity”—is probably the most important for experimental psychology. So it was good news ...

Recommended for you

Visual clues we use during walking and when we use them

July 25, 2017
(Medical Xpress)—A trio of researchers with the University of Texas and Rensselaer Polytechnic Institute has discovered which phase of visual information processing during human walking is used most to guide the feet accurately. ...

Psychopaths are better at learning to lie, say researchers

July 25, 2017
Individuals with high levels of psychopathic traits are better at learning to lie than individuals who show few psychopathic traits, according to a study published in the open access journal Translational Psychiatry. The ...

Toddlers begin learning rules of reading, writing at very early age, study finds

July 25, 2017
Even the proudest of parents may struggle to find some semblance of meaning behind the seemingly random mish-mash of letters that often emerge from a toddler's first scribbled and scrawled attempts at putting words on paper.

Higher cognitive abilities linked to greater risk of stereotyping

July 24, 2017
People with higher cognitive abilities are more likely to learn and apply social stereotypes, finds a new study. The results, stemming from a series of experiments, show that those with higher cognitive abilities also more ...

Exposure to violence hinders short-term memory, cognitive control

July 24, 2017
Being exposed to and actively remembering violent episodes—even those that happened up to a decade before—hinders short-term memory and cognitive control, according to a study published in the Proceedings of the National ...

Researchers pave new path toward preventing obesity

July 24, 2017
People who experience unpredictable childhoods due to issues such as divorce, crime or frequent moves face a higher risk of becoming obese as adults, according to a new study by a Florida State University researcher.


Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (5) Nov 28, 2013
When replication is done using students who have already learned about the results of prior experiments, as most students obviously have learned, their responses should produce a statistical effect that is stronger than in the original experiment. Thus, what these new results exemplify is the ability of psychologists to continue to shoot themselves in the foot at the same time they have set their sights on scientific progress akin to the hard sciences like physic, and even biology, which are the basis for psychology but rarely considered in study design that shows behavioral affect sans the thermodynamics of biolophysical constraints or conserved molecular mechanisms of epigenetically effect behavior in species from microbes to man.
1 / 5 (6) Nov 28, 2013
More demonstration that "science" is rampant fraud.
How many people shelling out a hundred dollars an hour for for "treatment" or who see their tax dollars go to "social programs" know that the basis behind them is all hogwash?
And how "science" dotes on that garbage, excusing away irreproducibility as "experiment design error" or being performed in the wrong setting. When unconventional studies even get reproducible results it's not beyond "science" shills to say that proves it's a lie!
And how encouraging that "experiments" that produced irreproducible results repeatedly before suddenly are being reproducible. When using the method of anonymous "results" sent through a computer from anywhere! How soon before this is revealed to be all a swindle?
1.5 / 5 (6) Nov 28, 2013
The benefits of treatment by psychologists have been reported by individuals. Whether or not those benefits can be confirmed by study design that incorporates questionnaires is something different. So are the side-effect of pharmaceuticals designed to help but do no harm. Yet, what we've just seen is a visceral response that attacks nearly all experimental evidence as if there were no such thing as scientific pursuits.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.