Psychologists collaborate to reproduce experimental results

November 28, 2013 by Marcia Malory, Medical Xpress report

(Medical Xpress)—Psychological studies have a bad reputation for delivering results that researchers never reproduce. Repeatedly, psychologists have been unable to replicate the effects of classic studies. To test the reproducibility of psychological experiments, researchers from around the world worked together to repeat 13 studies, using thousands of subjects in different locations. Their collaborative work, the Many Labs Replication Project, reproduced the effects of 10 of these 13 studies. The research can be found online and will appear in a special issue of Social Psychology in Spring 2014.

In scientific research, reproducibility is essential. An irreproducible experiment might have a , or its results might be applicable only to a small population or within a particular setting. While direct replications of experiments are standard in other scientific disciplines, they are uncommon in psychology. The failure of psychologists to reproduce the results of classic studies and the recent discovery of instances of fraud have tarnished psychology's reputation as a legitimate science.

To see just how reliable psychological studies are, a team of more than 150 researchers in 36 laboratories, 25 in the United States and 11 in other countries, repeated 13 classic and contemporary , combined into one interactive experiment, on 6,344 different subjects. Some of the subjects completed the studies in the lab, while others completed them remotely over the Internet. Such a large collaborative project, covering a diverse group of subjects in a wide range of settings, is unusual in psychology, where a small group of in one lab will often be the only people conducting an experiment.

The scientists were able to reproduce the results of 10 of the 13 studies. In five of these, including a classic study on anchoring by Nobel laureate Daniel Kahneman, the statistical effect was stronger than in the original experiment. The anchoring study showed that the first piece of information you receive about a topic biases your future decisions.

They could not replicate the effects of two recent studies on social priming. Originally, one of these showed that exposing Americans to an American flag makes them more likely to express conservative opinions. The other showed that exposing people to money makes them more likely to endorse their country's current social system.

There was only weak support for the 13th study, which showed that imagining someone from another ethnic group makes you less likely to be prejudiced toward members of the group.

The nationality of the participants, and whether they completed the study online or in the lab, had no effect on the results. This means that reproducibility depended more on the perceived effect itself than on the setting or subject sample.

Explore further: Washing your hands makes you optimistic

More information:

Related Stories

Washing your hands makes you optimistic

October 24, 2013
Washing our hands influences how we think, judge and decide. This is what researchers were able to confirm through experiments over the last few years.

Repeatedly exposing yourself to a negative event may prevent it from affecting you

November 25, 2013
Psychology shows that it doesn't take much to put you in a bad mood. Just reading the morning news can do it. And being in a bad mood slows your reaction time, and affects your basic cognitive abilities like speech, writing, ...

A new approach to understanding research relevance

March 5, 2013
(Medical Xpress)—"Science is broken; let's fix it," says the University of Sydney's Associate Professor Alex Holcombe, who is part of a major new effort to improve the reliability of psychological research.

Great results in the psych lab -- but do they hold up in the field?

March 26, 2012
(Medical Xpress) -- How well do findings in the psychology lab generalize to real life? This criterion—“external validity”—is probably the most important for experimental psychology. So it was good news ...

Recommended for you

Short-course treatment for combat-related PTSD offers expedited path to recovery

January 23, 2018
Symptoms of Post-Traumatic Stress Disorder (PTSD) can be debilitating and standard treatment can take months, often leaving those affected unable to work or care for their families. But, a new study demonstrated that many ...

Curcumin improves memory and mood, study says

January 23, 2018
Lovers of Indian food, give yourselves a second helping: Daily consumption of a certain form of curcumin—the substance that gives Indian curry its bright color—improved memory and mood in people with mild, age-related ...

Priming can negate stressful aspects of negative sporting environments, study finds

January 23, 2018
The scene is ubiquitous in sports: A coach yells at players, creating an environment where winning is the sole focus and mistakes are punished. New research from the University of Kansas shows that when participants find ...

Social and emotional skills linked to better student learning

January 23, 2018
Students with well-developed and adaptive social and emotional behaviours are most likely to excel in school, according to UNSW researchers in educational psychology.

People with prosthetic arms less affected by common illusion

January 22, 2018
People with prosthetic arms or hands do not experience the "size-weight illusion" as strongly as other people, new research shows.

Intensive behavior therapy no better than conventional support in treating teenagers with antisocial behavior

January 19, 2018
Research led by UCL has found that intensive and costly multisystemic therapy is no better than conventional therapy in treating teenagers with moderate to severe antisocial behaviour.


Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (5) Nov 28, 2013
When replication is done using students who have already learned about the results of prior experiments, as most students obviously have learned, their responses should produce a statistical effect that is stronger than in the original experiment. Thus, what these new results exemplify is the ability of psychologists to continue to shoot themselves in the foot at the same time they have set their sights on scientific progress akin to the hard sciences like physic, and even biology, which are the basis for psychology but rarely considered in study design that shows behavioral affect sans the thermodynamics of biolophysical constraints or conserved molecular mechanisms of epigenetically effect behavior in species from microbes to man.
1 / 5 (6) Nov 28, 2013
More demonstration that "science" is rampant fraud.
How many people shelling out a hundred dollars an hour for for "treatment" or who see their tax dollars go to "social programs" know that the basis behind them is all hogwash?
And how "science" dotes on that garbage, excusing away irreproducibility as "experiment design error" or being performed in the wrong setting. When unconventional studies even get reproducible results it's not beyond "science" shills to say that proves it's a lie!
And how encouraging that "experiments" that produced irreproducible results repeatedly before suddenly are being reproducible. When using the method of anonymous "results" sent through a computer from anywhere! How soon before this is revealed to be all a swindle?
1.5 / 5 (6) Nov 28, 2013
The benefits of treatment by psychologists have been reported by individuals. Whether or not those benefits can be confirmed by study design that incorporates questionnaires is something different. So are the side-effect of pharmaceuticals designed to help but do no harm. Yet, what we've just seen is a visceral response that attacks nearly all experimental evidence as if there were no such thing as scientific pursuits.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.