Reliability of neuroscience research questioned

April 10, 2013, University of Bristol

New research has questioned the reliability of neuroscience studies, saying that conclusions could be misleading due to small sample sizes.

A team led by from the University of Bristol reviewed 48 articles on neuroscience which were published in 2011 and concluded that most had an average power of around 20 per cent – a finding which means the chance of the average study discovering the effect being investigated is only one in five.

The paper, being published in Nature Reviews Neuroscience today, reveals that small, low-powered studies are 'endemic' in neuroscience, producing unreliable research which is inefficient and wasteful.

It focuses on how low statistical power – caused by low sample size of studies, small effects being investigated, or both – can be misleading and produce more false scientific claims than high-powered studies.

It also illustrates how low power reduces a study's ability to detect any effects, and shows that when discoveries are claimed, they are more likely to be false or misleading.

The paper claims there is substantial evidence that a large proportion of research published in scientific literature may be unreliable as a consequence.

Another consequence is that the findings are overestimated because smaller studies consistently give more positive results than larger studies. This was found to be the case for studies using a diverse range of methods, including brain imaging, genetics and animal studies.

Kate Button, from the School of Social and Community Medicine, and Marcus Munafò, from the School of , led a team of researchers from Stanford University, the University of Virginia and the University of Oxford.

She said: "There's a lot of interest at the moment in improving the of science. We looked at neuroscience and found that, on average, studies had only around a 20 per cent chance of detecting the effects they were investigating, even if the effects are real. This has two important implications - many studies lack the ability to give definitive answers to the questions they are testing, and many claimed findings are likely to be incorrect or unreliable."

The study concludes that improving the standard of results in , and enabling them to be more easily reproduced, is a key priority and requires attention to well-established methodological principles.

It recommends that existing scientific practices can be improved with small changes or additions to methodologies, such as acknowledging any limitations in the interpretation of results; disclosing methods and findings transparently; and working collaboratively to increase the total sample size and power.

Explore further: Broad analysis of many radiation studies finds no exposure threshold that precludes harm to life

More information: 'Power failure: why small sample size undermines the reliability of neuroscience' by Katherine Button, John Ioannidis, Claire Mokrysz, Brian Nosek, Jonathan Flint, Emma Robinson and Marcus Munafo in Nature Reviews Neuroscience.

Related Stories

Broad analysis of many radiation studies finds no exposure threshold that precludes harm to life

November 15, 2012
(Medical Xpress)—Even the very lowest levels of radiation are harmful to life, scientists have concluded in the Cambridge Philosophical Society's journal Biological Reviews. Reporting the results of a wide-ranging analysis ...

Medical studies with striking results often prove false

October 24, 2012
If a medical study seems too good to be true, it probably is, according to a new analysis.

New study evaluates link between physical activity in middle age and onset of dementia in later life

July 20, 2012
(Medical Xpress) -- Dementia and cognitive impairment are important public health issues, due to the morbidity associated with deteriorating memory, and the cost of caring for patients by both families and health services.

Exercise has numerous beneficial effects on brain health and cognition, review suggests

July 25, 2011
It's no secret that exercise has numerous beneficial effects on the body. However, a bevy of recent research suggests that these positive effects also extend to the brain, influencing cognition. In a new review article highlighting ...

Recommended for you

Research reveals atomic-level changes in ALS-linked protein

January 18, 2018
For the first time, researchers have described atom-by-atom changes in a family of proteins linked to amyotrophic lateral sclerosis (ALS), a group of brain disorders known as frontotemporal dementia and degenerative diseases ...

Fragile X finding shows normal neurons that interact poorly

January 18, 2018
Neurons in mice afflicted with the genetic defect that causes Fragile X syndrome (FXS) appear similar to those in healthy mice, but these neurons fail to interact normally, resulting in the long-known cognitive impairments, ...

How your brain remembers what you had for dinner last night

January 17, 2018
Confirming earlier computational models, researchers at University of California San Diego and UC San Diego School of Medicine, with colleagues in Arizona and Louisiana, report that episodic memories are encoded in the hippocampus ...

Recording a thought's fleeting trip through the brain

January 17, 2018
University of California, Berkeley neuroscientists have tracked the progress of a thought through the brain, showing clearly how the prefrontal cortex at the front of the brain coordinates activity to help us act in response ...

Midbrain 'start neurons' control whether we walk or run

January 17, 2018
Locomotion comprises the most fundamental movements we perform. It is a complex sequence from initiating the first step, to stopping when we reach our goal. At the same time, locomotion is executed at different speeds to ...

Miles Davis is not Mozart: The brains of jazz and classical pianists work differently

January 16, 2018
Keith Jarret, world-famous jazz pianist, once answered in an interview when asked if he would ever be interested in doing a concert where he would play both jazz and classical music: "No, that's hilarious. [...] It's like ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.