Reliability of neuroscience research questioned

New research has questioned the reliability of neuroscience studies, saying that conclusions could be misleading due to small sample sizes.

A team led by from the University of Bristol reviewed 48 articles on neuroscience which were published in 2011 and concluded that most had an average power of around 20 per cent – a finding which means the chance of the average study discovering the effect being investigated is only one in five.

The paper, being published in Nature Reviews Neuroscience today, reveals that small, low-powered studies are 'endemic' in neuroscience, producing unreliable research which is inefficient and wasteful.

It focuses on how low statistical power – caused by low sample size of studies, small effects being investigated, or both – can be misleading and produce more false scientific claims than high-powered studies.

It also illustrates how low power reduces a study's ability to detect any effects, and shows that when discoveries are claimed, they are more likely to be false or misleading.

The paper claims there is substantial evidence that a large proportion of research published in scientific literature may be unreliable as a consequence.

Another consequence is that the findings are overestimated because smaller studies consistently give more positive results than larger studies. This was found to be the case for studies using a diverse range of methods, including brain imaging, genetics and animal studies.

Kate Button, from the School of Social and Community Medicine, and Marcus Munafò, from the School of , led a team of researchers from Stanford University, the University of Virginia and the University of Oxford.

She said: "There's a lot of interest at the moment in improving the of science. We looked at neuroscience and found that, on average, studies had only around a 20 per cent chance of detecting the effects they were investigating, even if the effects are real. This has two important implications - many studies lack the ability to give definitive answers to the questions they are testing, and many claimed findings are likely to be incorrect or unreliable."

The study concludes that improving the standard of results in , and enabling them to be more easily reproduced, is a key priority and requires attention to well-established methodological principles.

It recommends that existing scientific practices can be improved with small changes or additions to methodologies, such as acknowledging any limitations in the interpretation of results; disclosing methods and findings transparently; and working collaboratively to increase the total sample size and power.

More information: 'Power failure: why small sample size undermines the reliability of neuroscience' by Katherine Button, John Ioannidis, Claire Mokrysz, Brian Nosek, Jonathan Flint, Emma Robinson and Marcus Munafo in Nature Reviews Neuroscience.

Related Stories

Not so fast -- sex differences in the brain are overblown

date Oct 27, 2010

People love to speculate about differences between the sexes, and neuroscience has brought a new technology to this pastime. Brain imaging studies are published at a great rate, and some report sex differences in brain structure ...

New guidelines issued for reporting of genetic risk research

date Mar 28, 2011

(PhysOrg.com) -- Apples to Apples is more than just a popular card game. It’s an important concept when comparing the results of published scientific studies. It’s impossible to draw accurate conclusions, for example, ...

Recommended for you

AAN: phenytoin neuroprotective in optic neuritis

date 22 hours ago

(HealthDay)—Phenytoin appears to be neuroprotective in acute optic neuritis (AON), according to a study scheduled to be presented at the annual meeting of the American Academy of Neurology, held from April ...

How a jab to the ribs jolts the brain into action

date Apr 17, 2015

A short jab in the ribs instantly arouses a drowsy colleague during a long and dreary work meeting. A new study by Yale neurobiologists describes just what happens in the brain immediately following that ...

How do we hear time within sound?

date Apr 16, 2015

How does our auditory system represent time within a sound? A new study published in PLOS Computational Biology investigates how temporal acoustic patterns can be represented by neural activity within audito ...

User comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.