Larger sample sizes needed to increase reproducibility in neuroscience studies

June 7, 2018, University of Illinois at Urbana-Champaign
Credit: CC0 Public Domain

Small sample sizes in studies using functional MRI to investigate brain connectivity and function are common in neuroscience, despite years of warnings that such studies likely lack sufficient statistical power. A new analysis reveals that task-based fMRI experiments involving typical sample sizes of about 30 participants are only modestly replicable. This means that independent efforts to repeat the experiments are as likely to challenge as to confirm the original results.

The study, reported in the journal Communications Biology, also finds that task-based fMRI studies with sizes of up to 100 also fall short of being perfectly replicable.

Task-based fMRI studies track changes in blood oxygen levels in the brain while study subjects are engaged in cognitive tasks. The technique allows researchers to see which brain regions are recruited to perform specific tasks.

But those in the field of cognitive neuroscience have not agreed on specific standards for the design of task-based fMRI studies—in particular, how many study subjects are needed to ensure reliable findings. The new research aims to address this shortfall, researchers said.

"This study is the largest investigation of the role of sample size in the reproducibility of task-based fMRI methods," said University of Illinois psychology professor Aron Barbey, who conducted the research with Erick Paul, a former postdoctoral fellow at the U. of I.; Benjamin Turner, of Nanyang Technological University in Singapore; and Michael Miller, of the University of California, Santa Barbara.

"We systematically examined the reproducibility of studies at different sample sizes, from as few as 16 to as many as 121 participants, and across many different task-based fMRI approaches," Barbey said. "We found that reproducibility is modest in studies involving the standard number of 30 participants, and that much larger sample sizes may be needed to improve replicability."

The problem is widespread, Paul said. A majority fMRI studies published in cognitive neuroscience journals in 2011-14 involved fewer than 100 participants. The median number is about 30. The problem stems in part from the fact that fMRI studies are expensive.

"Even small studies can cost several tens of thousands of dollars, and funding systems are not generally set up to enable the routine collection of data from more than 100 participants," Paul said.

Other factors also contribute to the challenge. For example, researchers in neuroscience and psychological science may not understand how statistical analyses for fMRI differ from those in behavioral studies, and so may not accurately evaluate the statistical power of their fMRI analyses, Turner said.

"They may confuse the high retest accuracy of fMRI with high replicability," he said. "But that sort of accuracy refers only to retests of the same individuals engaged in the same tasks, not to similarity across separate small groups."

To overcome the problem of expense, Miller suggests that researchers pool their resources to conduct fMRI studies across numerous sites involving greater numbers of participants. This would likely result in more diverse participant pools. It also would advance the work of many scientists, laboratories and institutions at once, yielding more data per participant with potentially greater impact.

"Replicability is the foundation of scientific progress," the authors wrote. "Unfortunately, for a variety of reasons, many scientific fields are gripped by a crisis of irreproducibility. While some of the causes of the crisis are deeply woven into the academic landscape—incentives related to publication, funding and tenure—the most straightforward solution relates to statistical power."

"Our study points to a major problem in the field of , a field in which fMRI studies are a commonly used tool," Barbey said. "But this is a problem that we can solve by increasing the size of the samples we study."

Explore further: Novel fMRI applications in childhood epilepsy increase understanding of seizure impacts

More information: Benjamin O. Turner et al, Small sample sizes reduce the replicability of task-based fMRI studies, Communications Biology (2018). DOI: 10.1038/s42003-018-0073-z

Related Stories

Novel fMRI applications in childhood epilepsy increase understanding of seizure impacts

December 14, 2017
Functional magnetic resonance imaging (fMRI) has allowed researchers to map the memory functions that are often impaired within the brains of children with epilepsy. Additionally, a separate study of a novel application of ...

Similar brain connectivity during rest and tasks linked to better mental performance

August 16, 2016
A brain on task differs from a brain at rest. But, how much it differs could depend on the cognitive ability of the person whose brain is being studied. New research published August 17 in The Journal of Neuroscience suggests ...

Brain scans may reveal most effective anti-drug messages

December 7, 2017
What if you could look into the brains of potential drug abusers and see what messages would be most likely to persuade them to "just say no?"

Brain disconnections may contribute to Parkinson's hallucinations

September 27, 2017
Researchers have found that disconnections of brain areas involved in attention and visual processing may contribute to visual hallucinations in individuals with Parkinson's disease, according to a new study published online ...

Faster computations to visualize the finer details of brain activity in functional brain imaging

March 21, 2018
A computationally efficient data processing scheme will make it possible to see correlations in brain activity between different parts of the brain at unprecedented resolution. The statistics-based computation scheme, developed ...

Looking deeper into brain function

March 27, 2018
To uncover connections between brain regions and specific cognitive functions, neuroscientists have long made extensive use of techniques like functional resonance imaging (fMRI). First introduced in the 1990s, the method ...

Recommended for you

Researchers investigate changes in white matter in mice exposed to low-frequency brain stimulation

June 19, 2018
A team of researchers at the University of Oregon has learned more about the mechanism involved in mouse brain white matter changes as it responds to stimulation. In their paper published in Proceedings of the National Academy ...

Cell type and environment influence protein turnover in the brain

June 19, 2018
Scientists have revealed that protein molecules in the brain are broken down and replaced at different rates, depending on where in the brain they are.

Left, right and center: mapping emotion in the brain

June 19, 2018
According to a radical new model of emotion in the brain, a current treatment for the most common mental health problems could be ineffective or even detrimental to about 50 percent of the population.

Often overlooked glial cell is key to learning and memory

June 18, 2018
Glial cells surround neurons and provide support—not unlike hospital staff and nurses supporting doctors to keep operations running smoothly. These often-overlooked cells, which include oligodendrocytes and astrocytes, ...

Electrically stimulating the brain may restore movement after stroke

June 18, 2018
UC San Francisco scientists have improved mobility in rats that had experienced debilitating strokes by using electrical stimulation to restore a distinctive pattern of brain cell activity associated with efficient movement. ...

Neuroscientists map brain's response to cold touch

June 18, 2018
Carnegie Mellon University neuroscientists have mapped the feeling of cool touch to the brain's insula in a mouse model. The findings, published in the June 15 issue of Journal of Comparative Neurology, provide an experimental ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.