Reported 'neuroimage bias' not as strong as first believed

A few years back, there was concern that the developing field of neuroimaging – producing colorful images of brain activity – was potentially biasing scientists and the public alike. A series of well-publicized experiments published in 2008 indicated that members of the public could be swayed to place undue faith in scientific studies based simply on the fact that they were accompanied by neuroimages.

Not so fast, says Nick Schweitzer, assistant professor in Arizona State University's New College of Interdisciplinary Arts and Sciences and the lead researcher for the college's Law and Cognition Lab. Schweitzer and coauthors Denise Baker and Evan Risko recently published the article "Fooled by the brain: Re-examining the influence of neuroimages" in the scientific journal Cognition.

"The fear was that a weak or flawed piece of research could be taken seriously just because of the images that are included along with it," Schweitzer said. "Researchers could intentionally 'dress up' their results with these types of images. And if not researchers, imagine drug companies sending out literature to physicians accompanied by the same types of images."

Fortunately, the new study by Schweitzer and colleagues is joining a growing body of more recent evidence indicating neuroimage bias isn't nearly as strong as first believed.

The "Fooled by the brain" study replicated the 2008 experiments reporting a biasing effect of neuroimages on perceptions of scientific research among members of the public. Overall, Schweitzer and his colleagues were not able to replicate those results.

"This in no way means there was anything 'wrong' with those 2008 studies," Schweitzer said. "I'm certain that those researchers reported what they found."

Rather, Schweitzer and his colleagues theorize that bias effects have faded over the past few years. "One potential explanation for these failures to replicate is that, as people have become more exposed to neuroscience over the past five years, the persuasive punch of neuroimagery has dulled, and what might have been legitimate effects in 2008 simply no longer exist," they wrote in Cognition.

The sole experiment Schweitzer and colleagues conducted that actually showed neuroimage bias involved participants who read two fabricated research summaries that included faulty reasoning. One summary was accompanied by a high-quality 3-D neuroimage while the other wasn't, and after viewing both summaries, participants rated the scientific reasoning more highly for the study accompanied by the neuroimage.

These results are in contrast to a research design in which a group of participants is shown only one study and asked to rate it. Under those circumstances, Schweitzer found no neuroimage bias. "This means that neuroimages aren't particularly persuasive unless you have something to compare them to," he explained.

But Schweitzer points out that this comparison-related bias could be significant in a courtroom setting, where there are always competing experts, cross-examination and other back-and-forth elements that allow jurors to make relative judgments about the evidence.

In 2009-2010, as part of the Law and Neuroscience Project (a research consortium funded by the John D. and Catherine T. MacArthur Foundation), Schweitzer and a group of colleagues set out to look for the neuroimage bias within a courtroom context.

"We had fully expected to find it, especially having read those 2008 papers and knowing that those papers generated a lot of media coverage," Schweitzer said. "Quite surprisingly, we couldn't find it no matter how much we looked for it. Eventually, I started to wonder whether the broader idea of the neuroimage bias was in fact true. The best way to figure that out is to go back to the initial work that launched the idea, and see if you can get it to show up again."

That led Schweitzer, Baker and Risko (a former New College faculty member now at the University of Waterloo in Canada) to conduct the replication study resulting in the new publication in Cognition.

Baker was pursuing New College's master's degree in psychology while she, Schweitzer and Risko began the study; she since has moved on to ASU's doctoral program in the human and social dimensions of science and technology, where she continues her work with Schweitzer.

"As a master's student, I was allowed – and expected – to fully participate in the processes of development, data collection, data analysis, interpretation and writing," Baker said.

"When I have spoken to friends in other master's programs at other universities, their experiences, though generally positive, seemed narrowly focused on accomplishing what needed to be done to complete their thesis," she added. "My experience in the Law and Cognition Lab with Nick has been quite different. The lab culture is focused on conducting meaningful, relevant research and creating knowledge to share."

In the case of the research reported in "Fooled by the brain," Schweitzer says the results have reignited the need to explore the potential for neuroimage bias in legal contexts.

Schweitzer's primary interest is in how people within the legal system, including judges, lawyers and jurors, use science to make decisions. "Whether it's forensic science in a criminal trial, medical research in a trial about drug side effects or the judicial system itself using research to enact reforms, I am interested in whether that science is presented and perceived accurately," he said.