Jumping to scientific conclusions challenges biomedical research

Improving experimental design and statistical analyses alone will not solve the reproducibility crisis in science, argues Ray Dingledine in a societal impact article published in eNeuro. Repeating classic behavioral economics experiments with graduate- to senior-level researchers, the author finds scientists of all career stages are subject to the same biases as undergraduates when interpreting data.

In the 1960s and '70s, psychologists Daniel Kahneman and Amos Tversky demonstrated that people tend to engage in "fast thinking"—relying on preconceived notions and emotions—when making decisions in the face of new information. While this finding has had clear implications for economic decisions such as weighing the risks and rewards of a financial investment, Dingledine wondered whether it would also apply to scientific decision-making.

A survey of present-day graduate students, postdoctoral and senior research staff, and principal investigators across three basic science departments revealed that these scientists were just as likely as the university students tested by Kahneman and Tversky 45 years ago to jump to conclusions when grappling with unfamiliar data. Conducting a "premortem" before new experiments take place could help to identify these potential pitfalls and encourage scientists to think more like the forward-planning ancient Greek Titan Prometheus rather than his hasty brother Epimetheus.