The power of learning from experience

In a first study, researchers from the Max Planck Institute for Human Development and the University of the Balearic Islands analyzed the development of research on statistical reasoning from a historical perspective. In the 1960s, adults were still considered to be good intuitive statisticians. From the 1970s, however, this notion was overturned. Since then, studies on so-called "heuristics and biases" have concluded that human reasoning is prone to systematic errors. According to this approach, the heuristics—mental shortcuts or rules of thumb—that people use for statistical reasoning produce systematically biased judgments that are inconsistent with the rules of probability theory and statistics.

What has escaped attention until now, however, is that there was also a fundamental change in the design of the studies used to test adults' statistical reasoning. Since the 1970s, most of these studies have involved text-based tasks and descriptions. Experience-based tasks, in which participants were able to experience statistical information at first hand, became a thing of the past. Yet the new study is the first to draw a causal connection between the dramatic change in experimental methods and the fact that findings of poor statistical reasoning in adults have been accumulating ever since.

What does this finding mean for future studies on human statistical reasoning? Should researchers return to the tradition of experiential studies and avoid text-based descriptions? "In practice, we can't work exclusively with experience-based tasks, but we shouldn't rely solely on description-based tasks either. We need to be aware that the two methods produce qualitatively different results. Ideally, both should be used in combination," says Tomás Lejarraga, Associate Research Scientist in the Center for Adaptive Rationality at the Max Planck Institute for Human Development and Director of the Decision Science Laboratory at the University of the Balearic Islands.

Different studies for babies and primates

To further examine the relationship between task format and performance, in another study researchers from the Max Planck Institute for Human Development compared numerous publications on the statistical intuitions of very different groups of participants. They focused on babies and nonhuman primates—two groups that recent studies have shown to be surprisingly capable of statistical learning and inference.

Whereas studies tend to use symbolic, abstract descriptions to assess adults' statistical reasoning, infants or animals have to experience the statistical information at first hand by interacting with the environment—after all, they cannot read. For example, one way of testing babies' statistical intuitions is for an experimenter to draw a colored ball from an opaque box of balls. The content of the box is then revealed, and the length of time the babies spend looking at the balls in the box is recorded. Babies tend to look for longer when the sample drawn does not reflect the color distribution of the balls in the box than when the sample is consistent with that distribution. This suggests that babies already have a basic understanding of random sampling. Similar studies have been conducted with animals by presenting them with food items.

Adults' decisions have been shown to improve when they are able to experience probability information themselves—for example, when they play a lottery repeatedly. Studies have also shown that adults make fewer errors in judging statistical information when they are able to experience that information at first hand—for example, in a computer simulation.

Relevance for education policy

"Awareness of this description–experience gap in statistical reasoning is of major relevance for education policy and the development of teaching methods. For example, schools could teach statistics and probability by means of hands-on examples and simulations, rather than primarily through written tasks," says Christin Schulze, Senior Research Scientist in the Center for Adaptive Rationality at the Max Planck Institute for Human Development and lead author of the study comparing findings from research on babies, nonhuman primates, and adults.

Although the two studies take different approaches, their results are consistent: It makes a difference whether we learn about probabilities from descriptions or from experience. "Many of our everyday decisions have to be made under uncertainty, with the help of statistical intuitions. Our findings show that the statistical intuitions of primates, infants, and adults can be surprisingly good. The key factor is how we encounter statistical information. These findings indicate that our statistical intuitions are by no means as irrational as has long been suggested," says Ralph Hertwig, Director of the Center for Adaptive Rationality at the Max Planck Institute for Human Development.

The research was published in Cognition.