Before assigning responsibility, our minds simulate alternative outcomes, study shows

October 17, 2017 by Anne Trafton, Massachusetts Institute of Technology
“What’s really cool about eye tracking is it lets you see things that you’re not consciously aware of,” Professor Josh Tenenbaum says. “When psychologists and philosophers have proposed the idea of counterfactual simulation, they haven’t necessarily meant that you do this consciously. It’s something going on behind the surface, and eye tracking is able to reveal that.” Credit: Jose-Luis Olivares, MIT

How do people assign a cause to events they witness? Some philosophers have suggested that people determine responsibility for a particular outcome by imagining what would have happened if a suspected cause had not intervened.

This kind of reasoning, known as counterfactual simulation, is believed to occur in many situations. For example, soccer referees deciding whether a player should be credited with an "own goal"—a goal accidentally scored for the opposing team—must try to determine what would have happened had the player not touched the ball.

This process can be conscious, as in the soccer example, or unconscious, so that we are not even aware we are doing it. Using technology that tracks eye movements, cognitive scientists at MIT have now obtained the first direct evidence that people unconsciously use counterfactual simulation to imagine how a situation could have played out differently.

"This is the first time that we or anybody have been able to see those simulations happening online, to count how many a person is making, and show the correlation between those simulations and their judgments," says Josh Tenenbaum, a professor in MIT's Department of Brain and Cognitive Sciences, a member of MIT's Computer Science and Artificial Intelligence Laboratory, and the senior author of the new study.

Tobias Gerstenberg, a postdoc at MIT who will be joining Stanford's Psychology Department as an assistant professor next year, is the lead author of the paper, which appears in the Oct. 17 issue of Psychological Science. Other authors of the paper are MIT postdoc Matthew Peterson, Stanford University Associate Professor Noah Goodman, and University College London Professor David Lagnado.

Follow the ball

Until now, studies of counterfactual simulation could only use reports from people describing how they made judgments about responsibility, which offered only indirect evidence of how their minds were working.

Gerstenberg, Tenenbaum, and their colleagues set out to find more direct evidence by tracking people's eye movements as they watched two billiard balls collide. The researchers created 18 videos showing different possible outcomes of the collisions. In some cases, the collision knocked one of the balls through a gate; in others, it prevented the ball from doing so.

Before watching the videos, some participants were told that they would be asked to rate how strongly they agreed with statements related to ball A's effect on ball B, such as, "Ball A caused ball B to go through the gate." Other participants were asked simply what the outcome of the collision was.

Before assigning responsibility, our minds simulate alternative outcomes, study shows
In this video, two participants' eye-movements are tracked while they watch a video clip. The blue dot indicates where each participant is looking on the screen. The participant on the left was asked to judge whether they thought that ball B went through the middle of the gate. Participants asked this question mostly looked at the balls and tried to predict where ball B would go. The participant on the right was asked to judge whether ball A caused ball B to go through the gate. Participants asked this question tried to simulate where ball B would have gone if ball A hadn't been present in the scene. Credit: Tobias Gerstenberg

As the subjects watched the videos, the researchers were able to track their eye movements using an infrared light that reflects off the pupil and reveals where the eye is looking. This allowed the researchers, for the first time, to gain a window into how the mind imagines possible outcomes that did not occur.

"What's really cool about eye tracking is it lets you see things that you're not consciously aware of," Tenenbaum says. "When psychologists and philosophers have proposed the idea of counterfactual simulation, they haven't necessarily meant that you do this consciously. It's something going on behind the surface, and is able to reveal that."

The researchers found that when participants were asked questions about ball A's effect on the path of ball B, their eyes followed the course that ball B would have taken had ball A not interfered. Furthermore, the more uncertainty there was as to whether ball A had an effect on the outcome, the more often participants looked toward ball B's imaginary trajectory.

"It's in the close cases where you see the most counterfactual looks. They're using those looks to resolve the uncertainty," Tenenbaum says.

Participants who were asked only what the actual outcome had been did not perform the same eye movements along ball B's alternative pathway.

How people think

The researchers are now using this approach to study more complex situations in which people use counterfactual simulation to make judgments of causality.

"We think this process of counterfactual simulation is really pervasive," Gerstenberg says. "In many cases it may not be supported by , because there are many kinds of abstract counterfactual thinking that we just do in our mind. But the billiard-ball collisions lead to a particular kind of counterfactual where we can see it."

One example the researchers are studying is the following: Imagine C is headed for the gate, while balls A and B each head toward C. Either one could knock C off course, but A gets there first. Is B off the hook, or should it still bear some responsibility for the outcome?

"Part of what we are trying to do with this work is get a little bit more clarity on how people deal with these complex cases. In an ideal world, the work we're doing can inform the notions of causality that are used in the law," Gerstenberg says. "There is quite a bit of interaction between computer science, psychology, and legal science. We're all in the same game of trying to understand how people think about causation."

Explore further: How long do batters 'keep their eye on the ball'? Eye and head movements differ when swinging or taking a pitch

Related Stories

How long do batters 'keep their eye on the ball'? Eye and head movements differ when swinging or taking a pitch

August 3, 2017
Where are baseball batters looking during the fraction of a second when a pitched ball is in their air? Their visual tracking strategies differ depending on whether they're swinging at the pitch, reports a study in the August ...

New ball to showcase talent in World Cup

June 5, 2014
University of Adelaide physics experts believe the new soccer ball created for the 2014 FIFA World Cup starting next week is a "keepers' ball".

Expectation may be essential to memory formation

March 28, 2016
A theory that links memory encoding to expectations of future relevance may better explain how human memory works, according to a team of Penn State psychologists.

New insights into the one-in-a-million lightning called 'ball lightning'

August 7, 2013
One of the rare scientific reports on the rarest form of lightning—ball lightning—describes better ways of producing this mysterious phenomenon under the modern laboratory conditions needed to explain it. The new study ...

Will the new World Cup soccer ball bend? Physics plays a role in on-ground action

June 8, 2010
Physics experts at the University of Adelaide believe the new ball created for the 2010 World Cup, called the Jabulani, will play "harder and faster", bending more unpredictably than its predecessor.

Study: Children's understanding of alternative courses of events

December 16, 2013
The capacity to think about how the present would be if an event in the past had taken a different course is referred to by experts as counterfactual reasoning. A research project funded by the FWF plans to examine how this ...

Recommended for you

Beauty is simpler, and less special, than we realize

August 20, 2018
Beauty, long studied by philosophers, and more recently by scientists, is simpler than we might think, New York University psychology researchers have concluded in a new analysis. Their work, which appears in the journal ...

Bilingual children who speak native language at home have higher intelligence

August 20, 2018
Children who regularly use their native language at home while growing up in a different country have higher IQs, a new study has shown.

People are more honest when using a foreign tongue, research finds

August 17, 2018
New UChicago-led research suggests that someone who speaks in a foreign language is probably more credible than the average native speaker.

FDA approves brain stimulation device for OCD

August 17, 2018
(HealthDay)—A brain stimulation device to treat obsessive-compulsive disorder (OCD) has received approval for marketing Friday by the U.S. Food and Drug Administration.

Research eyes role of stress in mental illnesses

August 17, 2018
We all face stress in our lives. Even researchers seeking to understand why some people shrug it off while others face battles against disorders like depression or PTSD.

16 going on 66: Will you be the same person 50 years from now?

August 17, 2018
How much do you change between high school and retirement? The answer depends on whether you're comparing yourself to others or to your younger self.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.