Multisensory integration: When correlation implies causation

December 15, 2011
This is the experimental setup. Credit: Kopp/ Parise

In order to get a better picture of our surroundings, the brain has to integrate information from different senses, but how does it know which signals to combine? New research involving scientists from the Max Planck Institute for Biological Cybernetics, the Bernstein Center for Computational Neuroscience Tübingen, the University of Oxford, and the University of Bielefeld has demonstrated that humans exploit the correlation between the temporal structures of signals to decide which of them to combine and which to keep segregated. This research is about to be published in Current Biology.

Multisensory signals originating from the same distal event are often similar in nature. Think of fireworks on New Year's Eve, an object falling and bouncing on the floor, or the footsteps of a person walking down the street. The temporal structures of such visual and auditory events are always almost overlapping (i.e., they correlate), and we often effortlessly assume an underlying unity between our visual and auditory experiences. In fact, the similarity of temporal structure of multiple unisensory signals, rather than merely their temporal coincidence as it has been previously thought, provides a potentially powerful cue for the to determine whether or not multiple have a common cause.

Cesare Parise from the Max Planck Institute for in Tübingen and Bernstein Center for Computational Neuroscience Tübingen and his colleagues set out to examine the role of signal correlation in multisensory integration by asking people to localize a stream of beeps and flashes. Participants seated in front of a large screen where sounds (streams of noise bursts) and images (streams blurred blobs) were presented from different spatial locations. On some trials only visual or were presented, while on other trials visual and auditory stimuli were presented in combination. Critically, on combined audiovisual trials, the temporal structure of the visual and auditory stimuli could either be correlated or not. Participants were required to report the spatial position of such stimuli by moving a cursor controlled by a graphic tablet. In line with previous studies, participants were more precise when the auditory and visual streams were presented together than when they were presented in isolation. Notably, precision was even higher when auditory and visual streams were correlated, and closely approached the theoretical maximum.

These results demonstrate that humans optimally combine multiple sensory signals only when they correlate in time. Previous research has demonstrated that optimal integration only occurs when the brain is sure that the signals have a common underlying cause. These results therefore demonstrate that the brain uses the statistical correlation between the sensory signals to infer whether they have a common physical cause, and hence whether they provide redundant information that should be integrated.

The researchers suggest the brain has evolved this ability to combine potentially related information from different senses so it can effectively pick its way through the noisy environments of everyday life.

"It's why at a noisy cocktail party you can tell who is speaking with which voice," says Parise. "Our eyes and ears are continually taking in sensory information and our brains make of it all by merging together sights and sounds with similar temporal structures."

In spite of being a pervasive aspect of sensory processing, little is known about the low-level statistical determinants of multisensory integration for signals with complex dynamic temporal patterns. This research highlights the role of a key organizational principle for multisensory perceptual grouping. What at first glance appears to be a logical fallacy, namely inferring causation from correlation, turns out to be the rule in perception.

Explore further: Playing music alters the processing of multiple sensory stimuli in the brain

More information: Cesare V. Parise, Charles Spence, Marc O. Ernst: When Correlation Implies Causation in Multisensory Integration. Current Biology doi: 10.1016/j.cub.2011.11.039

Related Stories

Playing music alters the processing of multiple sensory stimuli in the brain

November 24, 2011
(Medical Xpress) -- Over the years pianists develop a particularly acute sense of the temporal correlation between the movements of the piano keys and the sound of the notes played. However, they are no better than non-musicians ...

Recommended for you

'Residual echo' of ancient humans in scans may hold clues to mental disorders

July 26, 2017
Researchers at the National Institute of Mental Health (NIMH) have produced the first direct evidence that parts of our brains implicated in mental disorders may be shaped by a "residual echo" from our ancient past. The more ...

Laser used to reawaken lost memories in mice with Alzheimer's disease

July 26, 2017
(Medical Xpress)—A team of researchers at Columbia University has found that applying a laser to the part of a mouse brain used for memory storage caused the mice to recall memories lost due to a mouse version of Alzheimer's ...

Cellular roots of anxiety identified

July 26, 2017
From students stressing over exams to workers facing possible layoffs, worrying about the future is a normal and universal experience. But when people's anticipation of bad things to come starts interfering with daily life, ...

Cognitive cross-training enhances learning, study finds

July 25, 2017
Just as athletes cross-train to improve physical skills, those wanting to enhance cognitive skills can benefit from multiple ways of exercising the brain, according to a comprehensive new study from University of Illinois ...

Brain disease seen in most football players in large report

July 25, 2017
Research on 202 former football players found evidence of a brain disease linked to repeated head blows in nearly all of them, from athletes in the National Football League, college and even high school.

Zebrafish study reveals clues to healing spinal cord injuries

July 25, 2017
Fresh insights into how zebrafish repair their nerve connections could hold clues to new therapies for people with spinal cord injuries.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.