Multisensory integration: When correlation implies causation

December 15, 2011, Max-Planck-Gesellschaft
This is the experimental setup. Credit: Kopp/ Parise

In order to get a better picture of our surroundings, the brain has to integrate information from different senses, but how does it know which signals to combine? New research involving scientists from the Max Planck Institute for Biological Cybernetics, the Bernstein Center for Computational Neuroscience Tübingen, the University of Oxford, and the University of Bielefeld has demonstrated that humans exploit the correlation between the temporal structures of signals to decide which of them to combine and which to keep segregated. This research is about to be published in Current Biology.

Multisensory signals originating from the same distal event are often similar in nature. Think of fireworks on New Year's Eve, an object falling and bouncing on the floor, or the footsteps of a person walking down the street. The temporal structures of such visual and auditory events are always almost overlapping (i.e., they correlate), and we often effortlessly assume an underlying unity between our visual and auditory experiences. In fact, the similarity of temporal structure of multiple unisensory signals, rather than merely their temporal coincidence as it has been previously thought, provides a potentially powerful cue for the to determine whether or not multiple have a common cause.

Cesare Parise from the Max Planck Institute for in Tübingen and Bernstein Center for Computational Neuroscience Tübingen and his colleagues set out to examine the role of signal correlation in multisensory integration by asking people to localize a stream of beeps and flashes. Participants seated in front of a large screen where sounds (streams of noise bursts) and images (streams blurred blobs) were presented from different spatial locations. On some trials only visual or were presented, while on other trials visual and auditory stimuli were presented in combination. Critically, on combined audiovisual trials, the temporal structure of the visual and auditory stimuli could either be correlated or not. Participants were required to report the spatial position of such stimuli by moving a cursor controlled by a graphic tablet. In line with previous studies, participants were more precise when the auditory and visual streams were presented together than when they were presented in isolation. Notably, precision was even higher when auditory and visual streams were correlated, and closely approached the theoretical maximum.

These results demonstrate that humans optimally combine multiple sensory signals only when they correlate in time. Previous research has demonstrated that optimal integration only occurs when the brain is sure that the signals have a common underlying cause. These results therefore demonstrate that the brain uses the statistical correlation between the sensory signals to infer whether they have a common physical cause, and hence whether they provide redundant information that should be integrated.

The researchers suggest the brain has evolved this ability to combine potentially related information from different senses so it can effectively pick its way through the noisy environments of everyday life.

"It's why at a noisy cocktail party you can tell who is speaking with which voice," says Parise. "Our eyes and ears are continually taking in sensory information and our brains make of it all by merging together sights and sounds with similar temporal structures."

In spite of being a pervasive aspect of sensory processing, little is known about the low-level statistical determinants of multisensory integration for signals with complex dynamic temporal patterns. This research highlights the role of a key organizational principle for multisensory perceptual grouping. What at first glance appears to be a logical fallacy, namely inferring causation from correlation, turns out to be the rule in perception.

Explore further: Playing music alters the processing of multiple sensory stimuli in the brain

More information: Cesare V. Parise, Charles Spence, Marc O. Ernst: When Correlation Implies Causation in Multisensory Integration. Current Biology doi: 10.1016/j.cub.2011.11.039

Related Stories

Playing music alters the processing of multiple sensory stimuli in the brain

November 24, 2011
(Medical Xpress) -- Over the years pianists develop a particularly acute sense of the temporal correlation between the movements of the piano keys and the sound of the notes played. However, they are no better than non-musicians ...

Recommended for you

Research reveals atomic-level changes in ALS-linked protein

January 18, 2018
For the first time, researchers have described atom-by-atom changes in a family of proteins linked to amyotrophic lateral sclerosis (ALS), a group of brain disorders known as frontotemporal dementia and degenerative diseases ...

Fragile X finding shows normal neurons that interact poorly

January 18, 2018
Neurons in mice afflicted with the genetic defect that causes Fragile X syndrome (FXS) appear similar to those in healthy mice, but these neurons fail to interact normally, resulting in the long-known cognitive impairments, ...

How your brain remembers what you had for dinner last night

January 17, 2018
Confirming earlier computational models, researchers at University of California San Diego and UC San Diego School of Medicine, with colleagues in Arizona and Louisiana, report that episodic memories are encoded in the hippocampus ...

Recording a thought's fleeting trip through the brain

January 17, 2018
University of California, Berkeley neuroscientists have tracked the progress of a thought through the brain, showing clearly how the prefrontal cortex at the front of the brain coordinates activity to help us act in response ...

Midbrain 'start neurons' control whether we walk or run

January 17, 2018
Locomotion comprises the most fundamental movements we perform. It is a complex sequence from initiating the first step, to stopping when we reach our goal. At the same time, locomotion is executed at different speeds to ...

Miles Davis is not Mozart: The brains of jazz and classical pianists work differently

January 16, 2018
Keith Jarret, world-famous jazz pianist, once answered in an interview when asked if he would ever be interested in doing a concert where he would play both jazz and classical music: "No, that's hilarious. [...] It's like ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.