Sight, Sound Processed Together and Earlier than Previously Thought

October 30, 2007

The area of the brain that processes sounds entering the ears also appears to process stimulus entering the eyes, providing a novel explanation for why many viewers believe that ventriloquists have thrown their voices to the mouths of their dummies.

More generally, these findings from Duke University Medical Center offer new insights into how the brain takes in and assembles a multitude of stimuli from the outside world. By studying monkeys, the researchers found that auditory and visual information is processed together before the combined signals make it to the brain's cortex, the analytical portion of the brain that assembles the stimuli from all the senses into coherent thoughts.

"The prevailing wisdom among brain scientists has been that each of the five senses – sight, hearing, smell, touch and taste – is governed by its own corresponding region of the brain," said Jennifer Groh, Ph.D., a neurobiologist in Duke's Center for Cognitive Neuroscience. "The view has been that each of these areas processes the information separately and sends that information to the cortex, which puts it all together at the end.

"Now, we are beginning to appreciate that it's not that simple," Groh continued. "Our results show that there are interactions between the sensory pathways that occur very early in the process, which implies that the integration of the different senses may be a more primitive process and one not requiring high-level brain functioning."

The results of Groh's experiments were published early online in the Proceedings of the National Academy of Sciences.

Groh has a particular interest in a tiny round structure in the brain known as the inferior colliculus. This structure, less than a half-inch in diameter, is located in the most primitive area of the brain. It is one of several early stops in the brain for signals leaving the ear, headed for the cortex.

"In our experiments, we found that this structure, which had been assumed to mainly process auditory information, actually responds to visual information as well," Groh explained. "In fact, about 64 percent of the neurons in the inferior colliculus can carry visual as well as auditory signals. This means that visual and auditory information gets combined quite early, and before the 'thinking part' of the brain can make sense of it."

That is why ventriloquism seems to work, she said. The association between the voice and the moving mouth of the dummy is made before the viewer consciously thinks about it. The same process may also explain why the words being spoken by a talking head on television appear to be coming out of the mouth, even though the television speakers are located to the side of the set.

"The eyes see the lips moving and the ears hear the sound and the brain immediately jumps to the conclusion about the origin of the voice," Groh said.

Groh said that it makes logical sense for hearing and vision to have some level of integration in the monkeys she studied, and in humans.

"We generally live in similar ecological niches; we are active during the day and tend to communicate vocally," she said. "The inferior colliculus is similar in both species, and with the advent of new imaging technology, like functional MRI, which can visualize brain regions in real time. We should be able to correlate what we're seeing in animal models with what happens in humans."

Groh and her team are now conducting experiments to determine whether or not one of the senses influences how the other is perceived.

Source: Duke University

Explore further: Important brain network for processing sensory perceptions elucidated

Related Stories

Important brain network for processing sensory perceptions elucidated

January 21, 2016
Every day, we constantly absorb information through our sensory organs, which the brain then needs to process correctly. The information initially reaches the main relay center, the thalamus, and then travels to the cerebral ...

Vision trumps hearing in study

September 11, 2013
A Duke University study used puppet-based comedy to demonstrate the complicated inner-workings of the brain and shows what every ventriloquist knows: The eye is more convincing than the ear.

Staying focused: Cortico-thalamic pathway filters relevant sensory cues from perceptual input

May 13, 2014
(Medical Xpress)—On the one hand, the nervous has limited computational capability – but at the same time, the sensory environment contains an immense amount of information. In this demanding situation, the brain somehow ...

Why we look at the puppet, not the ventriloquist

August 30, 2013
(Medical Xpress)—As ventriloquists have long known, your eyes can sometimes tell your brain where a sound is coming from more convincingly than your ears can.

How vision captures sound now somewhat uncertain

January 16, 2014
(Medical Xpress)—When listening to someone speak, we also rely on lip-reading and gestures to help us understand what the person is saying.

Recommended for you

Make way for hemoglobin

August 18, 2017
Every cell in the body, whether skin or muscle or brain, starts out as a generic cell that acquires its unique characteristics after undergoing a process of specialization. Nowhere is this process more dramatic than it is ...

Bio-inspired materials give boost to regenerative medicine

August 18, 2017
What if one day, we could teach our bodies to self-heal like a lizard's tail, and make severe injury or disease no more threatening than a paper cut?

Female mouse embryos actively remove male reproductive systems

August 17, 2017
A protein called COUP-TFII determines whether a mouse embryo develops a male reproductive tract, according to researchers at the National Institutes of Health and their colleagues at Baylor College of Medicine, Houston. The ...

Two-step process leads to cell immortalization and cancer

August 17, 2017
A mutation that helps make cells immortal is critical to the development of a tumor, but new research at the University of California, Berkeley suggests that becoming immortal is a more complicated process than originally ...

Are stem cells the link between bacteria and cancer?

August 17, 2017
Gastric carcinoma is one of the most common causes of cancer-related deaths, primarily because most patients present at an advanced stage of the disease. The main cause of this cancer is the bacterium Helicobacter pylori, ...

New Pathology Atlas maps genes in cancer to accelerate progress in personalized medicine

August 17, 2017
A new Pathology Atlas is launched today with an analysis of all human genes in all major cancers showing the consequence of their corresponding protein levels for overall patient survival. The difference in expression patterns ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.