How vision captures sound now somewhat uncertain

January 16, 2014 by Steve Hartsoe, Duke University
How vision captures sound now somewhat uncertain
Neurons in the eye-movement region of the brain use two different strategies for signaling the locations of sights and sounds. For visual stimuli, the neurons form a map akin to a "zone defense" in basketball. The location of a visual stimulus can be inferred from a "hill" of activity in this brain region, called the superior colliculus. For sounds, neurons throughout the region respond with a broad plateau of activity that isn't location-dependent, but varies in intensity. The "height" of the plateau acts like the dial on a meter to signal sound location. Plateau height on opposite sides of the brain may perform a neural game of tug-of-war to indicate where a sound is.

(Medical Xpress)—When listening to someone speak, we also rely on lip-reading and gestures to help us understand what the person is saying.

To link these sights and sounds, the has to know where each stimulus is located so it can coordinate processing of related visual and auditory aspects of the scene. That's how we can single out a conversation when it's one of many going on in a room.

While past research has shown that the brain creates a similar code for vision and hearing to integrate this information, Duke University researchers have found the opposite: neurons in a particular brain region respond differently, not similarly, based on whether the stimuli is visual or auditory.

The finding, which posted Jan. 15 in the journal PLOS ONE, provides insight into how vision captures the location of perceived sound.

The idea among brain researchers has been that the neurons in a brain area known as the superior colliculus employ a "zone defense" when signaling where stimuli are located. That is, each neuron monitors a particular region of an external scene and responds whenever a stimulus—either visual or auditory—appears in that location. Through teamwork, the ensemble of neurons provides coverage of the entire scene.

But the study by Duke researchers found that don't behave that way. When the target was a sound, the neurons responded as if playing a game of tug-of-war, said lead author Jennifer Groh, a professor of psychology and neuroscience at Duke.

"The neurons responded to nearly all sound locations. But how vigorously they responded depended on where the sound was," Groh said. "It's still teamwork, but a different kind. It's pretty cool that the neurons can use two different strategies, play two different games, at the same time."

Groh said the finding opens up a mystery: if neurons respond differently to visual and at similar locations in space, then the underlying mechanism of how vision captures sound is now somewhat uncertain.

"Which neurons are 'on' tells you where a is located, but how strongly they're 'on' tells you where an auditory stimulus is located," said Groh, who conducted the study with co-author Jung Ah Lee, a postdoctoral fellow at Duke.

"Both of these kinds of signals can be used to control behavior, like eye movements, but it is trickier to envision how one type of signal might directly influence the other."

The study involved assessing the responses of , located in the rostral superior colliculus of the midbrain, as two rhesus monkeys moved their eyes to visual and auditory targets.

The sensory targets—light-emitting diodes attached to the front of nine speakers—were placed 58 inches in front of the animals. The speakers were located from 24 degrees left to 24 degrees right of the monkey in 6-degree increments.

The researchers then measured the monkey's responses to bursts of white noise and the illuminating of the lights.

Groh said how the brain takes raw input of one form and converts it into something else "may be broadly useful for more cognitive processes."

"As we develop a better understanding of how those computations unfold it may help us understand a little bit more about how we think," she said.

Explore further: Why we look at the puppet, not the ventriloquist

More information: "Different Stimuli, Different Spatial Codes: A Visual Map and an Auditory Rate Code for Oculomotor Space in the Primate Superior Colliculus," Jennifer M. Groh, Jung Ah Lee, Duke University. PLOS ONE, January 2014. DOI: 10.1371/journal.pone.0085017

Related Stories

Why we look at the puppet, not the ventriloquist

August 30, 2013
(Medical Xpress)—As ventriloquists have long known, your eyes can sometimes tell your brain where a sound is coming from more convincingly than your ears can.

Rewired visual input to sound-processing part of the brain leads to compromised hearing

August 22, 2012
Scientists at Georgia State University have found that the ability to hear is lessened when, as a result of injury, a region of the brain responsible for processing sounds receives both visual and auditory inputs.

Vision trumps hearing in study

September 11, 2013
A Duke University study used puppet-based comedy to demonstrate the complicated inner-workings of the brain and shows what every ventriloquist knows: The eye is more convincing than the ear.

How visual representations are improved by reducing noise in the brain

November 6, 2013
Neuroscientist Suresh Krishna from the German Primate Center (DPZ) in cooperation with Annegret Falkner and Michael Goldberg at Columbia University, New York has revealed how the activity of neurons in an important area of ...

Brain wiring quiets the voice inside your head

September 3, 2013
During a normal conversation, your brain is constantly adjusting the volume to soften the sound of your own voice and boost the voices of others in the room.

Study aims to understand how, when the auditory system registers complex auditory-visual synchrony

October 23, 2013
Imagine the brain's delight when experiencing the sounds of Beethoven's "Moonlight Sonata" while simultaneously taking in a light show produced by a visualizer.

Recommended for you

How the brain tells our limbs apart

February 21, 2018
Legs and arms perform very different functions. Our legs are responsible primarily for repetitive locomotion, like walking and running. Our arms and hands, by contrast, must be able to execute many highly specialized jobs—picking ...

Cognitive benefits of 'young blood' linked to brain protein in mice

February 21, 2018
Loss of an enzyme that modifies gene activity to promote brain regeneration may be partly responsible for age-related cognitive decline, according to new research in laboratory mice by UC San Francisco scientists, who also ...

Schizophrenia a side effect of human development

February 21, 2018
Schizophrenia may have evolved as an "unwanted side effect" of the development of the complex human brain, a new study has found.

Therapeutic antibodies protected nerve–muscle connections in a mouse model of Lou Gehrig's disease

February 20, 2018
Amyotrophic lateral sclerosis (ALS), also known as Lou Gehrig's disease, causes lethal respiratory paralysis within several years of diagnosis. There are no effective treatments to slow or halt this devastating disease. Mouse ...

Brain liquefaction after stroke is toxic to surviving brain: study

February 20, 2018
Scientists have known for years that the brain liquefies after a stroke. If cut off from blood and oxygen for a long enough period, a portion of the brain will die, slowly morphing from a hard, rubbery substance into liquid ...

Brain aging may begin earlier than expected

February 20, 2018
Physicists have devised a new method of investigating brain function, opening a new frontier in the diagnoses of neurodegenerative and ageing related diseases.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.