Visual cues amplify sound

February 13, 2018, University College London
Visual cues amplify sound
Credit: AngelPictures, Source: Flickr

Looking at someone's lips is good for listening in noisy environments because it helps our brains amplify the sounds we're hearing in time with what we're seeing, finds a new UCL-led study.

The researchers say their findings, published in Neuron, could be relevant to people with or , as they tend to struggle hearing conversations in noisy places like a pub or restaurant.

The researchers found that visual information is integrated with at an earlier, more basic level than previously believed, independent of any conscious or attention-driven processes. When information from the eyes and ears is temporally coherent, the auditory cortex – the part of the brain responsible for interpreting what we hear – boosts the relevant sounds that tie in with what we're looking at.

"While the auditory cortex is focused on processing sounds, roughly a quarter of its neurons respond to light – we helped discover that a decade ago, and we've been trying to figure out why that's the case ever since," said the study's lead author, Dr. Jennifer Bizley (UCL Ear Institute).

In a 2015 study, she and her team found that people can pick apart two different sounds more easily if the one they're trying to focus on happens in time with a visual cue. For this latest study, the researchers presented the same auditory and visual stimuli to ferrets while recording their neural activity. When one of the auditory streams changed in amplitude in conjunction with changes in luminance of the visual stimulus, more of the neurons in the reacted to that sound.

"Looking at someone when they're speaking doesn't just help us hear because of our ability to recognise lip movements – we've shown it's beneficial at a lower level than that, as the timing of the movements aligned with the timing of the sounds tells our which sounds to represent more strongly. If you're trying to pick someone's voice out of background noise, that could be really helpful," said Dr. Bizley.

The researchers say their findings could help develop training strategies for people with , as they have had early success in helping people tap into their brain's ability to link up sound and sight. The findings could also help hearing aid and cochlear implant manufacturers develop smarter ways to amplify sound by linking it to the person's gaze direction.

The paper adds to evidence that people who are having trouble should get their eyes tested as well.

Explore further: Egocentric hearing: Study clarifies how we can tell where a sound is coming from

More information: Huriye Atilgan et al. Integration of Visual Information in Auditory Cortex Promotes Auditory Scene Analysis through Multisensory Binding, Neuron (2018). DOI: 10.1016/j.neuron.2017.12.034

Related Stories

Egocentric hearing: Study clarifies how we can tell where a sound is coming from

June 15, 2017
A new UCL and University of Nottingham study has found that most neurons in the brain's auditory cortex detect where a sound is coming from relative to the head, but some are tuned to a sound source's actual position in the ...

When the eyes move, the eardrums move, too

January 23, 2018
Simply moving the eyes triggers the eardrums to move too, says a new study by Duke University neuroscientists.

Rewired visual input to sound-processing part of the brain leads to compromised hearing

August 22, 2012
Scientists at Georgia State University have found that the ability to hear is lessened when, as a result of injury, a region of the brain responsible for processing sounds receives both visual and auditory inputs.

Research finds brain responses to lip-reading can benefit cochlear implant users

August 15, 2017
A world-first study has found that lip-reading may have a beneficial effect on the brain and on a person's ability to hear with a cochlear implant, contrary to what was previously believed.

Our ability to focus on one voice in crowds is triggered by voice pitch

October 10, 2017
Scientists have discovered that a group of neurons in the brain's auditory stem help us to tune into specific conversations in a crowded room.

Recommended for you

Research shows signalling mechanism in the brain shapes social aggression

October 19, 2018
Duke-NUS researchers have discovered that a growth factor protein, called brain-derived neurotrophic factor (BDNF), and its receptor, tropomyosin receptor kinase B (TrkB) affects social dominance in mice. The research has ...

Scientists discover the region of the brain that registers excitement over a preferred food option

October 19, 2018
At holiday buffets and potlucks, people make quick calculations about which dishes to try and how much to take of each. Johns Hopkins University neuroscientists have found a brain region that appears to be strongly connected ...

Gene plays critical role in noise-induced deafness

October 19, 2018
In experiments using mice, a team of UC San Francisco researchers has discovered a gene that plays an essential role in noise-induced deafness. Remarkably, by administering an experimental chemical—identified in a separate ...

How clutch molecules enable neuron migration

October 19, 2018
The brain can discriminate over 1 trillion odors. Once entering the nose, odor-related molecules activate olfactory neurons. Neuron signals first accumulate at the olfactory bulb before being passed on to activate the appropriate ...

How the brain makes rapid, fine adjustments in motor activity

October 18, 2018
Short-term motor learning appears not to require physical change in the brain Brain's premotor cortex may use a 'neural scratch pad' to calculate fine adjustments Brain can try different things in simulation without 'screwing ...

Scientists uncover how rare gene mutation affects brain development and memory

October 18, 2018
Researchers from the University of California, Irvine School of Medicine, have found that a rare gene mutation alters brain development in mice, impairing memory and disrupting the communication between nerve cells. They ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.