Why we look at the puppet, not the ventriloquist

Why we look at the puppet, not the ventriloquist
Graduate student Daniel Pages is co-author of a paper studying the role sight plays in hearing. Credit: Jennifer Groh

(Medical Xpress)—As ventriloquists have long known, your eyes can sometimes tell your brain where a sound is coming from more convincingly than your ears can.

A series of experiments in humans and monkeys by Duke University researchers has found that the does not require simultaneous visual and audio stimulation to locate the source of a sound. Rather, visual feedback obtained from trying to find a sound with the eyes had a stronger effect than presented at the same time as the audio, according to the Duke study.

The findings could help those with mild learn to localize voices better, improving their ability to communicate in , said Jennifer Groh, a professor of psychology and neuroscience at Duke.

Locating where a sound is coming from is partially learned with the aid of vision. Researchers sought to learn more about how the brain locates the source of a sound when the source is unclear and there are a number of possible visual matches.

"Our study is related to ventriloquism, in which the of a puppet's mouth 'captures' the sound of the puppeteer's voice," Groh said. "It is thought that one reason this illusion occurs is because vision normally teaches the brain how to tell where sounds are coming from. We investigated how the brain knows which visual stimulus should capture the location of a sound, such as why it is the puppet's mouth and not some other visual stimulus."

The study, which appears Thursday (Aug. 29) in the journal PLOS ONE, tested two competing . In one, the brain determines the location of a sound based on the simultaneous occurrence of audio and its visual source. In the other, the brain uses a "guess and check" method. In this scenario, visual feedback sent to the brain after the eye focuses on a sound affects how the eye searches for that sound in the future, possibly through the brain's reward-related .

In both paradigms, the visual stimulus—an LED—was displaced from the sound. Groh's team then looked for evidence that the LED caused a persistent mislocation of the sound.

"Surprisingly, we found that visual feedback exerts the more powerful effect on altering localization of sounds," Groh said. "This suggests that the active behavior of looking at the during a ventriloquism performance plays a role in causing the shift in where you hear the voice."

Participants in the study—11 humans and two rhesus monkeys—shifted their sight to a sound under different visual and audio scenarios.

In one scenario, called the "synchrony-only" task, a visual stimulus appeared at the same time as a sound but too briefly to provide feedback after an eye movement to that sound.

In another, the "feedback-only" task, the visual stimulus appeared during the execution of an eye movement to a sound, but was never on at the same time as the sound.

The study found that the "feedback-only task" exerted a much more powerful effect on the estimation of sound location, as measured with eye tracking, than did the other scenario. This suggests that those who have difficulty localizing sounds may benefit from practice involving eye movements.

On average, participants altered their eye movements in the direction of the lightsâ?? location to a greater degree, about a quarter of the way, when the was presented as feedback than when it was presented at the same time as the sound, the study found.

"This is about the brain's self-improvement skills," said co-author Daniel Pages, a graduate student in Psychology & Neuroscience at Duke. "What we're getting at is how the brain uses different types of information to improve how it does its job. In this case, it uses vision coupled with eye movements to improve hearing."

"We were surprised at how important the were," Groh said. "But finding sounds is really hard. Feedback about your performance is important for anything that is difficult, whether it is the B- you get on your homework or the error your eyes detect in localizing a ."

More information: Pages, D. and Groh, J. Looking at the Ventriloquist: Visual Outcome of eye Movements Calibrates Sound Localization, PLOS ONE, Aug. 29, 2013. DOI: 10.1371/journal.pone.0072562

Journal information: PLoS ONE
Provided by Duke University
Citation: Why we look at the puppet, not the ventriloquist (2013, August 30) retrieved 20 April 2024 from https://medicalxpress.com/news/2013-08-puppet-ventriloquist.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Rewired visual input to sound-processing part of the brain leads to compromised hearing


Feedback to editors