Even unconsciously, sound helps us see

“Imagine you are playing ping-pong with a friend. Your friend makes a serve. Information about where and when the ball hit the table is provided by both vision and hearing. Scientists have believed that each of the senses produces an estimate relevant for the task (in this example, about the location or time of the ball’s impact) and then these votes get combined subconsciously according to rules that take into account which sense is more reliable. And this is how the senses interact in how we perceive the world. However, our findings show that the senses of hearing and vision can also interact at a more basic level, before they each even produce an estimate,” says Ladan Shams, a UCLA professor of psychology, and the senior author of a new study appearing in the December issue of Psychological Science, a journal published by the Association for Psychological Science. “If we think of the perceptual system as a democracy where each sense is like a person casting a vote and all votes are counted (albeit with different weights) to reach a decision, what our study shows is that the voters talk to one another and influence one another even before each casts a vote.”

“The senses affect each other in many ways,” says cognitive neuroscientist Robyn Kim. There are connections between the auditory and visual portions of the brain and at the cognitive level. When the information from one sense is ambiguous, another sense can step in and clarify or ratify the perception. Now, for the first time, Kim, Megan Peters, and Ladan Shams, working at the University of California Los Angeles, have shown behavioral evidence that this interplay happens in the earliest workings of perception—not just before that logical decision-making stage, but before the pre-conscious combination of sensory information.

To demonstrate that one sense can affect another even before perception, the researchers showed 63 participants a bunch of dots on a screen, in two phases with a pause between them. In one phase, the dots moved around at random; in the other, some proportion moved together from right to left. The participants had to indicate in which phase the dots moved together horizontally. In experiment 1, the subjects were divided into three groups. While they looked at the dots, one group heard sound moving in the same direction as the right-to-left dots, and stationary sound in the random phase. A second group heard the same right-to-left sound in both phases. The third group heard the identical sound in both phases, but it moved in the opposite direction of the dots. In the second and third conditions, because the sound was exactly the same in both phases, it added no cognitively useful information about which phase had the leftward-moving dots. In experiment 2, each participant experienced trials in all three conditions.

The results: All did best under the first condition—when the sound moved only in the leftward-motion phase. The opposite-moving sound neither enhanced nor worsened the visual perception. But surprisingly, the uninformative sound—the one that traveled leftward both with the leftward-moving dots and also when the dots moved randomly—helped people correctly perceive when the dots were moving from one side to the other. Hearing enhanced seeing, even though the added sense couldn’t help them make the choice.

The study, says Kim, should add to our appreciation of the complexity of our senses. “Most of us understand that smell affects taste. But people tend to think that what they see is what they see and what they hear is what they hear.” The findings of this study offer “further evidence that, even at a non-conscious level, visual and auditory processes are not so straightforward,” she says. “Perception is actually a very complex thing affected by many factors.”

“This study shows that at least in regards to perception of moving objects, hearing and sight are deeply intertwined, to the degree that even when is completely irrelevant to the task, it still influences the way we see the world,” Shams says.

Related Stories

New evidence of an unrecognized visual process

Nov 24, 2011

(Medical Xpress) -- We don’t see only what meets the eye. The visual system constantly takes in ambiguous stimuli, weighs its options, and decides what it perceives. This normally happens effortlessly. Sometimes, however, ...

Memories may skew visual perception

Jul 20, 2011

Taking a trip down memory lane while you are driving could land you in a roadside ditch, new research indicates. Vanderbilt University psychologists have found that our visual perception can be contaminated by memories of ...

Recommended for you

Researchers unlock mystery of skin's sensory abilities

Dec 19, 2014

Humans' ability to detect the direction of movement of stimuli in their sensory world is critical to survival. Much of this stimuli detection comes from sight and sound, but little is known about how the ...

Tackling neurotransmission precision

Dec 18, 2014

Behind all motor, sensory and memory functions, calcium ions are in the brain, making those functions possible. Yet neuroscientists do not entirely understand how fast calcium ions reach their targets inside ...

User comments

Adjust slider to filter visible comments by rank

Display comments: newest first

Tausch
1 / 5 (1) Dec 03, 2011
Kudos
http://www.newsci...1501.jpg
This is why the concept depicted in the link doesn't delivery what is promised. The non-conscious "intertwine-ment" has not been properly understood or considered.

Now, for the first time, Kim, Megan Peters, and Ladan Shams, working at the University of California Los Angeles, have shown behavioral evidence that this interplay happens in the earliest workings of perceptionnot just before that logical decision-making stage, but before the pre-conscious combination of sensory information. - authors


Yes. This ability arose embryonic-ally. All perceptions in the brain originated from signals of the senses. The catch is: the senses during fetal development were not sending signals induce external stimuli - hence the non-conscious level arose from that stage of developing perceptions.
The 'internal' stimuli send to the brain were the differentiated cells comprising of the developing senses.
cont...
Tausch
1 / 5 (1) Dec 03, 2011
Kudos
http://www.newsci...und.html
This is why the concept depicted in the link doesn't delivery what is promised. The non-conscious "intertwine-ment" has not been properly understood or considered.

Now, for the first time, Kim, Megan Peters, and Ladan Shams, working at the University of California Los Angeles, have shown behavioral evidence that this interplay happens in the earliest workings of perceptionnot just before that logical decision-making stage, but before the pre-conscious combination of sensory information. - authors


Yes. This ability arose embryonic-ally. All perceptions in the brain originated from signals of the senses. The catch is: the senses during fetal development were not sending signals induce external stimuli - hence the non-conscious level arose from that stage of developing perceptions.
The 'internal' stimuli send to the brain were the differentiated cells
cont...
Tausch
1 / 5 (1) Dec 03, 2011
Kudos
See NewScientist Health 17Aug2010
Sensory hijack: rewiring brains to see with sound

The concept depicted in the above article doesn't delivery what is promised. The non-conscious "intertwine-ment" has not been properly understood or considered.

Now, for the first time, Kim, Megan Peters, and Ladan Shams, working at the University of California Los Angeles, have shown behavioral evidence that this interplay happens in the earliest workings of perceptionnot just before that logical decision-making stage, but before the pre-conscious combination of sensory information. - authors


Yes. Kudos again. This ability arose embryonic-ally. All perceptions in the brain originated from signals of the senses. The catch is: the senses during fetal development were not sending signals from external stimuli - hence the non-conscious level arose from that stage to develop 'perception'.
The 'internal' stimuli send to the brain were the differentiated cells comprising of...
cont...
Tausch
1 / 5 (1) Dec 03, 2011
Apologize profusely for triple post!!!

...cont...the senses. The second catch is: the senses do not have to be so far developed to carry out their future tasks before sending signals to the brain. The cells comprising of the senses send signals to the brain long before any of the senses saw or experienced 'action'(external stimuli).

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.