'Seeing' bodies with sound (no sight required)

March 6, 2014, Cell Press

People born unable to see are readily capable of learning to perceive the shape of the human body through soundscapes that translate images into sound, according to researchers who report their findings in the Cell Press journal Current Biology on March 6. With a little training, soundscapes representing the outlines and silhouettes of bodies cause the brain's visual cortex—and specifically an area dedicated in normally sighted people to processing body shapes—to light up with activity.

With no more than 70 hours of training on average, study participants could recognize the presence of a human form. What's more, they were able to detect the exact posture of the person in the image and imitate it.

"The idea is to replace information from a missing sense by using input from a different sense," explains Amir Amedi of The Hebrew University of Jerusalem. "It's just like bats and dolphins use sounds and echolocation to 'see' using their ears."

"Imagine for instance a diagonal line going down from left to right; if we use a descending musical scale—going on the piano from right to left—it will describe it nicely," continues Ella Striem-Amit, also of The Hebrew University. "And if the diagonal line is going up from left to right, then we use an ascending musical scale."

The researchers first taught people to perceive simple dots and lines. Then those individuals learned to connect the lines with junctions or curves, gradually working up to more and more complex images.

The success of the sensory substitution approach suggests great potential for its use in therapy more broadly, the researchers say.

"We're beginning to understand [that] the brain is more than a pure sensory machine," Amedi says. "It is a highly flexible task machine. The time has come to revive the focus on practical visual rehabilitation with sensory substitution devices."

In the current study, the researchers used an algorithm they call vOICe, which translates images in black and white. But they've since developed a newer algorithm, EyeMusic, which incorporates color information through the use of different musical instruments. The EyeMusic app is freely available on iTunes, where the researchers hope it can be useful to the blind and the general public alike.

Explore further: EyeMusic Sensory Substitution Device enables the blind to 'see' colors and shapes

More information: Current Biology, Striem-Amit et al.: "Visual cortex extrastriate body-selective area activation in congenitally blind people 'seeing' using sounds." dx.doi.org/10.1016/j.cub.2014.02.010

Related Stories

EyeMusic Sensory Substitution Device enables the blind to 'see' colors and shapes

February 4, 2014
Using auditory or tactile stimulation, Sensory Substitution Devices (SSDs) provide representations of visual information and can help the blind "see" colors and shapes. SSDs scan images and transform the information into ...

Activating the 'mind's eye': Scientists teach blind to read, recognize objects with sounds (w/ Video)

November 7, 2012
Common wisdom has it that if the visual cortex in the brain is deprived of visual information in early infanthood, it may never develop properly its functional specialization, making sight restoration later in life almost ...

Device converting images into music helps individuals without vision reach for objects in space

July 5, 2012
Sensory substitution devices (SSDs) use sound or touch to help the visually impaired perceive the visual scene surrounding them. The ideal SSD would assist not only in sensing the environment but also in performing daily ...

How blind can 'read' shown in new research

May 16, 2012
A method developed at the Hebrew University of Jerusalem for training blind persons to "see" through the use of a sensory substitution device (SSD) has enabled those using the system to actually "read" an eye chart with letter ...

A sonar vision system for the congenitally blind

November 29, 2012
A "sonar vision" system that enables people who are blind from birth to perceive the shape of a face, a house or even words and letters, is being developed by a team at the Hebrew University of Jerusalem. Using this device, ...

Recommended for you

Research reveals atomic-level changes in ALS-linked protein

January 18, 2018
For the first time, researchers have described atom-by-atom changes in a family of proteins linked to amyotrophic lateral sclerosis (ALS), a group of brain disorders known as frontotemporal dementia and degenerative diseases ...

Fragile X finding shows normal neurons that interact poorly

January 18, 2018
Neurons in mice afflicted with the genetic defect that causes Fragile X syndrome (FXS) appear similar to those in healthy mice, but these neurons fail to interact normally, resulting in the long-known cognitive impairments, ...

How your brain remembers what you had for dinner last night

January 17, 2018
Confirming earlier computational models, researchers at University of California San Diego and UC San Diego School of Medicine, with colleagues in Arizona and Louisiana, report that episodic memories are encoded in the hippocampus ...

Recording a thought's fleeting trip through the brain

January 17, 2018
University of California, Berkeley neuroscientists have tracked the progress of a thought through the brain, showing clearly how the prefrontal cortex at the front of the brain coordinates activity to help us act in response ...

Midbrain 'start neurons' control whether we walk or run

January 17, 2018
Locomotion comprises the most fundamental movements we perform. It is a complex sequence from initiating the first step, to stopping when we reach our goal. At the same time, locomotion is executed at different speeds to ...

Miles Davis is not Mozart: The brains of jazz and classical pianists work differently

January 16, 2018
Keith Jarret, world-famous jazz pianist, once answered in an interview when asked if he would ever be interested in doing a concert where he would play both jazz and classical music: "No, that's hilarious. [...] It's like ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

winthrom
not rated yet Mar 06, 2014
Need to put this on Android also

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.