Activating the 'mind's eye': Scientists teach blind to read, recognize objects with sounds (w/ Video)
Illustration demonstrates the concept of visual-to-auditory sensory substitution. Images from a camera are converted into soundscapes, using a predictable algorithm, allowing the user wearing earphones to listen to and then interpret the visual information coming from the camera. This enables blind people to decode the visual information within the sounds and perceive complex images of people, words and objects. Credit: Karin Kedar and Michael Gluhoded
Common wisdom has it that if the visual cortex in the brain is deprived of visual information in early infanthood, it may never develop properly its functional specialization, making sight restoration later in life almost impossible.
Scientists at the Hebrew University of Jerusalem and in France have now shown that blind people – using specialized photographic and sound equipment – can actually "see" and describe objects and even identify letters and words.
The new study by a team of researchers, led by Prof. Amir Amedi of the Edmond and Lily Safra Center for Brain Sciences and the Institute for Medical Research Israel-Canada at the Hebrew University and Ph.D. candidate Ella Striem-Amit, has demonstrated how this achievement is possible through the use of a unique training paradigm, using sensory substitution devices (SSDs).
SSDs are non-invasive sensory aids that provide visual information to the blind via their existing senses. For example, using a visual-to-auditory SSD in a clinical or everyday setting, users wear a miniature camera connected to a small computer (or smart phone) and stereo headphones.
This video is not supported by your browser at this time.
This is an example of one image from each of the visual categories used in the structured SSD training (geometric shapes, Hebrew letters, textures, body postures, everyday objects, houses, and faces) is demonstrated, along with its corresponding transformation to sound via the vOICe SSD (two repetitions of each 2-s-long soundscape). Credit: Striem-Amit et al., NeuronThe images are converted into "soundscapes," using a predictable algorithm, allowing the user to listen to and then interpret the visual information coming from the camera. The blind participants using this device reach a level of visual acuity technically surpassing the world-agreed criterion of the World Health Organization (WHO) for blindness, as published in a previous study by the same group.
The resulting sight, though not conventional in that it does not involve activation of the ophthalmological system of the body, is no less visual in the sense that it actually activates the visual identification network in the brain.
The study shows that following a dedicated (but relatively brief) 70 hours of unique training paradigm developed in the Amedi lab, the blind people could easily use SSDs to characterize images into object categories, such as images of faces, houses, body shapes, everyday objects and textures. They could also identify even more complex everyday objects—locating people's positions, identifying facial expressions, and even reading letters and words.
These unprecedented behavioral results are reported in the current issue of the prestigious neuroscience journal, Neuron.
This video is not supported by your browser at this time.
After the training program, participants could assign soundscapes to their visual categories and also determine multiple features of the stimulus (such as hairstyle in a face image, number of floors and windows in a house image, and body posture in a body-shape image), enabling them to differentiate between objects within categories. The movie depicts two examples of these abilities: a trained congenitally blind participant in the study as she reads a three-letter word and another participant as she identifies emotional facial expressions perceived through the SSD. Credit: Striem-Amit et al., NeuronThe Hebrew University study went on further to actually test what happens in the brain when the blind learn to see with sounds. Specifically, the group tested the ability of this high-acuity vision to activate the supposedly dormant visual cortex of the blind, even though it was taught to process the visual images through sounds only in adulthood.
Prof. Amedi, and Ella Striem-Amit used functional magnetic resonance imaging (fMRI) to measure the neural activity of people blind from birth as they "saw"—using the SSD—high-resolution images of letters, faces, houses, everyday objects and body-shapes. Surprisingly, not only was their visual cortex activated by the sounds, their brain showed selectivity for visual categories which characterize the normally developing, sighted brain.
A specific part of the brain, known as the Visual Word Form Area, or VWFA—that was first discovered in sighted people by Profs. Laurent Cohen and Stanislas Dehaene of Pitie-Salpétriere Hospital-INSERM-CEA, of France, co-authors of the current article—is normally very selective.
In sighted people, it has a role in reading, and is activated by seeing and reading letters more than by any other visual object category. Astonishingly, the same was found in this area in people deprived of sight. Their VWFA, after only tens of hours of training in SSD use, showed more activation for letters than for any of the other visual categories tested.
In fact, the VWFA was so plastic to change, that it showed increased activation for SSD letters after less than two hours of training by one of the study participants.
"The adult brain is more flexible that we thought," says Prof. Amedi. In fact, this and other recent research from various groups have demonstrated that multiple brain areas are not specific to their input sense (vision, audition or touch), but rather to the task, or computation they perform, which may be computed with various modalities. (This information was summarized in a recent review by the Amedi research group published in the journal Current Directions in Neurology.)
All of this suggests that in the blind, brain areas might potentially be "awakened" to processing visual properties and tasks even after years or maybe even lifelong blindness, if the proper technologies and training approaches are used, says Amedi.
The findings also give hope that reintroduced input into the visual centers of the blind brain could potentially restore vision, and that SSDs might be useful for visual rehabilitation.
"SSDs might help blind or visually-impaired individuals learn to process complex images, as done in this study, or they might be used as sensory interpreters that provide high-resolution, supportive, synchronous input to a visual signal arriving from an external device such as bionic eyes" says Prof. Amedi.
More information: Teaching the Blind to Read and Recognize Objects with Sounds, Neuron, DOI: 10.1016/j.neuron.2012.08.026
Journal reference: Neuron
Provided by Hebrew University of Jerusalem
- How blind can 'read' shown in new research May 16, 2012 | not rated yet | 0
- The brain as a 'task machine' Feb 17, 2011 | not rated yet | 0
- Brain doesn't need vision at all in order to 'read' material Feb 22, 2011 | not rated yet | 0
- Researchers find the blind use visual brain area to improve other senses Oct 06, 2010 | not rated yet | 0
- Device converting images into music helps individuals without vision reach for objects in space Jul 05, 2012 | not rated yet | 0
- Motion perception revisited: High Phi effect challenges established motion perception assumptions Apr 23, 2013 | 3 / 5 (2) | 2
- Anything you can do I can do better: Neuromolecular foundations of the superiority illusion (Update) Apr 02, 2013 | 4.5 / 5 (11) | 5
- The visual system as economist: Neural resource allocation in visual adaptation Mar 30, 2013 | 5 / 5 (2) | 9
- Separate lives: Neuronal and organismal lifespans decoupled Mar 27, 2013 | 4.9 / 5 (8) | 0
- Sizing things up: The evolutionary neurobiology of scale invariance Feb 28, 2013 | 4.8 / 5 (10) | 14
Pressure-volume curve: Elastic Recoil Pressure don't make sense
May 18, 2013 From pressure-volume curve of the lung and chest wall (attached photo), I don't understand why would the elastic recoil pressure of the lung is...
If you became brain-dead, would you want them to pull the plug?
May 17, 2013 I'd want the rest of me to stay alive. Sure it's a lousy way to live but it beats being all-the-way dead. Maybe if I make it 20 years they'll...
MRI bill question
May 15, 2013 Dear PFers, The hospital gave us a $12k bill for one MRI (head with contrast). The people I talked to at the hospital tell me that they do not...
Ratio of Hydrogen of Oxygen in Dessicated Animal Protein
May 13, 2013 As an experiment, for the past few months I've been consuming at least one portion of Jell-O or unflavored Knox gelatin per day. I'm 64, in very...
Alcohol and acetaminophen
May 13, 2013 Edit: sorry for the typo in the title , can't edit I looked around on google quite a bit and it's very hard to find precise information on the...
Marie Curie's leukemia
May 13, 2013 Does anyone know what might be the cause of Marie Curie's cancer
- More from Physics Forums - Medical Sciences
More news stories
For combat veterans suffering from post-traumatic stress disorder, 'fear circuitry' in the brain never rests
Chronic trauma can inflict lasting damage to brain regions associated with fear and anxiety. Previous imaging studies of people with post-traumatic stress disorder, or PTSD, have shown that these brain regions can over-or ...
Neuroscience May 18, 2013 | 5 / 5 (1) | 0 |
The neural machinery underlying our olfactory sense continues to be an enigma for neuroscience. A recent review in Neuron seeks to expand traditional ideas about how neurons in the olfactory bulb might encode information about ...
Neuroscience May 17, 2013 | 4 / 5 (1) | 0 |
(Medical Xpress)—What if the quality of your work depends more on your focus on the piano keys or canvas or laptop than your musical or painting or computing skills? If target users can be convinced, they ...
Neuroscience May 17, 2013 | 3.7 / 5 (3) | 0 |
Neurological disorders can have a devastating impact on the lives of sufferers and their families.
Neuroscience May 17, 2013 | 5 / 5 (1) | 0 |
If you're a left-brain thinker, chances are you use your right hand to hold your cell phone up to your right ear, according to a newly published study from Henry Ford Hospital in Detroit.
Neuroscience May 16, 2013 | 2 / 5 (2) | 0 |
Researchers at the University of Wisconsin have identified a potential new risk factor for obstructive sleep apnea: asthma. Using data from the National Institutes of Health (Heart, Lung, and Blood Institute)-funded Wisconsin ...
4 hours ago | not rated yet | 0 |
A new study looking at sleep-disordered breathing (SDB) and markers for Alzheimer's disease (AD) risk in cerebrospinal fluid (CSF) and neuroimaging adds to the growing body of research linking the two.
4 hours ago | 5 / 5 (1) | 0 |
In their quest to learn more about the variability of cells between and within tissues, biomedical scientists have devised tools capable of simultaneously measuring dozens of characteristics of individual ...
5 hours ago | 5 / 5 (1) | 0 |
Scientists at Johns Hopkins have turned their view of osteoarthritis (OA) inside out. Literally. Instead of seeing the painful degenerative disease as a problem primarily of the cartilage that cushions joints, ...
5 hours ago | 5 / 5 (2) | 0 |
Gourmands and foodies everywhere have long recognized ginger as a great way to add a little peppery zing to both sweet and savory dishes; now, a study from researchers at Columbia University shows purified components of the ...
4 hours ago | not rated yet | 0
The hunt for an HIV vaccine has gobbled up $8 billion in the past decade, and the failure of the most recent efficacy trial has delivered yet another setback to 26 years of efforts.
9 hours ago | not rated yet | 0