Here's what music sounds like through an auditory implant
For some people with severe hearing loss, it is possible to restore their hearing with an auditory implant (also known as cochlear implants). These electronic devices are surgically implanted into the inner ear, converting the sound from the world into electrical signals that are sent through the auditory nerve to the brain. The damaged parts of the ear are bypassed and people are – almost miraculously – able to hear again. With practice, auditory implant users emerge from a world of silence able to hear the doorbell, to use the phone, to talk and laugh with their friends. Unfortunately, though, music can be hard to enjoy. Smooth melodies become harsh buzzes, beeps and squawks.
People with auditory implants find that much of what they used to love about music is now absent. The implant is poor at conveying the pitch of voices and instruments, as well as the quality (timbre) of the music. This can make it hard to follow the melody, understand the lyrics, or separate one instrument from another. As you can hear in our simulation (below), almost all of the raw, untrammelled emotion that Ed Sheeran brings to his performance of Thinking Out Loud is lost, leaving the music abrasive and flat.
This poor transmission of music through the implant can have an enormous impact on people's quality of life. Music is all around us, not just at home or in concerts but also in the background in cafes, pubs, shops, TV shows and films. For people with auditory implants, this can make it hard to enjoy things they previously loved to do. People tell us that music is one of the main things they would like to be improved in their implant. This presents a challenge for engineers and scientists.
The trouble with music
In healthy hearing, the sound of music is captured by the activity of thousands of highly sensitive "hair cells" – sensory receptors that respond to minute changes in pressure in the ear, translating sound into electrical activity that can be interpreted by the brain. This extraordinary sensory system is able to code the tiny fluctuations in sound that we interpret as notes, instruments, timbre and emotional resonance. It is this complex coding that allows us to enjoy the melodic voice of Mr Sheeran. In an auditory implant, that system is replaced by a tiny number of micro-electrodes – usually between eight and 22. These electrodes are only able to transmit very crude pitch information, missing the more detailed sound information.
Over time, some people with auditory implants are able to adjust to their new hearing, finding ways to enjoy and love music again. They often find that they must actively learn to enjoy music again to adjust to their new experience. Others have decided to engage with it differently, reading the lyrics while they listen to improve their understanding. Because the implant is able to transmit rhythm much more effectively than pitch, some users find that they can only enjoy certain, more rhythmic genres of music (such as the Michael Jackson song in our simulation). Some, amazingly, have even learned to play instruments when using an implant.
New approaches may hold the key to helping people with implants enjoy music again. One possibility is modifying musical tracks or even writing entirely new music specifically for implants, with qualities that can be more easily transferred by existing technology. For example, researchers have found that increasing the volume of the vocals and removing harmonic instruments improves the experience of listening to pop music.
Another option is changing the way that sounds are processed by the implant before sending the signals to the auditory nerve. Several implant makers now advertise their cutting-edge processing as best for listening to music. However, most implant users are still unable to enjoy music.
It may be necessary to take a radically new approach. We think that the information bottleneck at the implant could be bypassed by providing sound information through the sense of touch. We have recently used this approach to improve implant users' ability to understand speech in complex sound environments – perhaps we can improve their experience of music too.