How the human brain detects the 'music' of speech

August 24, 2017, University of California, San Francisco
Researchers have identified neurons in the brain that help discern changes in relative pitch, playing an essential role in helping humans extract meaning from speech. Credit: Carla Schaffer / AAAS

Researchers at UC San Francisco have identified neurons in the human brain that respond to pitch changes in spoken language, which are essential to clearly conveying both meaning and emotion.

The study was published online August 24, 2017 in Science by the lab of Edward Chang, MD, a professor of neurological surgery at the UCSF Weill Institute for Neurosciences, and led by Claire Tang, a fourth-year graduate student in the Chang lab.

"One of the lab's missions is to understand how the converts sounds into meaning," Tang said. "What we're seeing here is that there are neurons in the brain's neocortex that are processing not just what words are being said, but how those words are said."

Changes in vocal during speech - part of what linguists call speech prosody - are a fundamental part of human communication, nearly as fundamental as melody to music. In tonal languages such as Mandarin Chinese, pitch changes can completely alter the meaning of a word, but even in a non-tonal language like English, differences in pitch can significantly change the meaning of a spoken sentence.

For instance, "Sarah plays soccer," in which "Sarah" is spoken with a descending pitch, can be used by a speaker to communicate that Sarah, rather than some other person, plays soccer; in contrast, "Sarah plays soccer" indicates that Sarah plays soccer, rather than some other game. And adding a rising tone at the end of a sentence ("Sarah plays soccer?") indicates that the sentence is a question.

The brain's ability to interpret these changes in tone on the fly is particularly remarkable, given that each speaker also has their own typical vocal pitch and style (that is, some people have low voices, others have high voices, and others seem to end even statements as if they were questions). Moreover, the brain must track and interpret these pitch changes while simultaneously parsing which consonants and vowels are being uttered, what words they form, and how those words are being combined into phrases and sentences—with all of this happening on a millisecond scale.

Neural populations in the human auditory cortex respond to changes of vocal pitch in speech. In this video, we show cortical activity recorded from a small region of the brain in response to three sentences. As the pitch of the sentence goes high, neural activity (shown in the color and size of the circle on the brain) increases. Credit: Claire Tang / UCSF

Previous studies in both humans and non-human primates have identified areas of the brain's frontal and temporal cortices that are sensitive to vocal pitch and intonation, but none have answered the question of how neurons in these regions detect and represent changes in pitch to inform the brain's interpretation of a speaker's meaning.

Distinct groups of neurons in the brain's temporal cortex distinguish speaker, phonetics, and intonation

Chang, a neurosurgeon at the UCSF Epilepsy Center, specializes in surgeries to remove brain tissue that causes seizures in patients with epilepsy. In some cases, to prepare for these operations, he places high-density arrays of tiny electrodes onto the surface of the patients' brains, both to help identify the location triggering the patients' seizures and to map out other important areas, such as those involved in language, to make sure the surgery avoids damaging them.

In the new study, Tang asked 10 volunteers awaiting surgery with these electrodes in place to listen to recordings of four sentences as spoken by three different synthesized voices:

"Humans value genuine behavior"

"Movies demand minimal energy"

"Reindeer are a visual animal"

"Lawyers give a relevant opinion"

The sentences were designed to have the same length and construction, and could be played with four different intonations: neutral, emphasizing the first word, emphasizing the third word, or as a question. You can see how these intonation changes alter the meaning of the sentence: "Humans [unlike Klingons] value genuine behavior;" "Humans value genuine [not insincere] behavior;" and "Humans value genuine behavior?" [Do they really?]

Tang and her colleagues monitored the electrical activity of neurons in a part of the volunteers' auditory cortices called the superior temporal gyrus (STG), which previous research had shown might play some role in processing speech prosody.

They found that some neurons in the STG could distinguish between the three synthesized speakers, primarily based on differences in their average vocal pitch range. Other neurons could distinguish between the four sentences, no matter which speaker was saying them, based on the different kinds of sounds (or phonemes) that made up the sentences ("reindeer" sounds different from "lawyers" no matter who's talking). And yet another group of neurons could distinguish between the four different intonation patterns. These neurons changed their activity depending on where the emphasis fell in the sentence, but didn't care which sentence it was or who was saying it.

To prove to themselves that they had cracked the brain's system for pulling intonation information from sentences, the team designed an algorithm to predict how neurons' response to any should change based on speaker, phonetics, and intonation and then used this model to predict how the volunteers' neurons would respond to hundreds of recorded sentences by different speakers. They showed that while the responsive to the different speakers were focused on absolute pitch of the speaker's voice, the ones responsive to intonation were more focused on relative pitch: how the pitch of the speaker's voice changed from moment to moment during the recording.

"To me this was one of the most exciting aspects of our study," Tang said. "We were able to show not just where prosody is encoded in the brain, but also how, by explaining the activity in terms of specific changes in vocal pitch."

These findings reveal how the brain begins to take apart the complex stream of sounds that make up speech and identify important cues about the meaning of what we're hearing, Tang says. Who is talking, what are they saying, and just as importantly, how are they saying it?

"Now, a major unanswered question is how the brain controls our vocal tracts to make these intonational speech sounds," said Chang, the paper's senior author. "We hope we can solve this mystery soon."

Volunteers from the UCSF epilepsy center enable deeper look into workings of human brain

The patients involved in the study were all at UCSF undergoing surgery for severe, untreatable epilepsy. Brain surgery is a powerful way to halt epilepsy in its tracks, potentially completely stopping seizures overnight, and its success is directly related to the accuracy with which a medical team can map the brain, identifying the exact pieces of tissue responsible for an individual's seizures and removing them.

The UCSF Comprehensive Epilepsy Center is a leader in the use of advanced intracranial monitoring to map out elusive seizure-causing brain regions. The mapping is done by surgically placing a flexible electrode array under the skull on the brain's outer surface or cortex and recording the brain's activity in order to pinpoint the parts of the brain responsible for triggering seizures. In a second surgery a few weeks later, the electrodes are removed and the unhealthy brain tissue that causes the seizures is removed.

This setting also permits a rare opportunity to ask basic questions about how the works, such as how it controls speaking. The neurological basis of speech motor control has remained unknown until now because scientists cannot study speech mechanisms in animals and because non-invasive imaging methods lack the ability to track the very rapid time course of the brain signals that drive the muscles that create speech, which change in hundredths of seconds.

But presurgical brain mapping can record neural activity directly, and can detect changes in electrical activity on the order of a few milliseconds.

Explore further: Pop-outs: How the brain extracts meaning from noise

More information: C. Tang el al., "Intonational speech prosody encoding in the human auditory cortex," Science (2017). science.sciencemag.org/cgi/doi … 1126/science.aam8577

Related Stories

Pop-outs: How the brain extracts meaning from noise

December 20, 2016
When you're suddenly able to understand someone despite their thick accent, or finally make out the lyrics of a song, your brain appears to be re-tuning to recognize speech that was previously incomprehensible.

Secrets of human speech uncovered: Study shows brain exerts symphony-like control of vocal tract during act of speaking

February 20, 2013
A team of researchers at UC San Francisco has uncovered the neurological basis of speech motor control, the complex coordinated activity of tiny brain regions that controls our lips, jaw, tongue and larynx as we speak.

Researchers ID network of neurons crucial for vocal learning

June 27, 2017
Researchers have identified a network of neurons that plays a vital role in learning vocalizations by aiding communication between motor and auditory regions of the brain.

Learning dialects shapes brain areas that process spoken language

October 18, 2013
Using advanced imaging to visualize brain areas used for understanding language in native Japanese speakers, a new study from the RIKEN Brain Science Institute finds that the pitch-accent in words pronounced in standard Japanese ...

Studies of epilepsy patients uncover clues to how the brain remembers

June 2, 2017
In a pair of studies, scientists at the National Institutes of Health explored how the human brain stores and retrieves memories. One study suggests that the brain etches each memory into unique firing patterns of individual ...

Scientists discover neural communication pathways for prosody

November 10, 2015
Researchers from the Leipzig-based Max Planck Institute for Human Cognitive and Brain Sciences have decoded, for the first time, the connecting pathways via which the areas of our brain communicate with one another when we ...

Recommended for you

New brain imaging research shows that when we expect something to hurt it does, even if the stimulus isn't so painful

November 14, 2018
Expect a shot to hurt and it probably will, even if the needle poke isn't really so painful. Brace for a second shot and you'll likely flinch again, even though—second time around—you should know better.

New clues to the origin and progression of multiple sclerosis

November 13, 2018
Mapping of a certain group of cells, known as oligodendrocytes, in the central nervous system of a mouse model of multiple sclerosis (MS), shows that they might have a significant role in the development of the disease. The ...

Mutations, CRISPR, and the biology behind movement disorders

November 12, 2018
Scientists at the RIKEN Center for Brain Science (CBS) in Japan have discovered how mutations related to a group of movement disorders produce their effects. Published in Proceedings of the National Academy of Sciences, the ...

In live brain function, researchers are finally seeing red

November 12, 2018
For years, green has been the most reliable hue for live brain imaging, but after using a new high-throughput screening method, researchers at the John B. Pierce Laboratory and the Yale School of Medicine, together with collaborators ...

Concussion tied to suicide risk

November 12, 2018
(HealthDay)—People who have experienced either a concussion or a mild traumatic brain injury are twice as likely to commit suicide than others, a new review suggests.

Autism is associated with zinc deficiency in early development—now a study links the two

November 9, 2018
The emergence of autism in children has not only been linked to genes encoding synaptic proteins—among others—but also environmental insults such as zinc deficiency. Although it is unclear whether zinc deficiency contributes ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.