Method used to model mental representations of vocal intonation. Participants consider the intonaion of many computer-generated recordings of 'bonjour' ('hello') and decide which, for example, inspire trust. Analysis of thousands of responses makes it possible to visually model a person's auditory mental representations. Credit: CNRS/ENS/AMU

You can hear the perfect hello. And now you can see it, too. Researchers from the CNRS, the ENS, and Aix-Marseille University have established an experimental method that unveils the filter—that is, mental representation—we use to judge people when hearing them say a word as simple as "hello." What is the ideal intonation for coming across as determined or trustworthy? This method has been used by these researchers for clinical purposes with stroke survivors, and it opens many new doors for the study of language perception. The team's findings are published in PNAS (March 26, 2018).

When people say hello for the fist time, do they seem friendly or hostile? The linguistic and social judgments people make when hearing speech are based on intonation. People form of others' personalities according to the acoustic qualities of their voices. For the first time ever, researchers have visually modeled these mental representations and compared the representations of different individuals.

To do this, they developed a computer program for voice manipulation called CLEESE. This software can take a recording of a single word and randomly generate thousands of other variant pronunciations that are all realistic, but each unique in their melody. Then, by analyzing participants' responses to these different pronunciations, the researchers were able to determine experimentally what intonation makes a hello seem sincere. To sound determined, a French speaker must pronounce bonjour (French for "hello") with a descending pitch, putting emphasis on the second syllable. On the other hand, to inspire trust, the pitch must rise quickly at the end of the word. Using this software, the team is thus able to visualize the "code" people use to judge others by their voices, and has shown that the same code applies no matter the sex of the listener or the speaker.

A 'bonjour' generated by a computer that inspires trust. Credit: CNRS/ENS/AMU

A 'bonjour' generated by a computer that inspires determination. Credit: CNRS/ENS/AMU

This method of investigation could be used to answer many other questions in the field of language perception. For example, how do these findings play out at sentence level? And do mental representations vary with the language being spoken? It may also serve to understand how emotions are represented by autistic individuals. To help others answer these questions, the research team has made CLEESE freely available here . The team members themselves have already found a clinical application for the program: to study how words are interpreted by survivors of a stroke, an event which can alter how they perceive vocal intonation. Whether for the purposes of medical monitoring or diagnosis, the researchers would like to use their method to detect anomalies in language perception and possibly make it a tool for patient rehabilitation.

More information: Emmanuel Ponsot el al., "Cracking the social code of speech prosody using reverse correlation," PNAS (2018). www.pnas.org/cgi/doi/10.1073/pnas.1716090115

Journal information: Proceedings of the National Academy of Sciences

Provided by CNRS