How sounds, shapes, speech and body movements convey emotion through one shared property

How sounds, shapes, speech and body movements convey emotion through one shared property
Characteristic angry and sad drawings from study 3, after smoothing and corner detection. Corners are marked with red '+' signs. Among the study's participants, angry drawings had a mean of 23.3 corners, while sad drawings had a mean of 6.6 corners. Credit: Graphic compiled by Beau Sievers. (Figure 5 from the paper).

Death metal band logos often have a spiky look while romance novel titles often have a swirly script. The jaggedness or curviness of a font can be used to express an emotional tone. A Dartmouth study published in the Proceedings of the Royal Society B finds that sounds, shapes, speech and body movements convey emotional arousal the same way across the senses. The findings explain why nearly anything can have an emotional tone, including art, architecture and music.

"Our study set out to better understand how we express and read , which is fundamental to our core . We wanted to see if there is a low-level mechanism that allows us to decode emotional arousal information from the movements and sounds that people and animals make," says lead author Beau Sievers, a postdoctoral student of psychology at Harvard University, who was a in the department of psychological and at Dartmouth at the time of the study. "Our results show how the spectral centroid, or the balance of high-frequency versus low-frequency energy present in sounds, shapes and movements, allows us to express and understand emotional arousal," adds Sievers.

The spectral centroid is essentially a multi-sensory measure of spikyness. The results explain why Zen gardens and brutalist architecture have very different emotional effects, as well as why things like clouds and lullabies seem to go together even though one is seen and the other is heard: We match them based on the spectral centroid.

"In a series of studies, we demonstrate that people automatically perceive the of whatever is coming into their ears and eyes and compute the average—the spectral centroid," explains senior author Thalia Wheatley, a professor of psychological and brain sciences at Dartmouth, and principal investigator of the Dartmouth Social Systems Laboratory . "This is how people quickly identify the amount of emotional arousal in a person's voices and movements but also in abstract shapes and sounds, such as why spiky shapes seem to convey higher arousal than rounded shapes," she added.

To test whether the spectral centroid is used to express and understand emotional arousal, the researchers conducted five mini-studies, some of which asked participants to make judgements about the emotional arousal of shapes, sounds and movements. The researchers tested if the spectral centroid of the stimulus could be used to predict participants' emotional arousal judgements. The following are highlights from three of the mini-studies:

  • The authors used a computer program to randomly create hundreds of shapes and sounds. Participants were asked to look at shapes and listen to sounds and judge their levels of emotional arousal. As the study reports, shapes and sounds which had a high spectral centroid were associated with high-arousal emotions (angry, excited), whereas the lower spectral centroid shapes and sounds were associated with low-arousal emotions (sad, peaceful).
  • Participants were asked to draw shapes that were angry, sad, excited, or peaceful. The researchers then estimated the spectral centroids of the drawings by counting how many corners they had. The results revealed that angry and excited shapes had between 17 and 24 corners on average, while sad and peaceful shapes had between 7 and 9 corners on average. The spectral centroid could be used to predict the emotional arousal of shapes with close to 80 percent accuracy.
  • The researchers examined real-world recordings of people's or of people speaking (in German), to see if the spectral centroid of the voices and movements could be used to predict participants judgements of emotional arousal. The researchers found that higher spectral centroids predicted judgements of higher emotional arousal.

The researchers explain that multi-sensory associations with emotions have been known for a long time but why they occur has been a mystery until now.

Explore further

Short-term use of opioids increases subjective pleasure

More information: Beau Sievers et al. A multi-sensory code for emotional arousal, Proceedings of the Royal Society B: Biological Sciences (2019). DOI: 10.1098/rspb.2019.0513
Provided by Dartmouth College
Citation: How sounds, shapes, speech and body movements convey emotion through one shared property (2019, July 10) retrieved 27 October 2020 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors

User comments