Researchers identify emotions based on brain activity

For the first time, scientists at Carnegie Mellon University have identified which emotion a person is experiencing based on brain activation. This image shows the average positions of brain regions used to identify emotional states. Credit: Carnegie Mellon University

For the first time, scientists at Carnegie Mellon University have identified which emotion a person is experiencing based on brain activity.

The study, which will be published in the June 19 issue of PLOS ONE, combines (fMRI) and machine learning to measure to accurately read emotions in individuals. Led by researchers in CMU's Dietrich College of Humanities and Social Sciences, the findings illustrate how the brain categorizes feelings, giving researchers the first reliable process to analyze emotions. Until now, research on emotions has been long stymied by the lack of reliable methods to evaluate them, mostly because people are often reluctant to honestly report their feelings. Further complicating matters is that many may not be consciously experienced.

Identifying emotions based on builds on previous discoveries by CMU's Marcel Just and Tom M. Mitchell, which used similar techniques to create a that identifies individuals' thoughts of concrete objects, often dubbed "mind reading."

"This research introduces a new method with potential to identify emotions without relying on people's ability to self-report," said Karim Kassam, assistant professor of social and decision sciences and lead author of the study. "It could be used to assess an individual's emotional response to almost any kind of stimulus, for example, a flag, a brand name or a political candidate."

One challenge for the research team was find a way to repeatedly and reliably evoke different emotional states from the participants. Traditional approaches, such as showing subjects emotion-inducing film clips, would likely have been unsuccessful because the impact of film clips diminishes with repeated display. The researchers solved the problem by recruiting actors from CMU's School of Drama.

"Our big breakthrough was my colleague Karim Kassam's idea of testing actors, who are experienced at cycling through emotional states. We were fortunate, in that respect, that CMU has a superb drama school," said George Loewenstein, the Herbert A. Simon University Professor of Economics and Psychology.

For the study, 10 actors were scanned at CMU's Scientific Imaging & Brain Research Center while viewing the words of nine emotions: anger, disgust, envy, fear, happiness, lust, pride, sadness and shame. While inside the fMRI scanner, the actors were instructed to enter each of these multiple times, in random order.

This video is not supported by your browser at this time.
For the first time, scientists at Carnegie Mellon University have identified which emotion a person is experiencing based on brain activity. Credit: Carnegie Mellon University

Another challenge was to ensure that the technique was measuring emotions per se, and not the act of trying to induce an emotion in oneself. To meet this challenge, a second phase of the study presented participants with pictures of neutral and disgusting photos that they had not seen before. The computer model, constructed from using statistical information to analyze the activation patterns gathered for 18 emotional words, had learned the emotion patterns from self-induced emotions. It was able to correctly identify the emotional content of photos being viewed using the of the viewers.

To identify emotions within the brain, the researchers first used the participants' neural activation patterns in early scans to identify the emotions experienced by the same participants in later scans. The computer model achieved a rank accuracy of 0.84. Rank accuracy refers to the percentile rank of the correct emotion in an ordered list of the computer model guesses; random guessing would result in a rank accuracy of 0.50.

Next, the team took the machine learning analysis of the self-induced emotions to guess which emotion the subjects were experiencing when they were exposed to the disgusting photographs. The computer model achieved a rank accuracy of 0.91. With nine emotions to choose from, the model listed disgust as the most likely emotion 60 percent of the time and as one of its top two guesses 80 percent of the time.

Finally, they applied machine learning analysis of neural activation patterns from all but one of the participants to predict the emotions experienced by the hold-out participant. This answers an important question: If we took a new individual, put them in the scanner and exposed them to an emotional stimulus, how accurately could we identify their emotional reaction? Here, the model achieved a rank accuracy of 0.71, once again well above the chance guessing level of 0.50.

"Despite manifest differences between people's psychology, different people tend to neurally encode emotions in remarkably similar ways," noted Amanda Markey, a graduate student in the Department of Social and Decision Sciences.

A surprising finding from the research was that almost equivalent accuracy levels could be achieved even when the computer model made use of activation patterns in only one of a number of different subsections of the human brain.

"This suggests that emotion signatures aren't limited to specific brain regions, such as the amygdala, but produce characteristic patterns throughout a number of brain regions," said Vladimir Cherkassky, senior research programmer in the Psychology Department.

The research team also found that while on average the model ranked the correct emotion highest among its guesses, it was best at identifying happiness and least accurate in identifying envy. It rarely confused positive and negative emotions, suggesting that these have distinct neural signatures. And, it was least likely to misidentify lust as any other emotion, suggesting that lust produces a pattern of neural activity that is distinct from all other experiences.

Just, the D.O. Hebb University Professor of Psychology, director of the university's Center for Cognitive Brain Imaging and leading neuroscientist, explained, "We found that three main organizing factors underpinned the emotion neural signatures, namely the positive or negative valence of the emotion, its intensity—mild or strong, and its sociality—involvement or non-involvement of another person. This is how emotions are organized in the brain."

In the future, the researchers plan to apply this new identification method to a number of challenging problems in emotion research, including identifying emotions that individuals are actively attempting to suppress and multiple emotions experienced simultaneously, such as the combination of joy and envy one might experience upon hearing about a friend's good fortune.

Groundbreaking discoveries such as identifying emotions based on neural activation patterns have helped to establish Carnegie Mellon as a world leader in brain and behavioral sciences. To build on its foundation of research excellence in psychology, neuroscience and computational science, CMU recently launched a Brain, Mind and Learning initiative to enhance the university's ability to innovate in the laboratory and continue to solve real-world problems.

Related Stories

Study finds brain system for emotional self-control

May 09, 2013

Different brain areas are activated when we choose to suppress an emotion, compared to when we are instructed to inhibit an emotion, according a new study from the UCL Institute of Cognitive Neuroscience and Ghent University.

Human emotion: We report our feelings in 3-D

Mar 26, 2013

Like it or not and despite the surrounding debate of its merits, 3-D is the technology du jour for movie-making in Hollywood. It now turns out that even our brains use 3 dimensions to communicate emotions.

Recommended for you

Intervention program helps prevent high-school dropouts

Oct 24, 2014

New research findings from a team of prevention scientists at Arizona State University demonstrates that a family-focused intervention program for middle-school Mexican American children leads to fewer drop-out rates and ...

Bilingualism over the lifespan

Oct 24, 2014

It's a scene that plays out every day in Montreal. On the bus, in schools, in the office and at home, conversations weave seamlessly back and forth between French and English, or one of the many other languages represented ...

User comments

Adjust slider to filter visible comments by rank

Display comments: newest first

RobertKarlStonjek
not rated yet Jun 20, 2013
Lust is an emotion???? Since when?

Pride is an emotion??? Since when?

I think they would do better to use more scientific terms such as 'effective states' than to consider categories in which the elements under observation do not even belong.

Emotion is sometimes categorised by facial expression, subjective feeling or behaviour. There is a different set of emotions for each of these classification methods.

On Lust, the most common form in the press is 'lust for money' which appears at least ten times as often in the press as sexual lust...