Facial expressions affect how we remember facial colors, finds study

The association between facial expressions and the memory color effect has been elucidated through a collaborative effort involving the Cognitive Neurotechnology Unit and the Visual Perception and Cognition Laboratory at the Department of Computer Science and Engineering at Toyohashi University of Technology. The memory color effect refers to the phenomenon in which knowledge of the typical color of a specific object (the memory color) influences the recognition of its actual color.

This study showed that angry and fearful faces were more strongly affected in terms of color recognition because of the memory color effect compared with neutral faces and that memory colors varied between expressions. The results of this study were published online in the Journal of Vision on May 31, 2024.

The face is an important characteristic for recognizing individuals, and as is shown by Japanese phrases such as "kaoiro wo ukagau" (Look at the complexion; i.e., be sensitive to someone's mood, read someone's countenance), facial color plays a key role in reading a person's emotions.

Recent research has shown that facial color changes an individual's judgment of expressions, with a reddish face tending to be regarded as angry, for example, even when faces with the same features are presented. However, it was not well understood whether everyday memories of facial colors or the memory colors formed by knowledge of the typical colors of specific objects also vary between expressions.

Hence, the research team focused on the phenomenon in which color recognition changes according to memory colors, known as the memory color effect, and used facial images with different expressions and colors to conduct a psychophysical experiment. Experimental participants were asked to select which color a face appeared to have from two options (the "typical color" and the "opposite color") for facial images presented to them.

The typical color means the color that the observer holds as knowledge about the object and refers to the skin color, among others, in the case of faces. The opposite color refers to the color located opposite the typical color in terms of hue.

The experiment used three expression images with an angry face, a neutral face, and a fearful face with different colors. The experiment was conducted in a dimly illuminated room maintained at a constant brightness, thus mitigating the influence of ambient brightness on the appearance of the colors.

The experiment results showed that angry and fearful faces that were actually achromatic (gray) tended to appear to be colored red-yellow, their typical color, more than achromatic neutral faces did. As red-yellow, the memory color for angry and fearful faces, has a higher saturation than for neutral faces, it is possible that the achromatic facial color may have tended to appear colored with the typical color.

This is similar to the reports of previous research that expressions introduce bias to the memorized facial color, and the recalled facial color was a red-yellow with a higher saturation than when it was actually observed.

First-year doctoral candidate in the Department of Computer Science and Engineering and the primary author for this study, Yuya Hasegawa, explains, "In general, the color that anger brings to mind is red, and red is also often used when expressing anger. In that case, do people then regularly and empirically remember angry faces as being redder than neutral faces? We supposed that if people change the color of faces depending on their expression when remembering them, the memory color should differ for each expression, which inspired this study."

These outcomes are the first to reveal that expressions exert an influence on faces at the level of the memory color. Memory and attention are closely associated. In the future, we will test whether attention tends to be directed toward "red angry faces" in preference to normal angry faces or red neutral faces and examine how to deepen our understanding of the mechanisms by which the remembered facial color differs according to the expression.