Appearance means more than sound when judging music, research finds

by Marcia Malory report
In a study by Harvard graduate Chia-Jung Tsay (above), nearly all participants — including highly trained musicians — were better able to identify the winners of classical music competitions by watching silent video clips than by listening to audio recordings. "In this case," says Tsay, "it suggests that the visual trumps the audio, even in a setting where audio information should matter much more.” Credit: Kris Snibbe/Harvard Staff Photographer

(Medical Xpress)—We expect musical experts to focus on sound when judging competitions, and experts believe that their judgments are based on what they hear, not what they see. However, a study by Chia-Jung Tsay of University College London shows that appearance is more important than sound when it comes to judging musical quality. Both experts and novices were better able to identify winners of classical music competitions by watching silent videos than by listening to audio recordings. The research appears in Proceedings of the National Academy of Sciences.

Professional musicians spend many years learning how to judge the of musical performances. They are trained to focus on sound during performances, and they consistently state that sound provides the most relevant information needed for evaluating music. However, research shows that often influences how we perceive and process sound.

To determine the extent to which appearance affects musical , Tsay conducted a series of experiments that involved identifying which musicians had won classical music contests. The first experiment tested subjects' perceptions of how judges make their decisions. When promised a cash bonus for choosing the winners of international classical music competitions, based on recordings of three finalists in each competition, most participants said they would prefer to use sound-only recordings rather than video-only recordings, indicating they believed the judges chose winners on the basis of sound.

Although the subjects believed that would enable them to better match the judges' decisions, further experiments proved otherwise. When participants with little or no classical were given both video and audio excerpts of the finalists' performances, they were significantly more likely to choose the correct winner when presented with silent video rather than audio excerpts. In fact, when they relied on sound-only recordings, the ability to select the correct winner was significantly below chance, while video-only recordings increased their accuracy rate to significantly above chance.

Experiments in which subjects were professional musicians, recruited from music conservatories, symphony orchestras, and professional music organizations, yielded similar results.

In another experiment, subjects were able to identify competition winners at a rate significantly above average simply by watching silent, black and white moving outlines.

Tsay says that the dominance of vision over our other senses may provide us with adaptive benefits. However, it may cause us to rely on appearance when making decisions that should not depend on visual cues, such as which employees to hire or which political leaders to select.

More information: Sight over sound in the judgment of music performance, PNAS, Published online before print August 19, 2013, DOI: 10.1073/pnas.1221454110

Abstract
Social judgments are made on the basis of both visual and auditory information, with consequential implications for our decisions. To examine the impact of visual information on expert judgment and its predictive validity for performance outcomes, this set of seven experiments in the domain of music offers a conservative test of the relative influence of vision versus audition. People consistently report that sound is the most important source of information in evaluating performance in music. However, the findings demonstrate that people actually depend primarily on visual information when making judgments about music performance. People reliably select the actual winners of live music competitions based on silent video recordings, but neither musical novices nor professional musicians were able to identify the winners based on sound recordings or recordings with both video and sound. The results highlight our natural, automatic, and nonconscious dependence on visual cues. The dominance of visual information emerges to the degree that it is overweighted relative to auditory information, even when sound is consciously valued as the core domain content.

Press release

Related Stories

Musicians take note of tune-writing app

May 06, 2013

Need some instant musical notation to remember that little tune you just came up with? A new mobile app created by a researcher from KTH Royal Institute of Technology makes it possible to score any melody ...

Recommended for you

Religious music brings benefit to seniors' mental health

13 minutes ago

A new article published online in The Gerontologist reports that among older Christians, listening to religious music is associated with a decrease in anxiety about death and increases in life satisfaction, self-e ...

User comments

Adjust slider to filter visible comments by rank

Display comments: newest first

beleg
not rated yet Aug 23, 2013
Interpretation is not consensus.
All cues are bias. The pinnacle of bias is judgement.