Appearance means more than sound when judging music, research finds

August 20, 2013 by Marcia Malory report
In a study by Harvard graduate Chia-Jung Tsay (above), nearly all participants — including highly trained musicians — were better able to identify the winners of classical music competitions by watching silent video clips than by listening to audio recordings. "In this case," says Tsay, "it suggests that the visual trumps the audio, even in a setting where audio information should matter much more.” Credit: Kris Snibbe/Harvard Staff Photographer

(Medical Xpress)—We expect musical experts to focus on sound when judging competitions, and experts believe that their judgments are based on what they hear, not what they see. However, a study by Chia-Jung Tsay of University College London shows that appearance is more important than sound when it comes to judging musical quality. Both experts and novices were better able to identify winners of classical music competitions by watching silent videos than by listening to audio recordings. The research appears in Proceedings of the National Academy of Sciences.

Professional musicians spend many years learning how to judge the of musical performances. They are trained to focus on sound during performances, and they consistently state that sound provides the most relevant information needed for evaluating music. However, research shows that often influences how we perceive and process sound.

To determine the extent to which appearance affects musical , Tsay conducted a series of experiments that involved identifying which musicians had won classical music contests. The first experiment tested subjects' perceptions of how judges make their decisions. When promised a cash bonus for choosing the winners of international classical music competitions, based on recordings of three finalists in each competition, most participants said they would prefer to use sound-only recordings rather than video-only recordings, indicating they believed the judges chose winners on the basis of sound.

Although the subjects believed that would enable them to better match the judges' decisions, further experiments proved otherwise. When participants with little or no classical were given both video and audio excerpts of the finalists' performances, they were significantly more likely to choose the correct winner when presented with silent video rather than audio excerpts. In fact, when they relied on sound-only recordings, the ability to select the correct winner was significantly below chance, while video-only recordings increased their accuracy rate to significantly above chance.

Experiments in which subjects were professional musicians, recruited from music conservatories, symphony orchestras, and professional music organizations, yielded similar results.

In another experiment, subjects were able to identify competition winners at a rate significantly above average simply by watching silent, black and white moving outlines.

Tsay says that the dominance of vision over our other senses may provide us with adaptive benefits. However, it may cause us to rely on appearance when making decisions that should not depend on visual cues, such as which employees to hire or which political leaders to select.

Explore further: Love of musical harmony is not nature but nurture

More information: Sight over sound in the judgment of music performance, PNAS, Published online before print August 19, 2013, DOI: 10.1073/pnas.1221454110

Social judgments are made on the basis of both visual and auditory information, with consequential implications for our decisions. To examine the impact of visual information on expert judgment and its predictive validity for performance outcomes, this set of seven experiments in the domain of music offers a conservative test of the relative influence of vision versus audition. People consistently report that sound is the most important source of information in evaluating performance in music. However, the findings demonstrate that people actually depend primarily on visual information when making judgments about music performance. People reliably select the actual winners of live music competitions based on silent video recordings, but neither musical novices nor professional musicians were able to identify the winners based on sound recordings or recordings with both video and sound. The results highlight our natural, automatic, and nonconscious dependence on visual cues. The dominance of visual information emerges to the degree that it is overweighted relative to auditory information, even when sound is consciously valued as the core domain content.

Press release

Related Stories

Recommended for you

Men more likely to be seen as 'creative thinkers'

September 28, 2015

People tend to associate the ability to think creatively with stereotypical masculine qualities, according to new research published in Psychological Science, a journal of the Association for Psychological Science. The findings ...

Babies time their smiles to make their moms smile in return

September 23, 2015

Why do babies smile when they interact with their parents? Could their smiles have a purpose? In the Sept. 23 issue of PLOS ONE, a team of computer scientists, roboticists and developmental psychologists confirm what most ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Aug 23, 2013
Interpretation is not consensus.
All cues are bias. The pinnacle of bias is judgement.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.