Appearance means more than sound when judging music, research finds

August 20, 2013 by Marcia Malory report
In a study by Harvard graduate Chia-Jung Tsay (above), nearly all participants — including highly trained musicians — were better able to identify the winners of classical music competitions by watching silent video clips than by listening to audio recordings. "In this case," says Tsay, "it suggests that the visual trumps the audio, even in a setting where audio information should matter much more.” Credit: Kris Snibbe/Harvard Staff Photographer

(Medical Xpress)—We expect musical experts to focus on sound when judging competitions, and experts believe that their judgments are based on what they hear, not what they see. However, a study by Chia-Jung Tsay of University College London shows that appearance is more important than sound when it comes to judging musical quality. Both experts and novices were better able to identify winners of classical music competitions by watching silent videos than by listening to audio recordings. The research appears in Proceedings of the National Academy of Sciences.

Professional musicians spend many years learning how to judge the of musical performances. They are trained to focus on sound during performances, and they consistently state that sound provides the most relevant information needed for evaluating music. However, research shows that often influences how we perceive and process sound.

To determine the extent to which appearance affects musical , Tsay conducted a series of experiments that involved identifying which musicians had won classical music contests. The first experiment tested subjects' perceptions of how judges make their decisions. When promised a cash bonus for choosing the winners of international classical music competitions, based on recordings of three finalists in each competition, most participants said they would prefer to use sound-only recordings rather than video-only recordings, indicating they believed the judges chose winners on the basis of sound.

Although the subjects believed that would enable them to better match the judges' decisions, further experiments proved otherwise. When participants with little or no classical were given both video and audio excerpts of the finalists' performances, they were significantly more likely to choose the correct winner when presented with silent video rather than audio excerpts. In fact, when they relied on sound-only recordings, the ability to select the correct winner was significantly below chance, while video-only recordings increased their accuracy rate to significantly above chance.

Experiments in which subjects were professional musicians, recruited from music conservatories, symphony orchestras, and professional music organizations, yielded similar results.

In another experiment, subjects were able to identify competition winners at a rate significantly above average simply by watching silent, black and white moving outlines.

Tsay says that the dominance of vision over our other senses may provide us with adaptive benefits. However, it may cause us to rely on appearance when making decisions that should not depend on visual cues, such as which employees to hire or which political leaders to select.

Explore further: Love of musical harmony is not nature but nurture

More information: Sight over sound in the judgment of music performance, PNAS, Published online before print August 19, 2013, DOI: 10.1073/pnas.1221454110

Social judgments are made on the basis of both visual and auditory information, with consequential implications for our decisions. To examine the impact of visual information on expert judgment and its predictive validity for performance outcomes, this set of seven experiments in the domain of music offers a conservative test of the relative influence of vision versus audition. People consistently report that sound is the most important source of information in evaluating performance in music. However, the findings demonstrate that people actually depend primarily on visual information when making judgments about music performance. People reliably select the actual winners of live music competitions based on silent video recordings, but neither musical novices nor professional musicians were able to identify the winners based on sound recordings or recordings with both video and sound. The results highlight our natural, automatic, and nonconscious dependence on visual cues. The dominance of visual information emerges to the degree that it is overweighted relative to auditory information, even when sound is consciously valued as the core domain content.

Press release

Related Stories

Love of musical harmony is not nature but nurture

February 14, 2013
Our love of music and appreciation of musical harmony is learnt and not based on natural ability – a new study by University of Melbourne researchers has found.

Recommended for you

New study rebuts the claim that antidepressants do not work

August 18, 2017
A theory that has gained considerable attention in international media, including Newsweek and the CBS broadcast 60 minutes, suggests that antidepressant drugs such as the SSRIs do not exert any actual antidepressant effect. ...

Should I stay or should I leave? Untangling what goes on when a relationship is being questioned

August 17, 2017
Knowing whether to stay in or leave a romantic relationship is often an agonizing experience and that ambivalence can have negative consequences for health and well-being.

Kids learn moral lessons more effectively from stories with humans than human-like animals

August 17, 2017
A study by researchers at the Ontario Institute for Studies in Education (OISE) at the University of Toronto found that four to six-year-olds shared more after listening to books with human characters than books with anthropomorphic ...

History of stress increases miscarriage risk, says new review

August 17, 2017
A history of exposure to psychological stress can increase the risk of miscarriage by upto 42 per cent, according to a new review.

Study finds children pay close attention to potentially threatening information, avoid eye contact when anxious

August 17, 2017
We spend a lot of time looking at the eyes of others for social cues – it helps us understand a person's emotions, and make decisions about how to respond to them. We also know that adults avoid eye contact when anxious. ...

Communicating in a foreign language takes emotion out of decision making

August 16, 2017
If you could save the lives of five people by pushing another bystander in front of a train to his death, would you do it? And should it make any difference if that choice is presented in a language you speak, but isn't your ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Aug 23, 2013
Interpretation is not consensus.
All cues are bias. The pinnacle of bias is judgement.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.