Seeing two figures in coordinated action helps brain pick out movements of one

September 7, 2006
Seeing two figures in coordinated action helps brain pick out movements of one
Researchers used colored dots to represent the movement of dancers (above) and fighters in a pair of videos illustrating visual discrimination.

A new study by vision scientists at the University of California, Berkeley, finds that the human visual system is better able to discriminate the movements of a single person when his or her actions are coordinated in a meaningful way with a second individual.

When shown a pair of figures in motion, for instance, the brain is better able to pick out an individual if it perceives that one person is throwing a punch while another is making a defensive block, the researchers said.

This is especially important when the view is somehow obscured or impaired, according to the study, published in the September issue of the journal Nature Neuroscience.

"Our study reveals a greater degree of complexity in human visual processing than previously understood," said Dennis Levi, UC Berkeley dean of optometry and principal investigator of the study. "When we watch two people interacting, knowing what one is doing helps us understand the actions of the other. We think of it as 'It takes two to Tango.'"

Other researchers on the team are Peter Neri, a UC Berkeley post-doctoral fellow in Levi's laboratory, and Jennifer Luu, an undergraduate who joined the group as part of the UC Berkeley Undergraduate Research Apprentice Program. Luu is now a UC Berkeley Ph.D. student in bioengineering.

This research provides insight into how accurately we can interpret what we see from grainy security cameras, particularly when identifying whether a crime is taking place. There is even research taking place on the development of artificial intelligence computer programs that can automatically detect which actions are suspicious.

At its most fundamental level, the human brain's ability to interpret and react to the action it sees within a fraction of a second is a matter of life and death, such as identifying that a tiger lunging forward with teeth bared is a threat and then deciding how best to get away.

But this skill - when not impaired by such disorders as autism - is also essential to our success as a social species, allowing us to determine whether someone is happy, sad or nervous based upon visual cues both subtle and obvious.

"Our study is the first to provide quantitative evidence that this ability to interpret action enhances the processing of visual discrimination," said Neri, the lead author.

To build their dataset of human actions, the researchers outfitted members of the UC Martial Arts Program and the UC Ballroom Dance Team with sports suits that each contained 13 lights marking key body parts, such as the head, hands, knees and feet. The "actors" were then asked to fight or dance in a dark room while the researchers filmed the movements.

Using motion capture technology, the lights in the video were converted into dots that the researchers could manipulate. The original samples of each action - fighting or dancing - lasted 22 seconds each. In the synchronized video files, figures' actions corresponded naturally with each other.

The researchers also created desynchronized files in which the movements of one figure filmed in the first 11 seconds were paired with those for the other figure in the second 11 seconds. In these files, the movements of the two figures were no longer coordinated with each other.

Four UC Berkeley undergraduate students with normal vision who had been recruited for the study were then shown 1.5- to 3-second sequences that were randomly selected from either the synchronized or desynchronized files. In half of the sequences, the researchers had further scrambled the dots of one figure and added many other dots to the sequence to mask the action. The researchers were able to vary the amount of dots to measure participants' tolerance for "noise" or "fog."

The participants had to determine whether the dots they were viewing represented one or two figures.

"We found that people can tolerate more masking dots and answer correctly when the figures' actions are interacting in a meaningful way," said Neri, who also participated in the study. "If the sequence is desynchronized, we can tolerate 10 dots. When viewing synchronized action, we can tolerate a minimum of 20 and up to 100 dots. Having the action coordinated improves the performance by at least a factor of two."

The researchers noted that the relationship between two figures held true whether the movements were coordinated but antagonistic, as in the fighting sequences, or coordinated and cooperative, as in the dancing movements.

"This study shows that the cortex encodes complex information about actions, and that there can be a clear evolutionary advantage - improved visual discrimination - to doing so," said Neri.

The researchers pointed out that the human brain is constantly adapting to limited amounts of visual information, enabling us to imagine a full picture of a tiger, for instance, even if trees and bushes obscure parts of its body.

The brain also allows us to understand when a group of dots or lines represents a human being. When presented with a group of dots in motion, the human visual system can distinguish whether the dots comprise a male or a female, and even judge the figure's emotional state, the researchers said.

"We used to talk about how the eye is a camera that passes information intact to the brain, but that's actually not how we see," said Levi. "Things are always changing before our eyes, and our brain is constantly making best guesses about what it's seeing."

When the view is somehow obscured, that task naturally becomes more difficult. This guessing game played by the visual system is also why the brain is susceptible to optical illusions, Levi added. This best-guess approach to interpreting what we see is also an important factor when considering eyewitness accounts of events, the researchers said.

Source: UC Berkeley, By Sarah Yang

Explore further: AI is unleashed to sift through data piles for readable reports

Related Stories

AI is unleashed to sift through data piles for readable reports

October 26, 2017
Bitcoin. Self-driving cars. Sea levels. CO2 removal. Breast cancer. Wheat forecasts. Factory riots. The flood of data and research is daunting thanks to a knowledge world turned digital.

At-home vision monitoring app may improve patient care

November 13, 2017
Patients with age-related macular degeneration or diabetic retinopathy who used a mobile application to test their vision at home got comparable results to in-office vision testing, according to research presented today at ...

Could Amazon's Alexa help you cut down on your drinking?

November 14, 2017
It can tell you the time, read you the news and even crack a joke or two, but could Amazon's Alexa play a role in your health too?

Human brain recalls visual features in reverse order than it detects them

October 9, 2017
Scientists at Columbia's Zuckerman Institute have contributed to solving a paradox of perception, literally upending models of how the brain constructs interpretations of the outside world. When observing a scene, the brain ...

The brain at work: Spotting half-hidden objects

September 19, 2017
How does a driver's brain realize that a stop sign is behind a bush when only a red edge is showing? Or how can a monkey suspect that the yellow sliver in the leaves is a round piece of fruit?

Researchers surprised to find how neural circuits identify information needed for decisions

November 6, 2013
While eating lunch you notice an insect buzzing around your plate. Its color and motion could both influence how you respond. If the insect was yellow and black you might decide it was a bee and move away. Conversely, you ...

Recommended for you

To forget or to remember? Memory depends on subtle brain signals, scientists find

November 22, 2017
The fragrance of hot pumpkin pie can bring back pleasant memories of holidays past, while the scent of an antiseptic hospital room may cause a shudder. The power of odors to activate memories both pleasing and aversive exists ...

Combination immunotherapy targets cancer resistance

November 22, 2017
Cancer immunotherapy drugs have had notable but limited success because in many cases, tumors develop resistance to treatment. But researchers at Yale and Stanford have identified an experimental antibody that overcomes this ...

Team constructs whole-brain map of electrical connections key to forming memories

November 22, 2017
A team of neuroscientists at the University of Pennsylvania has constructed the first whole-brain map of electrical connectivity in the brain based on data from nearly 300 neurosurgical patients with electrodes implanted ...

Alzheimer's Tau protein forms toxic complexes with cell membranes

November 22, 2017
The brains of patients with Alzheimer's disease contain characteristic tangles inside neurons. These tangles are formed when a protein called Tau aggregates into twisted fibrils. As a result, the neurons' transport systems ...

Pitch imperfect? How the brain decodes pitch may improve cochlear implants

November 22, 2017
Picture yourself with a friend in a crowded restaurant. The din of other diners, the clattering of dishes, the muffled notes of background music, the voice of your friend, not to mention your own – all compete for your ...

Study finds infection and schizophrenia symptom link

November 22, 2017
If a mother's immune system is activated by infection during pregnancy, it could result in critical cognitive deficits linked to schizophrenia in her offspring, a University of Otago study has revealed.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.