Overturning widely held ideas: Visual attention drawn to meaning, not what stands out

September 25, 2017, UC Davis
Conventional thinking on visual attention is that our attention is automatically drawn to "salient" objects that stand out from the background. Researchers at the UC Davis Center for Mind and Brain mapped hundreds of images (examples far left) by eye tracking (center left), "meaning" (center right) and "salience" or outstanding features (far left). Statistical analysis shows that eyes are drawn to "meaningful" areas, not necessarily those that are most outstanding. Credit: John Henderson and Taylor Hayes, UC Davis

Our visual attention is drawn to parts of a scene that have meaning, rather than to those that are salient or "stick out," according to new research from the Center for Mind and Brain at the University of California, Davis. The findings, published Sept. 25 in the journal Nature Human Behavior, overturn the widely-held model of visual attention.

"A lot of people will have to rethink things," said Professor John Henderson, who led the research. "The saliency hypothesis really is the dominant view."

Our eyes we perceive a wide field of view in front of us, but we only focus our attention on a small part of this field. How do we decide where to direct our attention, without thinking about it?

The dominant theory in attention studies is "visual salience," Henderson said. Salience means things that "stick out" from the background, like colorful berries on a background of leaves or a brightly lit object in a room.

Saliency is relatively easy to measure. You can map the amount of saliency in different areas of a picture by measuring relative contrast or brightness, for example.

Henderson called this the "magpie theory" our attention is drawn to bright and shiny objects.

"It becomes obvious, though, that it can't be right," he said, otherwise we would constantly be distracted.

Making a Map of Meaning

Henderson and postdoctoral researcher Taylor Hayes set out to test whether attention is guided instead by how "meaningful" we find an area within our view. They first had to construct "meaning maps" of test scenes, where different parts of the had different levels of meaning to an observer.

To make their meaning map, Henderson and Hayes took images of scenes, broke them up into overlapping circular tiles, and submitted the individual tiles to the online crowdsourcing service Mechanical Turk, asking users to rate the tiles for meaning.

By tallying the votes of Mechanical Turk users they were able to assign levels of meaning to different areas of an image and create a meaning map comparable to a saliency map of the same scene.

Next, they tracked the eye movements of volunteers as they looked at the scene. Those eyetracks gave them a map of what parts of the image attracted the most attention. This "attention map" was closer to the meaning map than the salience map, Henderson said.

In Search of Meaning

Henderson and Hayes don't yet have firm data on what makes part of a scene meaningful, although they have some ideas. For example, a cluttered table or shelf attracted more attention than a highly salient splash of sunlight on a wall. With further work, they hope to develop a "taxonomy of ," Henderson said.

Although the research is aimed at a fundamental understanding of how visual works, there could be some near-term applications, Henderson said, for example in developing automated visual systems that allow computers to scan security footage or to automatically identify or caption images online.

Explore further: Key advance: Neuroscientists get a new look into how we read

More information: John M. Henderson et al, Meaning-based guidance of attention in scenes as revealed by meaning maps, Nature Human Behaviour (2017). DOI: 10.1038/s41562-017-0208-0

Related Stories

Key advance: Neuroscientists get a new look into how we read

April 7, 2016
Neuroscientists at UC Davis have come up with a way to observe brain activity during natural reading. It's the first time researchers have been able to study the brain while reading actual texts, instead of individual words, ...

Timing is (almost) everything: Memory enhanced when events occur at behaviorally relevant times

March 15, 2010
What determines whether a scene is remembered or forgotten? According to a study published this week in the open access journal PLoS Biology, memory for visual scenes may not depend on attention level or what a scene contains, ...

How visual attention selects important information

November 14, 2016
Researchers at Tohoku University have revealed multiple functions of visual attention, the process of selecting important information from retinal images.

When making meaning of the world, the brain is a multi-tasker

January 23, 2012
(Medical Xpress) -- How does the brain confer meaning on the things we perceive in the world? “Many of us favor the theory that, whether it comes in through the eyes or ears, through reading [or other stimuli], it’s ...

Study offers new clue on how brain processes visual information, provides insight into neural mechanisms of attention

July 23, 2012
Ever wonder how the human brain, which is constantly bombarded with millions of pieces of visual information, can filter out what's unimportant and focus on what's most useful?

Study discovers clues into how eyes search

June 17, 2009
Like the robots in the "Terminator" movies, our eyes move methodically through a scene when seeking out an object. If we don't immediately find what we're searching for, our attention leaves the already-scanned area behind ...

Recommended for you

White matter repair and traumatic brain injury

September 20, 2018
Traumatic brain injury (TBI) is a leading cause of death and disability in the U.S., contributing to about 30 percent of all injury deaths, according to the CDC. TBI causes damage to both white and gray matter in the brain, ...

Genomic dark matter activity connects Parkinson's and psychiatric diseases

September 20, 2018
Dopamine neurons are located in the midbrain, but their tendril-like axons can branch far into the higher cortical areas, influencing how we move and how we feel. New genetic evidence has revealed that these specialized cells ...

Full, but still feasting: Mouse study reveals how urge to eat overpowers a signal to stop

September 20, 2018
Almost everyone knows the feeling. You're at a restaurant or a holiday meal, and your stomach is telling you it's full, so logically you know you should stop eating.

Gambling monkeys help scientists find brain area linked to high-risk behavior

September 20, 2018
Monkeys who learned how to gamble have helped researchers pinpoint an area of the brain key to one's willingness to make risky decisions.

Researchers discover the way we see an image depends on 'where we are'

September 20, 2018
A study conducted by a Fight for Sight-funded researcher has discovered that the way we see an image changes depending on where we are. The results were published in Nature on 11 September 2018.

'Gut sense' is hardwired, not hormonal

September 20, 2018
If you've ever felt nauseous before an important presentation, or foggy after a big meal, then you know the power of the gut-brain connection.

3 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

RobertKarlStonjek
5 / 5 (1) Sep 25, 2017
"Saliency is relatively easy to measure."

...like searching for your lost keys under the street lamp, not because you lost them there, but because that is where the light is...

This is a recurring theme in many areas of scientific research. We can defer to the logic that it makes sense to study those areas more likely to yield results but note that in areas like consciousness studies the real issues of consciousness are typically left in the 'too hard basket', which is acceptable, but then non-conscious processes (like visual rivalry, reflexive responses etc) are raised to the level of consciousness, which is unacceptable.
thingumbobesquire
not rated yet Sep 26, 2017
In a great Rembrandt painting meaning and salience are deliberately interwoven as in a fugue to create dramatic tension.
Chris_Reeve
Sep 26, 2017
This comment has been removed by a moderator.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.