Mountain splendor? Scientists know where your eyes will look

December 4, 2018, Yale University
The top row shows images that our participants viewed in the fMRI scanner. The bottom row shows neural reconstructions from the participants' brain activity that predict where people will direct their eyes, overlaid on top of the images. Note that just the colored overlay predicting eye movements is reconstructed from brain activity and not the image underlay. The brighter white and pink areas in the reconstructions show where people directed their attention. Credit: Chun Lab Yale

Using precise brain measurements, Yale researchers predicted how people's eyes move when viewing natural scenes, an advance in understanding the human visual system that can improve a host of artificial intelligence efforts, such as the development of driverless cars, said the researchers.

"We are visual beings and knowing how the rapidly computes where to look is fundamentally important," said Yale's Marvin Chun, Richard M. Colgate Professor of Psychology, professor of neuroscience and co-author of new research published Dec. 4 in the journal Nature Communications.

Eye movements have been extensively studied, and researchers can tell with some certainty where a gaze will be directed at different elements in the environment. What hasn't been understood is how the brain orchestrates this ability, which is so fundamental to survival.

In a previous example of "mind reading," Chun's group successfully reconstructed facial images viewed while people were scanned in an MRI machine, based on their brain imaging data alone.

In the new paper, Chun and lead author Thomas P. O'Connell took a similar approach and showed that by analyzing the brain responses to complex, natural scenes, they could predict where people would direct their attention and gaze. This was made possible by analyzing the brain data with deep convolutional neural networks—models that are extensively used in artificial intelligence (AI).

"The work represents a perfect marriage of neuroscience and data science," Chun said.

The findings have a myriad of potential applications—such as testing competing systems that categorize images and guide driverless cars.

"People can see better than AI systems can," Chun said. "Understanding how the brain performs its complex calculations is an ultimate goal of neuroscience and benefits AI efforts."

Explore further: Researchers demonstrate 'mind-reading' brain-decoding tech

Related Stories

Researchers demonstrate 'mind-reading' brain-decoding tech

October 23, 2017
Researchers have demonstrated how to decode what the human brain is seeing by using artificial intelligence to interpret fMRI scans from people watching videos, representing a sort of mind-reading technology.

Researchers reconstruct facial images locked in a viewer's mind

March 25, 2014
Using only data from an fMRI scan, researchers led by a Yale University undergraduate have accurately reconstructed images of human faces as viewed by other people.

Dissecting artificial intelligence to better understand the human brain

March 25, 2018
In the natural world, intelligence takes many forms. It could be a bat using echolocation to expertly navigate in the dark, or an octopus quickly adapting its behavior to survive in the deep ocean. Likewise, in the computer ...

Artificial intelligence helps reveal how people process abstract thought

October 9, 2018
As artificial intelligence becomes more sophisticated, much of the public attention has focused on how successfully these technologies can compete against humans at chess and other strategy games. A philosopher from the University ...

Malicious brain cell identified—surprising finding fills gap in understanding astrocytes' role in brain disease

September 13, 2018
Astrocytes—the star-shaped cells of our brain—are very busy. Their job description includes maintaining the blood-brain barrier, removing excess neurotransmitters, repairing brain tissue and more.

Neural net activations are aligned with gamma band activity of the human visual cortex

August 16, 2018
Researchers at the University of Tartu's Computational Neuroscience Lab, in Estonia, have discovered that activations of deep convolutional neural networks are aligned with the gamma band activity of the human visual cortex. ...

Recommended for you

The importins of anxiety

December 11, 2018
According to some estimates, up to one in three people around the world may experience severe anxiety in their lifetime. In a study described today in Cell Reports, researchers at the Weizmann Institute of Science have revealed ...

Neurons in the brain work as a team to guide movement of arms, hands

December 11, 2018
The apparent simplicity of picking up a cup of coffee or turning a doorknob belies the complex sequence of calculations and processes that the brain must undergo to identify the location of an item in space, move the arm ...

How returning to a prior context briefly heightens memory recall

December 11, 2018
Whether it's the pleasant experience of returning to one's childhood home over the holidays or the unease of revisiting a site that proved unpleasant, we often find that when we return to a context where an episode first ...

The richer the reward, the faster you'll likely move to reach it, study shows

December 11, 2018
If you are wondering how long you personally are willing to stand in line to buy that hot new holiday gift, scientists at Johns Hopkins Medicine say the answer may be found in the biological rules governing how animals typically ...

Using neurofeedback to prevent PTSD in soldiers

December 11, 2018
A team of researchers from Israel, the U.S. and the U.K. has found that using neurofeedback could prevent soldiers from experiencing PTSD after engaging in emotionally difficult situations. In their paper published in the ...

Study: Age, race differences determine risk of stroke in women and men

December 11, 2018
A new study found that, between the ages of 45 and 74 years, white women were less likely to have a stroke than white men, but at age 75 and older, there was no difference in stroke risk between white women and men. In contrast, ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

FredJose
1 / 5 (1) Dec 04, 2018
Imitation is the sincerest form of flattery - or so the saying goes.
It would of course be even more sincere to acknowledge that the genius of the Creator is very much worth imitating, especially given that we are made in His image!
No evolution required. In fact, evolution is dead in the water and cannot in any way, shape or form conceive of, design or construct the highly complex systems in biological living organisms.
To think that there is such a god of evolution is to commit the proverbial sin of idolatry.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.