Mountain splendor? Scientists know where your eyes will look

December 4, 2018, Yale University
The top row shows images that our participants viewed in the fMRI scanner. The bottom row shows neural reconstructions from the participants' brain activity that predict where people will direct their eyes, overlaid on top of the images. Note that just the colored overlay predicting eye movements is reconstructed from brain activity and not the image underlay. The brighter white and pink areas in the reconstructions show where people directed their attention. Credit: Chun Lab Yale

Using precise brain measurements, Yale researchers predicted how people's eyes move when viewing natural scenes, an advance in understanding the human visual system that can improve a host of artificial intelligence efforts, such as the development of driverless cars, said the researchers.

"We are visual beings and knowing how the rapidly computes where to look is fundamentally important," said Yale's Marvin Chun, Richard M. Colgate Professor of Psychology, professor of neuroscience and co-author of new research published Dec. 4 in the journal Nature Communications.

Eye movements have been extensively studied, and researchers can tell with some certainty where a gaze will be directed at different elements in the environment. What hasn't been understood is how the brain orchestrates this ability, which is so fundamental to survival.

In a previous example of "mind reading," Chun's group successfully reconstructed facial images viewed while people were scanned in an MRI machine, based on their brain imaging data alone.

In the new paper, Chun and lead author Thomas P. O'Connell took a similar approach and showed that by analyzing the brain responses to complex, natural scenes, they could predict where people would direct their attention and gaze. This was made possible by analyzing the brain data with deep convolutional neural networks—models that are extensively used in artificial intelligence (AI).

"The work represents a perfect marriage of neuroscience and data science," Chun said.

The findings have a myriad of potential applications—such as testing competing systems that categorize images and guide driverless cars.

"People can see better than AI systems can," Chun said. "Understanding how the brain performs its complex calculations is an ultimate goal of neuroscience and benefits AI efforts."

Explore further: Researchers demonstrate 'mind-reading' brain-decoding tech

Related Stories

Researchers demonstrate 'mind-reading' brain-decoding tech

October 23, 2017
Researchers have demonstrated how to decode what the human brain is seeing by using artificial intelligence to interpret fMRI scans from people watching videos, representing a sort of mind-reading technology.

Researchers reconstruct facial images locked in a viewer's mind

March 25, 2014
Using only data from an fMRI scan, researchers led by a Yale University undergraduate have accurately reconstructed images of human faces as viewed by other people.

Dissecting artificial intelligence to better understand the human brain

March 25, 2018
In the natural world, intelligence takes many forms. It could be a bat using echolocation to expertly navigate in the dark, or an octopus quickly adapting its behavior to survive in the deep ocean. Likewise, in the computer ...

Artificial intelligence helps reveal how people process abstract thought

October 9, 2018
As artificial intelligence becomes more sophisticated, much of the public attention has focused on how successfully these technologies can compete against humans at chess and other strategy games. A philosopher from the University ...

Malicious brain cell identified—surprising finding fills gap in understanding astrocytes' role in brain disease

September 13, 2018
Astrocytes—the star-shaped cells of our brain—are very busy. Their job description includes maintaining the blood-brain barrier, removing excess neurotransmitters, repairing brain tissue and more.

Neural net activations are aligned with gamma band activity of the human visual cortex

August 16, 2018
Researchers at the University of Tartu's Computational Neuroscience Lab, in Estonia, have discovered that activations of deep convolutional neural networks are aligned with the gamma band activity of the human visual cortex. ...

Recommended for you

Scientists identify method to study resilience to pain

December 14, 2018
Scientists at the Yale School of Medicine and Veterans Affairs Connecticut Healthcare System have successfully demonstrated that it is possible to pinpoint genes that contribute to inter-individual differences in pain.

Parents' brain activity 'echoes' their infant's brain activity when they play together

December 13, 2018
When infants are playing with objects, their early attempts to pay attention to things are accompanied by bursts of high-frequency activity in their brain. But what happens when parents play together with them? New research, ...

Researchers discover abundant source for neuronal cells

December 13, 2018
USC researchers seeking a way to study genetic activity associated with psychiatric disorders have discovered an abundant source of human cells—the nose.

In the developing brain, scientists find roots of neuropsychiatric diseases

December 13, 2018
The most comprehensive genomic analysis of the human brain ever undertaken has revealed new insights into the changes it undergoes through development, how it varies among individuals, and the roots of neuropsychiatric illnesses ...

Researchers find the cause of and cure for brain injury associated with gut condition

December 13, 2018
Using a mouse model of necrotizing enterocolitis (NEC)—a potentially fatal condition that causes a premature infant's gut to suddenly die—researchers at Johns Hopkins say they have uncovered the molecular causes of the ...

How the brain tells you to scratch that itch

December 13, 2018
It's a maddening cycle that has affected us all: it starts with an itch that triggers scratching, but scratching only makes the itchiness worse. Now, researchers have revealed the brain mechanism driving this uncontrollable ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

FredJose
1 / 5 (1) Dec 04, 2018
Imitation is the sincerest form of flattery - or so the saying goes.
It would of course be even more sincere to acknowledge that the genius of the Creator is very much worth imitating, especially given that we are made in His image!
No evolution required. In fact, evolution is dead in the water and cannot in any way, shape or form conceive of, design or construct the highly complex systems in biological living organisms.
To think that there is such a god of evolution is to commit the proverbial sin of idolatry.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.