Mountain splendor? Scientists know where your eyes will look

Mountain splendor? Scientists know where your eyes will look
The top row shows images that our participants viewed in the fMRI scanner. The bottom row shows neural reconstructions from the participants' brain activity that predict where people will direct their eyes, overlaid on top of the images. Note that just the colored overlay predicting eye movements is reconstructed from brain activity and not the image underlay. The brighter white and pink areas in the reconstructions show where people directed their attention. Credit: Chun Lab Yale

Using precise brain measurements, Yale researchers predicted how people's eyes move when viewing natural scenes, an advance in understanding the human visual system that can improve a host of artificial intelligence efforts, such as the development of driverless cars, said the researchers.

"We are visual beings and knowing how the rapidly computes where to look is fundamentally important," said Yale's Marvin Chun, Richard M. Colgate Professor of Psychology, professor of neuroscience and co-author of new research published Dec. 4 in the journal Nature Communications.

Eye movements have been extensively studied, and researchers can tell with some certainty where a gaze will be directed at different elements in the environment. What hasn't been understood is how the brain orchestrates this ability, which is so fundamental to survival.

In a previous example of "mind reading," Chun's group successfully reconstructed facial images viewed while people were scanned in an MRI machine, based on their brain imaging data alone.

In the new paper, Chun and lead author Thomas P. O'Connell took a similar approach and showed that by analyzing the brain responses to complex, natural scenes, they could predict where people would direct their attention and gaze. This was made possible by analyzing the brain data with deep convolutional neural networks—models that are extensively used in artificial intelligence (AI).

"The work represents a perfect marriage of neuroscience and data science," Chun said.

The findings have a myriad of potential applications—such as testing competing systems that categorize images and guide driverless cars.

"People can see better than AI systems can," Chun said. "Understanding how the brain performs its complex calculations is an ultimate goal of neuroscience and benefits AI efforts."

Explore further

Researchers demonstrate 'mind-reading' brain-decoding tech

Journal information: Nature Communications

Provided by Yale University
Citation: Mountain splendor? Scientists know where your eyes will look (2018, December 4) retrieved 22 May 2019 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors

User comments

Dec 04, 2018
Imitation is the sincerest form of flattery - or so the saying goes.
It would of course be even more sincere to acknowledge that the genius of the Creator is very much worth imitating, especially given that we are made in His image!
No evolution required. In fact, evolution is dead in the water and cannot in any way, shape or form conceive of, design or construct the highly complex systems in biological living organisms.
To think that there is such a god of evolution is to commit the proverbial sin of idolatry.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more