Researchers reconstruct facial images locked in a viewer's mind

March 25, 2014, Yale University
Researchers reconstruct facial images locked in a viewer’s mind
Credit: Alan Cowen

Using only data from an fMRI scan, researchers led by a Yale University undergraduate have accurately reconstructed images of human faces as viewed by other people.

"It is a form of mind reading," said Marvin Chun, professor of psychology, cognitive science and neurobiology and an author of the paper in the journal Neuroimage.

The increased level of sophistication of fMRI scans has already enabled scientists to use data from scans taken as individuals view scenes and predict whether a subject was, for instance, viewing a beach or city scene, an animal or a building.

"But they can only tell you they are viewing an animal or a building, not what animal or building," Chun said. "This is a different level of sophistication."

One of Chun's students, Alan S. Cowen, then a Yale junior now pursing an advanced degree at the University of California at Berkeley, wanted to know whether it would be possible to reconstruct a human face from patterns of brain activity. The task was daunting, because faces are more similar to each other than buildings. Also large areas of the brain are recruited in the processing of human faces, a testament to its importance in survival.

"We perceive faces in a much greater level of detail than we perceive other things," Cowen said.

Working with funding from the Yale Provost's office, Cowen and post doctoral researcher Brice Kuhl, now an assistant professor at New York University, showed six subjects 300 different "training" faces while undergoing fMRI scans. They used the data to create a sort of statistical library of how those brains responded to individual faces. They then showed the six subjects new sets of faces while they were undergoing scans. Taking that fMRI data alone, researchers used their statistical library to reconstruct the faces their subjects were viewing.

Cowen said the accuracy of these facial reconstructions will increase with time and he envisions they can be used as a research tool, for instance in studying how autistic children respond to .

Chun said the study shows the value of funding research ambitions of Yale undergraduates.

"I would never have received external funding for this, it was too novel," Chun said.

Explore further: Brain differences seen in depressed preschoolers

Related Stories

Brain differences seen in depressed preschoolers

July 1, 2013
(Medical Xpress)—A key brain structure that regulates emotions works differently in preschoolers with depression compared with their healthy peers, according to new research at Washington University School of Medicine in ...

Racial blends - easy on the eyes until you categorise

January 9, 2014
New light has been shed on how our minds judge multi-racial individuals, thanks to a collaborative research project between the University of Otago and the University of California, San Diego.

What's in a face? Researchers find patterns of neural activity in brain region that plays role in recognizing traits

October 14, 2013
When you meet people for the first time, what's the first thing you think you notice? Is it their hair color, or eye color? Maybe it's whether they're wearing a suit or a T-shirt and jeans, or whether they have a firm handshake.

Researchers point to digital gains in human recognition

December 26, 2013
Human beings are highly efficient at recognizing familiar faces, even from very poor quality images.

Recommended for you

Research reveals atomic-level changes in ALS-linked protein

January 18, 2018
For the first time, researchers have described atom-by-atom changes in a family of proteins linked to amyotrophic lateral sclerosis (ALS), a group of brain disorders known as frontotemporal dementia and degenerative diseases ...

Fragile X finding shows normal neurons that interact poorly

January 18, 2018
Neurons in mice afflicted with the genetic defect that causes Fragile X syndrome (FXS) appear similar to those in healthy mice, but these neurons fail to interact normally, resulting in the long-known cognitive impairments, ...

How your brain remembers what you had for dinner last night

January 17, 2018
Confirming earlier computational models, researchers at University of California San Diego and UC San Diego School of Medicine, with colleagues in Arizona and Louisiana, report that episodic memories are encoded in the hippocampus ...

Recording a thought's fleeting trip through the brain

January 17, 2018
University of California, Berkeley neuroscientists have tracked the progress of a thought through the brain, showing clearly how the prefrontal cortex at the front of the brain coordinates activity to help us act in response ...

Midbrain 'start neurons' control whether we walk or run

January 17, 2018
Locomotion comprises the most fundamental movements we perform. It is a complex sequence from initiating the first step, to stopping when we reach our goal. At the same time, locomotion is executed at different speeds to ...

Neuroscientists suggest a model for how we gain volitional control of what we hold in our minds

January 16, 2018
Working memory is a sort of "mental sketchpad" that allows you to accomplish everyday tasks such as calling in your hungry family's takeout order and finding the bathroom you were just told "will be the third door on the ...


Adjust slider to filter visible comments by rank

Display comments: newest first

Captain Stumpy
5 / 5 (1) Mar 25, 2014
pure speculation, mind you, but IMHO there seems to be definite possibilities for this technology in the future law enforcement/psychology/profiling etc arena, starting with lie detection and expanding into criminal and abnormal psychology (think rape, pedophilia, serial killers)
imaging and being able to "see" what the offender/suspect see's during the recount of the criminal act (or during an interview) would give researchers/criminal psychologists/profilers/law enforcement etc quite a lot to work with
then there is the possibilities for victims as well... an image from the victim (non invasive fMRI) would be better than a traumatic recount where a sketch artist tries to reconstruct a face... just take it directly from the victim

just my opinion mind you

5 / 5 (1) Mar 25, 2014
Once the contents of our minds is no longer considered sacrosanct, all is lost.
not rated yet Mar 25, 2014
5 / 5 (1) Mar 25, 2014
Once they get more sophisticated I'll do the world a favor and let them be entertained by some of my crazy dreams lol
3 / 5 (1) Mar 26, 2014
Why would faces appear anywhere in the brain in the same form as seen in a photograph? This is the problem of infinite regress (the homunculus problem) in that some downstream process must now look at the reconstructed face. But why the extra step? Why not just skip the reconstruction phase and go directly to breaking down the image for interpretation?

As there are more than ten times as many neurons innovating the earliest visual processing area than coming from the retina it is likely that any 'recognised' aspect of the image is replaced with one from memory leaving only totally mysterious, novel or new features of the face.

This is the process most probably intercepted by the fMRI scan. If this is the case then faces of familiar people will be altered far more than faces of strangers (because recognition cues/feedback will be more elaborate). For instance a person that is always smiling may appear in the fMRI scan to be smiling even if the photograph shows no smile.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.