Researchers reconstruct facial images locked in a viewer's mind

March 25, 2014
Researchers reconstruct facial images locked in a viewer’s mind
Credit: Alan Cowen

Using only data from an fMRI scan, researchers led by a Yale University undergraduate have accurately reconstructed images of human faces as viewed by other people.

"It is a form of mind reading," said Marvin Chun, professor of psychology, cognitive science and neurobiology and an author of the paper in the journal Neuroimage.

The increased level of sophistication of fMRI scans has already enabled scientists to use data from scans taken as individuals view scenes and predict whether a subject was, for instance, viewing a beach or city scene, an animal or a building.

"But they can only tell you they are viewing an animal or a building, not what animal or building," Chun said. "This is a different level of sophistication."

One of Chun's students, Alan S. Cowen, then a Yale junior now pursing an advanced degree at the University of California at Berkeley, wanted to know whether it would be possible to reconstruct a human face from patterns of brain activity. The task was daunting, because faces are more similar to each other than buildings. Also large areas of the brain are recruited in the processing of human faces, a testament to its importance in survival.

"We perceive faces in a much greater level of detail than we perceive other things," Cowen said.

Working with funding from the Yale Provost's office, Cowen and post doctoral researcher Brice Kuhl, now an assistant professor at New York University, showed six subjects 300 different "training" faces while undergoing fMRI scans. They used the data to create a sort of statistical library of how those brains responded to individual faces. They then showed the six subjects new sets of faces while they were undergoing scans. Taking that fMRI data alone, researchers used their statistical library to reconstruct the faces their subjects were viewing.

Cowen said the accuracy of these facial reconstructions will increase with time and he envisions they can be used as a research tool, for instance in studying how autistic children respond to .

Chun said the study shows the value of funding research ambitions of Yale undergraduates.

"I would never have received external funding for this, it was too novel," Chun said.

Explore further: Brain differences seen in depressed preschoolers

Related Stories

Brain differences seen in depressed preschoolers

July 1, 2013
(Medical Xpress)—A key brain structure that regulates emotions works differently in preschoolers with depression compared with their healthy peers, according to new research at Washington University School of Medicine in ...

Racial blends - easy on the eyes until you categorise

January 9, 2014
New light has been shed on how our minds judge multi-racial individuals, thanks to a collaborative research project between the University of Otago and the University of California, San Diego.

What's in a face? Researchers find patterns of neural activity in brain region that plays role in recognizing traits

October 14, 2013
When you meet people for the first time, what's the first thing you think you notice? Is it their hair color, or eye color? Maybe it's whether they're wearing a suit or a T-shirt and jeans, or whether they have a firm handshake.

Researchers point to digital gains in human recognition

December 26, 2013
Human beings are highly efficient at recognizing familiar faces, even from very poor quality images.

Recommended for you

'Residual echo' of ancient humans in scans may hold clues to mental disorders

July 26, 2017
Researchers at the National Institute of Mental Health (NIMH) have produced the first direct evidence that parts of our brains implicated in mental disorders may be shaped by a "residual echo" from our ancient past. The more ...

Cellular roots of anxiety identified

July 26, 2017
From students stressing over exams to workers facing possible layoffs, worrying about the future is a normal and universal experience. But when people's anticipation of bad things to come starts interfering with daily life, ...

Laser used to reawaken lost memories in mice with Alzheimer's disease

July 26, 2017
(Medical Xpress)—A team of researchers at Columbia University has found that applying a laser to the part of a mouse brain used for memory storage caused the mice to recall memories lost due to a mouse version of Alzheimer's ...

Brain disease seen in most football players in large report

July 25, 2017
Research on 202 former football players found evidence of a brain disease linked to repeated head blows in nearly all of them, from athletes in the National Football League, college and even high school.

Zebrafish study reveals clues to healing spinal cord injuries

July 25, 2017
Fresh insights into how zebrafish repair their nerve connections could hold clues to new therapies for people with spinal cord injuries.

Lutein may counter cognitive aging, study finds

July 25, 2017
Spinach and kale are favorites of those looking to stay physically fit, but they also could keep consumers cognitively fit, according to a new study from University of Illinois researchers.

5 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

Captain Stumpy
5 / 5 (1) Mar 25, 2014
pure speculation, mind you, but IMHO there seems to be definite possibilities for this technology in the future law enforcement/psychology/profiling etc arena, starting with lie detection and expanding into criminal and abnormal psychology (think rape, pedophilia, serial killers)
imaging and being able to "see" what the offender/suspect see's during the recount of the criminal act (or during an interview) would give researchers/criminal psychologists/profilers/law enforcement etc quite a lot to work with
then there is the possibilities for victims as well... an image from the victim (non invasive fMRI) would be better than a traumatic recount where a sketch artist tries to reconstruct a face... just take it directly from the victim

just my opinion mind you

Mayday
5 / 5 (1) Mar 25, 2014
Once the contents of our minds is no longer considered sacrosanct, all is lost.
XavieX
not rated yet Mar 25, 2014
1984
TheKnowItAll
5 / 5 (1) Mar 25, 2014
Once they get more sophisticated I'll do the world a favor and let them be entertained by some of my crazy dreams lol
RobertKarlStonjek
3 / 5 (1) Mar 26, 2014
Why would faces appear anywhere in the brain in the same form as seen in a photograph? This is the problem of infinite regress (the homunculus problem) in that some downstream process must now look at the reconstructed face. But why the extra step? Why not just skip the reconstruction phase and go directly to breaking down the image for interpretation?

As there are more than ten times as many neurons innovating the earliest visual processing area than coming from the retina it is likely that any 'recognised' aspect of the image is replaced with one from memory leaving only totally mysterious, novel or new features of the face.

This is the process most probably intercepted by the fMRI scan. If this is the case then faces of familiar people will be altered far more than faces of strangers (because recognition cues/feedback will be more elaborate). For instance a person that is always smiling may appear in the fMRI scan to be smiling even if the photograph shows no smile.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.