Computer can read letters directly from the brain

August 19, 2013

By analysing MRI images of the brain with an elegant mathematical model, it is possible to reconstruct thoughts more accurately than ever before. In this way, researchers from Radboud University Nijmegen have succeeded in determining which letter a test subject was looking at. The journal Neuroimage has accepted the article, which will be published soon.

A preliminary version of the article can be read online.

Functional MRI scanners have been used in cognition research primarily to determine which are active while test subjects perform a specific task. The question is simple: is a particular region on or off? A research group at the Donders Institute for Brain, Cognition and Behaviour at Radboud University has gone a step further: they have used data from the to determine what a test subject is looking at.

The researchers 'taught' a how small volumes of 2x2x2 mm from the brain scans – known as voxels – respond to individual pixels. By combining all the information about the pixels from the voxels, it became possible to reconstruct the image viewed by the subject. The result was not a clear image, but a somewhat fuzzy speckle pattern. In this study, the researchers used hand-written letters.

Prior knowledge improves model performance

'After this we did something new," says lead researcher Marcel van Gerven. "We gave the model prior knowledge: we taught it what letters look like. This improved the recognition of the letters enormously. The model compares the letters to determine which one corresponds most exactly with the speckle image, and then pushes the results of the image towards that letter. The result was the actual letter, a true reconstruction."

"Our approach is similar to how we believe the brain itself combines prior knowledge with sensory information. For example, you can recognise the lines and curves in this article as letters only after you have learned to read. And this is exactly what we are looking for: models that show what is happening in the brain in a realistic fashion. We hope to improve the models to such an extent that we can also apply them to the working memory or to subjective experiences such as dreams or visualisations. Reconstructions indicate whether the model you have created approaches reality."

Improved resolution; more possibilities

"In our further research we will be working with a more powerful MRI scanner," explains Sanne Schoenmakers, who is working on a thesis about decoding thoughts. "Due to the higher resolution of the scanner, we hope to be able to link the model to more detailed images. We are currently linking images of letters to 1200 voxels in the brain; with the more powerful scanner we will link images of faces to 15,000 voxels."

Explore further: Australian scientists map mouse brains in greatest detail yet

More information: Linear reconstruction of perceived images from human brain activity, Neuroimage, in press.

Related Stories

MRI fast track for Alzheimer's diagnosis

July 18, 2013

A new approach to analyzing MRI brain scans could help speed up the diagnosis of Alzheimer's and other degenerative diseases, according to research published to be published in the International Journal of Computational Science ...

Fear: A justified response or faulty wiring?

June 4, 2013

Fear is one of the most primal feelings known to man and beast. As we develop in society and learn, fear is hard coded into our neural circuitry through the amygdala, a small, almond-shaped nuclei of neurons within the medial ...

Brain imaging reveals the movies in our mind

September 22, 2011

Imagine tapping into the mind of a coma patient, or watching one's own dream on YouTube. With a cutting-edge blend of brain imaging and computer simulation, scientists at the University of California, Berkeley, are bringing ...

Recommended for you

Rat brain atlas provides MR images for stereotaxic surgery

October 21, 2016

Boris Odintsov, senior research scientist at the Biomedical Imaging Center at the Beckman Institute for Advanced Science and Technology at the University of Illinois in Urbana-Champaign, and Thomas Brozoski, research professor ...

ALS study reveals role of RNA-binding proteins

October 20, 2016

Although only 10 percent of amyotrophic lateral sclerosis (ALS) cases are hereditary, a significant number of them are caused by mutations that affect proteins that bind RNA, a type of genetic material. University of California ...

Imaging technique maps serotonin activity in living brains

October 20, 2016

Serotonin is a neurotransmitter that's partly responsible for feelings of happiness and for mood regulation in humans. This makes it a common target for antidepressants, which block serotonin from being reabsorbed by neurons ...

Overcoming egocentricity increases self-control

October 19, 2016

Neurobiological models of self-control usually focus on brain mechanisms involved in impulse control and emotion regulation. Recent research at the University of Zurich shows that the mechanism for overcoming egocentricity ...


Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (1) Aug 20, 2013
Okay, how about "Computer can tell what letter is being looked at". The article title made it sound like it could read minds.
not rated yet Aug 20, 2013
We are still a long way away from reading thoughts or emotions - formulated analytical syntagms that contain both cognitive and emotional dynamics. Although we may progressively approach better resolution in such techniques I actually hope that thisfunctionality will prove impossible to achieve with perfection in the long run. The ability to read someone's mind appears like the ultimate invasion of privacy and seems a violation of nature that should never be allowed to reach full application.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.