Computer can read letters directly from the brain

August 19, 2013

By analysing MRI images of the brain with an elegant mathematical model, it is possible to reconstruct thoughts more accurately than ever before. In this way, researchers from Radboud University Nijmegen have succeeded in determining which letter a test subject was looking at. The journal Neuroimage has accepted the article, which will be published soon.

A preliminary version of the article can be read online.

Functional MRI scanners have been used in cognition research primarily to determine which are active while test subjects perform a specific task. The question is simple: is a particular region on or off? A research group at the Donders Institute for Brain, Cognition and Behaviour at Radboud University has gone a step further: they have used data from the to determine what a test subject is looking at.

The researchers 'taught' a how small volumes of 2x2x2 mm from the brain scans – known as voxels – respond to individual pixels. By combining all the information about the pixels from the voxels, it became possible to reconstruct the image viewed by the subject. The result was not a clear image, but a somewhat fuzzy speckle pattern. In this study, the researchers used hand-written letters.

Prior knowledge improves model performance

'After this we did something new," says lead researcher Marcel van Gerven. "We gave the model prior knowledge: we taught it what letters look like. This improved the recognition of the letters enormously. The model compares the letters to determine which one corresponds most exactly with the speckle image, and then pushes the results of the image towards that letter. The result was the actual letter, a true reconstruction."

"Our approach is similar to how we believe the brain itself combines prior knowledge with sensory information. For example, you can recognise the lines and curves in this article as letters only after you have learned to read. And this is exactly what we are looking for: models that show what is happening in the brain in a realistic fashion. We hope to improve the models to such an extent that we can also apply them to the working memory or to subjective experiences such as dreams or visualisations. Reconstructions indicate whether the model you have created approaches reality."

Improved resolution; more possibilities

"In our further research we will be working with a more powerful MRI scanner," explains Sanne Schoenmakers, who is working on a thesis about decoding thoughts. "Due to the higher resolution of the scanner, we hope to be able to link the model to more detailed images. We are currently linking images of letters to 1200 voxels in the brain; with the more powerful scanner we will link images of faces to 15,000 voxels."

Explore further: Brain imaging reveals the movies in our mind

More information: Linear reconstruction of perceived images from human brain activity, Neuroimage, in press.

Related Stories

Brain imaging reveals the movies in our mind

September 22, 2011

Imagine tapping into the mind of a coma patient, or watching one's own dream on YouTube. With a cutting-edge blend of brain imaging and computer simulation, scientists at the University of California, Berkeley, are bringing ...

Fear: A justified response or faulty wiring?

June 4, 2013

Fear is one of the most primal feelings known to man and beast. As we develop in society and learn, fear is hard coded into our neural circuitry through the amygdala, a small, almond-shaped nuclei of neurons within the medial ...

MRI fast track for Alzheimer's diagnosis

July 18, 2013

A new approach to analyzing MRI brain scans could help speed up the diagnosis of Alzheimer's and other degenerative diseases, according to research published to be published in the International Journal of Computational Science ...

Recommended for you

Drug used to treat cancer appears to sharpen memory

October 2, 2015

Can you imagine a drug that would make it easier to learn a language, sharpen your memory and help those with dementia and Alzheimer's disease by rewiring the brain and keeping neurons alive?

Brain chemical aids tic control in Tourette Syndrome

October 1, 2015

A chemical in the brain could potentially be harnessed to help young people with Tourette Syndrome (TS) to overcome the physical and vocal tics associated with the neurological disorder, say researchers.


Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (1) Aug 20, 2013
Okay, how about "Computer can tell what letter is being looked at". The article title made it sound like it could read minds.
not rated yet Aug 20, 2013
We are still a long way away from reading thoughts or emotions - formulated analytical syntagms that contain both cognitive and emotional dynamics. Although we may progressively approach better resolution in such techniques I actually hope that thisfunctionality will prove impossible to achieve with perfection in the long run. The ability to read someone's mind appears like the ultimate invasion of privacy and seems a violation of nature that should never be allowed to reach full application.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.