Computer can read letters directly from the brain

By analysing MRI images of the brain with an elegant mathematical model, it is possible to reconstruct thoughts more accurately than ever before. In this way, researchers from Radboud University Nijmegen have succeeded in determining which letter a test subject was looking at. The journal Neuroimage has accepted the article, which will be published soon.

A preliminary version of the article can be read online.

Functional MRI scanners have been used in cognition research primarily to determine which are active while test subjects perform a specific task. The question is simple: is a particular region on or off? A research group at the Donders Institute for Brain, Cognition and Behaviour at Radboud University has gone a step further: they have used data from the to determine what a test subject is looking at.

The researchers 'taught' a how small volumes of 2x2x2 mm from the brain scans – known as voxels – respond to individual pixels. By combining all the information about the pixels from the voxels, it became possible to reconstruct the image viewed by the subject. The result was not a clear image, but a somewhat fuzzy speckle pattern. In this study, the researchers used hand-written letters.

Prior knowledge improves model performance

'After this we did something new," says lead researcher Marcel van Gerven. "We gave the model prior knowledge: we taught it what letters look like. This improved the recognition of the letters enormously. The model compares the letters to determine which one corresponds most exactly with the speckle image, and then pushes the results of the image towards that letter. The result was the actual letter, a true reconstruction."

"Our approach is similar to how we believe the brain itself combines prior knowledge with sensory information. For example, you can recognise the lines and curves in this article as letters only after you have learned to read. And this is exactly what we are looking for: models that show what is happening in the brain in a realistic fashion. We hope to improve the models to such an extent that we can also apply them to the working memory or to subjective experiences such as dreams or visualisations. Reconstructions indicate whether the model you have created approaches reality."

Improved resolution; more possibilities

"In our further research we will be working with a more powerful MRI scanner," explains Sanne Schoenmakers, who is working on a thesis about decoding thoughts. "Due to the higher resolution of the scanner, we hope to be able to link the model to more detailed images. We are currently linking images of letters to 1200 voxels in the brain; with the more powerful scanner we will link images of faces to 15,000 voxels."

More information: Linear reconstruction of perceived images from human brain activity, Neuroimage, in press. dx.doi.org/10.1016/j.neuroimage.2013.07.043

Related Stories

MRI fast track for Alzheimer's diagnosis

Jul 18, 2013

A new approach to analyzing MRI brain scans could help speed up the diagnosis of Alzheimer's and other degenerative diseases, according to research published to be published in the International Journal of Computational Sc ...

New method to magnify digital images is 700 times faster

Jun 21, 2013

Aránzazu Jurío-Munárriz, a graduate in computer engineering from the NUP/UPNA-Public University of Navarre, has in her PhD thesis presented new methods for improving two of the most widespread means used in digital image ...

Fear: A justified response or faulty wiring?

Jun 04, 2013

Fear is one of the most primal feelings known to man and beast. As we develop in society and learn, fear is hard coded into our neural circuitry through the amygdala, a small, almond-shaped nuclei of neurons within the medial ...

Recommended for you

Why your favourite song takes you down memory lane

Aug 28, 2014

Music triggers different functions of the brain, which helps explain why listening to a song you like might be enjoyable but a favourite song may plunge you into nostalgia, scientists said on Thursday.

Transcranial Magnetic Stimulation of brain boosts memory

Aug 28, 2014

Stimulating a particular region in the brain via non-invasive delivery of electrical current using magnetic pulses, called Transcranial Magnetic Stimulation, improves memory, reports a new Northwestern Medicine ...

User comments

Adjust slider to filter visible comments by rank

Display comments: newest first

zorro6204
5 / 5 (1) Aug 20, 2013
Okay, how about "Computer can tell what letter is being looked at". The article title made it sound like it could read minds.
Tachyon8491
not rated yet Aug 20, 2013
We are still a long way away from reading thoughts or emotions - formulated analytical syntagms that contain both cognitive and emotional dynamics. Although we may progressively approach better resolution in such techniques I actually hope that thisfunctionality will prove impossible to achieve with perfection in the long run. The ability to read someone's mind appears like the ultimate invasion of privacy and seems a violation of nature that should never be allowed to reach full application.