Computer can read letters directly from the brain

August 19, 2013

By analysing MRI images of the brain with an elegant mathematical model, it is possible to reconstruct thoughts more accurately than ever before. In this way, researchers from Radboud University Nijmegen have succeeded in determining which letter a test subject was looking at. The journal Neuroimage has accepted the article, which will be published soon.

A preliminary version of the article can be read online.

Functional MRI scanners have been used in cognition research primarily to determine which are active while test subjects perform a specific task. The question is simple: is a particular region on or off? A research group at the Donders Institute for Brain, Cognition and Behaviour at Radboud University has gone a step further: they have used data from the to determine what a test subject is looking at.

The researchers 'taught' a how small volumes of 2x2x2 mm from the brain scans – known as voxels – respond to individual pixels. By combining all the information about the pixels from the voxels, it became possible to reconstruct the image viewed by the subject. The result was not a clear image, but a somewhat fuzzy speckle pattern. In this study, the researchers used hand-written letters.

Prior knowledge improves model performance

'After this we did something new," says lead researcher Marcel van Gerven. "We gave the model prior knowledge: we taught it what letters look like. This improved the recognition of the letters enormously. The model compares the letters to determine which one corresponds most exactly with the speckle image, and then pushes the results of the image towards that letter. The result was the actual letter, a true reconstruction."

"Our approach is similar to how we believe the brain itself combines prior knowledge with sensory information. For example, you can recognise the lines and curves in this article as letters only after you have learned to read. And this is exactly what we are looking for: models that show what is happening in the brain in a realistic fashion. We hope to improve the models to such an extent that we can also apply them to the working memory or to subjective experiences such as dreams or visualisations. Reconstructions indicate whether the model you have created approaches reality."

Improved resolution; more possibilities

"In our further research we will be working with a more powerful MRI scanner," explains Sanne Schoenmakers, who is working on a thesis about decoding thoughts. "Due to the higher resolution of the scanner, we hope to be able to link the model to more detailed images. We are currently linking images of letters to 1200 voxels in the brain; with the more powerful scanner we will link images of faces to 15,000 voxels."

Explore further: Brain imaging reveals the movies in our mind

More information: Linear reconstruction of perceived images from human brain activity, Neuroimage, in press.

Related Stories

Brain imaging reveals the movies in our mind

September 22, 2011

Imagine tapping into the mind of a coma patient, or watching one's own dream on YouTube. With a cutting-edge blend of brain imaging and computer simulation, scientists at the University of California, Berkeley, are bringing ...

Fear: A justified response or faulty wiring?

June 4, 2013

Fear is one of the most primal feelings known to man and beast. As we develop in society and learn, fear is hard coded into our neural circuitry through the amygdala, a small, almond-shaped nuclei of neurons within the medial ...

MRI fast track for Alzheimer's diagnosis

July 18, 2013

A new approach to analyzing MRI brain scans could help speed up the diagnosis of Alzheimer's and other degenerative diseases, according to research published to be published in the International Journal of Computational Science ...

Recommended for you

Umbilical cells help eye's neurons connect

November 24, 2015

Cells isolated from human umbilical cord tissue have been shown to produce molecules that help retinal neurons from the eyes of rats grow, connect and survive, according to Duke University researchers working with Janssen ...

Brain connections predict how well you can pay attention

November 24, 2015

During a 1959 television appearance, Jack Kerouac was asked how long it took him to write his novel On The Road. His response – three weeks – amazed the interviewer and ignited an enduring myth that the book was composed ...

No cable spaghetti in the brain

November 24, 2015

Our brain is a mysterious machine. Billions of nerve cells are connected such that they store information as efficiently as books are stored in a well-organized library. To this date, many details remain unclear, for instance ...

Neurons encoding hand shapes identified in human brain

November 23, 2015

Neural prosthetic devices, which include small electrode arrays implanted in the brain, can allow paralyzed patients to control the movement of a robotic limb, whether that limb is attached to the individual or not. In May ...

Wireless sensor enables study of traumatic brain injury

November 23, 2015

A new system that uses a wireless implant has been shown to record for the first time how brain tissue deforms when subjected to the kind of shock that causes blast-induced trauma commonly seen in combat veterans.


Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (1) Aug 20, 2013
Okay, how about "Computer can tell what letter is being looked at". The article title made it sound like it could read minds.
not rated yet Aug 20, 2013
We are still a long way away from reading thoughts or emotions - formulated analytical syntagms that contain both cognitive and emotional dynamics. Although we may progressively approach better resolution in such techniques I actually hope that thisfunctionality will prove impossible to achieve with perfection in the long run. The ability to read someone's mind appears like the ultimate invasion of privacy and seems a violation of nature that should never be allowed to reach full application.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.