Computer can read letters directly from the brain

August 19, 2013

By analysing MRI images of the brain with an elegant mathematical model, it is possible to reconstruct thoughts more accurately than ever before. In this way, researchers from Radboud University Nijmegen have succeeded in determining which letter a test subject was looking at. The journal Neuroimage has accepted the article, which will be published soon.

A preliminary version of the article can be read online.

Functional MRI scanners have been used in cognition research primarily to determine which are active while test subjects perform a specific task. The question is simple: is a particular region on or off? A research group at the Donders Institute for Brain, Cognition and Behaviour at Radboud University has gone a step further: they have used data from the to determine what a test subject is looking at.

The researchers 'taught' a how small volumes of 2x2x2 mm from the brain scans – known as voxels – respond to individual pixels. By combining all the information about the pixels from the voxels, it became possible to reconstruct the image viewed by the subject. The result was not a clear image, but a somewhat fuzzy speckle pattern. In this study, the researchers used hand-written letters.

Prior knowledge improves model performance

'After this we did something new," says lead researcher Marcel van Gerven. "We gave the model prior knowledge: we taught it what letters look like. This improved the recognition of the letters enormously. The model compares the letters to determine which one corresponds most exactly with the speckle image, and then pushes the results of the image towards that letter. The result was the actual letter, a true reconstruction."

"Our approach is similar to how we believe the brain itself combines prior knowledge with sensory information. For example, you can recognise the lines and curves in this article as letters only after you have learned to read. And this is exactly what we are looking for: models that show what is happening in the brain in a realistic fashion. We hope to improve the models to such an extent that we can also apply them to the working memory or to subjective experiences such as dreams or visualisations. Reconstructions indicate whether the model you have created approaches reality."

Improved resolution; more possibilities

"In our further research we will be working with a more powerful MRI scanner," explains Sanne Schoenmakers, who is working on a thesis about decoding thoughts. "Due to the higher resolution of the scanner, we hope to be able to link the model to more detailed images. We are currently linking images of letters to 1200 voxels in the brain; with the more powerful scanner we will link images of faces to 15,000 voxels."

Explore further: Australian scientists map mouse brains in greatest detail yet

More information: Linear reconstruction of perceived images from human brain activity, Neuroimage, in press. dx.doi.org/10.1016/j.neuroimage.2013.07.043

Related Stories

Australian scientists map mouse brains in greatest detail yet

April 29, 2013
(Medical Xpress)—Hopes for a cure for many brain diseases may rest on the humble mouse, now that scientists can map the rodents' brains more thoroughly than ever before.

Weight loss improves memory and alters brain activity in overweight women

June 17, 2013
Memory improves in older, overweight women after they lose weight by dieting, and their brain activity actually changes in the regions of the brain that are important for memory tasks, a new study finds. The results were ...

MRI fast track for Alzheimer's diagnosis

July 18, 2013
A new approach to analyzing MRI brain scans could help speed up the diagnosis of Alzheimer's and other degenerative diseases, according to research published to be published in the International Journal of Computational Science ...

Fear: A justified response or faulty wiring?

June 4, 2013
Fear is one of the most primal feelings known to man and beast. As we develop in society and learn, fear is hard coded into our neural circuitry through the amygdala, a small, almond-shaped nuclei of neurons within the medial ...

Brain imaging reveals the movies in our mind

September 22, 2011
Imagine tapping into the mind of a coma patient, or watching one's own dream on YouTube. With a cutting-edge blend of brain imaging and computer simulation, scientists at the University of California, Berkeley, are bringing ...

Recommended for you

Small but distinct differences among species mark evolution of human brain

November 23, 2017
The most dramatic divergence between humans and other primates can be found in the brain, the primary organ that gives our species its identity.

Team constructs whole-brain map of electrical connections key to forming memories

November 22, 2017
A team of neuroscientists at the University of Pennsylvania has constructed the first whole-brain map of electrical connectivity in the brain based on data from nearly 300 neurosurgical patients with electrodes implanted ...

To forget or to remember? Memory depends on subtle brain signals, scientists find

November 22, 2017
The fragrance of hot pumpkin pie can bring back pleasant memories of holidays past, while the scent of an antiseptic hospital room may cause a shudder. The power of odors to activate memories both pleasing and aversive exists ...

Pitch imperfect? How the brain decodes pitch may improve cochlear implants

November 22, 2017
Picture yourself with a friend in a crowded restaurant. The din of other diners, the clattering of dishes, the muffled notes of background music, the voice of your friend, not to mention your own – all compete for your ...

New research suggests high-intensity exercise boosts memory

November 22, 2017
The health advantages of high-intensity exercise are widely known but new research from McMaster University points to another major benefit: better memory.

Schizophrenia originates early in pregnancy, 'mini-brain' research suggests

November 20, 2017
Symptoms of schizophrenia usually appear in adolescence or young adulthood, but new research reveals that the brain disease likely begins very early in development, toward the end of the first trimester of pregnancy. The ...

2 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

zorro6204
5 / 5 (1) Aug 20, 2013
Okay, how about "Computer can tell what letter is being looked at". The article title made it sound like it could read minds.
Tachyon8491
not rated yet Aug 20, 2013
We are still a long way away from reading thoughts or emotions - formulated analytical syntagms that contain both cognitive and emotional dynamics. Although we may progressively approach better resolution in such techniques I actually hope that thisfunctionality will prove impossible to achieve with perfection in the long run. The ability to read someone's mind appears like the ultimate invasion of privacy and seems a violation of nature that should never be allowed to reach full application.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.