Scientists decode brain signals nearly at speed of perception

January 28, 2016
Brain signals representing activity spurred by visual stimuli experienced by the subjects in this study. In this example, images of human faces generated more brain activity than images of houses. (This was not the result in every case.) Credit: Illustration by Kai Miller and Brian Donohue

Using electrodes implanted in the temporal lobes of awake patients, scientists have decoded brain signals at nearly the speed of perception. Further, analysis of patients' neural responses to two categories of visual stimuli - images of faces and houses - enabled the scientists to subsequently predict which images the patients were viewing, and when, with better than 95 percent accuracy.

The research is published today in PLOS Computational Biology.

University of Washington computational neuroscientist Rajesh Rao and UW Medicine neurosurgeon Jeff Ojemann, working their student Kai Miller and with colleagues in Southern California and New York, conducted the study.

"We were trying to understand, first, how the human brain perceives objects in the temporal lobe, and second, how one could use a computer to extract and predict what someone is seeing in ?" explained Rao. He is a UW professor of computer science and engineering, and he directs the National Science Foundation's Center for Sensorimotor Engineering, headquartered at UW.

"Clinically, you could think of our result as a proof of concept toward building a communication mechanism for patients who are paralyzed or have had a stroke and are completely locked-in," he said.

The study involved seven epilepsy patients receiving care at Harborview Medical Center in Seattle. Each was experiencing not relieved by medication, Ojemann said, so each had undergone surgery in which their brains' were implanted - temporarily, for about a week - with electrodes to try to locate the seizures' focal points.

"They were going to get the electrodes no matter what; we were just giving them additional tasks to do during their hospital stay while they are otherwise just waiting around," Ojemann said.

Temporal lobes process sensory input and are a common site of epileptic seizures. Situated behind mammals' eyes and ears, the lobes are also involved in Alzheimer's and dementias and appear somewhat more vulnerable than other brain structures to head traumas, he said.

In the experiment, the electrodes from multiple temporal-lobe locations were connected to powerful computational software that extracted two characteristic properties of the brain signal: "event-related potentials" and "broadband spectral changes."

Rao characterized the former as likely arising from "hundreds of thousands of neurons being co-activated when an image is first presented," and the latter as "continued processing after the initial wave of information."

The subjects, watching a computer monitor, were shown a random sequence of pictures - brief (400 millisecond) flashes of images of human faces and houses, interspersed with blank gray screens. Their task was to watch for an image of an upside-down house.

"We got different responses from different (electrode) locations; some were sensitive to faces and some were sensitive to houses," Rao said.

The computational software sampled and digitized the brain signals 1,000 times per second to extract their characteristics. The software also analyzed the data to determine which combination of electrode locations and signal types correlated best with what each subject actually saw.

In that way it yielded highly predictive information.

By training an algorithm on the subjects' responses to the (known) first two-thirds of the images, the researchers could examine the representing the final third of the images, whose labels were unknown to them, and predict with 96 percent accuracy whether and when (within 20 milliseconds) the subjects were seeing a house, a face or a gray screen.

This accuracy was attained only when event-related potentials and broadband changes were combined for prediction, which suggests they carry complementary information.

"Traditionally scientists have looked at single neurons," Rao said. "Our study gives a more global picture, at the level of very large networks of neurons, of how a person who is awake and paying attention perceives a complex visual object."

The scientists' technique, he said, is a steppingstone for brain mapping, in that it could be used to identify in real time which locations of the brain are sensitive to types of information.

Lead author of the study is Kai Miller, a neurosurgery resident and physicist at Stanford University who obtained his M.D. and Ph.D. at the UW. Other collaborators were Dora Hermes, a Stanford postdoctoral fellow in neuroscience, and Gerwin Schalk, a neuroscientist at the Wadsworth Institute in New York.

"The computational tools that we developed can be applied to studies of motor function, studies of epilepsy, studies of memory. The math behind it, as applied to the biological, is fundamental to learning," Ojemann said.

Explore further: Using network science to help pinpoint source of seizures

More information: PLOS Computational Biology, dx.doi.org/10.1371/journal.pcbi.1004660

Related Stories

Using network science to help pinpoint source of seizures

December 17, 2015
The ability to reliably pinpoint the anatomical source of epileptic seizures, different for each patient, remains elusive. One third of patients do not respond to medication and an alternative can be surgery to locate and ...

Computer model could hold key to personalized epilepsy treatment

December 10, 2015
A computer model that identifies the parts of a person's brain responsible for epileptic seizures could be used to design personalised surgical procedures, researchers say.

Epileptic seizures can propagate using functional brain networks

April 2, 2013
The seizures that affect people with temporal-lobe epilepsy usually start in a region of the brain called the hippocampus. But they are often able to involve other areas outside the temporal lobe, propagating via anatomically ...

Study suggests REM sleep helps the brain capture snapshots of dream images

August 12, 2015
When we sleep, we experience our most vivid dreams and vigorous brain activity during the rapid eye movement, or REM phase. Although scientists have long suspected that our eyes flicker in response to what our unconscious ...

Forget Freud, research on dream imagery may help us understand consciousness

August 13, 2015
Rapid eye movements (REMs) during sleep may contribute to the visual part of our dreams by acting as a switch from image to image, researchers have found. The study, which measured the activity of individual cells in the ...

Researchers work on device to let paralyzed limbs move

January 5, 2016
Researchers are making progress in a quest to create a device that would allow people who have been paralyzed by injury or stroke to move their limbs.

Recommended for you

Afternoon slump in reward response

August 21, 2017
Activation of a reward-processing brain region peaks in the morning and evening and dips at 2 p.m., finds a study of healthy young men published in The Journal of Neuroscience. This finding may parallel the drop in alertness ...

Research reveals 'exquisite selectivity' of neuronal wiring in the cerebral cortex

August 21, 2017
The brain's astonishing anatomical complexity has been appreciated for over 100 years, when pioneers first trained microscopes on the profusion of branching structures that connect individual neurons. Even in the tiniest ...

Researchers find monkey brain structure that decides if viewed objects are new or unidentified

August 18, 2017
A team of researchers working at the University of Tokyo School of Medicine has found what they believe is the part of the monkey brain that decides if something that is being viewed is recognizable. In their paper published ...

Artificial neural networks decode brain activity during performed and imagined movements

August 18, 2017
Artificial intelligence has far outpaced human intelligence in certain tasks. Several groups from the Freiburg excellence cluster BrainLinks-BrainTools led by neuroscientist private lecturer Dr. Tonio Ball are showing how ...

Study of nervous system cells can help to understand degenerative diseases

August 18, 2017
The results of a new study show that many of the genes expressed by microglia differ between humans and mice, which are frequently used as animal models in research on Alzheimer's disease and other neurodegenerative disorders.

How whip-like cell appendages promote bodily fluid flow

August 18, 2017
Researchers at Nagoya University have identified a molecule that enables cell appendages called cilia to beat in a coordinated way to drive the flow of fluid around the brain; this prevents the accumulation of this fluid, ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.