Brain imaging reveals the movies in our mind
This set of paired images provided by Shinji Nishimoto of the University of California, Berkeley on Wednesday, Sept. 21, 2011 shows original video images, upper row, and those images reconstructed by computer from brain scans. While volunteers watched movie clips, a scanner watched their brains. And from their brain activity, a computer made rough reconstructions of what they viewed. Scientists reported that result Thursday, Sept. 22, 2011 and speculated such an approach might be able to reveal dreams and hallucinations someday. In the future, it might help stroke victims or others who have no other way to communicate, said Jack Gallant, a neuroscientist at the University of California, Berkeley, and co-author of the paper. (University of California, Berkeley, Shinji Nishimoto)
Imagine tapping into the mind of a coma patient, or watching one's own dream on YouTube. With a cutting-edge blend of brain imaging and computer simulation, scientists at the University of California, Berkeley, are bringing these futuristic scenarios within reach.
Using functional Magnetic Resonance Imaging (fMRI) and computational models, UC Berkeley researchers have succeeded in decoding and reconstructing people's dynamic visual experiences in this case, watching Hollywood movie trailers.
As yet, the technology can only reconstruct movie clips people have already viewed. However, the breakthrough paves the way for reproducing the movies inside our heads that no one else sees, such as dreams and memories, according to researchers.
"This is a major leap toward reconstructing internal imagery," said Professor Jack Gallant, a UC Berkeley neuroscientist and coauthor of the study to be published online Sept. 22 in the journal Current Biology. "We are opening a window into the movies in our minds."
Eventually, practical applications of the technology could include a better understanding of what goes on in the minds of people who cannot communicate verbally, such as stroke victims, coma patients and people with neurodegenerative diseases.
It may also lay the groundwork for brain-machine interface so that people with cerebral palsy or paralysis, for example, can guide computers with their minds.
However, researchers point out that the technology is decades from allowing users to read others' thoughts and intentions, as portrayed in such sci-fi classics as "Brainstorm," in which scientists recorded a person's sensations so that others could experience them.
Previously, Gallant and fellow researchers recorded brain activity in the visual cortex while a subject viewed black-and-white photographs. They then built a computational model that enabled them to predict with overwhelming accuracy which picture the subject was looking at.
In their latest experiment, researchers say they have solved a much more difficult problem by actually decoding brain signals generated by moving pictures.
"Our natural visual experience is like watching a movie," said Shinji Nishimoto, lead author of the study and a post-doctoral researcher in Gallant's lab. "In order for this technology to have wide applicability, we must understand how the brain processes these dynamic visual experiences."
Nishimoto and two other research team members served as subjects for the experiment, because the procedure requires volunteers to remain still inside the MRI scanner for hours at a time.
They watched two separate sets of Hollywood movie trailers, while fMRI was used to measure blood flow through the visual cortex, the part of the brain that processes visual information. On the computer, the brain was divided into small, three-dimensional cubes known as volumetric pixels, or "voxels."
"We built a model for each voxel that describes how shape and motion information in the movie is mapped into brain activity," Nishimoto said.
The brain activity recorded while subjects viewed the first set of clips was fed into a computer program that learned, second by second, to associate visual patterns in the movie with the corresponding brain activity.
Brain activity evoked by the second set of clips was used to test the movie reconstruction algorithm. This was done by feeding 18 million seconds of random YouTube videos into the computer program so that it could predict the brain activity that each film clip would most likely evoke in each subject.
Finally, the 100 clips that the computer program decided were most similar to the clip that the subject had probably seen were merged to produce a blurry yet continuous reconstruction of the original movie.
Reconstructing movies using brain scans has been challenging because the blood flow signals measured using fMRI change much more slowly than the neural signals that encode dynamic information in movies, researchers said. For this reason, most previous attempts to decode brain activity have focused on static images.
"We addressed this problem by developing a two-stage model that separately describes the underlying neural population and blood flow signals," Nishimoto said.
Ultimately, Nishimoto said, scientists need to understand how the brain processes dynamic visual events that we experience in everyday life.
"We need to know how the brain works in naturalistic conditions," he said. "For that, we need to first understand how the brain works while we are watching movies."
Current Biology: http://www.cell.com/current-biology
Movie reconstructions: http://bit.ly/nzm9Tw
Provided by University of California - Berkeley
- Brain cell electrical activity studied Aug 04, 2005 | not rated yet | 0
- Echoes discovered in early visual brain areas play role in working memory Feb 18, 2009 | not rated yet | 0
- Brain scans used to predict behavior Nov 30, 2005 | not rated yet | 0
- New study examines brain processes behind facial recognition Apr 18, 2011 | not rated yet | 0
- Brain's timing linked with timescales of the natural visual world Sep 05, 2007 | not rated yet | 0
- Motion perception revisited: High Phi effect challenges established motion perception assumptions Apr 23, 2013 | 3 / 5 (2) | 2
- Anything you can do I can do better: Neuromolecular foundations of the superiority illusion (Update) Apr 02, 2013 | 4.5 / 5 (11) | 5
- The visual system as economist: Neural resource allocation in visual adaptation Mar 30, 2013 | 5 / 5 (2) | 9
- Separate lives: Neuronal and organismal lifespans decoupled Mar 27, 2013 | 4.9 / 5 (8) | 0
- Sizing things up: The evolutionary neurobiology of scale invariance Feb 28, 2013 | 4.8 / 5 (10) | 14
Pressure-volume curve: Elastic Recoil Pressure don't make sense
1 hour ago From pressure-volume curve of the lung and chest wall (attached photo), I don't understand why would the elastic recoil pressure of the lung is...
If you became brain-dead, would you want them to pull the plug?
17 hours ago I'd want the rest of me to stay alive. Sure it's a lousy way to live but it beats being all-the-way dead. Maybe if I make it 20 years they'll...
MRI bill question
May 15, 2013 Dear PFers, The hospital gave us a $12k bill for one MRI (head with contrast). The people I talked to at the hospital tell me that they do not...
Ratio of Hydrogen of Oxygen in Dessicated Animal Protein
May 13, 2013 As an experiment, for the past few months I've been consuming at least one portion of Jell-O or unflavored Knox gelatin per day. I'm 64, in very...
Alcohol and acetaminophen
May 13, 2013 Edit: sorry for the typo in the title , can't edit I looked around on google quite a bit and it's very hard to find precise information on the...
Marie Curie's leukemia
May 13, 2013 Does anyone know what might be the cause of Marie Curie's cancer
- More from Physics Forums - Medical Sciences
More news stories
For combat veterans suffering from post-traumatic stress disorder, 'fear circuitry' in the brain never rests
Chronic trauma can inflict lasting damage to brain regions associated with fear and anxiety. Previous imaging studies of people with post-traumatic stress disorder, or PTSD, have shown that these brain regions can over-or ...
Neuroscience 3 hours ago | not rated yet | 0 |
The neural machinery underlying our olfactory sense continues to be an enigma for neuroscience. A recent review in Neuron seeks to expand traditional ideas about how neurons in the olfactory bulb might encode information about ...
Neuroscience 14 hours ago | not rated yet | 0 |
(Medical Xpress)—What if the quality of your work depends more on your focus on the piano keys or canvas or laptop than your musical or painting or computing skills? If target users can be convinced, they ...
Neuroscience 14 hours ago | 3.7 / 5 (3) | 0 |
Neurological disorders can have a devastating impact on the lives of sufferers and their families.
Neuroscience 19 hours ago | 5 / 5 (1) | 0 |
If you're a left-brain thinker, chances are you use your right hand to hold your cell phone up to your right ear, according to a newly published study from Henry Ford Hospital in Detroit.
Neuroscience May 16, 2013 | 2 / 5 (2) | 0 |
Big names in medicine are set to give an upbeat assessment of the war on AIDS on Tuesday, 30 years after French researchers identified the virus that causes the disease.
2 hours ago | not rated yet | 0
In 2008 researchers from the University of Southern Denmark showed that the drug thioridazine, which has previously been used to treat schizophrenia, is also a powerful weapon against antibiotic-resistant bacteria such as ...
11 hours ago | 3.7 / 5 (3) | 0 |
(Medical Xpress)—Working with lab mice models of multiple sclerosis (MS), UC Davis scientists have detected a novel molecular target for the design of drugs that could be safer and more effective than current FDA-approved ...
12 hours ago | 5 / 5 (2) | 0 |
In order to avoid harms associated with alcohol consumption, in 2009 the National Institute on Alcohol Abuse and Alcoholism issued guidelines that define low-risk drinking. These guidelines differ for men and women: no more ...
12 hours ago | not rated yet | 0 |
Treatment for alcohol use disorders works best if the patient actively understands and incorporates the interventions provided in the clinic. Multiple factors can influence both the type and degree of neurocognitive abnormalities ...
12 hours ago | not rated yet | 0 |
(HealthDay)—Few randomized clinical trials have been done to assess clinical prediction rules for patients with lower back pain, and the trials that have been done are of low quality and do not provide ...
13 hours ago | not rated yet | 0 |