Paralyzed individuals control robotic arms to reach and grasp using brain computer interface (w/ Video)
In a clinical trial, a woman used the BrainGate system to mentally control a robotic arm and reach for a drink. Credit: The BrainGate Collaboration
In an ongoing clinical trial, a paralyzed woman was able to reach for and sip from a drink on her own for the first time in nearly 15 years by using her thoughts to direct a robotic arm. The trial is evaluating the safety and feasibility of an investigational device called the BrainGate neural interface system. This is a type of brain-computer interface (BCI) intended to put robotics and other assistive technology under the brain's control.
On April 12, 2011, nearly fifteen years after she became paralyzed and unable to speak, a woman controlled a robotic arm by thinking about moving her arm and hand to lift a bottle of coffee to her mouth and take a drink. That achievement is one of the advances in brain-computer interfaces restorative neurotechnology and assistive robot technology described in the May 17 edition of the journal Nature by the BrainGate2 collaboration of researchers at the Department of Veterans Affairs, Brown University, Massachusetts General Hospital, Harvard Medical School., and the German Aerospace Center (DLR).
A 58-year-old woman, "S3," and a 66-year-old man, "T2," participated in the study. They had each been paralyzed by a brainstem stroke years earlier which left them with no functional control of their limbs. In the research, the participants used neural activity to directly control two different robotic arms, one developed by the DLR Institute of Robotics and Mechatronics and the other by DEKA Research and Development Corp., to perform reaching and grasping tasks across a broad three-dimensional space. The BrainGate2 (www.braingate2.org) pilot clinical trial employs the investigational BrainGate system initially developed at Brown University, in which a baby aspirin-sized device with a grid of 96 tiny electrodes is implanted in the motor cortexa part of the brain that is involved in voluntary movement. The electrodes are close enough to individual neurons to record the neural activity associated with intended movement. An external computer translates the pattern of impulses across a population of neurons into commands to operate assistive devices, such as the DLR and DEKA robot arms used in the study now reported in Nature.
BrainGate participants have previously demonstrated neurally based two-dimensional point-and-click control of a cursor on a computer screen and rudimentary control of simple robotic devices.
The study represents the first demonstration and the first peer-reviewed report of people with tetraplegia using brain signals to control a robotic arm in three-dimensional space to complete a task usually performed by their arm. Specifically, S3 and T2 controlled the arms to reach for and grasp foam targets that were placed in front of them using flexible supports. In addition, S3 used the DLR robot to pick up a bottle of coffee, bring it to her mouth, issue a command to tip it, drink through a straw, and return the bottle to the table. Her BrainGate-enabled, robotic-arm control during the drinking task required a combination of two-dimensional movements across a table top plus a "grasp" command to either grasp and lift or tilt the robotic hand.
"Our goal in this research is to develop technology that will restore independence and mobility for people with paralysis or limb loss," said lead author Dr. Leigh Hochberg, a neuroengineer and critical care neurologist who holds appointments at the Department of Veterans Affairs, Brown University, Massachusetts General Hospital and Harvard. He is the Sponsor-Investigator for the BrainGate2 pilot clinical trial. "We have much more work to do, but the encouraging progress of this research is demonstrated not only in the reach-and-grasp data, but even more so in S3's smile when she served herself coffee of her own volition for the first time in almost 15 years."
This video is not supported by your browser at this time.
This is a video showing and discussing results published in Nature from the BrainGate2 pilot clinical trial. Credit: Brown University/Braingate2.orgPartial funding for this work comes from the VA, which is committed to improving the lives of injured Veterans. "VA is honored to have played a role in this exciting and promising area of research," said VA Secretary Eric Shinseki. "Today's announcement represents a great step forward toward improving the quality of life for Veterans and others who have either lost limbs or are paralyzed."
Hochberg adds that even after nearly 15 years, a part of the brain essentially "disconnected" from its original target by a brainstem stroke was still able to direct the complex, multidimensional movement of an external arm in this case, a robotic limb. The researchers also noted that S3 was able to perform the tasks more than five years after the investigational BrainGate electrode array was implanted. This sets a new benchmark for how long implanted brain-computer interface electrodes have remained viable and provided useful command signals.
John Donoghue, the VA and Brown neuroscientist who pioneered BrainGate more than a decade ago and who is co-senior author of the study, said the paper shows how far the field of brain-computer interfaces has come since the first demonstrations of computer control with BrainGate.
"This paper reports an important advance by rigorously demonstrating in more than one participant that precise three-dimensional neural control of robot arms is not only possible, but also repeatable," said Donoghue, who directs the Brown Institute for Brain Science. "We've moved significantly closer to returning everyday functions, like serving yourself a sip of coffee, usually performed effortlessly by the arm and hand, for people who are unable to move their own limbs. We are also encouraged to see useful control more than five years after implant of the BrainGate array in one of our participants. This work is a critical step toward realizing the long-term goal of creating a neurotechnology that will restore movement, control and independence to people with paralysis or limb loss."
In the research, the robots acted as a substitute for each participant's paralyzed arm. The robotic arms responded to the participants' intent to move as they imagined reaching for each foam target. The robot hand grasped the target when the participants imagined a hand squeeze. Because the diameter of the targets was more than half the width of the robot hand openings, the task required the participants to exert precise control. (Movies of these actions are also available on the Nature website.)
The BrainGate array, which is implanted on the motor cortex, comprises nearly 100 electrodes on a chip the size of a baby aspirin. Credit: braingate2.orgIn 158 trials over four days, S3 was able to touch the target within an allotted time in 48.8 percent of the cases using the DLR robotic arm and hand and 69.2 percent of the cases with the DEKA arm and hand, which has the wider grasp. In 45 trials using the DEKA arm, T2 touched the target 95.6 percent of the time. Of the successful touches, S3 grasped the target 43.6 percent of the time with the DLR arm and 66.7 percent of the time with the DEKA arm. T2's grasp succeeded 62.2 percent of the time.
T2 performed the session in this study on his fourth day of interacting with the arm; the prior three sessions were focused on system development. Using his eyes to indicate each letter, he later described his control of the arm: "I just imagined moving my own arm and the [DEKA] arm moved where I wanted it to go."
The study used two advanced robotic arms: the DLR Light-Weight Robot III with DLR five-fingered hand and the DEKA Arm System. The DLR LWR-III, which is designed to assist in recreating actions like the human arm and hand and to interact with human users, could be valuable as an assistive robotic device for people with various disabilities. Patrick van der Smagt, head of bionics and assistive robotics at DLR, director of biomimetic robotics and machine learning labs at DLR and the Technische Universität München, and a co-senior author on the paper said: "This is what we were hoping for with this arm. We wanted to create an arm that could be used intuitively by varying forms of control. The arm is already in use by numerous research labs around the world who use its unique interaction and safety capabilities. This is a compelling demonstration of the potential utility of the arm by a person with paralysis."
DEKA Research and Development developed the DEKA Arm System for amputees, through funding from the United States Defense Advanced Research Projects Agency (DARPA). Dean Kamen, founder of DEKA said, "One of our dreams for the Luke Arm [as the DEKA Arm System is known informally] since its inception has been to provide a limb that could be operated not only by external sensors, but also by more directly thought-driven control. We're pleased about these results and for the continued research being done by the group at the VA, Brown and MGH." The research is aimed at learning how the DEKA arm might be controlled directly from the brain, potentially allowing amputees to more naturally control this prosthetic limb.
Over the past two years, VA has been conducting an optimization study of the DEKA prosthetic arm at several sites, with the cooperation of Veterans and active duty service members who have lost an arm. Feedback from the study is helping DEKA engineers to refine the artificial arm's design and function. "Brain-computer interfaces, such as BrainGate, have the potential to provide an unprecedented level of functional control over prosthetic arms of the future," said Joel Kupersmith, MD, VA Chief Research and Development Officer. "This innovation is an example of federal collaboration at its finest."
Story Landis, director of the National Institute of Neurological Disorders and Stroke, which funded the work in part, noted: "This technology was made possible by decades of investment and research into how the brain controls movement. It's been thrilling to see the technology evolve from studies of basic neurophysiology and move into clinical trials, where it is showing significant promise for people with brain injuries and disorders."
More information: Hochberg LR et al. "Reach and grasp by people with tetraplegia using a neurally controlled robotic arm." Nature, May 17, 2012, Vol. 485, pp. 372-375.
Journal reference: Nature
Provided by Brown University
- Brain-computer interface, developed at Brown, begins new clinical trial Jun 10, 2009 | not rated yet | 0
- BrainGate neural interface system reaches 1,000-day performance milestone Mar 24, 2011 | not rated yet | 0
- Stanford joins BrainGate team developing brain-computer interface to aid people with paralysis Nov 14, 2011 | not rated yet | 0
- 'Smart' prosthetics: restoring independence to people with disabilities Feb 16, 2007 | not rated yet | 0
- Robot arm improves performance of brain-controlled device Dec 14, 2010 | not rated yet | 0
- Motion perception revisited: High Phi effect challenges established motion perception assumptions Apr 23, 2013 | 3 / 5 (2) | 2
- Anything you can do I can do better: Neuromolecular foundations of the superiority illusion (Update) Apr 02, 2013 | 4.5 / 5 (11) | 5
- The visual system as economist: Neural resource allocation in visual adaptation Mar 30, 2013 | 5 / 5 (2) | 9
- Separate lives: Neuronal and organismal lifespans decoupled Mar 27, 2013 | 4.9 / 5 (8) | 0
- Sizing things up: The evolutionary neurobiology of scale invariance Feb 28, 2013 | 4.8 / 5 (10) | 14
Pressure-volume curve: Elastic Recoil Pressure don't make sense
May 18, 2013 From pressure-volume curve of the lung and chest wall (attached photo), I don't understand why would the elastic recoil pressure of the lung is...
If you became brain-dead, would you want them to pull the plug?
May 17, 2013 I'd want the rest of me to stay alive. Sure it's a lousy way to live but it beats being all-the-way dead. Maybe if I make it 20 years they'll...
MRI bill question
May 15, 2013 Dear PFers, The hospital gave us a $12k bill for one MRI (head with contrast). The people I talked to at the hospital tell me that they do not...
Ratio of Hydrogen of Oxygen in Dessicated Animal Protein
May 13, 2013 As an experiment, for the past few months I've been consuming at least one portion of Jell-O or unflavored Knox gelatin per day. I'm 64, in very...
Alcohol and acetaminophen
May 13, 2013 Edit: sorry for the typo in the title , can't edit I looked around on google quite a bit and it's very hard to find precise information on the...
Marie Curie's leukemia
May 13, 2013 Does anyone know what might be the cause of Marie Curie's cancer
- More from Physics Forums - Medical Sciences
More news stories
In their quest to learn more about the variability of cells between and within tissues, biomedical scientists have devised tools capable of simultaneously measuring dozens of characteristics of individual ...
Medical research 12 hours ago | 5 / 5 (3) | 0 |
In 2008 researchers from the University of Southern Denmark showed that the drug thioridazine, which has previously been used to treat schizophrenia, is also a powerful weapon against antibiotic-resistant bacteria such as ...
Medical research May 17, 2013 | 4 / 5 (4) | 0 |
Scientists investigating the interaction of a group of proteins in the brain responsible for protecting nerve cells from damage have identified a new target that could increase cell survival.
Medical research May 17, 2013 | 5 / 5 (1) | 0
New findings by researchers carrying out experiments at the U.S. Department of Energy Office of Science's Advanced Photon Source (APS) help explain why some drugs that interact with two kinds of human serotonin ...
Medical research May 17, 2013 | 4 / 5 (1) | 0 |
Scientists at Johns Hopkins have turned their view of osteoarthritis (OA) inside out. Literally. Instead of seeing the painful degenerative disease as a problem primarily of the cartilage that cushions joints, ...
12 hours ago | 4.8 / 5 (4) | 0 |
Researchers at the University of Wisconsin have identified a potential new risk factor for obstructive sleep apnea: asthma. Using data from the National Institutes of Health (Heart, Lung, and Blood Institute)-funded Wisconsin ...
11 hours ago | 5 / 5 (1) | 0 |
A new study looking at sleep-disordered breathing (SDB) and markers for Alzheimer's disease (AD) risk in cerebrospinal fluid (CSF) and neuroimaging adds to the growing body of research linking the two.
12 hours ago | 5 / 5 (2) | 0 |
The hunt for an HIV vaccine has gobbled up $8 billion in the past decade, and the failure of the most recent efficacy trial has delivered yet another setback to 26 years of efforts.
16 hours ago | not rated yet | 0
Gourmands and foodies everywhere have long recognized ginger as a great way to add a little peppery zing to both sweet and savory dishes; now, a study from researchers at Columbia University shows purified components of the ...
11 hours ago | 5 / 5 (1) | 0
The devastating effect of Alzheimer's disease on bilingual people has been thrown into focus in Canada, where the sudden loss of a second language can leave sufferers feeling like strangers in their own country.
14 hours ago | not rated yet | 0