Monkeys feel, move virtual objects using only their brains (w/ video)

A rhesus macaque monkey is seen in Hong Kong in April 2011. Monkeys implanted with brain electrodes were able to see and move a virtual object and sense the texture of what they saw, a step forward in the quest to help the severely paralysed touch the outside world once more.

(Medical Xpress) -- In a first ever demonstration of a two-way interaction between a primate brain and a virtual body, two monkeys trained at the Duke University Center for Neuroengineering learned to employ brain activity alone to move an avatar hand and identify the texture of virtual objects.

"Someday in the near future, quadriplegic patients will take advantage of this technology not only to move their arms and hands and to walk again, but also to sense the texture of objects placed in their hands, or experience the nuances of the terrain on which they stroll with the help of a wearable ," said Miguel Nicolelis, M.D., Ph.D., professor of neurobiology at Duke University Medical Center and co-director of the Duke Center for , who was senior author of the study.

This video is not supported by your browser at this time.

Without moving any part of their real bodies, the monkeys used their electrical brain activity to direct the virtual hands of an avatar to the surface of and, upon contact, were able to differentiate their textures.

Although the virtual objects employed in this study were visually identical, they were designed to have different artificial textures that could only be detected if the animals explored them with virtual hands controlled directly by their brain's electrical activity.

The texture of the virtual objects was expressed as a pattern of minute transmitted to the monkeys' brains. Three different electrical patterns corresponded to each of three different object textures.

Because no part of the animal's real body was involved in the operation of this brain-machine-brain interface (BMBI), these experiments suggest that in the future patients severely paralyzed due to a spinal cord lesion may take advantage of this technology, not only to regain mobility, but also to have their restored, said Nicolelis, who was senior author of the study published in the journal Nature on Oct. 5.

"This is the first demonstration of a brain-machine-brain interface that establishes a direct, bidirectional link between a brain and a virtual body," Nicolelis said. "In this BMBI, the virtual body is controlled directly by the animal's brain activity, while its virtual hand generates tactile feedback information that is signaled via direct electrical microstimulation of another region of the animal's cortex."

"We hope that in the next few years this technology could help to restore a more autonomous life to many patients who are currently locked in without being able to move or experience any tactile sensation of the surrounding world," Nicolelis said.

"This is also the first time we've observed a brain controlling a virtual arm that explores objects while the brain simultaneously receives electrical feedback signals that describe the fine texture of objects 'touched' by the monkey's newly acquired virtual hand," Nicolelis said. "Such an interaction between the brain and a virtual avatar was totally independent of the animal's real body, because the animals did not move their real arms and hands, nor did they use their real skin to touch the objects and identify their texture. It's almost like creating a new sensory channel through which the brain can resume processing information that cannot reach it anymore through the real body and peripheral nerves."

The combined of populations of 50-200 neurons in the monkey's motor cortex controlled the steering of the avatar arm, while thousands of neurons in the primary tactile cortex were simultaneously receiving continuous electrical feedback from the virtual hand's palm that let the monkey discriminate between objects, based on their texture alone.

"The remarkable success with non-human primates is what makes us believe that humans could accomplish the same task much more easily in the near future," Nicolelis said.

It took one monkey only four attempts and another nine attempts before they learned how to select the correct object during each trial. Several tests demonstrated that the monkeys were actually sensing the object and not selecting them randomly.

The findings provide further evidence that it may be possible to create a robotic exoskeleton that severely paralyzed patients could wear in order to explore and receive feedback from the outside world, Nicolelis said. Such an exoskeleton would be directly controlled by the patient's voluntary brain activity in order to allow the patient to move autonomously. Simultaneously, sensors distributed across the exoskeleton would generate the type of tactile feedback needed for the patient's to identify the texture, shape and temperature of objects, as well as many features of the surface upon which they walk.

This overall therapeutic approach is the one chosen by the Walk Again Project, an international, non-profit consortium, established by a team of Brazilian, American, Swiss, and German scientists, which aims at restoring full body mobility to quadriplegic patients through a brain-machine-brain interface implemented in conjunction with a full-body robotic exoskeleton.

The international scientific team recently proposed to carry out its first public demonstration of such an autonomous exoskeleton during the opening game of the 2014 FIFA Soccer World Cup that will be held in Brazil.

More information: O’Doherty, J. E. et al. Nature advance online publication, dx.doi.org/10.1038/nature10489 (2011).

Related Stories

Monkey's Thoughts Make Robot Walk from Across the Globe

Jan 17, 2008

In a first-of-its-kind experiment, the brain activity of a monkey has been used to control the real-time walking patterns of a robot halfway around the world, according to researchers at Duke University Medical Center.

How to leave your body

Feb 20, 2011

Leave your body and shake hands with yourself, gain an extra limb or change into a robot for a while. Swedish neuroscientist Henrik Ehrsson has demonstrated that the brain's image of the body is negotiable. Applications stretch ...

Robotic exoskeleton replaces muscle work

Feb 08, 2007

A robotic exoskeleton controlled by the wearer's own nervous system could help users regain limb function, which is encouraging news for people with partial nervous system impairment, say University of Michigan researchers.

Recommended for you

Turning off depression in the brain

17 hours ago

Scientists have traced vulnerability to depression-like behaviors in mice to out-of-balance electrical activity inside neurons of the brain's reward circuit and experimentally reversed it – but there's ...

Rapid whole-brain imaging with single cell resolution

18 hours ago

A major challenge of systems biology is understanding how phenomena at the cellular scale correlate with activity at the organism level. A concerted effort has been made especially in the brain, as scientists are aiming to ...

User comments

Adjust slider to filter visible comments by rank

Display comments: newest first

Isaacsname
not rated yet Oct 05, 2011
Oh yeah. Wait until they can do this with all sorts of animals. Insects even. Add some simple operant conditioning ala Skinner and they could be assigned vitually endless menial tasks while interacting with simple machine/computer interfaces.

The integration of technology and nature is the future, when design is based on nature in the first place, and our tech is emmulation at best ( imitation is the sincerest form of flattery ), integration and merging of the two will yield optimizations.

Pirouette
not rated yet Oct 05, 2011
This brings to mind an episode of "Star Trek" from the 1960s whereby Spock hijacks the Enterprise in order to bring Captain Pike, who was rendered a total invalid in a wheelchair by an enemy, to a planet of indigenous beautiful female Extraterrestrials. The E.T.s have a particular gift of mentally stimulating Captain Pike to believe that he is walking about on the planet and interacting (ahem) with the "girls", even though he, in reality, is still in his wheelchair. Spock is court-martialed but is exonerated when it is discovered that his true motive was humanitarian in nature. Carry on, researchers!!
jselin
not rated yet Oct 06, 2011
@Isaacsname

How about ants collecting gold flakes or other minerals of interest. I wonder how much energy surplus they carry or if their food foraging needs to be constant
Isaacsname
5 / 5 (1) Oct 06, 2011
@Isaacsname

How about ants collecting gold flakes or other minerals of interest. I wonder how much energy surplus they carry or if their food foraging needs to be constant


Sure, yes, finding landmines, etc. All animals, and even insects to some extent are inately curious about things. The specific dynamics of the interfaces I think will be easier than the translation of purpose and necessity. Just because we can finally interface with other life doesn't mean we would know what to say, this is the next hill, imo.

I don't think it will come any easier until we can resolve exactly who WE are, and how we see ourselves in relation to other life.