Researchers reveal how the brain remains focused on long-term goals

August 4, 2013, Massachusetts Institute of Technology
brain

As anyone who has traveled with young children knows, maintaining focus on distant goals can be a challenge. A new study from MIT suggests how the brain achieves this task, and indicates that the neurotransmitter dopamine may signal the value of long-term rewards. The findings may also explain why patients with Parkinson's disease—in which dopamine signaling is impaired—often have difficulty in sustaining motivation to finish tasks.

The work is described this week in the journal Nature.

Previous studies have linked to rewards, and have shown that show brief bursts of activity when animals receive an unexpected reward. These dopamine signals are believed to be important for reinforcement learning, the process by which an animal learns to perform actions that lead to reward.

Taking the long view

In most studies, that reward has been delivered within a few seconds. In real life, though, gratification is not always immediate: Animals must often travel in search of food, and must maintain motivation for a distant goal while also responding to more immediate cues. The same is true for humans: A driver on a long road trip must remain focused on reaching a final destination while also reacting to traffic, stopping for snacks, and entertaining children in the back seat.

The MIT team, led by Institute Professor Ann Graybiel—who is also an investigator at MIT's McGovern Institute for Brain Research—decided to study how dopamine changes during a maze task approximating work for . The researchers trained rats to navigate a maze to reach a reward. During each trial a rat would hear a tone instructing it to turn either right or left at an intersection to find a chocolate milk reward.

Rather than simply measuring the activity of dopamine-containing neurons, the MIT researchers wanted to measure how much dopamine was released in the , a known to be important in reinforcement learning. They teamed up with Paul Phillips of the University of Washington, who has developed a technology called fast-scan cyclic voltammetry (FSCV) in which tiny, implanted, carbon-fiber electrodes allow continuous measurements of dopamine concentration based on its electrochemical fingerprint.

"We adapted the FSCV method so that we could measure dopamine at up to four different sites in the brain simultaneously, as animals moved freely through the maze," explains first author Mark Howe, a former graduate student with Graybiel who is now a postdoc in the Department of Neurobiology at Northwestern University. "Each probe measures the concentration of extracellular dopamine within a tiny volume of brain tissue, and probably reflects the activity of thousands of nerve terminals."

Gradual increase in dopamine

From previous work, the researchers expected that they might see pulses of dopamine released at different times in the trial, "but in fact we found something much more surprising," Graybiel says: The level of dopamine increased steadily throughout each trial, peaking as the animal approached its goal—as if in anticipation of a reward.

The rats' behavior varied from trial to trial—some runs were faster than others, and sometimes the animals would stop briefly—but the dopamine signal did not vary with running speed or trial duration. Nor did it depend on the probability of getting a reward, something that had been suggested by previous studies.

"Instead, the dopamine signal seems to reflect how far away the rat is from its goal," Graybiel explains. "The closer it gets, the stronger the signal becomes." The researchers also found that the size of the signal was related to the size of the expected reward: When rats were trained to anticipate a larger gulp of chocolate milk, the dopamine signal rose more steeply to a higher final concentration.

In some trials the T-shaped maze was extended to a more complex shape, requiring animals to run further and to make extra turns before reaching a reward. During these trials, the dopamine signal ramped up more gradually, eventually reaching the same level as in the shorter maze. "It's as if the animal were adjusting its expectations, knowing that it had further to go," Graybiel says.

An 'internal guidance system'

"This means that dopamine levels could be used to help an animal make choices on the way to the goal and to estimate the distance to the goal," says Terrence Sejnowski of the Salk Institute, a computational neuroscientist who is familiar with the findings but who was not involved with the study. "This 'internal guidance system' could also be useful for humans, who also have to make choices along the way to what may be a distant goal."

One question that Graybiel hopes to examine in future research is how the signal arises within the brain. Rats and other animals form cognitive maps of their spatial environment, with so-called "place cells" that are active when the animal is in a specific location. "As our rats run the maze repeatedly," she says, "we suspect they learn to associate each point in the maze with its distance from the reward that they experienced on previous runs."

As for the relevance of this research to humans, Graybiel says, "I'd be shocked if something similar were not happening in our own brains." It's known that Parkinson's patients, in whom dopamine signaling is impaired, often appear to be apathetic, and have difficulty in sustaining motivation to complete a long task. "Maybe that's because they can't produce this slow ramping dopamine signal," Graybiel says.

Explore further: Two dimensions of value: Dopamine neurons represent reward but not aversiveness

More information: Prolonged dopamine signalling in striatum signals proximity and value of distant rewards, Nature, DOI: 10.1038/nature12475

Related Stories

Two dimensions of value: Dopamine neurons represent reward but not aversiveness

August 1, 2013
To make decisions, we need to estimate the value of sensory stimuli and motor actions, their "goodness" and "badness." We can imagine that good and bad are two ends of a single continuum, or dimension, of value. This would ...

Brain implant aims to stifle drug highs

July 29, 2013
What happens if addicts get no high from the drugs they take? Researchers at Case Western Reserve and Illinois State universities have received a $390,000 National Institute on Drug Abuse grant to help answer the question.

Breaking down the Parkinson's pathway: How affected brain cells respond during different behavioral tasks

March 13, 2013
The key hallmark of Parkinson's disease is a slowdown of movement caused by a cutoff in the supply of dopamine to the brain region responsible for coordinating movement. While scientists have understood this general process ...

Extroverts have more sensitive brain-reward system

July 11, 2013
Extroverts may be more outgoing and cheerful in part because of their brain chemistry, reports a study by Cornell neuroscientists.

Reward linked to image is enough to activate brain's visual cortex

March 21, 2013
Once rhesus monkeys learn to associate a picture with a reward, the reward by itself becomes enough to alter the activity in the monkeys' visual cortex. This finding was made by neurophysiologists Wim Vanduffel and John Arsenault ...

Alcoholism could be linked to a hyper-active brain dopamine system

August 2, 2013
Research from McGill University suggests that people who are vulnerable to developing alcoholism exhibit a distinctive brain response when drinking alcohol, according to a new study by Prof. Marco Leyton, of McGill University's ...

Recommended for you

Research reveals atomic-level changes in ALS-linked protein

January 18, 2018
For the first time, researchers have described atom-by-atom changes in a family of proteins linked to amyotrophic lateral sclerosis (ALS), a group of brain disorders known as frontotemporal dementia and degenerative diseases ...

Fragile X finding shows normal neurons that interact poorly

January 18, 2018
Neurons in mice afflicted with the genetic defect that causes Fragile X syndrome (FXS) appear similar to those in healthy mice, but these neurons fail to interact normally, resulting in the long-known cognitive impairments, ...

How your brain remembers what you had for dinner last night

January 17, 2018
Confirming earlier computational models, researchers at University of California San Diego and UC San Diego School of Medicine, with colleagues in Arizona and Louisiana, report that episodic memories are encoded in the hippocampus ...

Recording a thought's fleeting trip through the brain

January 17, 2018
University of California, Berkeley neuroscientists have tracked the progress of a thought through the brain, showing clearly how the prefrontal cortex at the front of the brain coordinates activity to help us act in response ...

Midbrain 'start neurons' control whether we walk or run

January 17, 2018
Locomotion comprises the most fundamental movements we perform. It is a complex sequence from initiating the first step, to stopping when we reach our goal. At the same time, locomotion is executed at different speeds to ...

A 'touching sight': How babies' brains process touch builds foundations for learning

January 16, 2018
Touch is the first of the five senses to develop, yet scientists know far less about the baby's brain response to touch than to, say, the sight of mom's face, or the sound of her voice.

2 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

Gmr
5 / 5 (2) Aug 04, 2013
Take a look at the "anhedonia" component of depression as well. It's helped sometimes by administration of dopaminergic compounds.
kochevnik
3 / 5 (4) Aug 04, 2013
An amazing find! Someone should compile a compendium of these useful hormones such as cortisol and oxytocin and serotonin. It seems neurobiology is outdating old cannons as fast as computer science

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.