This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

peer-reviewed publication

trusted source

proofread

Study sheds new light on the contribution of dopamine to reinforcement learning

Study sheds new light on the contribution of dopamine to reinforcement learning
A physiologically-relevant frequency of dopamine stimulation (20Hz) does not function as a meaningful reward, however, high-frequency dopamine stimulation (50Hz) functions as a reward that is encoded as a specific sensory event. Top: Histological verification with A) bilateral Cre-dependent ChR2 expression in TH-Cre rats, B) colocalization of TH and virus expression approached ~90%, and C) schematic of minimum and maximum virus expression and fiber placement. Left column: Schematic illustrating the task design using one counterbalancing example, which consisted of D) Pavlovian conditioning, E) Instrumental Conditioning, and F) the PIT test. Rats first learnt that two auditory cues (e.g. click and white noise) led to two outcomes (e.g. dopamine stimulation and pellets), then they learnt to perform two lever presses that led to the two outcomes. Finally, rats were presented with the two auditory cues and given an opportunity to press either lever, without reward feedback. Middle column: G) Rats in the 20Hz group (n=6) showed an increase in food-port entries during the pellet-paired stimulus, but not the dopamine-paired stimulus. These rats showed equivalent increases in locomotor activity across learning to both stimuli. H) During instrumental conditioning, where rats learned to make lever presses for the two outcomes, rats in the 20Hz group showed robust lever pressing responses for the pellets, but not the dopamine stimulation. I) In the final PIT test, when the pellet-paired cue is presented, these rats showed significant elevations in responding on the pellet-paired lever, indicating specific PIT. However, they did not show PIT for the dopamine paired cue. Right column: J) Rats in the 50Hz group (n=5) showed increases in food port entries during the pellet-paired stimulus, but not the dopamine-paired stimulus. Increases in locomotor activity across learning was similar for both the dopamine- and pellet-paired stimulus. K) During instrumental training, the 50Hz group showed robust lever pressing for both the dopamine stimulation and the pellets. L) In the final PIT test, the dopamine- and pellet-paired stimuli both produced robust specific PIT. Error bars =SEM. Credit: Millard et al.

The neurotransmitter dopamine has often been linked to pleasure-seeking behaviors and making stimuli paired with rewards (e.g., food, drinks) valuable. Nonetheless, the processes through which this key chemical messenger contributes to learning have not yet been fully elucidated.

Researchers at University of California Los Angeles, University of Sydney, and the State University of New Jersey recently carried out a study aimed at better understanding how (i.e., brain cells supporting the production of dopamine) support reward-based learning. Their findings, published in Nature Neuroscience, suggest that rather than representing the value attributed to different stimuli, these neurons contribute to the formation of new mental associations between stimuli and reward (or other neutral stimuli), which help us form cognitive maps of our environment.

"Our recent research has shown that firing of dopamine neurons act as the brain's teaching signal," Melissa Sharpe, co-author of the paper, told Medical Xpress. "This occurs whenever something new or salient happens, which helps us learn to associate events together to make a new memory. Critically, we have shown that dopamine neurons do this without making things 'valuable' or 'good' in and of themselves."

This work is at odds with past studies that have defined dopamine as the neurotransmitter producing "happiness" or "pleasure." However, if dopaminergic neurons do not carry value signals, they should be unable to attribute positive or pleasurable qualities to specific experiences or actions.

"We were wondering, if dopamine neurons don't carry a value signal, then how do they support intracranial self-stimulation, which suggests dopamine neurons carry a value signal?" Dr. Sharpe explained. "Our experiments were thus aimed at answering the question: If dopamine neurons do indeed carry value in the context of intra-cranial self-stimulation, what is the cognitive representation that allows [them] to do so?"

To answer this research question, Dr. Sharpe and her colleagues carried out a series of experiments on rats. During these experiments, they employed a Pavlovian-to-Instrumental transfer procedure, a well-known experimental test designed to elucidate the cognitive representations driving animal or human behavior.

"We teach rats that a cue (e.g., a tone or click) leads to a particular outcome (e.g., dopamine stimulation or a food pellet)," Dr. Sharpe said. "So, when the tone or click is played, one of these outcomes occurs (e.g., tone —> dopamine stimulation). Then, we teach them that they can earn these outcomes by pressing one of two levers. If the tone makes them think of the 'specific' outcome it was paired with (e.g., dopamine stimulation), they will selectively increase pressing of the lever associated with the dopamine stimulation (and not the food)."

The experiments run by Dr. Sharpe and her colleagues yielded several interesting findings. First, the researchers found that a physiological firing rate of dopamine neurons did not support intracranial self-stimulation in a way that would suggest dopamine neurons carry a value signal.

However, they observed that if they made dopaminergic neurons fire above this physiological rate, the firing of these neurons could function as a sensory-specific goal towards which the animals would exhibit behavior. That is, high frequency of firing in dopamine neurons could function as a reward that ultimately drove the rats to engage in the pleasure-seeking behaviors associated with the so-called Pavlovian-to-Instrumental transfer effect.

"This suggests that when dopamine neurons fire in , they're not making things valuable," Dr. Sharpe explained. "Instead, they function to help us form new memories or how things in our environment are related. In a case where dopamine neurons fire more than they are supposed to (e.g., when taking drugs of abuse), this may be encoded in the brain as a rewarding event that makes us more likely to seek out drugs in future."

Overall, this recent study by Dr. Sharpe and her colleagues could greatly contribute to the understanding of dopamine and its role in reward-based (i.e., reinforcement) learning. In particular, their findings suggest that dopamine neurons do not carry value signals that attach pleasure or happiness to stimuli in the environment. In the future, they could pave the way for additional experiments aimed at further validating the team's findings or examining the unique contribution of specific dopamine-producing neural circuits.

"Our team is now interested in how different dopamine circuits contribute to different types of learning and how this helps us to create a complex but unified representation of our environment," Dr. Sharpe added.

More information: Samuel J. Millard et al, Cognitive representations of intracranial self-stimulation of midbrain dopamine neurons depend on stimulation frequency, Nature Neuroscience (2024). DOI: 10.1038/s41593-024-01643-1

Journal information: Nature Neuroscience

© 2024 Science X Network

Citation: Study sheds new light on the contribution of dopamine to reinforcement learning (2024, June 2) retrieved 20 July 2024 from https://medicalxpress.com/news/2024-05-contribution-dopamine.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Study finds that dopamine projections to the amygdala contribute to encoding identity-specific reward memories

64 shares

Feedback to editors