Study sheds new light on the contribution of dopamine to reinforcement learning

The neurotransmitter dopamine has often been linked to pleasure-seeking behaviors and making stimuli paired with rewards (e.g., food, drinks) valuable. Nonetheless, the processes through which this key chemical messenger contributes to learning have not yet been fully elucidated.

Researchers at University of California Los Angeles, University of Sydney, and the State University of New Jersey recently carried out a study aimed at better understanding how dopaminergic neurons (i.e., brain cells supporting the production of dopamine) support reward-based learning. Their findings, published in Nature Neuroscience, suggest that rather than representing the value attributed to different stimuli, these neurons contribute to the formation of new mental associations between stimuli and reward (or other neutral stimuli), which help us form cognitive maps of our environment.

"Our recent research has shown that firing of dopamine neurons act as the brain's teaching signal," Melissa Sharpe, co-author of the paper, told Medical Xpress. "This occurs whenever something new or salient happens, which helps us learn to associate events together to make a new memory. Critically, we have shown that dopamine neurons do this without making things 'valuable' or 'good' in and of themselves."

This work is at odds with past studies that have defined dopamine as the neurotransmitter producing "happiness" or "pleasure." However, if dopaminergic neurons do not carry value signals, they should be unable to attribute positive or pleasurable qualities to specific experiences or actions.

"We were wondering, if dopamine neurons don't carry a value signal, then how do they support intracranial self-stimulation, which suggests dopamine neurons carry a value signal?" Dr. Sharpe explained. "Our experiments were thus aimed at answering the question: If dopamine neurons do indeed carry value in the context of intra-cranial self-stimulation, what is the cognitive representation that allows [them] to do so?"

To answer this research question, Dr. Sharpe and her colleagues carried out a series of experiments on rats. During these experiments, they employed a Pavlovian-to-Instrumental transfer procedure, a well-known experimental test designed to elucidate the cognitive representations driving animal or human behavior.

"We teach rats that a cue (e.g., a tone or click) leads to a particular outcome (e.g., dopamine stimulation or a food pellet)," Dr. Sharpe said. "So, when the tone or click is played, one of these outcomes occurs (e.g., tone —> dopamine stimulation). Then, we teach them that they can earn these outcomes by pressing one of two levers. If the tone makes them think of the 'specific' outcome it was paired with (e.g., dopamine stimulation), they will selectively increase pressing of the lever associated with the dopamine stimulation (and not the food)."

The experiments run by Dr. Sharpe and her colleagues yielded several interesting findings. First, the researchers found that a physiological firing rate of dopamine neurons did not support intracranial self-stimulation in a way that would suggest dopamine neurons carry a value signal.

However, they observed that if they made dopaminergic neurons fire above this physiological rate, the firing of these neurons could function as a sensory-specific goal towards which the animals would exhibit behavior. That is, high frequency of firing in dopamine neurons could function as a reward that ultimately drove the rats to engage in the pleasure-seeking behaviors associated with the so-called Pavlovian-to-Instrumental transfer effect.

"This suggests that when dopamine neurons fire in everyday life, they're not making things valuable," Dr. Sharpe explained. "Instead, they function to help us form new memories or how things in our environment are related. In a case where dopamine neurons fire more than they are supposed to (e.g., when taking drugs of abuse), this may be encoded in the brain as a rewarding event that makes us more likely to seek out drugs in future."

Overall, this recent study by Dr. Sharpe and her colleagues could greatly contribute to the understanding of dopamine and its role in reward-based (i.e., reinforcement) learning. In particular, their findings suggest that dopamine neurons do not carry value signals that attach pleasure or happiness to stimuli in the environment. In the future, they could pave the way for additional experiments aimed at further validating the team's findings or examining the unique contribution of specific dopamine-producing neural circuits.

"Our team is now interested in how different dopamine circuits contribute to different types of learning and how this helps us to create a complex but unified representation of our environment," Dr. Sharpe added.

© 2024 Science X Network