Dopamine fluctuations in distinct brain subregions predict rewards over a range of time-scales

Previous neuroscience studies suggest that transient increases in brain dopamine are critical signals for learning about reward, and the motivation to obtain more rewards. Researchers at University of California San Francisco carried out a study exploring transient increases in dopamine in specific subregions of the striatum, a part of the brain known to support reward-based learning and decision-making.

Their findings, published in Nature Neuroscience, suggest that dopamine transient fluctuations in three different parts of striatum reflect reward predictions over different time horizons (i.e., a fraction of a second, tens of seconds and hundreds of seconds later, respectively).

"A dopamine pulse may indicate that we have found ourselves in a better situation than we expected, and so our prior estimates of reward need to be updated," Josh Berke, lead author of the paper, told Medical Xpress. "This 'reward prediction error' theory has been very influential, in part because it connects brain activity patterns to certain computations in machine learning, yet there are several aspects of this theory that are, at best, incomplete."

The recent study by Berke and his collaborators was aimed at addressing two problematic aspects of this 'reward prediction error' theory. The first of these is the unclear definition of the term 'reward prediction,' which does not specify when a reward is expected (e.g., within one second, one minute, one hour, etc.).

"A second problem is that dopamine signals were originally thought to be broadcast uniformly throughout the forebrain, but more recent studies have found different dopamine signals at different places in the brain," Berke said. "So, does this indicate that we need many different theories to explain these different dopamine signals?"

To shed light on unclear aspects of reward prediction error theory, Berke and his colleagues carried out experiments on rats using a recently developed molecular sensor. This sensor utilizes a genetically engineered protein that changes its fluorescence when bound to dopamine.

"We deployed this sensor in three different areas within the rat striatum, the brain area that receives the strongest dopamine input," Berke explained. "These different areas are part of distinct, large-scale loop circuits in the brain, which process different types of information."

The researchers examined dopamine fluctuations, highlighted by their molecular sensor, as rats performed a range of behavioral tasks. The rats received rewards at varying rates and heard cues that indicated upcoming rewards with different delays and probabilities.

Interestingly, they found that dopamine fluctuations in each of the three striatal subregions they examined reflected different time scales for the rats' expectation of rewards.

"In one area, most concerned with motor control, dopamine fluctuates frequently, and the response to a reward-predictive cue is strong only if it predicts reward delivery within a fraction of a second," Berke said. "A second striatal area seems to care about rewards within tens of seconds, and a third about hundreds of seconds. We think there may be a continuous gradient of reward prediction time scales, involving parallel circuits within the brain."

The different time scales and associated striatal subregions unveiled by Berke and his colleagues could explain previously reported puzzling experimental observations recording different dopamine signals, all with a single theoretical framework. In addition, the existence of different reward prediction time scales could underpin some of the complex and apparently incoherent behaviors observed in animals.

"For example, when singing a song, there is very little time separation between moving our vocal cords and hearing the pleasant (or otherwise) result," Berke said. "This feedback must be fast for effective learning. However, sometimes we make choices and don't find out the results for a considerable time. We need brain mechanisms to overcome this time gap, to determine whether the choice was a good one."

The new insight gathered by this team of researchers could help to better understand the link between dopamine transients in striatal subregions and reward-based learning. In addition, they could shed light on some of the processes underlying decision-making.

"Often, we make resolutions about how to act over the long term (e.g., to lose weight), but when faced with an immediate choice we don't act accordingly," Berke explained.

"This mismatch has long been studied and was considered 'irrational,' However, it's possible that this is an inevitable consequence of having multiple decision sub-systems operating in parallel, each concerned with a different duration of the future. As potential outcomes get closer in time, more sub-systems get involved, and push for short-term results."

This study could soon inform the development of new theoretical models explaining the prediction of rewards at different time scales. Meanwhile, Berke and his collaborators plan to build on their findings to conduct further experiments looking at dopamine signals and their interaction with other neural circuits.

"We're now investigating how these dopamine signals interact with other circuit components, as part of developing a richer understanding of how these circuits work and how they go wrong in disorders such as addiction, Parkinson's Disease, and Tourette Syndrome. We also have an active program studying how we imagine future possibilities and adjust our behavior accordingly."

© 2024 Science X Network