The image on the left shows all neurons (the black dots) in the rat striatum, a part of the brain that is involved in higher-level decision-making. The image on the right shows just the cholinergic interneurons. There are far fewer black dots because cholinergic interneurons make up only 1 to 2 percent of the neurons in the striatum. It is these neurons that influence mental flexibility. (The large white spots are bundles of nerves.) Credit: OIST

Behavioral flexibility—the ability to change strategy when the rules change—is controlled by specific neurons in the brain, Researchers at the Okinawa Institute of Science and Technology Graduate University (OIST) have confirmed. Cholinergic interneurons are rare—they make up just one to two percent of the neurons in the striatum, a key part of the brain involved with higher-level decision-making. Scientists have suspected they play a role in changing strategies, and researchers at OIST recently confirmed this with experiments. Their findings were published in The Journal of Neuroscience.

"Not much is known about these ," said Sho Aoki, a post-doctoral researcher at OIST and lead author of the paper. "But we now have clear evidence that they play a key role in remaining flexible in this ever-changing world."

Previous studies tried to identify the role of cholinergic interneurons by recording during behavioral tasks. While that can strongly indicate specific neurons are correlated with a particular behavior, it is not definitive. In this study, Aoki killed cholinergic interneurons with a toxin that directly targets them, and then observed how rats reacted to rule changes compared with normal rats with intact neurons. "Our experiments show direct causation, not correlation," Aoki said.

Rats with and without damaged neurons were given tasks for several weeks—they had to press either lever A or B to get a sugar pellet reward. During the first few days, Lever A always resulted in a reward. Both groups of rats had no problem learning the initial strategy to get the sugar pellet—press Lever A.

But then, the rules of the game changed. A novel stimulus was introduced—a light flashed above the correct lever, which oscillated between Lever A and B. To get their sugar fix, the rats had to shift strategy and pay attention to this new information. While normal rats quickly responded to the light, rats with damaged neurons could not. The latter group continued to repeat the strategy they had already learned, and were disinclined to explore what the light meant.

A schematic shows the two key behavioral tests presented to the rats. In both tests, rats were initially required to go to a right lever for a sugar pellet reward, and subsequently they had to change a strategy following a light cue that directs a correct lever. In one of the tests, there was initially no light when rats chose a right lever. But then a light started to shine above the correct lever, which switched randomly between left and right. In another test, a light shone above the either lever, but was irrelevant to the correct lever choice. Sometimes it lined up with the correct lever, and sometimes it did not. However, then the light did line up with the correct choice. In both tests, rats with damaged cholinergic interneurons could learn the initial strategy, but could not change strategies to adapt to the new rules. Credit: OIST

In another test, a light cue that had been flashing in a meaningless pattern during the initial learning phase switched to signaling the correct lever to push for reward. This meant to maximize rewards, and the animals should now pay attention to a stimulus they previously ignored. Again, the control rats had no problem adapting to this rule change, but the damaged rats stuck to their original strategy, even though it meant fewer rewards. They also decreased exploring what might increase their chance of success.

Interestingly, with neurons damaged in the dorsomedial part of the striatum had greater difficulty paying attention to previously irrelevant light cues. Rats with neurons damaged in the ventral part of the striatum had a harder time reacting to novel stimulants.

"This indicates that cholinergic interneurons throughout the striatum play a common role, namely inhibiting old rules and encouraging exploration, but different regions of the striatum are activated depending on the situation and type of stimulus," Aoki said.

The research findings might help researchers and medical professionals who investigate aging. "Since cholinergic interneurons degenerate with age, this work may provide a clue for understanding the decline in mental flexibility that occurs with advancing age," said professor Jeff Wickens, head of OIST's Neurobiology Research Unit and senior paper author.

Journal information: Journal of Neuroscience