Neuroscientist uses AI to map learning, decision-making, to discover how brains work

With trial and error, repetition and praise, when a puppy hears "Sit," they learn what they're expected to do. That's reinforcement learning, and it's a complex subject that fascinates neuroscientist Ryoma Hattori, Ph.D., who recently joined The Herbert Wertheim UF Scripps Institute for Biomedical Innovation & Technology.

Hattori focuses on understanding and mapping reinforcement learning and how the brain integrates information to make decisions. He also studies how the brain comprehends numbers. What seems simple on its face is actually stunningly complex. The human brain has approximately 86 billion neurons, which make more than 100 trillion connections.

Hattori says many factors play into the decision-making process. Something as simple as deciding where to eat may involve a matrix of memories and judgments, and hence, many areas of the brain. One restaurant has good food and service, another, so-so. One has higher prices, another is cheaper. Experience provides the inputs that must be assigned values and considered for the decision to be made.

"It's very difficult to integrate all of these processes, and yet, somehow, our brains do that," Hattori says.

Understanding the mechanisms that underlie this process may prove important in addressing psychiatric and autism spectrum disorders, he notes.

"Many psychiatric diseases and neurological disorders feature some impairment in decision-making," he says.

Modeling how multiple brain areas interact to process reinforcing experiences and guide decision-making is an interesting challenge, he says. Hattori uses many research techniques to gather data, including large-scale 2-photon imaging, virtual reality-based experiments, and optogenetics, a method for using light to manipulate neural activity. Computational modeling is increasingly a valuable tool to understand complex animal behaviors and brain dynamics, Hattori says.

Hattori and colleagues are developing artificial intelligence to assist with their research. It's a two-way relationship: AI helps advance the neuroscience discoveries, and the neuroscience discoveries may also help improve the AI.

"Both the brain and AI are made of neural networks that perform computations and learn using neural activity dynamics and synaptic plasticity," Hattori says. "They receive external inputs, process the information and output an action. Then, the outcome of the action guides the learning by the network. The similarity gives us an opportunity to use AI as a neural network model for certain behaviors."

Hattori recently moved to The Wertheim UF Scripps campus in Jupiter, Florida, following a postdoctoral fellowship at the University of California, San Diego. He earned his doctorate in molecular and cellular biology at Harvard University in 2016.

An assistant professor in the institute's neuroscience department, he's also a recipient of many awards, including the Warren Alpert Distinguished Scholar award and the Simons Foundation SFARI Bridge-to-Independence award.

His wife is a neuroscientist as well, Mariko Hattori, Ph.D. She recently joined the lab of Kirill Martemyanov, Ph.D., chair of the neuroscience department, as a postdoctoral researcher. The Hattoris have a 15-month old son, and enjoy taking him to the ocean when they're not in their labs.

The Jupiter community has become a great magnet for neuroscientists, they said. The Wertheim UF Scripps' strong program is joined by the neighboring Florida Atlantic University Stiles-Nicholson Brain Institute and the Max Planck Florida Institute for Neuroscience.

The Hattoris collaborated with Max Planck's scientific director, Ryohei Yasuda, Ph.D., on a recently published Nature Neuroscience paper about the role of a brain region called the orbitofrontal cortex in the acquisition of generalized knowledge.

The scientists found multiple layers of learning at work in mouse adaptation to new environments, with different time scales. The mouse learning mechanisms resembled those of a computer model of reinforcement learning that was developed by AI researchers.

"We can gain insights into brain mechanisms from AI. Also, as we better understand the brain mechanisms for decision-making and learning, we may be able to transfer the knowledge to AI models," Ryoma Hattori says. "I hope my research projects contribute to understanding of the brain, and also contribute to development of AI with better performance in the machine learning community as well."