Of mice and mental models: Neuroscientific implications of risk-optimized behavior in the mouse

(Medical Xpress) -- Regardless of an organism’s biological complexity, every encephalized animal continuously makes under-informed behavioral choices that can have serious consequences. Despite its ubiquity, however, there’s a long-standing question about its neurological basis – namely, whether these choices are made through probabilistic world models constructed by the brain, or by reinforcement of learned associations. Recently, however, scientists in the Department of Psychology at Rutgers University found that reinforcement cannot account for the rapidity with which mice modify their behavior when the chance of a given phenomenon changes. The researchers say this indicates that mice may have primordially-evolved neural capabilities to represent likelihood and perform calculations that optimize their resulting behavior – and therefore that such genetic mechanisms can be investigated and manipulated by genetic and other procedures.

In conducting their research, Prof. Randy Gallistel and doctoral student Aaron Kheifets had to first address a key challenge in identifying estimates of stochastic parameters versus reinforcement-driven processes as the behavior-optimizing mechanism in the laboratory mice studied (the c57bl/6j strain of Mus musculus, the common house mouse, from Jackson Labs). “Because both processes can lead to approximately optimal behavior in the long run,” Gallistel tells Medical Xpress, “one has to focus on the short run – that is, on the course of the transition in behavior. The problem in this case is that the transition is a change in the distribution of switch latencies.” A distribution of switch latencies is composed of a great many temporal discriminations on the part of the subject observed over a long sequence of trials, so this distribution can be used to prove that the process generating the distribution changed abruptly.

“Fortunately,” Gallistel continues, “it was obvious from simple inspection of the raw data that there was an abrupt change. The challenge was to develop a mathematical analysis that confirmed this. Meeting this challenge required the use of Bayesian methods, which are just now beginning to be applied to behavioral data. In addition, we had to develop analyses showing that differential reinforcement could not explain the transition.” The team therefore applied Bayesian methods of analysis to the determination of the parameters of a transition function for a 4-parameter mixture distribution.

“Also,” Gallistel adds, “a graphical means of displaying the raw data in such a way as to make the basic phenomenon visually apparent was required. To this end, we devised a figure with a huge number of bits per square centimeter – that is, it shows an enormous amount of readily graspable information in a small space.”

There are several ways the researchers are augmenting their current investigation. “We’re working on automating the process of decision-making during the experiment,” Gallistel illustrates, “so as to improve the efficiency.” They’re also adding external temporal noise by varying the objective pay-off latencies (the durations between which the mouse must discriminate) to see how external uncertainty (random variation in the objective intervals) and internal uncertainty (uncertainty about the timing of the experienced intervals) interact.

The scientists state that their findings suggest that neural mechanisms for estimating probabilities and calculating relative risk are phylogenetically ancient. “Mice and humans have not shared a common ancestor since before the extinction of the dinosaurs,” notes Gallistel. “Thus, the fact that both mice and humans have well-developed brain mechanisms for calculating risk indicates that those mechanisms were present in their common ancestor.” This also suggests, he says, that this finding means that such mechanisms may be explored through genetic and other invasive procedures in genetically manipulated mice and other laboratory animals.

“A common strategy in modern mechanism-oriented biological research,” says Gallistel,” is to use the enormous power of combined classical and molecular genetics to discover the molecular, cellular and systems realization of basic mechanisms. A classic illustration of such use comes from the pioneering work of Seymour Benzer¹ and his students on the circadian clock. By the 1970s, a great deal of behavioral and physiological evidence had accumulated that organisms of all kinds, even bacteria, have an internal clock that regulates their physiology and, in animals, their behavior – for example, the sleep-wake cycle, the ingestion cycle, and, indeed, almost every aspect of physiology and behavior.”

However, until Benzer's work, no one had the faintest idea what the actual mechanism of such a clock might look like or where to look for it. “What this mechanism might possibly be was so mysterious that many scientists did not believe that there really was a clock,” Gallistel points out. ”They thought it was an emergent phenomenon, which is scientific jargon for a phenomenon that does not have a mechanism in any simple sense of the term, but rather emerges from mysterious and ineffable interactions between many different mechanisms. Many contemporary neuroscientists see memory as such a phenomenon."

“When I was a graduate student,” Gallistel recalls, “I became familiar with the extensive behavioral evidence of an internal clock in, for example, bees, and I argued to some of my fellow graduate students that there had to be an honest-to-God clock in the brain. I well remember one of them saying in sarcastic disbelief, ‘You mean if you took the top of the skull off, you could see the hands going around?’”

Benzer searched for fruit flies that had heritable genetic malfunctions so that they either did not have a clock, or had an abnormal clock that ran too fast or too slow. “Some very eminent colleagues of his – such as the Nobelist Max Delbruck – thought he was crazy to embark on such a wild goose chase, but he and his students soon found what they had set out to look for,” Gallistel continues. “They then used classical genetic techniques to localize these genes to small regions of certain chromosomes and that made it possible to use molecular genetics techniques to find the exact location of these genes and to establish the base-pair sequences in these genes. That, in turn, enables molecular biologists to identify the protein coded for by these genes and to fashion all kinds of every more sophisticated molecular tools that have enabled legions of other scientists to establish what is by now a quite detailed and constantly improved story about this molecular clock and where it is located and how it works. This is now in all the textbooks and it is a great triumph of reductionist biology.”

Gallistel adds that because Benzer’s strategy has proven so powerful, there are now techniques for inducing mutations in mice for the express purpose of finding mice with heritable malfunctions in any of the thousands of different physiological and behavioral processes about whose underlying mechanisms we’re still ignorant. The general strategy is to test hundreds or even thousands of these mutant mice, looking for mice with a heritable malfunction in the phenomenon one is interested in. “In our case,” explains Gallistel, “this would be a heritable malfunction in risk assessment – either an inability to assess risk at all, or, more interestingly, a systematic inaccurate estimation of risk. One exploits the vast knowledge we have accumulated about mouse genetics and the thousands of genetically distinct inbred strains of mice to localize the gene whose mutation produces the heritable anomaly. One then uses molecular techniques, which have become radically more efficient and fast than they were in Benzer's day, to sequence the gene. One can then use an incredible array of tools and techniques that have been developed over the last several decades by molecular biologists to locate the cells where the gene is expressed, thereby opening up the investigation of the cellular and systems biology. One also can manipulate the gene itself in order to alter the functioning of the mechanism in ways that enable us to understand how the mechanism works at the molecular level.”

Gallistel notes that this strategy only works if the phenomenon being investigated is robustly present and readily measured in animals like the mouse, the zebra fish, or the fruit fly – the species chiefly used in the pursuit of this strategy. “If you think only humans – and maybe only college trained humans – can correctly estimate probabilities and correctly calculate risks, then there is no way you can use this strategy. However, we’ve shown that mice can correctly estimate probabilities and correctly calculate risks – and that their ability to do so can be assessed in completely automated behavioral tests that require very little human labor, and that can be run on hundreds of mice simultaneously. In other words, there is now a way to apply Benzer's strategy to the mechanisms that mediate the brain's ability to estimate probabilities and calculate risks – and the molecular and cellular bases of these abilities are as mysterious to us at this time, as were the molecular and cellular bases of the daily clock in the 1970s.”

Gallistel adds that other research and application areas might benefit from their findings. “Probabilities are simple quantities and the calculation of risk requires the application of arithmetic operations to these quantities,” he concludes. “The ability to represent quantities and apply arithmetic operations to them is a foundation of mental activity. Pursuit of these avenues could lead to an understanding of the physical bases for our ability to think.”

Copyright 2012 Medical Xpress
All rights reserved. This material may not be published, broadcast, rewritten or redistributed in whole or part without the express written permission of PhysOrg.com.