Of mice and mental models: Neuroscientific implications of risk-optimized behavior in the mouse

May 25, 2012 by Stuart Mason Dambrot feature
The experimental environment. In the switch task, a trial proceeds as follows: 1: Light in the Trial-Initiation Hopper signals that the mouse may initiate a trial. 2: The mouse approaches and pokes into the trial-initiation hopper, extinguishing the light there and turning on the lights in the two feeding hoppers (trial onset). 3: The mouse goes to the short-latency hopper and pokes into it. 4: If, after 3 s have elapsed since the trial onset, poking in the short-latency hopper does not deliver a pellet, the mouse switches to the long-latency hopper, where it gets a pellet there in response to the first poke at or after 9 s since the trial onset. Lights in both feeding hoppers extinguish either at pellet delivery or when an erroneously timed poke occurs. Short trials last about 3 s and long trials about 9 s, whether reinforced or not: if the mouse is poking in the short hopper at the end of a 3-s trial, it gets a pellet and the trial ends; if it is poking in the 9-s hopper, it does not get a pellet and the trial ends at 3 s. Similarly, long trials end at 9 s: if the mouse is poking in the 9-s hopper, it gets a pellet; if in the 3-s hopper, it does not. A switch latency is the latency of the last poke in the short hopper before the mouse switches to the long hopper. Only the switch latencies from long trials are analyzed. Copyright © PNAS, doi: 10.1073/pnas.1205131109

(Medical Xpress) -- Regardless of an organism’s biological complexity, every encephalized animal continuously makes under-informed behavioral choices that can have serious consequences. Despite its ubiquity, however, there’s a long-standing question about its neurological basis – namely, whether these choices are made through probabilistic world models constructed by the brain, or by reinforcement of learned associations. Recently, however, scientists in the Department of Psychology at Rutgers University found that reinforcement cannot account for the rapidity with which mice modify their behavior when the chance of a given phenomenon changes. The researchers say this indicates that mice may have primordially-evolved neural capabilities to represent likelihood and perform calculations that optimize their resulting behavior – and therefore that such genetic mechanisms can be investigated and manipulated by genetic and other procedures.

In conducting their research, Prof. Randy Gallistel and doctoral student Aaron Kheifets had to first address a key challenge in identifying estimates of stochastic parameters versus reinforcement-driven processes as the behavior-optimizing mechanism in the laboratory studied (the c57bl/6j strain of Mus musculus, the common house mouse, from Jackson Labs). “Because both processes can lead to approximately optimal in the long run,” Gallistel tells Medical Xpress, “one has to focus on the short run – that is, on the course of the transition in behavior. The problem in this case is that the transition is a change in the distribution of switch latencies.” A distribution of switch latencies is composed of a great many temporal discriminations on the part of the subject observed over a long sequence of trials, so this distribution can be used to prove that the process generating the distribution changed abruptly.

“Fortunately,” Gallistel continues, “it was obvious from simple inspection of the raw data that there was an abrupt change. The challenge was to develop a that confirmed this. Meeting this challenge required the use of Bayesian methods, which are just now beginning to be applied to behavioral data. In addition, we had to develop analyses showing that differential reinforcement could not explain the transition.” The team therefore applied Bayesian methods of analysis to the determination of the parameters of a transition function for a 4-parameter mixture distribution.

“Also,” Gallistel adds, “a graphical means of displaying the raw data in such a way as to make the basic phenomenon visually apparent was required. To this end, we devised a figure with a huge number of bits per square centimeter – that is, it shows an enormous amount of readily graspable information in a small space.”

There are several ways the researchers are augmenting their current investigation. “We’re working on automating the process of decision-making during the experiment,” Gallistel illustrates, “so as to improve the efficiency.” They’re also adding external temporal noise by varying the objective pay-off latencies (the durations between which the mouse must discriminate) to see how external uncertainty (random variation in the objective intervals) and internal uncertainty (uncertainty about the timing of the experienced intervals) interact.

The scientists state that their findings suggest that neural mechanisms for estimating probabilities and calculating relative risk are phylogenetically ancient. “Mice and humans have not shared a common ancestor since before the extinction of the dinosaurs,” notes Gallistel. “Thus, the fact that both mice and humans have well-developed mechanisms for calculating risk indicates that those mechanisms were present in their common ancestor.” This also suggests, he says, that this finding means that such mechanisms may be explored through genetic and other invasive procedures in genetically manipulated mice and other laboratory animals.

“A common strategy in modern mechanism-oriented biological research,” says Gallistel,” is to use the enormous power of combined classical and molecular genetics to discover the molecular, cellular and systems realization of basic mechanisms. A classic illustration of such use comes from the pioneering work of Seymour Benzer1 and his students on the circadian clock. By the 1970s, a great deal of behavioral and physiological evidence had accumulated that organisms of all kinds, even bacteria, have an internal clock that regulates their physiology and, in animals, their behavior – for example, the sleep-wake cycle, the ingestion cycle, and, indeed, almost every aspect of physiology and behavior.”

However, until Benzer's work, no one had the faintest idea what the actual mechanism of such a clock might look like or where to look for it. “What this mechanism might possibly be was so mysterious that many scientists did not believe that there really was a clock,” Gallistel points out. ”They thought it was an emergent phenomenon, which is scientific jargon for a phenomenon that does not have a mechanism in any simple sense of the term, but rather emerges from mysterious and ineffable interactions between many different mechanisms. Many contemporary neuroscientists see memory as such a phenomenon."

“When I was a graduate student,” Gallistel recalls, “I became familiar with the extensive behavioral evidence of an internal clock in, for example, bees, and I argued to some of my fellow graduate students that there had to be an honest-to-God clock in the brain. I well remember one of them saying in sarcastic disbelief, ‘You mean if you took the top of the skull off, you could see the hands going around?’

Benzer searched for fruit flies that had heritable genetic malfunctions so that they either did not have a clock, or had an abnormal clock that ran too fast or too slow. “Some very eminent colleagues of his – such as the Nobelist Max Delbruck – thought he was crazy to embark on such a wild goose chase, but he and his students soon found what they had set out to look for,” Gallistel continues. “They then used classical genetic techniques to localize these genes to small regions of certain chromosomes and that made it possible to use molecular genetics techniques to find the exact location of these genes and to establish the base-pair sequences in these genes. That, in turn, enables molecular biologists to identify the protein coded for by these genes and to fashion all kinds of every more sophisticated molecular tools that have enabled legions of other scientists to establish what is by now a quite detailed and constantly improved story about this molecular clock and where it is located and how it works. This is now in all the textbooks and it is a great triumph of reductionist biology.”

Gallistel adds that because Benzer’s strategy has proven so powerful, there are now techniques for inducing mutations in mice for the express purpose of finding mice with heritable malfunctions in any of the thousands of different physiological and behavioral processes about whose underlying mechanisms we’re still ignorant. The general strategy is to test hundreds or even thousands of these mutant mice, looking for mice with a heritable malfunction in the phenomenon one is interested in. “In our case,” explains Gallistel, “this would be a heritable malfunction in risk assessment – either an inability to assess risk at all, or, more interestingly, a systematic inaccurate estimation of risk. One exploits the vast knowledge we have accumulated about mouse genetics and the thousands of genetically distinct inbred strains of mice to localize the gene whose mutation produces the heritable anomaly. One then uses molecular techniques, which have become radically more efficient and fast than they were in Benzer's day, to sequence the gene. One can then use an incredible array of tools and techniques that have been developed over the last several decades by molecular biologists to locate the cells where the gene is expressed, thereby opening up the investigation of the cellular and systems biology. One also can manipulate the gene itself in order to alter the functioning of the mechanism in ways that enable us to understand how the mechanism works at the molecular level.”

Gallistel notes that this strategy only works if the phenomenon being investigated is robustly present and readily measured in animals like the mouse, the zebra fish, or the fruit fly – the species chiefly used in the pursuit of this strategy. “If you think only humans – and maybe only college trained humans – can correctly estimate probabilities and correctly calculate risks, then there is no way you can use this strategy. However, we’ve shown that mice can correctly estimate probabilities and correctly calculate risks – and that their ability to do so can be assessed in completely automated behavioral tests that require very little human labor, and that can be run on hundreds of mice simultaneously. In other words, there is now a way to apply Benzer's strategy to the mechanisms that mediate the brain's ability to estimate probabilities and calculate risks – and the molecular and cellular bases of these abilities are as mysterious to us at this time, as were the molecular and cellular bases of the daily clock in the 1970s.”

Gallistel adds that other research and application areas might benefit from their findings. “Probabilities are simple quantities and the calculation of risk requires the application of arithmetic operations to these quantities,” he concludes. “The ability to represent quantities and apply arithmetic operations to them is a foundation of mental activity. Pursuit of these avenues could lead to an understanding of the physical bases for our ability to think.”

Explore further: Spatial configuration can spark deja vu, psychology study reveals

More information: Mice take calculated risks, PNAS May 16, 2012, Published online before print May 16, 2012, doi: 10.1073/pnas.1205131109

1Related: Clock Mutants of Drosophila melanogaster, PNAS September 1, 1971 vol. 68 no. 9 2112-2116

Related Stories

Spatial configuration can spark deja vu, psychology study reveals

May 24, 2012
(Medical Xpress) -- Déjà vu - that strange feeling of having experienced something before - is more likely to occur when a scene's spatial layout resembles one in memory, according to groundbreaking new research ...

Brain oscillations reveal that our senses do not experience the world continuously

May 14, 2012
(Medical Xpress) -- It has long been suspected that humans do not experience the world continuously, but rather in rapid snapshots.

Device helps with Sudden Infant Death Syndrome detection

May 9, 2012
University of Texas at Arlington researchers have obtained a patent for a device aimed at saving babies’ lives through improved and rapid detection of Sudden Infant Death Syndrome.

Dreamless nights: Brain activity during nonrapid eye movement sleep

April 9, 2012
(Medical Xpress) -- The link between dreaming and rapid eye movement (REM) sleep are well understood – but the fact that consciousness is reduced during nonrapid eye movement (NREM) sleep is not. Recently, scientists ...

Recommended for you

Theory: Flexibility is at the heart of human intelligence

November 19, 2017
Centuries of study have yielded many theories about how the brain gives rise to human intelligence. Some neuroscientists think intelligence springs from a single region or neural network. Others argue that metabolism or the ...

Investigating patterns of degeneration in Alzheimer's disease

November 17, 2017
Alzheimer's disease (AD) is known to cause memory loss and cognitive decline, but other functions of the brain can remain intact. The reasons cells in some brain regions degenerate while others are protected is largely unknown. ...

Study may point to new treatment approach for ASD

November 17, 2017
Using sophisticated genome mining and gene manipulation techniques, researchers at Vanderbilt University Medical Center (VUMC) have solved a mystery that could lead to a new treatment approach for autism spectrum disorder ...

Paraplegic rats walk and regain feeling after stem cell treatment

November 16, 2017
Engineered tissue containing human stem cells has allowed paraplegic rats to walk independently and regain sensory perception. The implanted rats also show some degree of healing in their spinal cords. The research, published ...

Brain implant tested in human patients found to improve memory recall

November 15, 2017
(Medical Xpress)—A team of researchers with the University of Southern California and the Wake Forest School of Medicine has conducted experiments involving implanting electrodes into the brains of human volunteers to see ...

Researchers identify potential mediator for social memory formation

November 15, 2017
Research by a group of scientists at the Department of Physiology, Yong Loo Lin School of Medicine, National University of Singapore (NUS Medicine) have discovered that a tiny brain region plays a critical role in the formation ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.