Motoring ahead with rodents

How neural circuits underlie the acquisition and control of learned motor behaviors has traditionally been explored in monkeys and, more recently, songbirds. The development of genetic tools for functional circuit analysis in rodents, the availability of transgenic animals with well characterized phenotypes, and the relative ease with which rats and mice can be trained to perform various motor tasks, make rodents attractive models for exploring the neural circuit mechanisms underlying the acquisition and production of learned motor skills. Here we discuss the advantages and drawbacks of this approach, review recent trends and results, and outline possible strategies for wider adoption of rodents as a model system for complex motor learning.     

Automated Visual Cognitive Tasks for Recording Neural Activity Using a Floor Projection Maze

Neuropsychological tasks used in primates to investigate mechanisms of learning and memory are typically visually guided cognitive tasks. We have developed visual cognitive tasks for rats using the Floor Projection Maze^1,2 that are optimized for visual abilities of rats permitting stronger comparisons of experimental findings with other species.In order to investigate neural correlates of learning and memory, we have integrated electrophysiological recordings into fully automated cognitive tasks on the Floor Projection Maze^1,2. Behavioral software interfaced with an animal tracking system allows monitoring of the animal''s behavior with precise control of image presentation and reward contingencies for better trained animals. Integration with an in vivo electrophysiological recording system enables examination of behavioral correlates of neural activity at selected epochs of a given cognitive task.We describe protocols for a model system that combines automated visual presentation of information to rodents and intracranial reward with electrophysiological approaches. Our model system offers a sophisticated set of tools as a framework for other cognitive tasks to better isolate and identify specific mechanisms contributing to particular cognitive processes.     

Protocol for Studying Extinction of Conditioned Fear in Naturally Cycling Female Rats

Extinction of conditioned fear has been extensively studied in male rodents. Recently, there have been an increasing number of studies indicating that neural mechanisms for certain behavioral tasks and response behaviors are different in females and males. Using females in research studies can represent a challenge because of the variation of gonadal hormones during their estrous cycle. This protocol describes well-established procedures that are useful in investigating the role of estrogen in fear extinction memory consolidation in female rats. Phase of the estrous cycle and exogenous estrogen administration prior to extinction training can influence extinction recall 24 hr later. The vaginal swabbing technique for estrous phase identification described here aids the examination and manipulation of naturally cycling gonadal hormones. The use of this basic rodent model may further delineate the mechanisms by which estrogen can modulate fear extinction memory in females.     

Optimization of the touchscreen paired-associate learning (PAL) task for mice and its dorsal hippocampal dependency

Two versions of the touchscreen paired-associate learning (PAL) task have been developed for rodents: same PAL (sPAL) and different PAL (dPAL). These tasks are very important in studying murine models of Alzheimer’s disease and schizophrenia, and have also been used to test object-location memory in various studies. However, the relatively long time needed for the tasks (approx. 50 days for mice) limits their widespread use. By giving training that was more intensive with a higher number of trials, we shortened the time required for learning saturation in sPAL and dPAL to about one-third of the time required for the generally used protocol. Furthermore, by applying a reduced number of objects and trial types for sPAL, we developed a simplified version of sPAL, termed 2-object sPAL, in which mice could reach the fully learned level in 6 days. Our pharmacological experiments indicate that the dorsal hippocampal CA1 region is crucial for the performance of the two PAL tasks with the new protocols and the new 2-object sPAL. This work has significantly enhanced the usefulness of the touchscreen PAL tasks to increase the speed of learning, but they remain highly hippocampus-dependent object-location memory tasks.     

Reinforcement Learning on Slow Features of High-Dimensional Input Streams

Humans and animals are able to learn complex behaviors based on a massive stream of sensory information from different modalities. Early animal studies have identified learning mechanisms that are based on reward and punishment such that animals tend to avoid actions that lead to punishment whereas rewarded actions are reinforced. However, most algorithms for reward-based learning are only applicable if the dimensionality of the state-space is sufficiently small or its structure is sufficiently simple. Therefore, the question arises how the problem of learning on high-dimensional data is solved in the brain. In this article, we propose a biologically plausible generic two-stage learning system that can directly be applied to raw high-dimensional input streams. The system is composed of a hierarchical slow feature analysis (SFA) network for preprocessing and a simple neural network on top that is trained based on rewards. We demonstrate by computer simulations that this generic architecture is able to learn quite demanding reinforcement learning tasks on high-dimensional visual input streams in a time that is comparable to the time needed when an explicit highly informative low-dimensional state-space representation is given instead of the high-dimensional visual input. The learning speed of the proposed architecture in a task similar to the Morris water maze task is comparable to that found in experimental studies with rats. This study thus supports the hypothesis that slowness learning is one important unsupervised learning principle utilized in the brain to form efficient state representations for behavioral learning.     

Using deep neural networks to evaluate object vision tasks in rats

In the last two decades rodents have been on the rise as a dominant model for visual neuroscience. This is particularly true for earlier levels of information processing, but a number of studies have suggested that also higher levels of processing such as invariant object recognition occur in rodents. Here we provide a quantitative and comprehensive assessment of this claim by comparing a wide range of rodent behavioral and neural data with convolutional deep neural networks. These networks have been shown to capture hallmark properties of information processing in primates through a succession of convolutional and fully connected layers. We find that performance on rodent object vision tasks can be captured using low to mid-level convolutional layers only, without any convincing evidence for the need of higher layers known to simulate complex object recognition in primates. Our approach also reveals surprising insights on assumptions made before, for example, that the best performing animals would be the ones using the most abstract representations–which we show to likely be incorrect. Our findings suggest a road ahead for further studies aiming at quantifying and establishing the richness of representations underlying information processing in animal models at large.     

Neural plasticity in high-level visual cortex underlying object perceptual learning

With intensive training, human can achieve impressive behavioral improvement on various perceptual tasks. This phenomenon, termed perceptual learning, has long been considered as a hallmark of the plasticity of sensory neural system. Not surprisingly, high-level vision, such as object perception, can also be improved by perceptual learning. Here we review recent psychophysical, electrophysiological, and neuroimaging studies investigating the effects of training on object selective cortex, such as monkey inferior temporal cortex and human lateral occipital area. Evidences show that learning leads to an increase in object selectivity at the single neuron level and/or the neuronal population level. These findings indicate that high-level visual cortex in humans is highly plastic and visual experience can strongly shape neural functions of these areas. At the end of the review, we discuss several important future directions in this area.     

A Fully Automated Rodent Conditioning Protocol for Sensorimotor Integration and Cognitive Control Experiments

Rodents have been traditionally used as a standard animal model in laboratory experiments involving a myriad of sensory, cognitive, and motor tasks. Higher cognitive functions that require precise control over sensorimotor responses such as decision-making and attentional modulation, however, are typically assessed in nonhuman primates. Despite the richness of primate behavior that allows multiple variants of these functions to be studied, the rodent model remains an attractive, cost-effective alternative to primate models. Furthermore, the ability to fully automate operant conditioning in rodents adds unique advantages over the labor intensive training of nonhuman primates while studying a broad range of these complex functions.Here, we introduce a protocol for operantly conditioning rats on performing working memory tasks. During critical epochs of the task, the protocol ensures that the animal''s overt movement is minimized by requiring the animal to ''fixate'' until a Go cue is delivered, akin to nonhuman primate experimental design. A simple two alternative forced choice task is implemented to demonstrate the performance. We discuss the application of this paradigm to other tasks.     

Unsupervised clustering and centroid estimation using dynamic competitive learning

In this paper, an unsupervised learning algorithm is developed. Two versions of an artificial neural network, termed a differentiator, are described. It is shown that our algorithm is a dynamic variation of the competitive learning found in most unsupervised learning systems. These systems are frequently used for solving certain pattern recognition tasks such as pattern classification and k-means clustering. Using computer simulation, it is shown that dynamic competitive learning outperforms simple competitive learning methods in solving cluster detection and centroid estimation problems. The simulation results demonstrate that high quality clusters are detected by our method in a short training time. Either a distortion function or the minimum spanning tree method of clustering is used to verify the clustering results. By taking full advantage of all the information presented in the course of training in the differentiator, we demonstrate a powerful adaptive system capable of learning continuously changing patterns.     

Beetles, boxes and brain cells: neural mechanisms underlying valuation and learning

Sensory cues in the environment can predict the availability of reward. Through experience, humans and animals learn these predictions and use them to guide their actions. For example, we can learn to discriminate chanterelles from ordinary champignons through experience. Assuming the development of a taste for the complex and lingering flavors of chanterelles, we therefore learn to value the same action--picking mushrooms--differentially depending upon the appearance of a mushroom. One major goal of cognitive neuroscience is to understand the neural mechanisms that underlie this sort of learning. Because the acquisition of rewards motivates much behavior, recent efforts have focused on describing the neural signals related to learning the value of stimuli and actions. Neurons in the basal ganglia, in midbrain dopamine areas, in frontal and parietal cortices and in other brain areas, all modulate their activity in relation to aspects of learning. By training monkeys on various behavioral tasks, recent studies have begun to characterize how neural signals represent distinct processes, such as the timing of events, motivation, absolute (objective) and relative (subjective) valuation, and the formation of associative links between stimuli and potential actions. In addition, a number of studies have either further characterized dopamine signals or sought to determine how such signaling might interact with target structures, such as the striatum and rhinal cortex, to underlie learning.     

A review of the generalization of auditory learning

The ability to detect and discriminate attributes of sounds improves with practice. Determining how such auditory learning generalizes to stimuli and tasks that are not encountered during training can guide the development of training regimens used to improve hearing abilities in particular populations as well as provide insight into the neural mechanisms mediating auditory performance. Here we review the newly emerging literature on the generalization of auditory learning, focusing on behavioural investigations of generalization on basic auditory tasks in human listeners. The review reveals a variety of generalization patterns across different trained tasks that can not be summarized with a simple rule, and a diversity of views about the definition, evaluation and interpretation of generalization.     

The octopus: a model for a comparative analysis of the evolution of learning and memory mechanisms

Comparative analysis of brain function in invertebrates with sophisticated behaviors, such as the octopus, may advance our understanding of the evolution of the neural processes that mediate complex behaviors. Until the last few years, this approach was infeasible due to the lack of neurophysiological tools for testing the neural circuits mediating learning and memory in the brains of octopus and other cephalopods. Now, for the first time, the adaptation of modern neurophysiological methods to the study of the central nervous system of the octopus allows this avenue of research. The emerging results suggest that a convergent evolutionary process has led to the selection of vertebrate-like neural organization and activity-dependent long-term synaptic plasticity. As octopuses and vertebrates are very remote phylogenetically, this convergence suggests the importance of the shared properties for the mediation of learning and memory.     

Estrogen Enhances Performance of Female Rats during Acquisition of a Radial Arm Maze

Estrogen can influence the expression of behaviors not associated directly with reproduction, including learning and memory. However, the effects of estrogen on learning and memory in mammals are complex, dependent on a variety of factors. The radial arm maze is a traditional experimental task that takes advantage of the natural foraging strategy of rats and provides an appropriate measure for studying the effects of estrogen on working memory in this species. In the experiments reported here, ovariectomized rats were implanted subcutaneously with 5-mm Silastic capsules containing 25% estradiol diluted with cholesterol. Control females received 5-mm Silastic capsules containing 100% cholesterol. Results of three separate experiments demonstrated that estradiol administered by Silastic implants for 30 days prior to eight-arm radial maze training, during the 24 days of maze training, or both significantly improved working memory performance compared to females treated with cholesterol alone, as indicated by improved arm choice accuracy over trials. The positive effect of estradiol exposure prior to training suggests that estrogen may induce neuronal changes that persist beyond the period of exposure with functional consequences for behavior.     

Adaptive Robotic Control Driven by a Versatile Spiking Cerebellar Network

The cerebellum is involved in a large number of different neural processes, especially in associative learning and in fine motor control. To develop a comprehensive theory of sensorimotor learning and control, it is crucial to determine the neural basis of coding and plasticity embedded into the cerebellar neural circuit and how they are translated into behavioral outcomes in learning paradigms. Learning has to be inferred from the interaction of an embodied system with its real environment, and the same cerebellar principles derived from cell physiology have to be able to drive a variety of tasks of different nature, calling for complex timing and movement patterns. We have coupled a realistic cerebellar spiking neural network (SNN) with a real robot and challenged it in multiple diverse sensorimotor tasks. Encoding and decoding strategies based on neuronal firing rates were applied. Adaptive motor control protocols with acquisition and extinction phases have been designed and tested, including an associative Pavlovian task (Eye blinking classical conditioning), a vestibulo-ocular task and a perturbed arm reaching task operating in closed-loop. The SNN processed in real-time mossy fiber inputs as arbitrary contextual signals, irrespective of whether they conveyed a tone, a vestibular stimulus or the position of a limb. A bidirectional long-term plasticity rule implemented at parallel fibers-Purkinje cell synapses modulated the output activity in the deep cerebellar nuclei. In all tasks, the neurorobot learned to adjust timing and gain of the motor responses by tuning its output discharge. It succeeded in reproducing how human biological systems acquire, extinguish and express knowledge of a noisy and changing world. By varying stimuli and perturbations patterns, real-time control robustness and generalizability were validated. The implicit spiking dynamics of the cerebellar model fulfill timing, prediction and learning functions.     

The neurobiology of social attachment: A comparative approach to behavioral, neuroanatomical, and neurochemical studies

The formation and maintenance of social bonds in adulthood is an essential component of human health. However studies investigating the underlying neurobiology of such behaviors have been scarce. Microtine rodents offer a unique comparative animal model to explore the neural processes responsible for pair bonding and its associated behaviors. Studies using monogamous prairie voles and other related species have recently offered insight into the neuroanatomical, neurobiological, and neurochemical underpinnings of social attachment. In this review, we will discuss the utility of the microtine rodents in comparative studies by exploring their natural history and social behavior in the laboratory. We will then summarize the data implicating vasopressin, oxytocin, and dopamine in the regulation of pair bonding. Finally, we will discuss the ways in which these neurochemical systems may interact to mediate this complex behavior.     

Hippocampus, ageing, and taste memories

Previous studies have shown that ageing may induce deficits in hippocampal-dependent learning and memory tasks, the spatial task being most extensively applied in rats. It is proposed that taste learning and memory tasks may assist in understanding the ageing of memory systems, giving access to a more complete picture. Taste learning tasks allow us to explore a variety of learning phenomena in safe and aversive memories using similar behavioral procedures. In demanding the same sensory, response, and motivational requirements, this approach provides reliable comparisons between the performance of hippocampal lesioned and aged rats in different types of memory. Present knowledge on the effect of both ageing and hippocampal damage in complex taste learning phenomena is reviewed. Besides inducing deficits in hippocampal-dependent phenomena, such as blocking of conditioned taste aversion, while at the same time leaving intact nonhippocampal-dependent effects, such as latent inhibition, ageing is also associated with an increased neophobia by previous aversive taste memories and enhanced taste aversion conditioning which cannot be explained by age-related changes in taste or visceral distress sensitivity. In all, the results indicate a peculiar organization of the memory systems during aging that cannot be explained by a general cognitive decline or exclusively by the decay of the hippocampal function.     

A 3D-Video-Based Computerized Analysis of Social and Sexual Interactions in Rats

A large number of studies have analyzed social and sexual interactions between rodents in relation to neural activity. Computerized video analysis has been successfully used to detect numerous behaviors quickly and objectively; however, to date only 2D video recording has been used, which cannot determine the 3D locations of animals and encounters difficulties in tracking animals when they are overlapping, e.g., when mounting. To overcome these limitations, we developed a novel 3D video analysis system for examining social and sexual interactions in rats. A 3D image was reconstructed by integrating images captured by multiple depth cameras at different viewpoints. The 3D positions of body parts of the rats were then estimated by fitting skeleton models of the rats to the 3D images using a physics-based fitting algorithm, and various behaviors were recognized based on the spatio-temporal patterns of the 3D movements of the body parts. Comparisons between the data collected by the 3D system and those by visual inspection indicated that this system could precisely estimate the 3D positions of body parts for 2 rats during social and sexual interactions with few manual interventions, and could compute the traces of the 2 animals even during mounting. We then analyzed the effects of AM-251 (a cannabinoid CB1 receptor antagonist) on male rat sexual behavior, and found that AM-251 decreased movements and trunk height before sexual behavior, but increased the duration of head-head contact during sexual behavior. These results demonstrate that the use of this 3D system in behavioral studies could open the door to new approaches for investigating the neuroscience of social and sexual behavior.     

Shifts in preferred learning strategy across the estrous cycle in female rats

The current status of the effects of ovarian steroids on learning and memory remains somewhat unclear, despite a large undertaking to evaluate these effects. What is emerging from this literature is that estrogen, and perhaps progesterone, influences learning and memory, but does so in a task-dependent manner. Previously, we have shown that ovariectomized rats given acute treatments of estrogen acquire allocentric or "place" tasks more easily than do rats deprived of estrogen, but acquire egocentric or "response" learning tasks more slowly than do those deprived of hormone, suggesting that estrogen treatment may bias the strategy a rat is able to use to solve tasks. To determine if natural fluctuations in ovarian hormones influence cognitive strategy, we tested whether strategy use fluctuated across the estrous cycle in reproductively intact female rats. We found that in two tasks in which rats freely choose the strategy used to solve the task, rats were more likely to use place strategies at proestrous, that is, when ovarian steroids are high. Conversely, estrous rats were biased toward response strategies. The data suggest that natural fluctuations in ovarian steroids may bias the neural system used and thus the cognitive strategies chosen during learning and memory.     

A Within-Animal Comparison of Skilled Forelimb Assessments in Rats

A variety of skilled reaching tasks have been developed to evaluate forelimb function in rodent models. The single pellet skilled reaching task and pasta matrix task have provided valuable insight into recovery of forelimb function in models of neurological injury and disease. Recently, several automated measures have been developed to reduce the cost and time burden of forelimb assessment in rodents. Here, we provide a within-subject comparison of three common forelimb assessments to allow direct evaluation of sensitivity and efficiency across tasks. Rats were trained to perform the single pellet skilled reaching task, the pasta matrix task, and the isometric pull task. Once proficient on all three tasks, rats received an ischemic lesion of motor cortex and striatum to impair use of the trained limb. On the second week post-lesion, all three tasks measured a significant deficit in forelimb function. Performance was well-correlated across tasks. By the sixth week post-lesion, only the isometric pull task measured a significant deficit in forelimb function, suggesting that this task is more sensitive to chronic impairments. The number of training days required to reach asymptotic performance was longer for the isometric pull task, but the total experimenter time required to collect and analyze data was substantially lower. These findings suggest that the isometric pull task represents an efficient, sensitive measure of forelimb function to facilitate preclinical evaluation in models of neurological injury and disease.