Category learning in rodents using touchscreen-based tasks

Categorization is a fundamental cognitive function that organizes our experiences into meaningful “chunks.” This category knowledge can then be generalized to novel stimuli and situations. Multiple clinical populations, including people with Parkinson's disease, amnesia, autism, ADHD and schizophrenia, have impairments in the acquisition and use of categories. Although rodent research is well suited for examining the neural mechanisms underlying cognitive functions, many rodent cognitive tasks have limited translational value. To bridge this gap, we use touchscreens to permit greater flexibility in stimulus presentation and task design, track key dependent measures, and minimize experimenter involvement. Touchscreens offer a valuable tool for creating rodent cognitive tasks that are directly comparable to tasks used with humans. Touchscreen tasks are also readily used with cutting-edge neuroscientific methods that are difficult to do in humans such as optogenetics, chemogenetics, neurophysiology and calcium imaging (using miniscopes). In this review, we show advantages of touchscreen-based tasks for studying category learning in rats. We also address multiple factors for consideration when designing category learning tasks, including the limitations of the rodent visual system, experimental design, and analysis strategies.     

Shifts in preferred learning strategy across the estrous cycle in female rats

The current status of the effects of ovarian steroids on learning and memory remains somewhat unclear, despite a large undertaking to evaluate these effects. What is emerging from this literature is that estrogen, and perhaps progesterone, influences learning and memory, but does so in a task-dependent manner. Previously, we have shown that ovariectomized rats given acute treatments of estrogen acquire allocentric or "place" tasks more easily than do rats deprived of estrogen, but acquire egocentric or "response" learning tasks more slowly than do those deprived of hormone, suggesting that estrogen treatment may bias the strategy a rat is able to use to solve tasks. To determine if natural fluctuations in ovarian hormones influence cognitive strategy, we tested whether strategy use fluctuated across the estrous cycle in reproductively intact female rats. We found that in two tasks in which rats freely choose the strategy used to solve the task, rats were more likely to use place strategies at proestrous, that is, when ovarian steroids are high. Conversely, estrous rats were biased toward response strategies. The data suggest that natural fluctuations in ovarian steroids may bias the neural system used and thus the cognitive strategies chosen during learning and memory.     

Dual Reward Prediction Components Yield Pavlovian Sign- and Goal-Tracking

Reinforcement learning (RL) has become a dominant paradigm for understanding animal behaviors and neural correlates of decision-making, in part because of its ability to explain Pavlovian conditioned behaviors and the role of midbrain dopamine activity as reward prediction error (RPE). However, recent experimental findings indicate that dopamine activity, contrary to the RL hypothesis, may not signal RPE and differs based on the type of Pavlovian response (e.g. sign- and goal-tracking responses). In this study, we address this discrepancy by introducing a new neural correlate for learning reward predictions; the correlate is called “cue-evoked reward”. It refers to a recall of reward evoked by the cue that is learned through simple cue-reward associations. We introduce a temporal difference learning model, in which neural correlates of the cue itself and cue-evoked reward underlie learning of reward predictions. The animal''s reward prediction supported by these two correlates is divided into sign and goal components respectively. We relate the sign and goal components to approach responses towards the cue (i.e. sign-tracking) and the food-tray (i.e. goal-tracking) respectively. We found a number of correspondences between simulated models and the experimental findings (i.e. behavior and neural responses). First, the development of modeled responses is consistent with those observed in the experimental task. Second, the model''s RPEs were similar to dopamine activity in respective response groups. Finally, goal-tracking, but not sign-tracking, responses rapidly emerged when RPE was restored in the simulated models, similar to experiments with recovery from dopamine-antagonist. These results suggest two complementary neural correlates, corresponding to the cue and its evoked reward, form the basis for learning reward predictions in the sign- and goal-tracking rats.     

Adding preferred color to a conventional reward method improves the memory of zebrafish in the T-maze behavior model

Zebrafish have become a useful model for studying behavior and cognitive functions. Recent studies have shown that zebrafish have natural color preference and the ability to form associative memories with visual perception. It is well known that visual perception enhances memory recall in humans, and we suggest that a similar phenomenon occurs in zebrafish. This study proposes that adding a visual perception component to a conventional reward method would enhance memory recall in zebrafish. We found that zebrafish showed greater preference for red cellophane over yellow in the training session but could not remember the preferred place in the memory test. However, the test memory recall was greater when the zebrafish were exposed to the red cellophane with a food reward during the training session, when compared with the use of food reward only. Furthermore, the red cellophane with food reward group showed more predictable memory recall than the food reward only group. These results propose that visual perception can increase memory recall by enhancing the consolidation processes. We suggest that color-cued learning with food reward is a more discriminative method than food reward alone for examining the cognitive changes in the zebrafish.

Abbreviations: WM: working memory; LTM: long-term memory     

Cortical and Hippocampal Correlates of Deliberation during Model-Based Decisions for Rewards in Humans

How do we use our memories of the past to guide decisions we''ve never had to make before? Although extensive work describes how the brain learns to repeat rewarded actions, decisions can also be influenced by associations between stimuli or events not directly involving reward — such as when planning routes using a cognitive map or chess moves using predicted countermoves — and these sorts of associations are critical when deciding among novel options. This process is known as model-based decision making. While the learning of environmental relations that might support model-based decisions is well studied, and separately this sort of information has been inferred to impact decisions, there is little evidence concerning the full cycle by which such associations are acquired and drive choices. Of particular interest is whether decisions are directly supported by the same mnemonic systems characterized for relational learning more generally, or instead rely on other, specialized representations. Here, building on our previous work, which isolated dual representations underlying sequential predictive learning, we directly demonstrate that one such representation, encoded by the hippocampal memory system and adjacent cortical structures, supports goal-directed decisions. Using interleaved learning and decision tasks, we monitor predictive learning directly and also trace its influence on decisions for reward. We quantitatively compare the learning processes underlying multiple behavioral and fMRI observables using computational model fits. Across both tasks, a quantitatively consistent learning process explains reaction times, choices, and both expectation- and surprise-related neural activity. The same hippocampal and ventral stream regions engaged in anticipating stimuli during learning are also engaged in proportion to the difficulty of decisions. These results support a role for predictive associations learned by the hippocampal memory system to be recalled during choice formation.     

Multisensory Control of Multimodal Behavior: Do the Legs Know What the Tongue Is Doing?

Understanding of adaptive behavior requires the precisely controlled presentation of multisensory stimuli combined with simultaneous measurement of multiple behavioral modalities. Hence, we developed a virtual reality apparatus that allows for simultaneous measurement of reward checking, a commonly used measure in associative learning paradigms, and navigational behavior, along with precisely controlled presentation of visual, auditory and reward stimuli. Rats performed a virtual spatial navigation task analogous to the Morris maze where only distal visual or auditory cues provided spatial information. Spatial navigation and reward checking maps showed experience-dependent learning and were in register for distal visual cues. However, they showed a dissociation, whereby distal auditory cues failed to support spatial navigation but did support spatially localized reward checking. These findings indicate that rats can navigate in virtual space with only distal visual cues, without significant vestibular or other sensory inputs. Furthermore, they reveal the simultaneous dissociation between two reward-driven behaviors.     

Neural correlates of the difference between working memory speed and simple sensorimotor speed: an fMRI study

The difference between the speed of simple cognitive processes and the speed of complex cognitive processes has various psychological correlates. However, the neural correlates of this difference have not yet been investigated. In this study, we focused on working memory (WM) for typical complex cognitive processes. Functional magnetic resonance imaging data were acquired during the performance of an N-back task, which is a measure of WM for typical complex cognitive processes. In our N-back task, task speed and memory load were varied to identify the neural correlates responsible for the difference between the speed of simple cognitive processes (estimated from the 0-back task) and the speed of WM. Our findings showed that this difference was characterized by the increased activation in the right dorsolateral prefrontal cortex (DLPFC) and the increased functional interaction between the right DLPFC and right superior parietal lobe. Furthermore, the local gray matter volume of the right DLPFC was correlated with participants' accuracy during fast WM tasks, which in turn correlated with a psychometric measure of participants' intelligence. Our findings indicate that the right DLPFC and its related network are responsible for the execution of the fast cognitive processes involved in WM. Identified neural bases may underlie the psychometric differences between the speed with which subjects perform simple cognitive tasks and the speed with which subjects perform more complex cognitive tasks, and explain the previous traditional psychological findings.     

Visual associative learning in restrained honey bees with intact antennae

A restrained honey bee can be trained to extend its proboscis in response to the pairing of an odor with a sucrose reward, a form of olfactory associative learning referred to as the proboscis extension response (PER). Although the ability of flying honey bees to respond to visual cues is well-established, associative visual learning in restrained honey bees has been challenging to demonstrate. Those few groups that have documented vision-based PER have reported that removing the antennae prior to training is a prerequisite for learning. Here we report, for a simple visual learning task, the first successful performance by restrained honey bees with intact antennae. Honey bee foragers were trained on a differential visual association task by pairing the presentation of a blue light with a sucrose reward and leaving the presentation of a green light unrewarded. A negative correlation was found between age of foragers and their performance in the visual PER task. Using the adaptations to the traditional PER task outlined here, future studies can exploit pharmacological and physiological techniques to explore the neural circuit basis of visual learning in the honey bee.     

Brain Imaging Investigation of the Impairing Effect of Emotion on Cognition

Emotions can impact cognition by exerting both enhancing (e.g., better memory for emotional events) and impairing (e.g., increased emotional distractibility) effects (reviewed in ¹). Complementing our recent protocol ² describing a method that allows investigation of the neural correlates of the memory-enhancing effect of emotion (see also ^{1, 3-5}), here we present a protocol that allows investigation of the neural correlates of the detrimental impact of emotion on cognition. The main feature of this method is that it allows identification of reciprocal modulations between activity in a ventral neural system, involved in ''hot'' emotion processing (HotEmo system), and a dorsal system, involved in higher-level ''cold'' cognitive/executive processing (ColdEx system), which are linked to cognitive performance and to individual variations in behavior (reviewed in ¹). Since its initial introduction ⁶, this design has proven particularly versatile and influential in the elucidation of various aspects concerning the neural correlates of the detrimental impact of emotional distraction on cognition, with a focus on working memory (WM), and of coping with such distraction ^7,11, in both healthy ^8-11 and clinical participants ^12-14.     

A Fully Automated High-Throughput Training System for Rodents

Addressing the neural mechanisms underlying complex learned behaviors requires training animals in well-controlled tasks, an often time-consuming and labor-intensive process that can severely limit the feasibility of such studies. To overcome this constraint, we developed a fully computer-controlled general purpose system for high-throughput training of rodents. By standardizing and automating the implementation of predefined training protocols within the animal’s home-cage our system dramatically reduces the efforts involved in animal training while also removing human errors and biases from the process. We deployed this system to train rats in a variety of sensorimotor tasks, achieving learning rates comparable to existing, but more laborious, methods. By incrementally and systematically increasing the difficulty of the task over weeks of training, rats were able to master motor tasks that, in complexity and structure, resemble ones used in primate studies of motor sequence learning. By enabling fully automated training of rodents in a home-cage setting this low-cost and modular system increases the utility of rodents for studying the neural underpinnings of a variety of complex behaviors.     

Quantifying Cognitive Decrements Caused by Cranial Radiotherapy

With the exception of survival, cognitive impairment stemming from the clinical management of cancer is a major factor dictating therapeutic outcome. For many patients afflicted with CNS and non-CNS malignancies, radiotherapy and chemotherapy offer the best options for disease control. These treatments however come at a cost, and nearly all cancer survivors (~11 million in the US alone as of 2006) incur some risk for developing cognitive dysfunction, with the most severe cases found in patients subjected to cranial radiotherapy (~200,000/yr) for the control of primary and metastatic brain tumors¹. Particularly problematic are pediatric cases, whose long-term survival plagued with marked cognitive decrements results in significant socioeconomic burdens². To date, there are still no satisfactory solutions to this significant clinical problem.We have addressed this serious health concern using transplanted stem cells to combat radiation-induced cognitive decline in athymic rats subjected to cranial irradiation³. Details of the stereotaxic irradiation and the in vitro culturing and transplantation of human neural stem cells (hNSCs) can be found in our companion paper (Acharya et al., JoVE reference). Following irradiation and transplantation surgery, rats are then assessed for changes in cognition, grafted cell survival and expression of differentiation-specific markers 1 and 4-months after irradiation. To critically evaluate the success or failure of any potential intervention designed to ameliorate radiation-induced cognitive sequelae, a rigorous series of quantitative cognitive tasks must be performed. To accomplish this, we subject our animals to a suite of cognitive testing paradigms including novel place recognition, water maze, elevated plus maze and fear conditioning, in order to quantify hippocampal and non-hippocampal learning and memory. We have demonstrated the utility of these tests for quantifying specific types of cognitive decrements in irradiated animals, and used them to show that animals engrafted with hNSCs exhibit significant improvements in cognitive function³.The cognitive benefits derived from engrafted human stem cells suggest that similar strategies may one day provide much needed clinical recourse to cancer survivors suffering from impaired cognition. Accordingly, we have provided written and visual documentation of the critical steps used in our cognitive testing paradigms to facilitate the translation of our promising results into the clinic.     

Glutamate,learning and dementia-selection of evidence

Summary Initial suggestions on the involvement of glutamate in memory came from electrophysiological studies on LTP that is blocked by NMDA-antagonists. Then Morris and colleagues (1986) provided the first evidence that icv infusion of the competitive NMDA antagonist 2-amino-5-phosphonovaleric acid (APV) to rats, inhibits both LTP in vivo and spatial learning in a Morris water maze. This was followed by a great amount of evidence confirming the initial finding in various learning tasks. The present paper is devoted to critical review of the literature focusing on the following problems: which glutamate receptors are involved?, in which tests NMDA antagonists inhibit learning?; which types of memory are affected?; which brain structures are involved?; do NMDA receptor antagonists invariably impair learning?; is the effect of NMDA receptors antagonists on learning specific?; does the stimulation of NMDA receptors result in cognitive enhancement?.     

Social Environment Influences Performance in a Cognitive Task in Natural Variants of the Foraging Gene

In Drosophila melanogaster, natural genetic variation in the foraging gene affects the foraging behaviour of larval and adult flies, larval reward learning, adult visual learning, and adult aversive training tasks. Sitters (for ^s) are more sedentary and aggregate within food patches whereas rovers (for^R) have greater movement within and between food patches, suggesting that these natural variants are likely to experience different social environments. We hypothesized that social context would differentially influence rover and sitter behaviour in a cognitive task. We measured adult rover and sitter performance in a classical olfactory training test in groups and alone. All flies were reared in groups, but fly training and testing were done alone and in groups. Sitters trained and tested in a group had significantly higher learning performances compared to sitters trained and tested alone. Rovers performed similarly when trained and tested alone and in a group. In other words, rovers learning ability is independent of group training and testing. This suggests that sitters may be more sensitive to the social context than rovers. These differences in learning performance can be altered by pharmacological manipulations of PKG activity levels, the foraging (for) gene''s gene product. Learning and memory is also affected by the type of social interaction (being in a group of the same strain or in a group of a different strain) in rovers, but not in sitters. These results suggest that for mediates social learning and memory in D. melanogaster.     

Generalization of Auditory Sensory and Cognitive Learning in Typically Developing Children

Despite the well-established involvement of both sensory (“bottom-up”) and cognitive (“top-down”) processes in literacy, the extent to which auditory or cognitive (memory or attention) learning transfers to phonological and reading skills remains unclear. Most research has demonstrated learning of the trained task or even learning transfer to a closely related task. However, few studies have reported “far-transfer” to a different domain, such as the improvement of phonological and reading skills following auditory or cognitive training. This study assessed the effectiveness of auditory, memory or attention training on far-transfer measures involving phonological and reading skills in typically developing children. Mid-transfer was also assessed through untrained auditory, attention and memory tasks. Sixty 5- to 8-year-old children with normal hearing were quasi-randomly assigned to one of five training groups: attention group (AG), memory group (MG), auditory sensory group (SG), placebo group (PG; drawing, painting), and a control, untrained group (CG). Compliance, mid-transfer and far-transfer measures were evaluated before and after training. All trained groups received 12 x 45-min training sessions over 12 weeks. The CG did not receive any intervention. All trained groups, especially older children, exhibited significant learning of the trained task. On pre- to post-training measures (test-retest), most groups exhibited improvements on most tasks. There was significant mid-transfer for a visual digit span task, with highest span in the MG, relative to other groups. These results show that both sensory and cognitive (memory or attention) training can lead to learning in the trained task and to mid-transfer learning on a task (visual digit span) within the same domain as the trained tasks. However, learning did not transfer to measures of language (reading and phonological awareness), as the PG and CG improved as much as the other trained groups. Further research is required to investigate the effects of various stimuli and lengths of training on the generalization of sensory and cognitive learning to literacy skills.     

Neural plasticity in high-level visual cortex underlying object perceptual learning

With intensive training, human can achieve impressive behavioral improvement on various perceptual tasks. This phenomenon, termed perceptual learning, has long been considered as a hallmark of the plasticity of sensory neural system. Not surprisingly, high-level vision, such as object perception, can also be improved by perceptual learning. Here we review recent psychophysical, electrophysiological, and neuroimaging studies investigating the effects of training on object selective cortex, such as monkey inferior temporal cortex and human lateral occipital area. Evidences show that learning leads to an increase in object selectivity at the single neuron level and/or the neuronal population level. These findings indicate that high-level visual cortex in humans is highly plastic and visual experience can strongly shape neural functions of these areas. At the end of the review, we discuss several important future directions in this area.     

蜜蜂对复合气味中特征因素的选择性记忆巩固的研究

目的 蜜蜂天生具有丰富的嗅觉辨识能力,觅食、交配、导航以及社交活动均依赖其嗅觉系统,是研究嗅觉感知和学习记忆的行为及神经机制的理想模型。蜜蜂既能够将某个复合气味作为一个整体也可以将复合气味的各组成成分进行辨别和区分,但是在特征依赖的联合记忆中依据何种原则进行加工并存储到长期记忆还不清楚。方法 本文利用特征阳性（feature positive：AB+,B-）和特征阴性（feature negative：AB-,B+）的奖赏性嗅觉条件化,训练蜜蜂对复合气味和成分气味的辨别,并检测蜜蜂对复合气味（AB）、成分气味（B）以及特征气味（A）的中长时记忆（3 h）和长时记忆（24 h）。结果 在特征阳性的奖赏性嗅觉条件化中,蜜蜂对训练过的气味可以形成稳定的中长时和长时记忆,并且对复合气味中的特征气味的记忆与复合气味的记忆呈现高度相似。但在特征阴性的奖赏性嗅觉条件化中,蜜蜂虽能够在3 h和24 h对训练过的两种气味具有显著的伸喙反应差异,且对特征阴性的气味无显著反应,但对复合气味的反应随时间的推移而增加。结论 实验结果表明,蜜蜂选择性地将与奖赏信息联合出现的气味巩固到长时记忆中,但并未依据特征成分加工储存到长时记忆中。奖赏信息预示着食物源,与生存息息相关,表明对环境信息进行选择性的记忆巩固加工并储存可能是低等动物高效地编码生存相关信息的重要策略。     

Reinforcement Learning on Slow Features of High-Dimensional Input Streams

Humans and animals are able to learn complex behaviors based on a massive stream of sensory information from different modalities. Early animal studies have identified learning mechanisms that are based on reward and punishment such that animals tend to avoid actions that lead to punishment whereas rewarded actions are reinforced. However, most algorithms for reward-based learning are only applicable if the dimensionality of the state-space is sufficiently small or its structure is sufficiently simple. Therefore, the question arises how the problem of learning on high-dimensional data is solved in the brain. In this article, we propose a biologically plausible generic two-stage learning system that can directly be applied to raw high-dimensional input streams. The system is composed of a hierarchical slow feature analysis (SFA) network for preprocessing and a simple neural network on top that is trained based on rewards. We demonstrate by computer simulations that this generic architecture is able to learn quite demanding reinforcement learning tasks on high-dimensional visual input streams in a time that is comparable to the time needed when an explicit highly informative low-dimensional state-space representation is given instead of the high-dimensional visual input. The learning speed of the proposed architecture in a task similar to the Morris water maze task is comparable to that found in experimental studies with rats. This study thus supports the hypothesis that slowness learning is one important unsupervised learning principle utilized in the brain to form efficient state representations for behavioral learning.     

Operant Procedures for Assessing Behavioral Flexibility in Rats

Executive functions consist of multiple high-level cognitive processes that drive rule generation and behavioral selection. An emergent property of these processes is the ability to adjust behavior in response to changes in one’s environment (i.e., behavioral flexibility). These processes are essential to normal human behavior, and may be disrupted in diverse neuropsychiatric conditions, including schizophrenia, alcoholism, depression, stroke, and Alzheimer’s disease. Understanding of the neurobiology of executive functions has been greatly advanced by the availability of animal tasks for assessing discrete components of behavioral flexibility, particularly strategy shifting and reversal learning. While several types of tasks have been developed, most are non-automated, labor intensive, and allow testing of only one animal at a time. The recent development of automated, operant-based tasks for assessing behavioral flexibility streamlines testing, standardizes stimulus presentation and data recording, and dramatically improves throughput. Here, we describe automated strategy shifting and reversal tasks, using operant chambers controlled by custom written software programs. Using these tasks, we have shown that the medial prefrontal cortex governs strategy shifting but not reversal learning in the rat, similar to the dissociation observed in humans. Moreover, animals with a neonatal hippocampal lesion, a neurodevelopmental model of schizophrenia, are selectively impaired on the strategy shifting task but not the reversal task. The strategy shifting task also allows the identification of separate types of performance errors, each of which is attributable to distinct neural substrates. The availability of these automated tasks, and the evidence supporting the dissociable contributions of separate prefrontal areas, makes them particularly well-suited assays for the investigation of basic neurobiological processes as well as drug discovery and screening in disease models.     

Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology

Neurons in a small number of brain structures detect rewards and reward-predicting stimuli and are active during the expectation of predictable food and liquid rewards. These neurons code the reward information according to basic terms of various behavioural theories that seek to explain reward-directed learning, approach behaviour and decision-making. The involved brain structures include groups of dopamine neurons, the striatum including the nucleus accumbens, the orbitofrontal cortex and the amygdala. The reward information is fed to brain structures involved in decision-making and organisation of behaviour, such as the dorsolateral prefrontal cortex and possibly the parietal cortex. The neural coding of basic reward terms derived from formal theories puts the neurophysiological investigation of reward mechanisms on firm conceptual grounds and provides neural correlates for the function of rewards in learning, approach behaviour and decision-making.     

Adaptive properties of differential learning rates for positive and negative outcomes

The concept of the reward prediction error—the difference between reward obtained and reward predicted—continues to be a focal point for much theoretical and experimental work in psychology, cognitive science, and neuroscience. Models that rely on reward prediction errors typically assume a single learning rate for positive and negative prediction errors. However, behavioral data indicate that better-than-expected and worse-than-expected outcomes often do not have symmetric impacts on learning and decision-making. Furthermore, distinct circuits within cortico-striatal loops appear to support learning from positive and negative prediction errors, respectively. Such differential learning rates would be expected to lead to biased reward predictions and therefore suboptimal choice performance. Contrary to this intuition, we show that on static “bandit” choice tasks, differential learning rates can be adaptive. This occurs because asymmetric learning enables a better separation of learned reward probabilities. We show analytically how the optimal learning rate asymmetry depends on the reward distribution and implement a biologically plausible algorithm that adapts the balance of positive and negative learning rates from experience. These results suggest specific adaptive advantages for separate, differential learning rates in simple reinforcement learning settings and provide a novel, normative perspective on the interpretation of associated neural data.