Perceptual learning and decision-making in human medial frontal cortex

The dominant view that perceptual learning is accompanied by changes in early sensory representations has recently been challenged. Here we tested the idea that perceptual learning can be accounted for by reinforcement learning involving changes in higher decision-making areas. We trained subjects on an orientation discrimination task involving feedback over 4 days, acquiring fMRI data on the first and last day. Behavioral improvements were well explained by a reinforcement learning model in which learning leads to enhanced readout of sensory information, thereby establishing noise-robust representations of decision variables. We find stimulus orientation encoded in early visual and higher cortical regions such as lateral parietal cortex and anterior cingulate cortex (ACC). However, only activity patterns in the ACC tracked changes in decision variables during learning. These results provide strong evidence for perceptual learning-related changes in higher order areas and suggest that perceptual and reward learning are based on a common neurobiological mechanism.     

Amphetamine-Induced Sensitization Has Little Effect on Multiple Learning Paradigms and Fails to Rescue Mice with a Striatal Learning Defect

Behavioral sensitization to psychostimulants such as amphetamine (AMPH) is associated with synaptic modifications that are thought to underlie learning and memory. Because AMPH enhances extracellular dopamine in the striatum where dopamine and glutamate signaling are essential for learning, one might expect that the molecular and morphological changes that occur in the striatum in response to AMPH, including changes in synaptic plasticity, would affect learning. To ascertain whether AMPH sensitization affects learning, we tested wild-type mice and mice lacking NMDA receptor signaling in striatal medium spiny neurons in several different learning tests (motor learning, Pavlovian association, U-maze escape test with strategy shifting) with or without prior sensitization to AMPH. Prior sensitization had minimal effect on learning in any of these paradigms in wild-type mice and failed to restore learning in mutant mice, despite the fact that the mutant mice became sensitized by the AMPH treatment. We conclude that the changes in synaptic plasticity and many other signaling events that occur in response to AMPH sensitization are dissociable from those involved in learning the tasks used in our experiments.     

A parameter control method in reinforcement learning to rapidly follow unexpected environmental changes

In order to rapidly follow unexpected environmental changes, we propose a parameter control method in reinforcement learning that changes each of learning parameters in appropriate directions. We determine each appropriate direction on the basis of relationships between behaviors and neuromodulators by considering an emergency as a key word. Computer experiments show that the agents using our proposed method could rapidly respond to unexpected environmental changes, not depending on either two reinforcement learning algorithms (Q-learning and actor-critic (AC) architecture) or two learning problems (discontinuous and continuous state-action problems).     

Hippocampus and context in classical conditioning.

Recent evidence suggests that contextual learning encompasses a variety of changes in learning and performance processes. Only some of these changes depend on the hippocampus. Specialized functions proposed for the hippocampus in contextual learning include the construction and consolidation of contextual memory representations, incidental contextual learning, and inhibitory contextual learning.     

鸣禽鸣叫学习的神经生物学机制

鸣禽鸣叫具有复杂的神经生理和生化基础,表现为一种复杂的学习过程。鸣啭控制系统是研究神经系统与学习、行为和发育关系的重要模型。而鸣禽鸣叫学习行为与鸣啭控制系统内长时程增强效应、神经元超微结构的改变和神经核团内的电活动、激素水平高低及其周期性变化、神经元再生或改变、即早基因的表达等方面密切相关。对鸣禽鸣叫的神经生物学机制进行了综述。     

Interrelated mechanisms in reward and learning

This brief review is focused on recent work in our laboratory, in which we assayed nicotine-induced neurotransmitter changes, comparing it to changes induced by other compounds and examined the receptor systems and their interactions that mediate the changes. The primary aim of our studies is to examine the role of neurotransmitter changes in reward and learning processes. We find that these processes are interlinked and interact in that reward-addiction mechanisms include processes of learning and learning-memory mechanisms include processes of reward. In spite being interlinked, the two processes have different functions and distinct properties and our long-term aim is to identify factors that control these processes and the differences among the processes. Here, we discuss reward processes, which we define as changes examined after administration of nicotine, cocaine or food, each of which induces changes in neurotransmitter levels and functions in cognitive areas as well as in reward areas. The changes are regionally heterogeneous and are drug or stimulus specific. They include changes in the transmitters assayed (catecholamines, amino acids, and acetylcholine) and also in their metabolites, hence, in addition to release, uptake and metabolism are involved. Many receptors modulate the response with direct and indirect effects. The involvement of many transmitters, receptors and their interactions and the stimulus specificity of the response indicated that each function, reward and learning represents the involvement of different pattern of changes with a different stimulus, therefore, many different learning and many different reward processes are active, which allow stimulus specific responses. The complex pattern of reward-induced changes in neurotransmitters is only a part of the multiple changes observed, but one which has a crucial and controlling function.     

Drifting towards a diffuse control model of exploratory motor learning: A comparison of global and within-trial performance measures

 Accurate measurement is crucial for understanding the processes that underlie exploratory patterns in motor learning. Accordingly, measures of learning should be sensitive to the changes that take place during skill acquisition. Most studies, however, use trial-based global measures that assess performance but do not actually measure gradual changes taking place within trials. The present study attempted to remedy this shortcoming by analysing a visual adaptation task, and comparing traditional global measures of learning with new, within-trial measures. Movement time was the only global measure sensitive to changes in the movement trajectory during learning. Three new measures were expected to reveal changes to the movement trajectory that are associated with learning: (i) the length of runs, (ii) change of trajectory angle in relation to the target, and (iii) drift (change in distance from the target). All three measures were sensitive to learning and indicated a gradual straightening of the movement trajectories over trials. Furthermore, three different methods to partition trajectories into segments were examined. The new within-trial measures, irrespective of partitioning method, prove promising for the development of a diffuse control model of exploratory learning. Received: 5 February 2001 / Accepted in revised form: 16 January 2002     

Learning about Time: Plastic Changes and Interindividual Brain Differences

Learning the timing of rapidly changing sensory events is crucial to construct a reliable representation of the environment and to efficiently control behavior. The neurophysiological mechanisms underlying the learning of time are unknown. We used functional and structural magnetic resonance imaging to investigate neurophysiological changes and individual brain differences underlying the learning of time in the millisecond range. We found that the representation of a trained visual temporal interval was associated with functional and structural changes in a sensory-motor network including occipital, parietal, and insular cortices, plus the cerebellum. We show that both types of neurophysiological changes correlated with changes of performance accuracy and that activity and gray-matter volume of sensorimotor cortices predicted individual learning abilities. These findings represent neurophysiological evidence of functional and structural plasticity associated with the learning of time in humans and highlight the role of sensory-motor circuits in the perceptual representation of time in the millisecond range.     

Neural Activations during Visual Sequence Learning Leave a Trace in Post-Training Spontaneous EEG

Recent EEG studies have shown that implicit learning involving specific cortical circuits results in an enduring local trace manifested as local changes in spectral power. Here we used a well characterized visual sequence learning task and high density-(hd-)EEG recording to determine whether also declarative learning leaves a post-task, local change in the resting state oscillatory activity in the areas involved in the learning process. Thus, we recorded hd-EEG in normal subjects before, during and after the acquisition of the order of a fixed spatial target sequence (VSEQ) and during the presentation of targets in random order (VRAN). We first determined the temporal evolution of spectral changes during VSEQ and compared it to VRAN. We found significant differences in the alpha and theta bands in three main scalp regions, a right occipito-parietal (ROP), an anterior-frontal (AFr), and a right frontal (RFr) area. The changes in frontal theta power during VSEQ were positively correlated with the learning rate. Further, post-learning EEG recordings during resting state revealed a significant increase in alpha power in ROP relative to a pre-learning baseline. We conclude that declarative learning is associated with alpha and theta changes in frontal and posterior regions that occur during the task, and with an increase of alpha power in the occipito-parietal region after the task. These post-task changes may represent a trace of learning and a hallmark of use-dependent plasticity.     

Expressions of multiple neuronal dynamics during sensorimotor learning in the motor cortex of behaving monkeys

Previous studies support the notion that sensorimotor learning involves multiple processes. We investigated the neuronal basis of these processes by recording single-unit activity in motor cortex of non-human primates (Macaca fascicularis), during adaptation to force-field perturbations. Perturbed trials (reaching to one direction) were practiced along with unperturbed trials (to other directions). The number of perturbed trials relative to the unperturbed ones was either low or high, in two separate practice schedules. Unsurprisingly, practice under high-rate resulted in faster learning with more pronounced generalization, as compared to the low-rate practice. However, generalization and retention of behavioral and neuronal effects following practice in high-rate were less stable; namely, the faster learning was forgotten faster. We examined two subgroups of cells and showed that, during learning, the changes in firing-rate in one subgroup depended on the number of practiced trials, but not on time. In contrast, changes in the second subgroup depended on time and practice; the changes in firing-rate, following the same number of perturbed trials, were larger under high-rate than low-rate learning. After learning, the neuronal changes gradually decayed. In the first subgroup, the decay pace did not depend on the practice rate, whereas in the second subgroup, the decay pace was greater following high-rate practice. This group shows neuronal representation that mirrors the behavioral performance, evolving faster but also decaying faster at learning under high-rate, as compared to low-rate. The results suggest that the stability of a new learned skill and its neuronal representation are affected by the acquisition schedule.     

Efficacy of repeated training at different stages of elaboration of a defensive reflex in the mouse

It has been found that the efficiency of additional learning is in no linear dependency on interval value between initial and repeated conditioning sessions. At consecutive interval extension the efficiency of additional session changes wave-like. The effect of repeated learning session directly correlates with successful reproduction of traces of initial reflex elaboration in the second session. It is suggested that fluctuations of reproduction of unstable reflexes and of efficiency of additional learning sessions depend on changes in spatial relations of bioelectrical processes in the cortex which accompany the additional learning.     

Associative fear learning enhances sparse network coding in primary sensory cortex

Several models of associative learning predict that stimulus processing changes during association formation. How associative learning reconfigures neural circuits in primary sensory cortex to "learn" associative attributes of a stimulus remains unknown. Using 2-photon in vivo calcium imaging to measure responses of networks of neurons in primary somatosensory cortex, we discovered that associative fear learning, in which whisker stimulation is paired with foot shock, enhances sparse population coding and robustness of the conditional stimulus, yet decreases total network activity. Fewer cortical neurons responded to stimulation of the trained whisker than in controls, yet their response strength was enhanced. These responses were not observed in mice exposed to a nonassociative learning procedure. Our results define how the cortical representation of a sensory stimulus is shaped by associative fear learning. These changes are proposed to enhance efficient sensory processing after associative learning.     

Computational model of motor learning and perceptual change

Motor learning in the context of arm reaching movements has been frequently investigated using the paradigm of force-field learning. It has been recently shown that changes to somatosensory perception are likewise associated with motor learning. Changes in perceptual function may be the reason that when the perturbation is removed following motor learning, the hand trajectory does not return to a straight line path even after several dozen trials. To explain the computational mechanisms that produce these characteristics, we propose a motor control and learning scheme using a simplified two-link system in the horizontal plane: We represent learning as the adjustment of desired joint-angular trajectories so as to achieve the reference trajectory of the hand. The convergence of the actual hand movement to the reference trajectory is proved by using a Lyapunov-like lemma, and the result is confirmed using computer simulations. The model assumes that changes in the desired hand trajectory influence the perception of hand position and this in turn affects movement control. Our computer simulations support the idea that perceptual change may come as a result of adjustments to movement planning with motor learning.     

Appetitive odor learning does not change olfactory coding in a subpopulation of honeybee antennal lobe neurons

Odors elicit spatio-temporal patterns of activity in the olfactory bulb of vertebrates and the antennal lobe of insects. There have been several reports of changes in these patterns following olfactory learning. These studies pose a conundrum: how can an animal learn to efficiently respond to a particular odor with an adequate response, if its primary representation already changes during this process? In this study, we offer a possible solution for this problem. We measured odor-evoked calcium responses in a subpopulation of uniglomerular AL output neurons in honeybees. We show that their responses to odors are remarkably resistant to plasticity following a variety of appetitive olfactory learning paradigms. There was no significant difference in the changes of odor-evoked activity between single and multiple trial forward or backward conditioning, differential conditioning, or unrewarded successive odor stimulation. In a behavioral learning experiment we show that these neurons are necessary for conditioned odor responses. We conclude that these uniglomerular projection neurons are necessary for reliable odor coding and are not modified by learning in this paradigm. The role that other projection neurons play in olfactory learning remains to be investigated.     

The 50s Cliff: Perceptuo-Motor Learning Rates across the Lifespan

We recently found that older adults show reduced learning rates when learning a new pattern of coordinated rhythmic movement. The purpose of this study was to extend that finding by examining the performance of all ages across the lifespan from the 20 s through to the 80 s to determine how learning rates change with age. We tested whether adults could learn to produce a novel coordinated rhythmic movement (90° relative phase) in a visually guided unimanual task. We determined learning rates to quantify changes in learning with age and to determine at what ages the changes occur. We found, as before, that learning rates of participants in their 70 s and 80 s were half those of participants in their 20 s. We also found a gradual slow decline in learning rate with age until approximately age 50, when there was a sudden drop to a reduced learning rate for the 60 though 80 year olds. We discuss possible causes for the “50 s cliff” in perceptuo-motor learning rates and suggest that age related deficits in perception of complex motions may be the key to understanding this result.     

Representational drift as a window into neural and behavioural plasticity

Large-scale recordings of neural activity over days and weeks have revealed that neural representations of familiar tasks, precepts and actions continually evolve without obvious changes in behaviour. We hypothesise that this steady drift in neural activity and accompanying physiological changes is due in part to the continuous application of a learning rule at the cellular and population level. Explicit predictions of this drift can be found in neural network models that use iterative learning to optimise weights. Drift therefore provides a measurable signal that can reveal systems–level properties of biological plasticity mechanisms, such as their precision and effective learning rates.     

Parallels between cerebellum- and amygdala-dependent conditioning

Recent evidence from cerebellum-dependent motor learning and amygdala-dependent fear conditioning indicates that, despite being mediated by different brain systems, these forms of learning might use a similar sequence of events to form new memories. In each case, learning seems to induce changes in two different groups of neurons. Changes in the first class of cells are induced very rapidly during the initial stages of learning, whereas changes in the second class of cells develop more slowly and are resistant to extinction. So, anatomically distinct cell populations might contribute differentially to the initial encoding and the long-term storage of memory in these two systems.     

Specificity of sensorimotor learning and the neural code: Neuronal representations in the primary motor cortex

Human studies show that the learning of a new sensorimotor mapping that requires adaptation to directional errors is local and generalizes poorly to untrained directions. We trained monkeys to learn new visuomotor rotations for only one target in space and recorded neuronal activity in the primary motor cortex before, during and after learning. Similar to humans, the monkeys showed poor transfer of learning to other directions, as observed by behavioral aftereffects for untrained directions. To test for internal representations underlying these changes, we compared two features of neuronal activity before and after learning: changes in firing rates and changes in information content. Specific elevations of firing rate were only observed in a subpopulation of cells in the motor cortex with directional properties corresponding to the locally learned rotation; namely cells only showed plasticity if their preferred direction was near the training one. We applied measures from information theory to probe for learning-related changes in the neuronal code. Single cells conveyed more information about the direction of movement and this specific improvement in encoding was correlated with an increase in the slope of the neurons' tuning curve. Further, the improved information after learning enabled a more accurate reconstruction of movement direction from neuronal populations. Our findings suggest a neural mechanism for the confined generalization of a newly acquired internal model by showing a tight relationship between the locality of learning and the properties of neurons. They also provide direct evidence for improvement in the neural code as a result of learning.     

A Mathematical Model of the Cerebellar-Olivary System II: Motor Adaptation Through Systematic Disruption of Climbing Fiber Equilibrium

The implications for motor learning of the model developed in the previous article are analyzed using idealized Pavlovian eyelid conditioning trials, a simple example of cerebellar motor learning. Results suggest that changes in grPkj synapses produced by a training trial disrupt equilibrium and lead to subsequent changes in the opposite direction that restore equilibrium. We show that these opposing phases would make the net plasticity at each grPkj synapse proportional to the change in its activity during the training trial, as influenced by a factor that precludes plasticity when changes in activity are inconsistent. This yields an expression for the component of granule cell activity that supports learning, the across-trials consistency vector, the square of which determines the expected rate of learning. These results suggest that the equilibrium maintained by the cerebellar-olivary system must be disrupted in a specific and systematic manner to promote cerebellar-mediated motor learning.     

Tracking Plasticity: Effects of Long-Term Rehearsal in Expert Dancers Encoding Music to Movement

Our knowledge of neural plasticity suggests that neural networks show adaptation to environmental and intrinsic change. In particular, studies investigating the neuroplastic changes associated with learning and practicing motor tasks have shown that practicing such tasks results in an increase in neural activation in several specific brain regions. However, studies comparing experts and non-experts suggest that experts employ less neuronal activation than non-experts when performing a familiar motor task. Here, we aimed to determine the long-term changes in neural networks associated with learning a new dance in professional ballet dancers over 34 weeks. Subjects visualized dance movements to music while undergoing fMRI scanning at four time points over 34-weeks. Results demonstrated that initial learning and performance at seven weeks led to increases in activation in cortical regions during visualization compared to the first week. However, at 34 weeks, the cortical networks showed reduced activation compared to week seven. Specifically, motor learning and performance over the 34 weeks showed the typical inverted-U-shaped function of learning. Further, our result demonstrate that learning of a motor sequence of dance movements to music in the real world can be visualized by expert dancers using fMRI and capture highly significant modeled fits of the brain network variance of BOLD signals from early learning to expert level performance.