A neural circuit model of flexible sensorimotor mapping: learning and forgetting on multiple timescales

Volitional behavior relies on the brain's ability to remap sensory flow to motor programs whenever demanded by a changed behavioral context. To investigate the circuit basis of such flexible behavior, we have developed a biophysically based decision-making network model of spiking neurons for arbitrary sensorimotor mapping. The model quantitatively reproduces behavioral and prefrontal single-cell data from an experiment in which monkeys learn visuomotor associations that are reversed unpredictably from time to time. We show that when synaptic modifications occur on multiple timescales, the model behavior becomes flexible only when needed: slow components of learning usually dominate the decision process. However, if behavioral contexts change frequently enough, fast components of plasticity take over, and the behavior exhibits a quick forget-and-learn pattern. This model prediction is confirmed by monkey data. Therefore, our work reveals a scenario for conditional associative learning that is distinct from instant switching between sets of well-established sensorimotor associations.     

Prediction error during retrospective revaluation of causal associations in humans: fMRI evidence in favor of an associative model of learning

Associative learning theory assumes that prediction error is a driving force in learning. A competing view, probabilistic contrast (PC) theory, is that learning and prediction error are unrelated. We tested a learning phenomenon that has proved troublesome for associative theory--retrospective revaluation--to evaluate these two models. We previously showed that activation in right lateral prefrontal cortex (PFC) provides a reliable signature for the presence of prediction error. Thus, if the associative view is correct, retrospective revaluation should be accompanied by right lateral PFC activation. PC theory would be supported by the absence of this activation. Right PFC and ventral striatal activation occurred during retrospective revaluation, supporting the associative account. Activations appeared to reflect the degree of revaluation, predicting later brain responses to revalued cues. Our results support a modified associative account of retrospective revaluation and demonstrate the potential of functional neuroimaging as a tool for evaluating competing learning models.     

A Predictive Reinforcement Model of Dopamine Neurons for Learning Approach Behavior

A neural network model of how dopamine and prefrontal cortex activity guides short- and long-term information processing within the cortico-striatal circuits during reward-related learning of approach behavior is proposed. The model predicts two types of reward-related neuronal responses generated during learning: (1) cell activity signaling errors in the prediction of the expected time of reward delivery and (2) neural activations coding for errors in the prediction of the amount and type of reward or stimulus expectancies. The former type of signal is consistent with the responses of dopaminergic neurons, while the latter signal is consistent with reward expectancy responses reported in the prefrontal cortex. It is shown that a neural network architecture that satisfies the design principles of the adaptive resonance theory of Carpenter and Grossberg (1987) can account for the dopamine responses to novelty, generalization, and discrimination of appetitive and aversive stimuli. These hypotheses are scrutinized via simulations of the model in relation to the delivery of free food outside a task, the timed contingent delivery of appetitive and aversive stimuli, and an asymmetric, instructed delay response task.     

Dynamics of population code for working memory in the prefrontal cortex

Some neurons (delay cells) in the prefrontal cortex elevate their activities throughout the time period during which the animal is required to remember past events and prepare future behavior, suggesting that working memory is mediated by continuous neural activity. It is unknown, however, how working memory is represented within a population of prefrontal cortical neurons. We recorded from neuronal ensembles in the prefrontal cortex as rats learned a new delayed alternation task. Ensemble activities changed in parallel with behavioral learning so that they increasingly allowed correct decoding of previous and future goal choices. In well-trained rats, considerable decoding was possible based on only a few neurons and after removing continuously active delay cells. These results show that neural activity in the prefrontal cortex changes dynamically during new task learning so that working memory is robustly represented and that working memory can be mediated by sequential activation of different neural populations.     

前额叶皮层在社交行为中的作用

社交行为对于个体身心健康和社会发展都极其重要。社交行为障碍已成为多种精神类疾病的典型临床表征,对个体的发展有严重不良影响。前额叶皮层作为调节社交行为的关键脑区之一,参与了社交、情绪、决策等高级功能,其内部神经元、神经胶质细胞的活动变化及相互作用对调节社交行为有着重要影响,而且前额叶皮层与其他脑区之间的协作也会影响不同的社会行为。本文回顾了前额叶皮层中神经元、神经胶质细胞以及脑区投射与社交行为关系的最新研究,系统综述了前额叶皮层在社交行为调节中的作用,以期为社交障碍的神经机制和有效诊疗提供参考。     

Behavioral planning in the prefrontal cortex

Recent studies have presented evidence that the prefrontal cortex plays a crucial role in every aspect of the cognitive processes necessary for behavioral planning: processing and integration of perceived or memorized information, associative learning, reward-based behavioral control, behavioral selection/decision-making and behavioral guidance. We propose that the creation of novel information is the means by which the prefrontal cortex operates to achieve executive control over behavioral planning. The prefrontal cortex is the site of operation of nodal points, where neural circuits integrate currently available or memorized information to generate the information that is necessary to perform an action. The prefrontal cortex also regulates the flow of information through multiple nodes to meet behavioral demands.     

Prefrontal cortex function, quasi-physiological stimuli, and synaptic plasticity

The prelimbic area of rat medial frontal cortex may be functionally analogous to human/primate dorsolateral prefrontal cortex. This area may be involved in selective attention to the external stimuli and the coupling of the attention to a repertory of actions. It was suggested that this function may rely on a form of long-term memory [Biol. Rev. 77 (2002) 563]. Indeed, during learning of this type of behavior, a portion of prelimbic neurons persistently change their firing characteristics [Prog. Brain Res. 126 (2000) 287]. It is therefore important to study long-term potentiation (LTP) and depression (LTD) in rat prelimbic neurons. In this article, the author first briefly reviews recent findings on the prefrontal cortex function and discusses that the prefrontal cortex may be involved in long-term memory. Second, the author will show some new results which indicate that quasi-physiological patterns of stimuli mimicking prelimbic neuronal activity during behavior can induce LTP in prelimbic pyramidal neuron synapses. These results suggest that prelimbic neuronal activity during behavior may lastingly modify prelimbic synaptic efficacy.     

Effects of subclinical depression on prefrontal–striatal model-based and model-free learning

Depression is characterized by deficits in the reinforcement learning (RL) process. Although many computational and neural studies have extended our knowledge of the impact of depression on RL, most focus on habitual control (model-free RL), yielding a relatively poor understanding of goal-directed control (model-based RL) and arbitration control to find a balance between the two. We investigated the effects of subclinical depression on model-based and model-free learning in the prefrontal–striatal circuitry. First, we found that subclinical depression is associated with the attenuated state and reward prediction error representation in the insula and caudate. Critically, we found that it accompanies the disrupted arbitration control between model-based and model-free learning in the predominantly inferior lateral prefrontal cortex and frontopolar cortex. We also found that depression undermines the ability to exploit viable options, called exploitation sensitivity. These findings characterize how subclinical depression influences different levels of the decision-making hierarchy, advancing previous conflicting views that depression simply influences either habitual or goal-directed control. Our study creates possibilities for various clinical applications, such as early diagnosis and behavioral therapy design.     

海马-前额叶神经回路与工作记忆

学习和记忆是神经科学研究的热点。已证实大脑中的海马、前额叶和海马−前额叶回路均参与工作记忆功能。本文对海马−前额叶回路的解剖和生理特性以及海马、前额叶和海马−前额叶回路在工作记忆中的作用的研究进展做一概述。     

Female Adolescents with Severe Substance and Conduct Problems Have Substantially Less Brain Gray Matter Volume

ObjectiveStructural neuroimaging studies have demonstrated lower regional gray matter volume in adolescents with severe substance and conduct problems. These research studies, including ours, have generally focused on male-only or mixed-sex samples of adolescents with conduct and/or substance problems. Here we compare gray matter volume between female adolescents with severe substance and conduct problems and female healthy controls of similar ages. Hypotheses: Female adolescents with severe substance and conduct problems will show significantly less gray matter volume in frontal regions critical to inhibition (i.e. dorsolateral prefrontal cortex and ventrolateral prefrontal cortex), conflict processing (i.e., anterior cingulate), valuation of expected outcomes (i.e., medial orbitofrontal cortex) and the dopamine reward system (i.e. striatum).MethodsWe conducted whole-brain voxel-based morphometric comparison of structural MR images of 22 patients (14-18 years) with severe substance and conduct problems and 21 controls of similar age using statistical parametric mapping (SPM) and voxel-based morphometric (VBM8) toolbox. We tested group differences in regional gray matter volume with analyses of covariance, adjusting for age and IQ at p<0.05, corrected for multiple comparisons at whole-brain cluster-level threshold.ResultsFemale adolescents with severe substance and conduct problems compared to controls showed significantly less gray matter volume in right dorsolateral prefrontal cortex, left ventrolateral prefrontal cortex, medial orbitofrontal cortex, anterior cingulate, bilateral somatosensory cortex, left supramarginal gyrus, and bilateral angular gyrus. Considering the entire brain, patients had 9.5% less overall gray matter volume compared to controls.ConclusionsFemale adolescents with severe substance and conduct problems in comparison to similarly aged female healthy controls showed substantially lower gray matter volume in brain regions involved in inhibition, conflict processing, valuation of outcomes, decision-making, reward, risk-taking, and rule-breaking antisocial behavior.     

Neural Computations Mediating One-Shot Learning in the Human Brain

Incremental learning, in which new knowledge is acquired gradually through trial and error, can be distinguished from one-shot learning, in which the brain learns rapidly from only a single pairing of a stimulus and a consequence. Very little is known about how the brain transitions between these two fundamentally different forms of learning. Here we test a computational hypothesis that uncertainty about the causal relationship between a stimulus and an outcome induces rapid changes in the rate of learning, which in turn mediates the transition between incremental and one-shot learning. By using a novel behavioral task in combination with functional magnetic resonance imaging (fMRI) data from human volunteers, we found evidence implicating the ventrolateral prefrontal cortex and hippocampus in this process. The hippocampus was selectively “switched” on when one-shot learning was predicted to occur, while the ventrolateral prefrontal cortex was found to encode uncertainty about the causal association, exhibiting increased coupling with the hippocampus for high-learning rates, suggesting this region may act as a “switch,” turning on and off one-shot learning as required.     

Subjecting elite athletes to inspiratory breathing load reveals behavioral and neural signatures of optimal performers in extreme environments

Background

It is unclear whether and how elite athletes process physiological or psychological challenges differently than healthy comparison subjects. In general, individuals optimize exercise level as it relates to differences between expected and experienced exertion, which can be conceptualized as a body prediction error. The process of computing a body prediction error involves the insular cortex, which is important for interoception, i.e. the sense of the physiological condition of the body. Thus, optimal performance may be related to efficient minimization of the body prediction error. We examined the hypothesis that elite athletes, compared to control subjects, show attenuated insular cortex activation during an aversive interoceptive challenge.

Methodology/Principal Findings

Elite adventure racers (n = 10) and healthy volunteers (n = 11) performed a continuous performance task with varying degrees of a non-hypercapnic breathing load while undergoing functional magnetic resonance imaging. The results indicate that (1) non-hypercapnic inspiratory breathing load is an aversive experience associated with a profound activation of a distributed set of brain areas including bilateral insula, dorsolateral prefrontal cortex and anterior cingulated; (2) adventure racers relative to comparison subjects show greater accuracy on the continuous performance task during the aversive interoceptive condition; and (3) adventure racers show an attenuated right insula cortex response during and following the aversive interoceptive condition of non-hypercapnic inspiratory breathing load.

Conclusions/Significance

These findings support the hypothesis that elite athletes during an aversive interoceptive condition show better performance and an attenuated insular cortex activation during the aversive experience. Interestingly, differential modulation of the right insular cortex has been found previously in elite military personnel and appears to be emerging as an important brain system for optimal performance in extreme environments.     

The avian 'prefrontal cortex' and cognition

Both mammals and birds can flexibly organize their behavior over time. In mammals, the mental operations generating this ability are called executive functions and are associated with the prefrontal cortex. The corresponding structure in birds is the nidopallium caudolaterale. Anatomical, neurochemical, electrophysiological and behavioral studies show these structures to be highly similar. The avian forebrain displays no lamination that corresponds to the mammalian neocortex, hence lamination does not seem to be a requirement for higher cognitive functions. Because all other aspects of the neural architecture of the mammalian and the avian prefrontal areas are extremely comparable, the freedom to create different neural architectures that generate prefrontal functions seems to be very limited.     

Motor learning in man: A review of functional and clinical studies

This chapter reviews results of clinical and functional imaging studies which investigated the time-course of cortical and subcortical activation during the acquisition of motor a skill. During the early phases of learning by trial and error, activation in prefrontal areas, especially in the dorsolateral prefrontal cortex, is has been reported. The role of these areas is presumably related to explicit working memory and the establishment of a novel association between visual cues and motor commands. Furthermore, motor associated areas of the right hemisphere and distributed cerebellar areas reveal strong activation during the early motor learning. Activation in superior-posterior parietal cortex presumably arises from visuospatial processes, while sensory feedback is coded in the anterior-inferior parietal cortex and the neocerebellar structures. With practice, motor associated areas of the left-hemisphere reveal increased activity. This shift to the left hemisphere has been observed regardless of the hand used during training, indicating a left-hemispheric dominance in the storage of visuomotor skills. Concerning frontal areas, learned actions of sequential character are represented in the caudal part of the supplementary motor area (SMA proper), whereas the lateral premotor cortex appears to be responsible for the coding of the association between visuo-spatial information and motor commands. Functional imaging studies which investigated the activation patterns of motor learning under implicit conditions identified for the first, a motor circuit which includes lateral premotor cortex and SMA proper of the left hemisphere and primary motor cortex, for the second, a cognitive loop which consists of basal ganglia structures of the right hemisphere. Finally, activity patterns of intermanual transfer are discussed. After right-handed training, activity in motor associated areas maintains during performance of the mirror version, but is increased during the performance of the original-oriented version with the left hand. In contrary, increased activity during the mirror reversed action, but not during the original-oriented performance of the untrained right hand is observed after left-handed training. These results indicate the transfer of acquired right-handed information which reflects the mirror symmetry of the body, whereas spatial information is mainly transferred after left-handed training. Taken together, a combined approach of clinical lesion studies and functional imaging is a promising tool for identifying the cerebral regions involved in the process of motor learning and provides insight into the mechanisms underlying the generalisation of actions.     

Astral microtubules control redistribution of dynein at the cell cortex to facilitate spindle positioning

Cytoplasmic dynein is recruited to the cell cortex in early mitosis, where it can generate pulling forces on astral microtubules to position the mitotic spindle. Recent work has shown that dynein displays a dynamic asymmetric cortical localization, and that dynein recruitment is negatively regulated by spindle pole-proximity. This results in oscillating dynein recruitment to opposite sides of the cortex to center the mitotic spindle. However, although the centrosome-derived signal that promotes displacement of dynein has been identified, it is currently unknown how dynein is re-recruited to the cortex once it has been displaced. Here we show that re-recruitment of cortical dynein requires astral microtubules. We find that microtubules are necessary for the sustained localized enrichment of dynein at the cortex. Furthermore, we show that stabilization of astral microtubules causes spindle misorientation, followed by mispositioning of dynein at the cortex. Thus, our results demonstrate the importance of astral microtubules in the dynamic regulation of cortical dynein recruitment in mitosis.     

Broca's area and the hierarchical organization of human behavior

The prefrontal cortex subserves executive control, i.e., the organization of action or thought in relation to internal goals. This brain region hosts a system of executive processes extending from premotor to the most anterior prefrontal regions that governs the temporal organization of behavior. Little is known, however, about the prefrontal executive system involved in the hierarchical organization of behavior. Here, we show using magnetic resonance imaging in humans that the posterior portion of the prefrontal cortex, including Broca's area and its homolog in the right hemisphere, contains a system of executive processes that control start and end states and the nesting of functional segments that combine in hierarchically organized action plans. Our results indicate that Broca's area and its right homolog process hierarchically structured behaviors regardless of their temporal organization, suggesting a fundamental segregation between prefrontal executive systems involved in the hierarchical and temporal organization of goal-directed behaviors.     

Progressive plasticity of auditory cortex during appetitive operant conditioning

In stimulus-response-outcome learning, different regions in the cortico-basal ganglia network are progressively involved according to the stage of learning. However, the involvement of sensory cortex remains ellusive even though massive cortical projections to the striatum imply its significant role in this learning. Here we show that the global tonotopic representation in the auditory cortex changed progressively depending on the stage of training in auditory operant conditioning. At the early stage, tone-responsive areas mainly in the core cortex expanded, while both the core and belt cortices shrank at the late stage as behavior became conditioned. Taken together with previous findings, this progressive global plasticity from the core to belt cortices suggests differentiated roles in these areas: the core cortex serves as a filter to better identify auditory objects for hierarchical computation within the belt cortex, while the belt stores auditory objects and affects decision making through direct projections to limbic system and higher association cortex. Thus, the progressive plasticity in the present study reflects a shift from identification to storage of a behaviorally relevant auditory object, which is potentially associated with a habitual behavior.     

Neural plasticity in the ageing brain

The mechanisms involved in plasticity in the nervous system are thought to support cognition, and some of these processes are affected during normal ageing. Notably, cognitive functions that rely on the medial temporal lobe and prefrontal cortex, such as learning, memory and executive function, show considerable age-related decline. It is therefore not surprising that several neural mechanisms in these brain areas also seem to be particularly vulnerable during the ageing process. In this review, we discuss major advances in our understanding of age-related changes in the medial temporal lobe and prefrontal cortex and how these changes in functional plasticity contribute to behavioural impairments in the absence of significant pathology.     

The involvement of the left motor cortex in learning of a novel action word lexicon

Current theoretical positions assume that action-related word meanings are established by functional connections between perisylvian language areas and the motor cortex (MC) according to Hebb's associative learning principle. To test this assumption, we probed the functional relevance of the left MC for learning of a novel action word vocabulary by disturbing neural plasticity in the MC with transcranial direct current stimulation (tDCS). In combination with tDCS, subjects learned a novel vocabulary of 76 concrete, body-related actions by means of an associative learning paradigm. Compared with a control condition with "sham" stimulation, cathodal tDCS reduced success rates in vocabulary acquisition, as shown by tests of novel action word translation into the native language. The analysis of learning behavior revealed a specific effect of cathodal tDCS on the ability to associatively couple actions with novel words. In contrast, we did not find these effects in control experiments, when tDCS was applied to the prefrontal cortex or when subjects learned object-related words. The present study lends direct evidence to the proposition that the left MC is causally involved in the acquisition of novel action-related words.     

A modified error backpropagation algorithm for complex-value neural networks

The complex-valued backpropagation algorithm has been widely used in fields of dealing with telecommunications, speech recognition and image processing with Fourier transformation. However, the local minima problem usually occurs in the process of learning. To solve this problem and to speed up the learning process, we propose a modified error function by adding a term to the conventional error function, which is corresponding to the hidden layer error. The simulation results show that the proposed algorithm is capable of preventing the learning from sticking into the local minima and of speeding up the learning.