概率类别学习的认知神经机制

概率类别学习是探讨人们如何习得线索与结果之间的"概率"关系.研究者借助天气预报等任务,探讨了概率类别学习的认知策略、无意识性及其与工作记忆和注意的关系,并借助脑成像技术和脑损伤病人,探讨了基底神经节、内侧颞叶、前额叶和顶叶等脑区在概率类别学习中的作用.但是,由于概率类别学习涉及内隐和外显学习系统的分离问题,目前对其相关研究结果和理论解释还存在很大争议,概率类别学习的认知神经机制仍有待进一步研究.     

Are we ready for a natural history of motor learning?

Here we argue that general principles with regard to the contributions of the cerebellum, basal ganglia, and primary motor cortex to motor learning can begin to be inferred from explicit comparison across model systems and consideration of phylogeny. Both the cerebellum and the basal ganglia have highly conserved circuit architecture in vertebrates. The cerebellum has consistently been shown to be necessary for adaptation of eye and limb movements. The precise contribution of the basal ganglia to motor learning remains unclear but one consistent finding is that they are necessary for early acquisition of novel sequential actions. The primary motor cortex allows independent control of joints and construction of new movement synergies. We suggest that this capacity of the motor cortex implies that it is a necessary locus for motor skill learning, which we argue is the ability to execute selected actions with increasing speed and precision.     

Basal ganglia neural mechanisms of natural movement sequences

Natural rodent grooming and other instinctive behavior serves as a natural model of complex movement sequences. Rodent grooming has syntactic (rule-driven) sequences and more random movement patterns. Both incorporate the same movements--only the serial structure differs. Recordings of neural activity in the dorsolateral striatum and the substantia nigra pars reticulata indicate preferential activation during syntactic sequences over more random sequences. Neurons that are responsive during syntactic grooming sequences are often unresponsive or have reverse activation profiles during kinematically similar movements that occur in flexible or random grooming sequences. Few neurons could be categorized as strictly movement related--instead they were activated only in the context of particular sequential patterns of movements. Particular sequential patterns included "syntactic chain" grooming sequences of paw, head, and body movements and also "warm-up" sequences, which consist of head and body/limb movements that precede locomotion after a period of quiet resting (Golani 1992). Activation during warm-up was less intense and less frequent than during grooming sequences, but both sequences activated neurons above baseline levels, and the same neurons sometimes responded to both sequences. The fact that striatal neurons code 2 natural sequences which are made up of different constituent movements suggests that the basal ganglia may have a generalized role in sequence control. The basal ganglia are modulated by the context of the sequence and may play an executive function in the complex natural patterns of sequenced behaviour.     

Modulation by dopamine of human basal ganglia involvement in feedback control of movement

We learn new motor tasks by trial and error, repeating what works best and avoiding past mistakes. To repeat what works best we must register a satisfactory outcome, and in a study [1] we showed the existence of an evoked activity in the basal ganglia that correlates with accuracy of task performance and is associated with reiteration of successful motor parameters in subsequent movements. Here we report evidence that the signaling of positive trial outcome relies on dopaminergic input to the basal ganglia, by recording from the subthalamic nucleus (STN) in patients with nigrostriatal denervation due to Parkinson's Disease (PD) who have undergone functional neurosurgery. Correlations between subthalamic evoked activities and trial accuracy were weak and behavioral performance remained poor while patients were untreated; however, both improved after the dopamine prodrug levodopa was re-introduced. The results suggest that the midbrain dopaminergic system may be important, not only in signaling explicit positive outcomes or rewards in tasks requiring choices between options [2,3], but also in trial-to-trial learning and in reinforcing the selection of optimal parameters in more automatic motor control.     

Informativeness in stimulus-response modelling

The concept of informativeness is introduced in the context of A.J. Clark's pioneering "concentration-action" studies, in relation to the stimulus-response function of the accepted theory of agonist/receptor reactions and their observable consequences. The dominance of so-called "null-methods" in the existing literature is noted and contrasted with an informativeness analysis in which the unknown but estimable stimulus-response function plays an explicit r?le. Some statistical advantages of the latter approach are emphasised.     

Dissociating explicit timing from temporal expectation with fMRI

Explicit timing is engaged whenever subjects make a deliberate estimate of discrete duration in order to compare it with a previously memorised standard. Conversely, implicit timing is engaged, even without a specific instruction to time, whenever sensorimotor information is temporally structured and can be used to predict the duration of future events. Both emergent timing (motor) and temporal expectation (perceptual) are forms of implicit timing. Recent fMRI studies demonstrate discrete neural substrates for explicit and implicit timing. Specifically, basal ganglia are activated almost invariably by explicit timing, with co-activation of prefrontal, premotor and cerebellar areas being more context-dependent. Conversely, implicit perceptual timing (or "temporal expectation") recruits cortical action circuits, comprising inferior parietal and premotor areas, highlighting its role in the optimisation of prospective behaviour.     

Complementary roles of basal ganglia and cerebellum in learning and motor control

The classical notion that the basal ganglia and the cerebellum are dedicated to motor control has been challenged by the accumulation of evidence revealing their involvement in non-motor, cognitive functions. From a computational viewpoint, it has been suggested that the cerebellum, the basal ganglia, and the cerebral cortex are specialized for different types of learning: namely, supervised learning, reinforcement learning and unsupervised learning, respectively. This idea of learning-oriented specialization is helpful in understanding the complementary roles of the basal ganglia and the cerebellum in motor control and cognitive functions.     

Shaping of object representations in the human medial temporal lobe based on temporal regularities

Regularities are gradually represented in cortex after extensive experience [1], and yet they can influence behavior after minimal exposure [2, 3]. What kind of representations support such rapid statistical learning? The medial temporal lobe (MTL) can represent information from even a single experience [4], making it a good candidate system for assisting in initial learning about regularities. We combined anatomical segmentation of the MTL, high-resolution fMRI, and multivariate pattern analysis to identify representations of objects in cortical and hippocampal areas of human MTL, assessing how these representations were shaped by exposure to regularities. Subjects viewed a continuous visual stream containing hidden temporal relationships-pairs of objects that reliably appeared nearby in time. We compared the pattern of blood oxygen level-dependent activity evoked by each object before and after this exposure, and found that perirhinal cortex, parahippocampal cortex, subiculum, CA1, and CA2/CA3/dentate gyrus (CA2/3/DG) encoded regularities by increasing the representational similarity of their constituent objects. Most regions exhibited bidirectional associative shaping, whereas CA2/3/DG represented regularities in a forward-looking predictive manner. These findings suggest that object representations in MTL come to mirror the temporal structure of the environment, supporting rapid and incidental statistical learning.     

Speed/accuracy trade-off between the habitual and the goal-directed processes

Instrumental responses are hypothesized to be of two kinds: habitual and goal-directed, mediated by the sensorimotor and the associative cortico-basal ganglia circuits, respectively. The existence of the two heterogeneous associative learning mechanisms can be hypothesized to arise from the comparative advantages that they have at different stages of learning. In this paper, we assume that the goal-directed system is behaviourally flexible, but slow in choice selection. The habitual system, in contrast, is fast in responding, but inflexible in adapting its behavioural strategy to new conditions. Based on these assumptions and using the computational theory of reinforcement learning, we propose a normative model for arbitration between the two processes that makes an approximately optimal balance between search-time and accuracy in decision making. Behaviourally, the model can explain experimental evidence on behavioural sensitivity to outcome at the early stages of learning, but insensitivity at the later stages. It also explains that when two choices with equal incentive values are available concurrently, the behaviour remains outcome-sensitive, even after extensive training. Moreover, the model can explain choice reaction time variations during the course of learning, as well as the experimental observation that as the number of choices increases, the reaction time also increases. Neurobiologically, by assuming that phasic and tonic activities of midbrain dopamine neurons carry the reward prediction error and the average reward signals used by the model, respectively, the model predicts that whereas phasic dopamine indirectly affects behaviour through reinforcing stimulus-response associations, tonic dopamine can directly affect behaviour through manipulating the competition between the habitual and the goal-directed systems and thus, affect reaction time.     

Self-Organizing Task Modules and Explicit Coordinate Systems in a Neural Network Model for 3-D Saccades

The goal of this study was to train an artificial neural network to generate accurate saccades in Listing's plane and then determine how the hidden units performed the visuomotor transformation. A three-layer neural network was successfully trained, using back-prop, to take in oculocentric retinal error vectors and three-dimensional eye orientation and to generate the correct head-centric motor error vector within Listing's plane. Analysis of the hidden layer of trained networks showed that explicit representations of desired target direction and eye orientation were not employed. Instead, the hidden-layer units consistently divided themselves into four parallel modules: a dominant "vector-propagation" class (approximately 50% of units) with similar visual and motor tuning but negligible position sensitivity and three classes with specific spatial relations between position, visual, and motor tuning. Surprisingly, the vector-propagation units, and only these, formed a highly precise and consistent orthogonal coordinate system aligned with Listing's plane. Selective "lesions" confirmed that the vector-propagation module provided the main drive for saccade magnitude and direction, whereas a balance between activity in the other modules was required for the correct eye-position modulation. Thus, contrary to popular expectation, error-driven learning in itself was sufficient to produce a "neural" algorithm with discrete functional modules and explicit coordinate systems, much like those observed in the real saccade generator.     

Resting-State fMRI Activity Predicts Unsupervised Learning and Memory in an Immersive Virtual Reality Environment

In the real world, learning often proceeds in an unsupervised manner without explicit instructions or feedback. In this study, we employed an experimental paradigm in which subjects explored an immersive virtual reality environment on each of two days. On day 1, subjects implicitly learned the location of 39 objects in an unsupervised fashion. On day 2, the locations of some of the objects were changed, and object location recall performance was assessed and found to vary across subjects. As prior work had shown that functional magnetic resonance imaging (fMRI) measures of resting-state brain activity can predict various measures of brain performance across individuals, we examined whether resting-state fMRI measures could be used to predict object location recall performance. We found a significant correlation between performance and the variability of the resting-state fMRI signal in the basal ganglia, hippocampus, amygdala, thalamus, insula, and regions in the frontal and temporal lobes, regions important for spatial exploration, learning, memory, and decision making. In addition, performance was significantly correlated with resting-state fMRI connectivity between the left caudate and the right fusiform gyrus, lateral occipital complex, and superior temporal gyrus. Given the basal ganglia''s role in exploration, these findings suggest that tighter integration of the brain systems responsible for exploration and visuospatial processing may be critical for learning in a complex environment.     

Do Basal Ganglia Amplify Willed Action by Stochastic Resonance? A Model

Basal ganglia are usually attributed a role in facilitating willed action, which is found to be impaired in Parkinson''s disease, a pathology of basal ganglia. We hypothesize that basal ganglia possess the machinery to amplify will signals, presumably weak, by stochastic resonance. Recently we proposed a computational model of Parkinsonian reaching, in which the contributions from basal ganglia aid the motor cortex in learning to reach. The model was cast in reinforcement learning framework. We now show that the above basal ganglia computational model has all the ingredients of stochastic resonance process. In the proposed computational model, we consider the problem of moving an arm from a rest position to a target position: the two positions correspond to two extrema of the value function. A single kick (a half-wave of sinusoid, of sufficiently low amplitude) given to the system in resting position, succeeds in taking the system to the target position, with high probability, only at a critical noise level. But for suboptimal noise levels, the model arm''s movements resemble Parkinsonian movement symptoms like akinetic rigidity (low noise) and dyskinesias (high noise).     

The Striatum: Where Skills and Habits Meet

After more than a century of work concentrating on the motor functions of the basal ganglia, new ideas have emerged, suggesting that the basal ganglia also have major functions in relation to learning habits and acquiring motor skills. We review the evidence supporting the role of the striatum in optimizing behavior by refining action selection and in shaping habits and skills as a modulator of motor repertoires. These findings challenge the notion that striatal learning processes are limited to the motor domain. The learning mechanisms supported by striatal circuitry generalize to other domains, including cognitive skills and emotion-related patterns of action.The nuclei and interconnections of the basal ganglia are widely recognized for modulating motor behavior. Whether measured at the neuronal or regional level, the activities of neurons in the basal ganglia correlate with many movement parameters, particularly those that influence the vigor of an action, such as force and velocity. Pathology within different basal ganglia circuits predictably leads to either hypokinetic or hyperkinetic movement disorders. In parallel, however, the basal ganglia, and especially the striatum, are now widely recognized as being engaged in activity related to learning. Interactions between the dopamine-containing neurons of the midbrain and their targets in the striatum are critical to this function. A fundamental question is how these two capacities—(motor behavior and reinforcement-based learning)—relate to each other and what role the striatum and other basal ganglia nuclei have in forming new behavioral repertoires. Here, we consider relevant physiological properties of the striatum by contrasting two common forms of adaptation found in all mammals: the acquisition of behavioral habits and physical skills.Without resorting to technical definitions, we all have an intuition of what habits and skills are. Tying one’s shoes after putting them on is something we consider a habit—part of a behavioral routine. The capacity to tie the laces properly is a skill. Habits and skills have many common features. Habits are consistent behaviors triggered by appropriate events (typically, but not always, external stimuli) occurring within particular contexts. Physical skills are changes in a physical repertoire: new combinations of movements that lead to new capacities for goal-directed action. Both habits and skills can leverage reward-based learning, particularly during their initial acquisition. In either instance, after sufficient experience, the need for reward becomes lower and lower. With sufficient practice, both lead to “automaticity” and a resilience against competing actions that might lead to unlearning.     

A neurophysiologic model for aggressive behavior in the cat

A neurophysiologic model for aggressive behavior in the cat is proposed. Stimulus-bound and seizure-bound aggression was evaluated in relation to limbic and basal ganglia induced seizures (after-discharges). Electrically induced limbic and basal ganglia after-discharges were used because they are known to implicate septohypothalamic sites from which aggression can be elicited by direct stimulation. The occurrence of behavioral aggression is correlated with the discharge characteristics of a single discharging system and with two interacting discharging systems. Aggression is composed of autonomic and somato-motor components which poses relatively low and high thresholds, respectively, for their activation. Aggression occurring during a combined septum and amygdala discharge was more intense and prolonged than with a septum discharge alone. Participation of a slow frequency discharging basal ganglia system activated seizure-bound aggression in an otherwise nonaggressive limbic seizure. The limbic and basal ganglia stimulations and after-discharges lowered the excitability threshold of the aggression system and made it more vulnerable to being activated by external stimuli, such as visual and auditory stimuli. These observations are reminiscent of patients with aggressive behavior associated with psychomotor seizures.     

Input to the lateral habenula from the basal ganglia is excitatory, aversive, and suppressed by serotonin

The lateral habenula (LHb) has recently been identified as a key regulator of the reward system by driving inhibition onto dopaminergic neurons. However, the nature and potential modulation of the major input to the LHb originating from the basal ganglia are poorly understood. Although the output of the basal ganglia is thought to be primarily inhibitory, here we show that transmission from the basal ganglia to the LHb is excitatory, glutamatergic, and suppressed by serotonin. Behaviorally, activation of this pathway is aversive, consistent with its role as an "antireward" signal. Our demonstration of an excitatory projection from the basal ganglia to the LHb explains how LHb-projecting basal ganglia neurons can have similar encoding properties as LHb neurons themselves. Our results also provide a link between antireward excitatory synapses and serotonin,?a neuromodulator implicated in depression.     

A network representation of response probability in the striatum

The striatum of the basal ganglia is considered a key structure in the learning circuitry of the brain. To analyze neural signals that underlie striatal plasticity, we recorded from an identifiable class of striatal interneurons as macaque monkeys underwent training in a range of conditioning and non-associative learning paradigms, and recorded eyeblink electromyographs as the measure of behavioral response. We found that the responses of these striatal interneurons were modifiable under all training conditions and that their population responses were tightly correlated with the probability that a given stimulus would evoke a behavioral response. Such a network signal, proportional to current response probability, could be crucial to the learning and decision functions of the basal ganglia.     

Complementary 'bottom-up' and 'top-down' approaches to basal ganglia function

Recently, two quite different approaches exemplifying 'bottom-up' and 'top-down' philosophies have shed new light on basal ganglia function. In vitro work using organotypic co-cultures has implicated the subthalamic nucleus (STN) and the external segment of the globus pallidus (GP(e)) as pacemakers for low-frequency bursting that is reminiscent of the activity produced in Parkinsonian tremor. A circuit essential for avian song learning has been identified as part of the basal ganglia with surprisingly well conserved cellular details; investigation of this system may help to address general issues of basal ganglia function.     

Multiple representations and algorithms for reinforcement learning in the cortico-basal ganglia circuit

Accumulating evidence shows that the neural network of the cerebral cortex and the basal ganglia is critically involved in reinforcement learning. Recent studies found functional heterogeneity within the cortico-basal ganglia circuit, especially in its ventromedial to dorsolateral axis. Here we review computational issues in reinforcement learning and propose a working hypothesis on how multiple reinforcement learning algorithms are implemented in the cortico-basal ganglia circuit using different representations of states, values, and actions.     

Surprised at all the entropy: hippocampal, caudate and midbrain contributions to learning from prediction errors

Influential concepts in neuroscientific research cast the brain a predictive machine that revises its predictions when they are violated by sensory input. This relates to the predictive coding account of perception, but also to learning. Learning from prediction errors has been suggested for take place in the hippocampal memory system as well as in the basal ganglia. The present fMRI study used an action-observation paradigm to investigate the contributions of the hippocampus, caudate nucleus and midbrain dopaminergic system to different types of learning: learning in the absence of prediction errors, learning from prediction errors, and responding to the accumulation of prediction errors in unpredictable stimulus configurations. We conducted analyses of the regions of interests' BOLD response towards these different types of learning, implementing a bootstrapping procedure to correct for false positives. We found both, caudate nucleus and the hippocampus to be activated by perceptual prediction errors. The hippocampal responses seemed to relate to the associative mismatch between a stored representation and current sensory input. Moreover, its response was significantly influenced by the average information, or Shannon entropy of the stimulus material. In accordance with earlier results, the habenula was activated by perceptual prediction errors. Lastly, we found that the substantia nigra was activated by the novelty of sensory input. In sum, we established that the midbrain dopaminergic system, the hippocampus, and the caudate nucleus were to different degrees significantly involved in the three different types of learning: acquisition of new information, learning from prediction errors and responding to unpredictable stimulus developments. We relate learning from perceptual prediction errors to the concept of predictive coding and related information theoretic accounts.     

Involvement of human basal ganglia in offline feedback control of voluntary movement

Practice makes perfect, but the neural substrates of trial-to-trial learning in motor tasks remain unclear. There is some evidence that the basal ganglia process feedback-related information to modify learning in essentially cognitive tasks , but the evidence that these key motor structures are involved in offline feedback-related improvement of performance in motor tasks is paradoxically limited. Lesion studies in adult zebra finches suggest that the avian basal ganglia are involved in the transmission or production of an error signal during song . However, patients with Huntington's disease, in which there is prominent basal ganglia dysfunction, are not impaired in error-dependent modulation of future trial performance . By directly recording from the subthalamic nucleus in patients with Parkinson's disease, we demonstrate that this nucleus processes error in trial performance at short latency. Local evoked activity is greatest in response to smallest errors and influences the programming of subsequent movements. Accordingly, motor parameters are least likely to change after the greatest evoked responses so that accurately performed trials tend to precede other accurate trials. This relationship is disrupted by electrical stimulation of the nucleus at high frequency. Thus, the human subthalamic nucleus is involved in feedback-based learning.