Primary and Secondary Rewards Differentially Modulate Neural Activity Dynamics during Working Memory

Background

Cognitive control and working memory processes have been found to be influenced by changes in motivational state. Nevertheless, the impact of different motivational variables on behavior and brain activity remains unclear.

Methodology/Principal Findings

The current study examined the impact of incentive category by varying on a within-subjects basis whether performance during a working memory task was reinforced with either secondary (monetary) or primary (liquid) rewards. The temporal dynamics of motivation-cognition interactions were investigated by employing an experimental design that enabled isolation of sustained and transient effects. Performance was dramatically and equivalently enhanced in each incentive condition, whereas neural activity dynamics differed between incentive categories. The monetary reward condition was associated with a tonic activation increase in primarily right-lateralized cognitive control regions including anterior prefrontal cortex (PFC), dorsolateral PFC, and parietal cortex. In the liquid condition, the identical regions instead showed a shift in transient activation from a reactive control pattern (primary probe-based activation) during no-incentive trials to proactive control (primary cue-based activation) during rewarded trials. Additionally, liquid-specific tonic activation increases were found in subcortical regions (amygdala, dorsal striatum, nucleus accumbens), indicating an anatomical double dissociation in the locus of sustained activation.

Conclusions/Significance

These different activation patterns suggest that primary and secondary rewards may produce similar behavioral changes through distinct neural mechanisms of reinforcement. Further, our results provide new evidence for the flexibility of cognitive control, in terms of the temporal dynamics of activation.     

Multiplicity of control in the basal ganglia: computational roles of striatal subregions

The basal ganglia, in particular the striatum, are central to theories of behavioral control, and often identified as a seat of action selection. Reinforcement learning (RL) models--which have driven much recent experimental work on this region--cast striatum as a dynamic controller, integrating sensory and motivational information to construct efficient and enriching behavioral policies. Befitting this informationally central role, the BG sit at the nexus of multiple anatomical 'loops' of synaptic projections, connecting a wide range of cortical and subcortical structures. Numerous pioneering anatomical studies conducted over the past several decades have meticulously catalogued these loops, and labeled them according to the inferred functions of the connected regions. The specific cotermina of the projections are highly localized to several different subregions of the striatum, leading to the suggestion that these subregions perform complementary but distinct functions. However, until recently, the dominant computational framework outlined only a bipartite, dorsal/ventral, division of striatum. We review recent computational and experimental advances that argue for a more finely fractionated delineation. In particular, experimental data provide extensive insight into unique functions subserved by the dorsomedial striatum (DMS). These functions appear to correspond well with theories of a 'model-based' RL subunit, and may also shed light on the suborganization of ventral striatum. Finally, we discuss the limitations of these ideas and how they point the way toward future refinements of neurocomputational theories of striatal function, bringing them into contact with other areas of computational theory and other regions of the brain.     

Reward Sensitivity Modulates Brain Activity in the Prefrontal Cortex,ACC and Striatum during Task Switching

Current perspectives on cognitive control acknowledge that individual differences in motivational dispositions may modulate cognitive processes in the absence of reward contingencies. This work aimed to study the relationship between individual differences in Behavioral Activation System (BAS) sensitivity and the neural underpinnings involved in processing a switching cue in a task-switching paradigm. BAS sensitivity was hypothesized to modulate brain activity in frontal regions, ACC and the striatum. Twenty-eight healthy participants underwent fMRI while performing a switching task, which elicited activity in fronto-striatal regions during the processing of the switch cue. BAS sensitivity was negatively associated with activity in the lateral prefrontal cortex, anterior cingulate cortex and the ventral striatum. Combined with previous results, our data indicate that BAS sensitivity modulates the neurocognitive processes involved in task switching in a complex manner depending on task demands. Therefore, individual differences in motivational dispositions may influence cognitive processing in the absence of reward contingencies.     

Caspr3-Deficient Mice Exhibit Low Motor Learning during the Early Phase of the Accelerated Rotarod Task

Caspr3 (Contactin-associated protein-like 3, Cntnap3) is a neural cell adhesion molecule belonging to the Caspr family. We have recently shown that Caspr3 is expressed abundantly between the first and second postnatal weeks in the mouse basal ganglia, including the striatum, external segment of the globus pallidus, subthalamic nucleus, and substantia nigra. However, its physiological role remains largely unknown. In this study, we conducted a series of behavioral analyses on Capsr3-knockout (KO) mice and equivalent wild-type (WT) mice to investigate the role of Caspr3 in brain function. No significant differences were observed in most behavioral traits between Caspr3-KO and WT mice, but we found that Caspr3-KO mice performed poorly during the early phase of the accelerated rotarod task in which latency to falling off a rod rotating with increasing velocity was examined. In the late phase, the performance of the Caspr3-KO mice caught up to the level of WT mice, suggesting that the deletion of Caspr3 caused a delay in motor learning. We then examined changes in neural activity after training on the accelerated rotarod by conducting immunohistochemistry using antibody to c-Fos, an indirect marker for neuronal activity. Experience of the accelerated rotarod task caused increases in the number of c-Fos-positive cells in the dorsal striatum, cerebellum, and motor cortex in both Caspr3-KO and WT mice, but the number of c-Fos-positive cells was significantly lower in the dorsal striatum of Caspr3-KO mice than in that of WT mice. The expression of c-Fos in the ventral striatum of Caspr3-KO and WT mice was not altered by the training. Our findings suggest that reduced activation of neural cells in the dorsal striatum in Caspr3-KO mice leads to a decline in motor learning in the accelerated rotarod task.     

Default Mode Network,Motor Network,Dorsal and Ventral Basal Ganglia Networks in the Rat Brain: Comparison to Human Networks Using Resting State-fMRI

Rodent models are developed to enhance understanding of the underlying biology of different brain disorders. However, before interpreting findings from animal models in a translational aspect to understand human disease, a fundamental step is to first have knowledge of similarities and differences of the biological systems studied. In this study, we analyzed and verified four known networks termed: default mode network, motor network, dorsal basal ganglia network, and ventral basal ganglia network using resting state functional MRI (rsfMRI) in humans and rats. Our work supports the notion that humans and rats have common robust resting state brain networks and that rsfMRI can be used as a translational tool when validating animal models of brain disorders. In the future, rsfMRI may be used, in addition to short-term interventions, to characterize longitudinal effects on functional brain networks after long-term intervention in humans and rats.     

Methylphenidate alters basal ganglia neurotensin systems through dopaminergic mechanisms: a comparison with cocaine treatment

Methylphenidate (MPD) is a psychostimulant widely used to treat behavioral problems such as attention deficit hyperactivity disorder. MPD competitively inhibits the dopamine (DA) transporter. Previous studies demonstrated that stimulants of abuse, such as cocaine (COC) and methamphetamine differentially alter rat brain neurotensin (NT) systems through DA mechanisms. As NT is a neuropeptide primarily associated with the regulation of the nigrostriatal and mesolimbic DA systems, the effect of MPD on NT-like immunoreactivity (NTLI) content in several basal ganglia regions was assessed. MPD, at doses of 2.0 or 10.0 mg/kg, s.c., significantly increased the NTLI contents in dorsal striatum, substantia nigra and globus pallidus; similar increases in NTLI were observed in these areas after administration of COC (30.0 mg/kg, i.p.). No changes in NTLI occurred within the nucleus accumbens, frontal cortex and ventral tegmental area following MPD treatment. In addition, the NTLI changes in basal ganglia regions induced by MPD were prevented when D(1) (SCH 23390) or D(2) (eticlopride) receptor antagonists were coadministered with MPD. MPD treatment also increased dynorphin (DYN) levels in basal ganglia structures. These findings provide evidence that basal ganglia, but not limbic, NT systems are significantly affected by MPD through D(1) and D(2) receptor mechanisms, and these NTLI changes are similar, but not identical to those which occurred with COC administration. In addition, the MPD effects on NT systems are mechanistically distinct from the effects of methamphetamine.     

Cellular mechanisms of striatum-dependent behavioral plasticity and drug addiction

The striatum has long been known to be involved in the control of motor behavior, since disruption of dopamine-mediated function in this brain structure is directly linked to Parkinson's disease and other disorders of movement. However, it is now accepted that both dorsal and ventral striatal nuclei are also essential for a variety of cognitive processes, which depend on reward-based stimulus-response learning. Since the neuroanatomical and neurochemical organization of dorsal and ventral striatum is only partially overlapping, it is likely that both common and nucleus-specific cellular and molecular events contribute to synaptic plasticity, learning and memory processes mediated by these cerebral structures. Alterations in cell signaling in the striatum may be particularly important in the response to both acute and chronic administration of drugs of abuse, resulting in maladaptive changes in the reward-based associative learning involved in addiction, withdrawal and relapse.     

Neural substrates for the motivational regulation of motor recovery after spinal-cord injury

It is believed that depression impedes and motivation enhances functional recovery after neuronal damage such as spinal-cord injury and stroke. However, the neuronal substrate underlying such psychological effects on functional recovery remains unclear. A longitudinal study of brain activation in the non-human primate model of partial spinal-cord injury using positron emission tomography (PET) revealed a contribution of the primary motor cortex (M1) to the recovery of finger dexterity through the rehabilitative training. Here, we show that activity of the ventral striatum, including the nucleus accumbens (NAc), which plays a critical role in processing of motivation, increased and its functional connectivity with M1 emerged and was progressively strengthened during the recovery. In addition, functional connectivities among M1, the ventral striatum and other structures belonging to neural circuits for processing motivation, such as the orbitofrontal cortex, anterior cingulate cortex and pedunculopontine tegmental nucleus were also strengthened during the recovery. These results give clues to the neuronal substrate for motivational regulation of motor learning required for functional recovery after spinal-cord injury.     

Imbalanced Decision Hierarchy in Addicts Emerging from Drug-Hijacked Dopamine Spiraling Circuit

Despite explicitly wanting to quit, long-term addicts find themselves powerless to resist drugs, despite knowing that drug-taking may be a harmful course of action. Such inconsistency between the explicit knowledge of negative consequences and the compulsive behavioral patterns represents a cognitive/behavioral conflict that is a central characteristic of addiction. Neurobiologically, differential cue-induced activity in distinct striatal subregions, as well as the dopamine connectivity spiraling from ventral striatal regions to the dorsal regions, play critical roles in compulsive drug seeking. However, the functional mechanism that integrates these neuropharmacological observations with the above-mentioned cognitive/behavioral conflict is unknown. Here we provide a formal computational explanation for the drug-induced cognitive inconsistency that is apparent in the addicts'' “self-described mistake”. We show that addictive drugs gradually produce a motivational bias toward drug-seeking at low-level habitual decision processes, despite the low abstract cognitive valuation of this behavior. This pathology emerges within the hierarchical reinforcement learning framework when chronic exposure to the drug pharmacologically produces pathologicaly persistent phasic dopamine signals. Thereby the drug hijacks the dopaminergic spirals that cascade the reinforcement signals down the ventro-dorsal cortico-striatal hierarchy. Neurobiologically, our theory accounts for rapid development of drug cue-elicited dopamine efflux in the ventral striatum and a delayed response in the dorsal striatum. Our theory also shows how this response pattern depends critically on the dopamine spiraling circuitry. Behaviorally, our framework explains gradual insensitivity of drug-seeking to drug-associated punishments, the blocking phenomenon for drug outcomes, and the persistent preference for drugs over natural rewards by addicts. The model suggests testable predictions and beyond that, sets the stage for a view of addiction as a pathology of hierarchical decision-making processes. This view is complementary to the traditional interpretation of addiction as interaction between habitual and goal-directed decision systems.     

Review. Neurobiological mechanisms for opponent motivational processes in addiction

The conceptualization of drug addiction as a compulsive disorder with excessive drug intake and loss of control over intake requires motivational mechanisms. Opponent process as a motivational theory for the negative reinforcement of drug dependence has long required a neurobiological explanation. Key neurochemical elements involved in reward and stress within basal forebrain structures involving the ventral striatum and extended amygdala are hypothesized to be dysregulated in addiction to convey the opponent motivational processes that drive dependence. Specific neurochemical elements in these structures include not only decreases in reward neurotransmission such as dopamine and opioid peptides in the ventral striatum, but also recruitment of brain stress systems such as corticotropin-releasing factor (CRF), noradrenaline and dynorphin in the extended amygdala. Acute withdrawal from all major drugs of abuse produces increases in reward thresholds, anxiety-like responses and extracellular levels of CRF in the central nucleus of the amygdala. CRF receptor antagonists block excessive drug intake produced by dependence. A brain stress response system is hypothesized to be activated by acute excessive drug intake, to be sensitized during repeated withdrawal, to persist into protracted abstinence and to contribute to stress-induced relapse. The combination of loss of reward function and recruitment of brain stress systems provides a powerful neurochemical basis for the long hypothesized opponent motivational processes responsible for the negative reinforcement driving addiction.     

Ligands of the NMDA receptor-associated glycine recognition site and motor behavior

Summary Motor behavior critically depends on glutamatergic functions in the basal ganglia (BG). The dorsal and ventral striatum — the main input structures of the BG - are involved in modulation of stereotyped sniffing behavior, locomotion, catalepsy and prepulse inhibition. The effects of the NMDA receptor have been well characterized in respect to motor behavior in the past. The function of the allosteric glycine site was however disregarded until now, because brain penetrating ligands were missing. The present study summarized the motor behavioral profile of several glycine site ligands (7chlorokynurenate, ACEA 1021, MRZ-2/576, (+) HA-966, D-cycloserine and felbamate). It is shown that through blockade of the glycine site of the NMDA receptor a distinct behavioral profile can be obtained.     

Role of anticipated reward in cognitive behavioral control

The lateral prefrontal cortex (LPFC), which is important for higher cognitive activity, is also concerned with motivational operations; this is exemplified by its activity in relation to expectancy of rewards. In the LPFC, motivational information is integrated with cognitive information, as demonstrated by the enhancement of working-memory-related activity by reward expectancy. Such activity would be expected to induce changes in attention and, subsequently, to modify behavioral performance. Recently, the effects of motivation and emotion on neural activities have been examined in several areas of the brain in relation to cognitive-task performance. Of these areas, the LPFC seems to have the most important role in adaptive goal-directed behavior, by sending top-down attention-control signals to other areas of the brain.     

Aversive Event Anticipation Affects Connectivity between the Ventral Striatum and the Orbitofrontal Cortex in an fMRI Avoidance Task

Ability to anticipate aversive events is important for avoiding dangerous or unpleasant situations. The motivation to avoid an event is influenced by the incentive salience of an event-predicting cue. In an avoidance fMRI task we used tone intensities to manipulate salience in order to study the involvement of the orbitofrontal cortex in processing of incentive salience. In the task, cues predicting either aversive or neutral avoidable tones were presented. Ventral striatum, amygdala and anterior insula activations were significantly stronger during presentation of cues for aversive than neutral tones. A psychophysiological interaction analysis showed stronger connectivity between the ventral striatum and the orbitofrontal cortex during aversive than neutral conditions. The present study shows an interaction between the ventral striatum, a structure previously linked to negative incentive salience, and the orbitofrontal cortex supporting a role for this region in processing salience. In addition, this study replicates previous findings suggesting that the task is robust.     

Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice

In choosing between different rewards expected after unequal delays, humans and animals often prefer the smaller but more immediate reward, indicating that the subjective value or utility of reward is depreciated according to its delay. Here, we show that neurons in the primate caudate nucleus and ventral striatum modulate their activity according to temporally discounted values of rewards with a similar time course. However, neurons in the caudate nucleus encoded the difference in the temporally discounted values of the two alternative targets more reliably than neurons in the ventral striatum. In contrast, neurons in the ventral striatum largely encoded the sum of the temporally discounted values, and therefore, the overall goodness of available options. These results suggest a more pivotal role for the dorsal striatum in action selection during intertemporal choice.     

Evidences for shared features in the organization of the basal ganglia in tetrapods: studies in amphibians.

In a series of recent studies, the organization of the basal ganglia of amphibians, more in particular their connectivity and chemoarchitecture, has been thoroughly analyzed. The pattern of organization found for the amphibian basal ganglia includes dorsal and ventral striatopallidal systems, reciprocal connections between the striatopallidal complex and structures derived from the diencephalic and mesencephalic parts of the basal plate (striatonigral and nigrostriatal projections), and descending pathways from the striatopallidal system to the midbrain tectum and the reticular formation of the brain stem. A comparative analysis of the organization of the basal ganglia in tetrapods strongly supports the notion that a primitive pattern was most likely present in ancestral tetrapods, and that many features can still be recognized in extant amphibians and amniotes.     

Neural Correlates of Sexual Cue Reactivity in Individuals with and without Compulsive Sexual Behaviours

Although compulsive sexual behaviour (CSB) has been conceptualized as a “behavioural” addiction and common or overlapping neural circuits may govern the processing of natural and drug rewards, little is known regarding the responses to sexually explicit materials in individuals with and without CSB. Here, the processing of cues of varying sexual content was assessed in individuals with and without CSB, focusing on neural regions identified in prior studies of drug-cue reactivity. 19 CSB subjects and 19 healthy volunteers were assessed using functional MRI comparing sexually explicit videos with non-sexual exciting videos. Ratings of sexual desire and liking were obtained. Relative to healthy volunteers, CSB subjects had greater desire but similar liking scores in response to the sexually explicit videos. Exposure to sexually explicit cues in CSB compared to non-CSB subjects was associated with activation of the dorsal anterior cingulate, ventral striatum and amygdala. Functional connectivity of the dorsal anterior cingulate-ventral striatum-amygdala network was associated with subjective sexual desire (but not liking) to a greater degree in CSB relative to non-CSB subjects. The dissociation between desire or wanting and liking is consistent with theories of incentive motivation underlying CSB as in drug addictions. Neural differences in the processing of sexual-cue reactivity were identified in CSB subjects in regions previously implicated in drug-cue reactivity studies. The greater engagement of corticostriatal limbic circuitry in CSB following exposure to sexual cues suggests neural mechanisms underlying CSB and potential biological targets for interventions.     

Functional Reorganization of Motor and Limbic Circuits after Exercise Training in a Rat Model of Bilateral Parkinsonism

Exercise training is widely used for neurorehabilitation of Parkinson’s disease (PD). However, little is known about the functional reorganization of the injured brain after long-term aerobic exercise. We examined the effects of 4 weeks of forced running wheel exercise in a rat model of dopaminergic deafferentation (bilateral, dorsal striatal 6-hydroxydopamine lesions). One week after training, cerebral perfusion was mapped during treadmill walking or at rest using [¹⁴C]-iodoantipyrine autoradiography. Regional cerebral blood flow-related tissue radioactivity (rCBF) was analyzed in three-dimensionally reconstructed brains by statistical parametric mapping. In non-exercised rats, lesions resulted in persistent motor deficits. Compared to sham-lesioned rats, lesioned rats showed altered functional brain activation during walking, including: 1. hypoactivation of the striatum and motor cortex; 2. hyperactivation of non-lesioned areas in the basal ganglia-thalamocortical circuit; 3. functional recruitment of the red nucleus, superior colliculus and somatosensory cortex; 4. hyperactivation of the ventrolateral thalamus, cerebellar vermis and deep nuclei, suggesting recruitment of the cerebellar-thalamocortical circuit; 5. hyperactivation of limbic areas (amygdala, hippocampus, ventral striatum, septum, raphe, insula). These findings show remarkable similarities to imaging findings reported in PD patients. Exercise progressively improved motor deficits in lesioned rats, while increasing activation in dorsal striatum and rostral secondary motor cortex, attenuating a hyperemia of the zona incerta and eliciting a functional reorganization of regions participating in the cerebellar-thalamocortical circuit. Both lesions and exercise increased activation in mesolimbic areas (amygdala, hippocampus, ventral striatum, laterodorsal tegmental n., ventral pallidum), as well as in related paralimbic regions (septum, raphe, insula). Exercise, but not lesioning, resulted in decreases in rCBF in the medial prefrontal cortex (cingulate, prelimbic, infralimbic). Our results in this PD rat model uniquely highlight the breadth of functional reorganizations in motor and limbic circuits following lesion and long-term, aerobic exercise, and provide a framework for understanding the neural substrates underlying exercise-based neurorehabilitation.     

Loss of Nkx2.1 homeobox gene function results in a ventral to dorsal molecular respecification within the basal telencephalon: evidence for a transformation of the pallidum into the striatum.

The telencephalon is organized into distinct longitudinal domains: the cerebral cortex and the basal ganglia. The basal ganglia primarily consists of a dorsal region (striatum) and a ventral region (pallidum). Within the telencephalon, the anlage of the pallidum expresses the Nkx2.1 homeobox gene. A mouse deficient in Nkx2.1 function does not form pallidal structures, lacks basal forebrain TrkA-positive neurons (probable cholinergic neurons) and has reduced numbers of cortical cells expressing GABA, DLX2 and calbindin that migrate from the pallidum through the striatum and into the cortex. We present evidence that these phenotypes result from a ventral-to-dorsal transformation of the pallidal primordium into a striatal-like anlage.     

fMRI signal changes in response to forced expiratory loading in congenital central hypoventilation syndrome.

Congenital central hypoventilation syndrome (CCHS) patients show impaired ventilatory responses to CO2 and hypoxia and reduced drive to breathe during sleep but retain appropriate breathing patterns in response to volition or increased exercise. Breath-by-breath influences on heart rate are also deficient. Using functional magnetic resonance imaging techniques, we examined responses over the brain to voluntary forced expiratory loading, a task that CCHS patients can perform but that results in impaired rapid heart rate variation patterns normally associated with the loading challenge. Increased signals emerged in control (n = 14) over CCHS (n = 13; ventilator dependent during sleep but not waking) subjects in the cingulate and right parietal cortex, cerebellar cortex and fastigial nucleus, and basal ganglia, whereas anterior cerebellar cortical sites and deep nuclei, dorsal midbrain, and dorsal pons showed increased signals in the patient group. The dorsal and ventral medulla showed delayed responses in CCHS patients. Primary motor and sensory areas bordering the central sulcus showed comparable responses in both groups. The delayed responses in medullary sensory and output regions and the aberrant reactions in cerebellar and pontine sensorimotor coordination areas suggest that rapid cardiorespiratory integration deficits in CCHS may stem from defects in these sites. Additional autonomic and perceptual motor deficits may derive from cingulate and parietal cortex aberrations.     

The Striatum: Where Skills and Habits Meet

After more than a century of work concentrating on the motor functions of the basal ganglia, new ideas have emerged, suggesting that the basal ganglia also have major functions in relation to learning habits and acquiring motor skills. We review the evidence supporting the role of the striatum in optimizing behavior by refining action selection and in shaping habits and skills as a modulator of motor repertoires. These findings challenge the notion that striatal learning processes are limited to the motor domain. The learning mechanisms supported by striatal circuitry generalize to other domains, including cognitive skills and emotion-related patterns of action.The nuclei and interconnections of the basal ganglia are widely recognized for modulating motor behavior. Whether measured at the neuronal or regional level, the activities of neurons in the basal ganglia correlate with many movement parameters, particularly those that influence the vigor of an action, such as force and velocity. Pathology within different basal ganglia circuits predictably leads to either hypokinetic or hyperkinetic movement disorders. In parallel, however, the basal ganglia, and especially the striatum, are now widely recognized as being engaged in activity related to learning. Interactions between the dopamine-containing neurons of the midbrain and their targets in the striatum are critical to this function. A fundamental question is how these two capacities—(motor behavior and reinforcement-based learning)—relate to each other and what role the striatum and other basal ganglia nuclei have in forming new behavioral repertoires. Here, we consider relevant physiological properties of the striatum by contrasting two common forms of adaptation found in all mammals: the acquisition of behavioral habits and physical skills.Without resorting to technical definitions, we all have an intuition of what habits and skills are. Tying one’s shoes after putting them on is something we consider a habit—part of a behavioral routine. The capacity to tie the laces properly is a skill. Habits and skills have many common features. Habits are consistent behaviors triggered by appropriate events (typically, but not always, external stimuli) occurring within particular contexts. Physical skills are changes in a physical repertoire: new combinations of movements that lead to new capacities for goal-directed action. Both habits and skills can leverage reward-based learning, particularly during their initial acquisition. In either instance, after sufficient experience, the need for reward becomes lower and lower. With sufficient practice, both lead to “automaticity” and a resilience against competing actions that might lead to unlearning.