Reward Anticipation in Ventral Striatum and Individual Sensitivity to Reward: A Pilot Study of a Child-Friendly fMRI Task

Reward processing has been implicated in developmental disorders. However, the classic task to probe reward anticipation, the monetary incentive delay task, has an abstract coding of reward and no storyline and may therefore be less appropriate for use with developmental populations. We modified the task to create a version appropriate for use with children. We investigated whether this child-friendly version could elicit ventral striatal activation during reward anticipation in typically developing children and young adolescents (aged 9.5–14.5). In addition, we tested whether our performance-based measure of reward sensitivity was associated with anticipatory activity in ventral striatum. Reward anticipation was related to activity in bilateral ventral striatum. Moreover, we found an association between individual reward sensitivity and activity in ventral striatum. We conclude that this task assesses ventral striatal activity in a child-friendly paradigm. The combination with a performance-based measure of reward sensitivity potentially makes the task a powerful tool for developmental imaging studies of reward processing.     

Ventromedial prefrontal cortex activation is associated with memory formation for predictable rewards

During reinforcement learning, dopamine release shifts from the moment of reward consumption to the time point when the reward can be predicted. Previous studies provide consistent evidence that reward-predicting cues enhance long-term memory (LTM) formation of these items via dopaminergic projections to the ventral striatum. However, it is less clear whether memory for items that do not precede a reward but are directly associated with reward consumption is also facilitated. Here, we investigated this question in an fMRI paradigm in which LTM for reward-predicting and neutral cues was compared to LTM for items presented during consumption of reliably predictable as compared to less predictable rewards. We observed activation of the ventral striatum and enhanced memory formation during reward anticipation. During processing of less predictable as compared to reliably predictable rewards, the ventral striatum was activated as well, but items associated with less predictable outcomes were remembered worse than items associated with reliably predictable outcomes. Processing of reliably predictable rewards activated the ventromedial prefrontal cortex (vmPFC), and vmPFC BOLD responses were associated with successful memory formation of these items. Taken together, these findings show that consumption of reliably predictable rewards facilitates LTM formation and is associated with activation of the vmPFC.     

Pavlovian reward prediction and receipt in schizophrenia: relationship to anhedonia

Reward processing abnormalities have been implicated in the pathophysiology of negative symptoms such as anhedonia and avolition in schizophrenia. However, studies examining neural responses to reward anticipation and receipt have largely relied on instrumental tasks, which may confound reward processing abnormalities with deficits in response selection and execution. 25 chronic, medicated outpatients with schizophrenia and 20 healthy controls underwent functional magnetic resonance imaging using a pavlovian reward prediction paradigm with no response requirements. Subjects passively viewed cues that predicted subsequent receipt of monetary reward or non-reward, and blood-oxygen-level-dependent signal was measured at the time of cue presentation and receipt. At the group level, neural responses to both reward anticipation and receipt were largely similar between groups. At the time of cue presentation, striatal anticipatory responses did not differ between patients and controls. Right anterior insula demonstrated greater activation for nonreward than reward cues in controls, and for reward than nonreward cues in patients. At the time of receipt, robust responses to receipt of reward vs. nonreward were seen in striatum, midbrain, and frontal cortex in both groups. Furthermore, both groups demonstrated responses to unexpected versus expected outcomes in cortical areas including bilateral dorsolateral prefrontal cortex. Individual difference analyses in patients revealed an association between physical anhedonia and activity in ventral striatum and ventromedial prefrontal cortex during anticipation of reward, in which greater anhedonia severity was associated with reduced activation to money versus no-money cues. In ventromedial prefrontal cortex, this relationship held among both controls and patients, suggesting a relationship between anticipatory activity and anhedonia irrespective of diagnosis. These findings suggest that in the absence of response requirements, brain responses to reward receipt are largely intact in medicated individuals with chronic schizophrenia, while reward anticipation responses in left ventral striatum are reduced in those patients with greater anhedonia severity.     

Reward circuitry activation by noxious thermal stimuli.

Using functional magnetic resonance imaging (fMRI), we observed that noxious thermal stimuli (46 degrees C) produce significant signal change in putative reward circuitry as well as in classic pain circuitry. Increases in signal were observed in the sublenticular extended amygdala of the basal forebrain (SLEA) and the ventral tegmentum/periaqueductal gray (VT/PAG), while foci of increased signal and decreased signal were observed in the ventral striatum and nucleus accumbens (NAc). Early and late phases were observed for signals in most brain regions, with early activation in reward related regions such as the SLEA, VT/PAG, and ventral striatum. In contrast, structures associated with somatosensory perception, including SI somatosensory cortex, thalamus, and insula, showed delayed activation. These data support the notion that there may be a shared neural system for evaluation of aversive and rewarding stimuli.     

Neural responses during anticipation of a primary taste reward

The aim of this study was to determine the brain regions involved in anticipation of a primary taste reward and to compare these regions to those responding to the receipt of a taste reward. Using fMRI, we scanned human subjects who were presented with visual cues that signaled subsequent reinforcement with a pleasant sweet taste (1 M glucose), a moderately unpleasant salt taste (0.2 M saline), or a neutral taste. Expectation of a pleasant taste produced activation in dopaminergic midbrain, posterior dorsal amygdala, striatum, and orbitofrontal cortex (OFC). Apart from OFC, these regions were not activated by reward receipt. The findings indicate that when rewards are predictable, brain regions recruited during expectation are, in part, dissociable from areas responding to reward receipt.     

Impact of Early Life Adversity on Reward Processing in Young Adults: EEG-fMRI Results from a Prospective Study over 25 Years

Several lines of evidence have implicated the mesolimbic dopamine reward pathway in altered brain function resulting from exposure to early adversity. The present study examined the impact of early life adversity on different stages of neuronal reward processing later in life and their association with a related behavioral phenotype, i.e. attention deficit/hyperactivity disorder (ADHD). 162 healthy young adults (mean age = 24.4 years; 58% female) from an epidemiological cohort study followed since birth participated in a simultaneous EEG-fMRI study using a monetary incentive delay task. Early life adversity according to an early family adversity index (EFA) and lifetime ADHD symptoms were assessed using standardized parent interviews conducted at the offspring''s age of 3 months and between 2 and 15 years, respectively. fMRI region-of-interest analysis revealed a significant effect of EFA during reward anticipation in reward-related areas (i.e. ventral striatum, putamen, thalamus), indicating decreased activation when EFA increased. EEG analysis demonstrated a similar effect for the contingent negative variation (CNV), with the CNV decreasing with the level of EFA. In contrast, during reward delivery, activation of the bilateral insula, right pallidum and bilateral putamen increased with EFA. There was a significant association of lifetime ADHD symptoms with lower activation in the left ventral striatum during reward anticipation and higher activation in the right insula during reward delivery. The present findings indicate a differential long-term impact of early life adversity on reward processing, implicating hyporesponsiveness during reward anticipation and hyperresponsiveness when receiving a reward. Moreover, a similar activation pattern related to lifetime ADHD suggests that the impact of early life stress on ADHD may possibly be mediated by a dysfunctional reward pathway.     

腹侧被盖区多巴胺能神经元对奖赏/厌恶刺激反应的异质性

中脑多巴胺奖赏系统,由腹侧被盖区及其投射靶区组成,参与药物依赖、精神疾病等病理过程的调控.奖赏和厌恶刺激是衡量上述病理过程的重要手段.一直以来,不同研究在该系统对奖赏和厌恶刺激的反应上存在分歧,越来越多的研究倾向于认为该系统,特别是腹侧被盖区多巴胺能神经元存在较大的异质性.本文从腹侧被盖区多巴胺能神经元判定标准、解剖定位和投射特异性等角度对其在奖赏和厌恶刺激中的功能异质性进行综述,并对未来研究方向进行展望.     

Early Effects of Reward Anticipation Are Modulated by Dopaminergic Stimulation

The abilities to predict future rewards and assess the value of reward delivery are crucial aspects of adaptive behavior. While the mesolimbic system, including dopaminergic midbrain, ventral striatum and prefrontal cortex have long been associated with reward processing, recent studies also indicate a prominent role of early visual brain regions. However, the precise underlying neural mechanisms still remain unclear. To address this issue, we presented participants with visual cues predicting rewards of high and low magnitudes and probability (2×2 factorial design), while neural activity was scanned using magnetoencephalography. Importantly, one group of participants received 150 mg of the dopamine precursor levodopa prior to the experiment, while another group received a placebo. For the placebo group, neural signals of reward probability (but not magnitude) emerged at ∼100 ms after cue presentation at occipital sensors in the event-related magnetic fields. Importantly, these probability signals were absent in the levodopa group indicating a close link. Moreover, levodopa administration reduced oscillatory power in the high (20–30 Hz) and low (13–20 Hz) beta band during both reward anticipation and delivery. Taken together, our findings indicate that visual brain regions are involved in coding prospective reward probability but not magnitude and that these effects are modulated by dopamine.     

Aversive Event Anticipation Affects Connectivity between the Ventral Striatum and the Orbitofrontal Cortex in an fMRI Avoidance Task

Ability to anticipate aversive events is important for avoiding dangerous or unpleasant situations. The motivation to avoid an event is influenced by the incentive salience of an event-predicting cue. In an avoidance fMRI task we used tone intensities to manipulate salience in order to study the involvement of the orbitofrontal cortex in processing of incentive salience. In the task, cues predicting either aversive or neutral avoidable tones were presented. Ventral striatum, amygdala and anterior insula activations were significantly stronger during presentation of cues for aversive than neutral tones. A psychophysiological interaction analysis showed stronger connectivity between the ventral striatum and the orbitofrontal cortex during aversive than neutral conditions. The present study shows an interaction between the ventral striatum, a structure previously linked to negative incentive salience, and the orbitofrontal cortex supporting a role for this region in processing salience. In addition, this study replicates previous findings suggesting that the task is robust.     

Sex Differences in the Neural Processing of Aversive Interoceptive Events: The Benefit of Relief

Do men and women process and experience unpleasant bodily states differently? We used fMRI to determine brain processing before, during and after an aversive respiratory stimulation. No sex difference emerged during anticipation or stimulation. However, after the offset of the stimulation, men but not women showed enhanced activation of brain regions that are important for interoception and reward processing. Moreover, this activation was highest in those males who rated the preceding stimulation as most unpleasant. These results indicate that men are particularly sensitive to reward associated with the termination of an aversive event, which may signal relief.     

Reward representations and reward-related learning in the human brain: insights from neuroimaging

This review outlines recent findings from human neuroimaging concerning the role of a highly interconnected network of brain areas including orbital and medial prefrontal cortex, amygdala, striatum and dopaminergic mid-brain in reward processing. Distinct reward-related functions can be attributed to different components of this network. Orbitofrontal cortex is involved in coding stimulus reward value and in concert with the amygdala and ventral striatum is implicated in representing predicted future reward. Such representations can be used to guide action selection for reward, a process that depends, at least in part, on orbital and medial prefrontal cortex as well as dorsal striatum.     

Review. Neurobiological mechanisms for opponent motivational processes in addiction

The conceptualization of drug addiction as a compulsive disorder with excessive drug intake and loss of control over intake requires motivational mechanisms. Opponent process as a motivational theory for the negative reinforcement of drug dependence has long required a neurobiological explanation. Key neurochemical elements involved in reward and stress within basal forebrain structures involving the ventral striatum and extended amygdala are hypothesized to be dysregulated in addiction to convey the opponent motivational processes that drive dependence. Specific neurochemical elements in these structures include not only decreases in reward neurotransmission such as dopamine and opioid peptides in the ventral striatum, but also recruitment of brain stress systems such as corticotropin-releasing factor (CRF), noradrenaline and dynorphin in the extended amygdala. Acute withdrawal from all major drugs of abuse produces increases in reward thresholds, anxiety-like responses and extracellular levels of CRF in the central nucleus of the amygdala. CRF receptor antagonists block excessive drug intake produced by dependence. A brain stress response system is hypothesized to be activated by acute excessive drug intake, to be sensitized during repeated withdrawal, to persist into protracted abstinence and to contribute to stress-induced relapse. The combination of loss of reward function and recruitment of brain stress systems provides a powerful neurochemical basis for the long hypothesized opponent motivational processes responsible for the negative reinforcement driving addiction.     

Neural encoding in ventral striatum during olfactory discrimination learning

A growing body of evidence implicates the ventral striatum in using information acquired through associative learning. The present study examined the activity of ventral striatal neurons in awake, behaving rats during go/no-go odor discrimination learning and reversal. Many neurons fired selectively to odor cues predictive of either appetitive (sucrose) or aversive (quinine) outcomes. Few neurons were selective when first exposed to the odors, but many acquired this differential activity as rats learned the significance of the cues. A substantial proportion of these neurons encoded the cues' learned motivational significance, and these neurons tended to reverse their firing selectivity after reversal of odor-outcome contingencies. Other neurons that became selectively activated during learning did not reverse, but instead appeared to encode specific combinations of cues and associated motor responses. The results support a role for ventral striatum in using the learned significance, both appetitive and aversive, of predictive cues to guide behavior.     

The role of the striatum in aversive learning and aversive prediction errors

Neuroeconomic studies of decision making have emphasized reward learning as critical in the representation of value-driven choice behaviour. However, it is readily apparent that punishment and aversive learning are also significant factors in motivating decisions and actions. In this paper, we review the role of the striatum and amygdala in affective learning and the coding of aversive prediction errors (PEs). We present neuroimaging results showing aversive PE-related signals in the striatum in fear conditioning paradigms with both primary (shock) and secondary (monetary loss) reinforcers. These results and others point to the general role for the striatum in coding PEs across a broad range of learning paradigms and reinforcer types.     

Serotonin differentially regulates short- and long-term prediction of rewards in the ventral and dorsal striatum

Background

The ability to select an action by considering both delays and amount of reward outcome is critical for maximizing long-term benefits. Although previous animal experiments on impulsivity have suggested a role of serotonin in behaviors requiring prediction of delayed rewards, the underlying neural mechanism is unclear.

Methodology/Principal Findings

To elucidate the role of serotonin in the evaluation of delayed rewards, we performed a functional brain imaging experiment in which subjects chose small-immediate or large-delayed liquid rewards under dietary regulation of tryptophan, a precursor of serotonin. A model-based analysis revealed that the activity of the ventral part of the striatum was correlated with reward prediction at shorter time scales, and this correlated activity was stronger at low serotonin levels. By contrast, the activity of the dorsal part of the striatum was correlated with reward prediction at longer time scales, and this correlated activity was stronger at high serotonin levels.

Conclusions/Significance

Our results suggest that serotonin controls the time scale of reward prediction by differentially regulating activities within the striatum.     

Coevolutionary relationship between striatum size and social play in nonhuman primates

The striatum is a region of the brain specifically tied to the experience and anticipation of pleasure, reward, appropriate behavioral sequencing, cognition, learning, and social modulation. Furthermore, the striatum is connected neurologically and functionally to other brain regions associated with the exhibition of social play, such as the neocortex, cerebellum, and limbic system. For these reasons, the striatum is especially interesting to researchers of play behavior. Moreover, the caudate-putamen area of the striatum has been specifically implicated in laboratory studies of social play behavior. This study uses the phylogenetic comparative method of independent contrasts to test for an evolutionary relationship between striatum volume and a measure of social play in nonhuman primates. Relative volume of the primate striatum correlates with rate of social, but not nonsocial, play behavior across species, suggesting a coevolution of traits. The pleasurable and procedural aspects of social play behavior may be mediated in part by the striatum and further to its connection to dopaminergic pathways in the primate brain.     

Decision-making during gambling: an integration of cognitive and psychobiological approaches

Gambling is a widespread form of entertainment that may afford unique insights into the interaction between cognition and emotion in human decision-making. It is also a behaviour that can become harmful, and potentially addictive, in a minority of individuals. This article considers the status of two dominant approaches to gambling behaviour. The cognitive approach has identified a number of erroneous beliefs held by gamblers, which cause them to over-estimate their chances of winning. The psychobiological approach has examined case-control differences between groups of pathological gamblers and healthy controls, and has identified dysregulation of brain areas linked to reward and emotion, including the ventromedial prefrontal cortex (vmPFC) and striatum, as well as alterations in dopamine neurotransmission. In integrating these two approaches, recent data are discussed that reveal anomalous recruitment of the brain reward system (including the vmPFC and ventral striatum) during two common cognitive distortions in gambling games: the near-miss effect and the effect of personal control. In games of chance, near-misses and the presence of control have no objective influence on the likelihood of winning. These manipulations appear to harness a reward system that evolved to learn skill-oriented behaviours, and by modulating activity in this system, these cognitive distortions may promote continued, and potentially excessive, gambling.     

Reward and Novelty Enhance Imagination of Future Events in a Motivational-Episodic Network

Thinking about personal future events is a fundamental cognitive process that helps us make choices in daily life. We investigated how the imagination of episodic future events is influenced by implicit motivational factors known to guide decision making. In a two-day functional magnetic resonance imaging (fMRI) study, we controlled learned reward association and stimulus novelty by pre-familiarizing participants with two sets of words in a reward learning task. Words were repeatedly presented and consistently followed by monetary reward or no monetary outcome. One day later, participants imagined personal future events based on previously rewarded, unrewarded and novel words. Reward association enhanced the perceived vividness of the imagined scenes. Reward and novelty-based construction of future events were associated with higher activation of the motivational system (striatum and substantia nigra/ ventral tegmental area) and hippocampus, and functional connectivity between these areas increased during imagination of events based on reward-associated and novel words. These data indicate that implicit past motivational experience contributes to our expectation of what the future holds in store.     

Amphetamine modulates human incentive processing

Research suggests that psychostimulants can physiologically alter dopamine kinetics in the ventral striatum (VS) and psychologically enhance mood and attention. Using event-related functional magnetic resonance imaging (fMRI), we conducted a within-subject, double-blind, placebo (PLAC)-controlled study of the effects of oral dextroamphetamine (AMPH, 0.25 mg/kg) treatment on brain activity and affect during incentive processing. In two counterbalanced scanning sessions 60-180 min after ingesting AMPH or PLAC, 8 healthy volunteers played a game involving anticipation and receipt of monetary gains and losses. Group and volume of interest analyses suggested that by enhancing tonic over phasic activation, AMPH treatment "equalized" levels of VS activity and positive arousal during anticipation of both gain and loss. These findings suggest that therapeutic effects of amphetamine on incentive processing may involve reducing the difference between anticipation of gains and losses.     

Subliminal instrumental conditioning demonstrated in the human brain

How the brain uses success and failure to optimize future decisions is a long-standing question in neuroscience. One computational solution involves updating the values of context-action associations in proportion to a reward prediction error. Previous evidence suggests that such computations are expressed in the striatum and, as they are cognitively impenetrable, represent an unconscious learning mechanism. Here, we formally test this by studying instrumental conditioning in a situation where we masked contextual cues, such that they were not consciously perceived. Behavioral data showed that subjects nonetheless developed a significant propensity to choose cues associated with monetary rewards relative to punishments. Functional neuroimaging revealed that during conditioning cue values and prediction errors, generated from a computational model, both correlated with activity in ventral striatum. We conclude that, even without conscious processing of contextual cues, our brain can learn their reward value and use them to provide a bias on decision making.