Signatures of Value Comparison in Ventral Striatum Neurons

The ventral striatum (VS), like its cortical afferents, is closely associated with processing of rewards, but the relative contributions of striatal and cortical reward systems remains unclear. Most theories posit distinct roles for these structures, despite their similarities. We compared responses of VS neurons to those of ventromedial prefrontal cortex (vmPFC) Area 14 neurons, recorded in a risky choice task. Five major response patterns observed in vmPFC were also observed in VS: (1) offer value encoding, (2) value difference encoding, (3) preferential encoding of chosen relative to unchosen value, (4) a correlation between residual variance in responses and choices, and (5) prominent encoding of outcomes. We did observe some differences as well; in particular, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, the close match between vmPFC and VS suggests that cortex and its striatal targets make overlapping contributions to economic choice.     

Reward-related FMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation

Long-term potentiation in the hippocampus can be enhanced and prolonged by dopaminergic inputs from midbrain structures such as the substantia nigra. This improved synaptic plasticity is hypothesized to be associated with better memory consolidation in the hippocampus. We used a condition that reliably elicits a dopaminergic response, reward anticipation, to study the relationship between activity of dopaminergic midbrain areas and hippocampal long-term memory in healthy adults. Pictures of object drawings that predicted monetary reward were associated with stronger fMRI activity in reward-related brain areas, including the substantia nigra, compared with non-reward-predicting pictures. Three weeks later, recollection and source memory were better for reward-predicting than for non-reward-predicting pictures. FMRI activity in the hippocampus and the midbrain was higher for reward-predicting pictures that were later recognized compared with later forgotten pictures. These data are consistent with the hypothesis that activation of dopaminergic midbrain regions enhances hippocampus-dependent memory formation, possibly by enhancing consolidation.     

Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology

Neurons in a small number of brain structures detect rewards and reward-predicting stimuli and are active during the expectation of predictable food and liquid rewards. These neurons code the reward information according to basic terms of various behavioural theories that seek to explain reward-directed learning, approach behaviour and decision-making. The involved brain structures include groups of dopamine neurons, the striatum including the nucleus accumbens, the orbitofrontal cortex and the amygdala. The reward information is fed to brain structures involved in decision-making and organisation of behaviour, such as the dorsolateral prefrontal cortex and possibly the parietal cortex. The neural coding of basic reward terms derived from formal theories puts the neurophysiological investigation of reward mechanisms on firm conceptual grounds and provides neural correlates for the function of rewards in learning, approach behaviour and decision-making.     

Reward-motivated learning: mesolimbic activation precedes memory formation

We examined anticipatory mechanisms of reward-motivated memory formation using event-related FMRI. In a monetary incentive encoding task, cues signaled high- or low-value reward for memorizing an upcoming scene. When tested 24 hr postscan, subjects were significantly more likely to remember scenes that followed cues for high-value rather than low-value reward. A monetary incentive delay task independently localized regions responsive to reward anticipation. In the encoding task, high-reward cues preceding remembered but not forgotten scenes activated the ventral tegmental area, nucleus accumbens, and hippocampus. Across subjects, greater activation in these regions predicted superior memory performance. Within subject, increased correlation between the hippocampus and ventral tegmental area was associated with enhanced long-term memory for the subsequent scene. These findings demonstrate that brain activation preceding stimulus encoding can predict declarative memory formation. The findings are consistent with the hypothesis that reward motivation promotes memory formation via dopamine release in the hippocampus prior to learning.     

Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice

In choosing between different rewards expected after unequal delays, humans and animals often prefer the smaller but more immediate reward, indicating that the subjective value or utility of reward is depreciated according to its delay. Here, we show that neurons in the primate caudate nucleus and ventral striatum modulate their activity according to temporally discounted values of rewards with a similar time course. However, neurons in the caudate nucleus encoded the difference in the temporally discounted values of the two alternative targets more reliably than neurons in the ventral striatum. In contrast, neurons in the ventral striatum largely encoded the sum of the temporally discounted values, and therefore, the overall goodness of available options. These results suggest a more pivotal role for the dorsal striatum in action selection during intertemporal choice.     

Direct activation of the ventral striatum in anticipation of aversive stimuli

The brain "reward" system, centered on the limbic ventral striatum, plays a critical role in the response to pleasure and pain. The ventral striatum is activated in animal and human studies during anticipation of appetitive/pleasurable events, but its role in aversive/painful events is less clear. Here we present data from three human fMRI studies based on aversive conditioning using unpleasant cutaneous electrical stimulation and show that the ventral striatum is reliably activated. This activation is observed during anticipation and is not a consequence of relief after the aversive event. Further, the ventral striatum is activated in anticipation regardless of whether there is an opportunity to avoid the aversive stimulus or not. Our data suggest that the ventral striatum, a crucial element of the brain "reward" system, is directly activated in anticipation of aversive stimuli.     

Neural Sensitivity to Absolute and Relative Anticipated Reward in Adolescents

Adolescence is associated with a dramatic increase in risky and impulsive behaviors that have been attributed to developmental differences in neural processing of rewards. In the present study, we sought to identify age differences in anticipation of absolute and relative rewards. To do so, we modified a commonly used monetary incentive delay (MID) task in order to examine brain activity to relative anticipated reward value (neural sensitivity to the value of a reward as a function of other available rewards). This design also made it possible to examine developmental differences in brain activation to absolute anticipated reward magnitude (the degree to which neural activity increases with increasing reward magnitude). While undergoing fMRI, 18 adolescents and 18 adult participants were presented with cues associated with different reward magnitudes. After the cue, participants responded to a target to win money on that trial. Presentation of cues was blocked such that two reward cues associated with $.20, $1.00, or $5.00 were in play on a given block. Thus, the relative value of the $1.00 reward varied depending on whether it was paired with a smaller or larger reward. Reflecting age differences in neural responses to relative anticipated reward (i.e., reference dependent processing), adults, but not adolescents, demonstrated greater activity to a $1 reward when it was the larger of the two available rewards. Adults also demonstrated a more linear increase in ventral striatal activity as a function of increasing absolute reward magnitude compared to adolescents. Additionally, reduced ventral striatal sensitivity to absolute anticipated reward (i.e., the difference in activity to medium versus small rewards) correlated with higher levels of trait Impulsivity. Thus, ventral striatal activity in anticipation of absolute and relative rewards develops with age. Absolute reward processing is also linked to individual differences in Impulsivity.     

Neural responses during anticipation of a primary taste reward

The aim of this study was to determine the brain regions involved in anticipation of a primary taste reward and to compare these regions to those responding to the receipt of a taste reward. Using fMRI, we scanned human subjects who were presented with visual cues that signaled subsequent reinforcement with a pleasant sweet taste (1 M glucose), a moderately unpleasant salt taste (0.2 M saline), or a neutral taste. Expectation of a pleasant taste produced activation in dopaminergic midbrain, posterior dorsal amygdala, striatum, and orbitofrontal cortex (OFC). Apart from OFC, these regions were not activated by reward receipt. The findings indicate that when rewards are predictable, brain regions recruited during expectation are, in part, dissociable from areas responding to reward receipt.     

Subliminal instrumental conditioning demonstrated in the human brain

How the brain uses success and failure to optimize future decisions is a long-standing question in neuroscience. One computational solution involves updating the values of context-action associations in proportion to a reward prediction error. Previous evidence suggests that such computations are expressed in the striatum and, as they are cognitively impenetrable, represent an unconscious learning mechanism. Here, we formally test this by studying instrumental conditioning in a situation where we masked contextual cues, such that they were not consciously perceived. Behavioral data showed that subjects nonetheless developed a significant propensity to choose cues associated with monetary rewards relative to punishments. Functional neuroimaging revealed that during conditioning cue values and prediction errors, generated from a computational model, both correlated with activity in ventral striatum. We conclude that, even without conscious processing of contextual cues, our brain can learn their reward value and use them to provide a bias on decision making.     

Potential vulnerabilities of neuronal reward, risk, and decision mechanisms to addictive drugs

How do addictive drugs hijack the brain's reward system? This review speculates how normal, physiological reward processes may be affected by addictive drugs. Addictive drugs affect acute responses and plasticity in dopamine neurons and postsynaptic structures. These effects reduce reward discrimination, increase the effects of reward prediction error signals, and enhance neuronal responses to reward-predicting stimuli, which may contribute to compulsion. Addictive drugs steepen neuronal temporal reward discounting and create temporal myopia that impairs the control of drug taking. Tonically enhanced dopamine levels may disturb working memory mechanisms necessary for assessing background rewards and thus may generate inaccurate neuronal reward predictions. Drug-induced working memory deficits may impair neuronal risk signaling, promote risky behaviors, and facilitate preaddictive drug use. Malfunctioning adaptive reward coding may lead to overvaluation of drug rewards. Many of these malfunctions may result in inadequate neuronal decision mechanisms and lead to choices biased toward drug rewards.     

Pavlovian reward prediction and receipt in schizophrenia: relationship to anhedonia

Reward processing abnormalities have been implicated in the pathophysiology of negative symptoms such as anhedonia and avolition in schizophrenia. However, studies examining neural responses to reward anticipation and receipt have largely relied on instrumental tasks, which may confound reward processing abnormalities with deficits in response selection and execution. 25 chronic, medicated outpatients with schizophrenia and 20 healthy controls underwent functional magnetic resonance imaging using a pavlovian reward prediction paradigm with no response requirements. Subjects passively viewed cues that predicted subsequent receipt of monetary reward or non-reward, and blood-oxygen-level-dependent signal was measured at the time of cue presentation and receipt. At the group level, neural responses to both reward anticipation and receipt were largely similar between groups. At the time of cue presentation, striatal anticipatory responses did not differ between patients and controls. Right anterior insula demonstrated greater activation for nonreward than reward cues in controls, and for reward than nonreward cues in patients. At the time of receipt, robust responses to receipt of reward vs. nonreward were seen in striatum, midbrain, and frontal cortex in both groups. Furthermore, both groups demonstrated responses to unexpected versus expected outcomes in cortical areas including bilateral dorsolateral prefrontal cortex. Individual difference analyses in patients revealed an association between physical anhedonia and activity in ventral striatum and ventromedial prefrontal cortex during anticipation of reward, in which greater anhedonia severity was associated with reduced activation to money versus no-money cues. In ventromedial prefrontal cortex, this relationship held among both controls and patients, suggesting a relationship between anticipatory activity and anhedonia irrespective of diagnosis. These findings suggest that in the absence of response requirements, brain responses to reward receipt are largely intact in medicated individuals with chronic schizophrenia, while reward anticipation responses in left ventral striatum are reduced in those patients with greater anhedonia severity.     

Lesions of the Fasciculus Retroflexus Alter Footshock-Induced cFos Expression in the Mesopontine Rostromedial Tegmental Area of Rats

Midbrain dopamine neurons are an essential part of the circuitry underlying motivation and reinforcement. They are activated by rewards or reward-predicting cues and inhibited by reward omission. The lateral habenula (lHb), an epithalamic structure that forms reciprocal connections with midbrain dopamine neurons, shows the opposite response being activated by reward omission or aversive stimuli and inhibited by reward-predicting cues. It has been hypothesized that habenular input to midbrain dopamine neurons is conveyed via a feedforward inhibitory pathway involving the GABAergic mesopontine rostromedial tegmental area. Here, we show that exposing rats to low-intensity footshock (four, 0.5 mA shocks over 20 min) induces cFos expression in the rostromedial tegmental area and that this effect is prevented by lesions of the fasciculus retroflexus, the principal output pathway of the habenula. cFos expression is also observed in the medial portion of the lateral habenula, an area that receives dense DA innervation via the fr and the paraventricular nucleus of the thalamus, a stress sensitive area that also receives dopaminergic input. High-intensity footshock (120, 0.8 mA shocks over 40 min) also elevates cFos expression in the rostromedial tegmental area, medial and lateral aspects of the lateral habenula and the paraventricular thalamus. In contrast to low-intensity footshock, increases in cFos expression within the rostromedial tegmental area are not altered by fr lesions suggesting a role for non-habenular inputs during exposure to highly aversive stimuli. These data confirm the involvement of the lateral habenula in modulating the activity of rostromedial tegmental area neurons in response to mild aversive stimuli and suggest that dopamine input may contribute to footshock- induced activation of cFos expression in the lateral habenula.     

Does Taste Matter? How Anticipation of Cola Brands Influences Gustatory Processing in the Brain

Brands surround us everywhere in daily life. Here we investigate the influences of brand cues on gustatory processing of the same beverage. Participants were led to believe that the brand that announced the administration of a Cola mixture provided correct information about the drink to come. We found stronger fMRI signal in right mOFC during weak compared to strong brand cues in a contrast of parametric modulation with subjective liking. When directly comparing the two strong brands cues, more activation in the right amygdala was found for Coca Cola cues compared with Pepsi Cola cues. During the taste phase the same beverage elicited stronger activation in left ventral striatum when it was previously announced by a strong compared with a weak brand. This effect was stronger in participants who drink Cola infrequently and might therefore point to a stronger reliance on brand cues in less experienced consumers. The present results reveal strong effects of brand labels on neural responses signalling reward.     

Evidence for human fronto-central gamma activity during long-term memory encoding of word sequences

Although human gamma activity (30-80 Hz) associated with visual processing is often reported, it is not clear to what extend gamma activity can be reliably detected non-invasively from frontal areas during complex cognitive tasks such as long term memory (LTM) formation. We conducted a memory experiment composed of 35 blocks each having three parts: LTM encoding, working memory (WM) maintenance and LTM retrieval. In the LTM encoding and WM maintenance parts, participants had to respectively encode or maintain the order of three sequentially presented words. During LTM retrieval subjects had to reproduce these sequences. Using magnetoencephalography (MEG) we identified significant differences in the gamma and beta activity. Robust gamma activity (55-65 Hz) in left BA6 (supplementary motor area (SMA)/pre-SMA) was stronger during LTM rehearsal than during WM maintenance. The gamma activity was sustained throughout the 3.4 s rehearsal period during which a fixation cross was presented. Importantly, the difference in gamma band activity correlated with memory performance over subjects. Further we observed a weak gamma power difference in left BA6 during the first half of the LTM rehearsal interval larger for successfully than unsuccessfully reproduced word triplets. In the beta band, we found a power decrease in left anterior regions during LTM rehearsal compared to WM maintenance. Also this suppression of beta power correlated with memory performance over subjects. Our findings show that an extended network of brain areas, characterized by oscillatory activity in different frequency bands, supports the encoding of word sequences in LTM. Gamma band activity in BA6 possibly reflects memory processes associated with language and timing, and suppression of beta activity at left frontal sensors is likely to reflect the release of inhibition directly associated with the engagement of language functions.     

Neural encoding in ventral striatum during olfactory discrimination learning

A growing body of evidence implicates the ventral striatum in using information acquired through associative learning. The present study examined the activity of ventral striatal neurons in awake, behaving rats during go/no-go odor discrimination learning and reversal. Many neurons fired selectively to odor cues predictive of either appetitive (sucrose) or aversive (quinine) outcomes. Few neurons were selective when first exposed to the odors, but many acquired this differential activity as rats learned the significance of the cues. A substantial proportion of these neurons encoded the cues' learned motivational significance, and these neurons tended to reverse their firing selectivity after reversal of odor-outcome contingencies. Other neurons that became selectively activated during learning did not reverse, but instead appeared to encode specific combinations of cues and associated motor responses. The results support a role for ventral striatum in using the learned significance, both appetitive and aversive, of predictive cues to guide behavior.     

Early Effects of Reward Anticipation Are Modulated by Dopaminergic Stimulation

The abilities to predict future rewards and assess the value of reward delivery are crucial aspects of adaptive behavior. While the mesolimbic system, including dopaminergic midbrain, ventral striatum and prefrontal cortex have long been associated with reward processing, recent studies also indicate a prominent role of early visual brain regions. However, the precise underlying neural mechanisms still remain unclear. To address this issue, we presented participants with visual cues predicting rewards of high and low magnitudes and probability (2×2 factorial design), while neural activity was scanned using magnetoencephalography. Importantly, one group of participants received 150 mg of the dopamine precursor levodopa prior to the experiment, while another group received a placebo. For the placebo group, neural signals of reward probability (but not magnitude) emerged at ∼100 ms after cue presentation at occipital sensors in the event-related magnetic fields. Importantly, these probability signals were absent in the levodopa group indicating a close link. Moreover, levodopa administration reduced oscillatory power in the high (20–30 Hz) and low (13–20 Hz) beta band during both reward anticipation and delivery. Taken together, our findings indicate that visual brain regions are involved in coding prospective reward probability but not magnitude and that these effects are modulated by dopamine.     

Reward Anticipation in Ventral Striatum and Individual Sensitivity to Reward: A Pilot Study of a Child-Friendly fMRI Task

Reward processing has been implicated in developmental disorders. However, the classic task to probe reward anticipation, the monetary incentive delay task, has an abstract coding of reward and no storyline and may therefore be less appropriate for use with developmental populations. We modified the task to create a version appropriate for use with children. We investigated whether this child-friendly version could elicit ventral striatal activation during reward anticipation in typically developing children and young adolescents (aged 9.5–14.5). In addition, we tested whether our performance-based measure of reward sensitivity was associated with anticipatory activity in ventral striatum. Reward anticipation was related to activity in bilateral ventral striatum. Moreover, we found an association between individual reward sensitivity and activity in ventral striatum. We conclude that this task assesses ventral striatal activity in a child-friendly paradigm. The combination with a performance-based measure of reward sensitivity potentially makes the task a powerful tool for developmental imaging studies of reward processing.     

Attribution and Expression of Incentive Salience Are Differentially Signaled by Ultrasonic Vocalizations in Rats

During Pavlovian incentive learning, the affective properties of rewards are thought to be transferred to their predicting cues. However, how rewards are represented emotionally in animals is widely unknown. This study sought to determine whether 50-kHz ultrasonic vocalizations (USVs) in rats may signal such a state of incentive motivation to natural, nutritional rewards. To this end, rats learned to anticipate food rewards and, across experiments, the current physiological state (deprived vs. sated), the type of learning mechanism recruited (Pavlovian vs. instrumental), the hedonic properties of UCS (low vs. high palatable food), and the availability of food reward (continued vs. discontinued) were manipulated. Overall, we found that reward-cues elicited 50-kHz calls as they were signaling a putative affective state indicative of incentive motivation in the rat. Attribution and expression of incentive salience, however, seemed not to be an unified process, and could be teased apart in two different ways: 1) under high motivational state (i.e., hunger), the attribution of incentive salience to cues occurred without being expressed at the USVs level, if reward expectations were higher than the outcome; 2) in all experiments when food rewards were devalued by satiation, reward cues were still able to elicit USVs and conditioned anticipatory activity although reward seeking and consumption were drastically weakened. Our results suggest that rats are capable of representing rewards emotionally beyond apparent, immediate physiological demands. These findings may have translational potential in uncovering mechanisms underlying aberrant and persistent motivation as observed in drug addiction, gambling, and eating disorders.     

Food reward in the absence of taste receptor signaling

Food palatability and hedonic value play central roles in nutrient intake. However, postingestive effects can influence food preferences independently of palatability, although the neurobiological bases of such mechanisms remain poorly understood. Of central interest is whether the same brain reward circuitry that is responsive to palatable rewards also encodes metabolic value independently of taste signaling. Here we show that trpm5-/- mice, which lack the cellular machinery required for sweet taste transduction, can develop a robust preference for sucrose solutions based solely on caloric content. Sucrose intake induced dopamine release in the ventral striatum of these sweet-blind mice, a pattern usually associated with receipt of palatable rewards. Furthermore, single neurons in this same ventral striatal region showed increased sensitivity to caloric intake even in the absence of gustatory inputs. Our findings suggest that calorie-rich nutrients can directly influence brain reward circuits that control food intake independently of palatability or functional taste transduction.     

Reduced sensitivity to immediate reward during decision-making in older than younger adults

We examined whether older adults differ from younger adults in the degree to which they favor immediate over delayed rewards during decision-making. To examine the neural correlates of age-related differences in delay discounting we acquired functional MR images while participants made decisions between smaller but sooner and larger but later monetary rewards. The behavioral results show age-related reductions in delay discounting. Less impulsive decision-making in older adults was associated with lower ventral striatal activations to immediate reward. Furthermore, older adults showed an overall higher percentage of delayed choices and reduced activity in the dorsal striatum than younger adults. This points to a reduced reward sensitivity of the dorsal striatum in older adults. Taken together, our findings indicate that less impulsive decision-making in older adults is due to a reduced sensitivity of striatal areas to reward. These age-related changes in reward sensitivity may result from transformations in dopaminergic neuromodulation with age.