Reward and Novelty Enhance Imagination of Future Events in a Motivational-Episodic Network

Thinking about personal future events is a fundamental cognitive process that helps us make choices in daily life. We investigated how the imagination of episodic future events is influenced by implicit motivational factors known to guide decision making. In a two-day functional magnetic resonance imaging (fMRI) study, we controlled learned reward association and stimulus novelty by pre-familiarizing participants with two sets of words in a reward learning task. Words were repeatedly presented and consistently followed by monetary reward or no monetary outcome. One day later, participants imagined personal future events based on previously rewarded, unrewarded and novel words. Reward association enhanced the perceived vividness of the imagined scenes. Reward and novelty-based construction of future events were associated with higher activation of the motivational system (striatum and substantia nigra/ ventral tegmental area) and hippocampus, and functional connectivity between these areas increased during imagination of events based on reward-associated and novel words. These data indicate that implicit past motivational experience contributes to our expectation of what the future holds in store.     

Ventromedial prefrontal cortex activation is associated with memory formation for predictable rewards

During reinforcement learning, dopamine release shifts from the moment of reward consumption to the time point when the reward can be predicted. Previous studies provide consistent evidence that reward-predicting cues enhance long-term memory (LTM) formation of these items via dopaminergic projections to the ventral striatum. However, it is less clear whether memory for items that do not precede a reward but are directly associated with reward consumption is also facilitated. Here, we investigated this question in an fMRI paradigm in which LTM for reward-predicting and neutral cues was compared to LTM for items presented during consumption of reliably predictable as compared to less predictable rewards. We observed activation of the ventral striatum and enhanced memory formation during reward anticipation. During processing of less predictable as compared to reliably predictable rewards, the ventral striatum was activated as well, but items associated with less predictable outcomes were remembered worse than items associated with reliably predictable outcomes. Processing of reliably predictable rewards activated the ventromedial prefrontal cortex (vmPFC), and vmPFC BOLD responses were associated with successful memory formation of these items. Taken together, these findings show that consumption of reliably predictable rewards facilitates LTM formation and is associated with activation of the vmPFC.     

Reward Anticipation in Ventral Striatum and Individual Sensitivity to Reward: A Pilot Study of a Child-Friendly fMRI Task

Reward processing has been implicated in developmental disorders. However, the classic task to probe reward anticipation, the monetary incentive delay task, has an abstract coding of reward and no storyline and may therefore be less appropriate for use with developmental populations. We modified the task to create a version appropriate for use with children. We investigated whether this child-friendly version could elicit ventral striatal activation during reward anticipation in typically developing children and young adolescents (aged 9.5–14.5). In addition, we tested whether our performance-based measure of reward sensitivity was associated with anticipatory activity in ventral striatum. Reward anticipation was related to activity in bilateral ventral striatum. Moreover, we found an association between individual reward sensitivity and activity in ventral striatum. We conclude that this task assesses ventral striatal activity in a child-friendly paradigm. The combination with a performance-based measure of reward sensitivity potentially makes the task a powerful tool for developmental imaging studies of reward processing.     

Studying Memory Encoding to Promote Reliable Engagement of the Medial Temporal Lobe at the Single-Subject Level

The medial temporal lobe (MTL)—comprising hippocampus and the surrounding neocortical regions—is a targeted brain area sensitive to several neurological diseases. Although functional magnetic resonance imaging (fMRI) has been widely used to assess brain functional abnormalities, detecting MTL activation has been technically challenging. The aim of our study was to provide an fMRI paradigm that reliably activates MTL regions at the individual level, thus providing a useful tool for future research in clinical memory-related studies. Twenty young healthy adults underwent an event-related fMRI study consisting of three encoding conditions: word-pairs, face-name associations and complex visual scenes. A region-of-interest analysis at the individual level comparing novel and repeated stimuli independently for each task was performed. The results of this analysis yielded activations in the hippocampal and parahippocampal regions in most of the participants. Specifically, 95% and 100% of participants showed significant activations in the left hippocampus during the face-name encoding and in the right parahippocampus, respectively, during scene encoding. Additionally, a whole brain analysis, also comparing novel versus repeated stimuli at the group level, showed mainly left frontal activation during the word task. In this group analysis, the face-name association engaged the HP and fusiform gyri bilaterally, along with the left inferior frontal gyrus, and the complex visual scenes activated mainly the parahippocampus and hippocampus bilaterally. In sum, our task design represents a rapid and reliable manner to study and explore MTL activity at the individual level, thus providing a useful tool for future research in clinical memory-related fMRI studies.     

Reward-related FMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation

Long-term potentiation in the hippocampus can be enhanced and prolonged by dopaminergic inputs from midbrain structures such as the substantia nigra. This improved synaptic plasticity is hypothesized to be associated with better memory consolidation in the hippocampus. We used a condition that reliably elicits a dopaminergic response, reward anticipation, to study the relationship between activity of dopaminergic midbrain areas and hippocampal long-term memory in healthy adults. Pictures of object drawings that predicted monetary reward were associated with stronger fMRI activity in reward-related brain areas, including the substantia nigra, compared with non-reward-predicting pictures. Three weeks later, recollection and source memory were better for reward-predicting than for non-reward-predicting pictures. FMRI activity in the hippocampus and the midbrain was higher for reward-predicting pictures that were later recognized compared with later forgotten pictures. These data are consistent with the hypothesis that activation of dopaminergic midbrain regions enhances hippocampus-dependent memory formation, possibly by enhancing consolidation.     

Neural Sensitivity to Absolute and Relative Anticipated Reward in Adolescents

Adolescence is associated with a dramatic increase in risky and impulsive behaviors that have been attributed to developmental differences in neural processing of rewards. In the present study, we sought to identify age differences in anticipation of absolute and relative rewards. To do so, we modified a commonly used monetary incentive delay (MID) task in order to examine brain activity to relative anticipated reward value (neural sensitivity to the value of a reward as a function of other available rewards). This design also made it possible to examine developmental differences in brain activation to absolute anticipated reward magnitude (the degree to which neural activity increases with increasing reward magnitude). While undergoing fMRI, 18 adolescents and 18 adult participants were presented with cues associated with different reward magnitudes. After the cue, participants responded to a target to win money on that trial. Presentation of cues was blocked such that two reward cues associated with $.20, $1.00, or $5.00 were in play on a given block. Thus, the relative value of the $1.00 reward varied depending on whether it was paired with a smaller or larger reward. Reflecting age differences in neural responses to relative anticipated reward (i.e., reference dependent processing), adults, but not adolescents, demonstrated greater activity to a $1 reward when it was the larger of the two available rewards. Adults also demonstrated a more linear increase in ventral striatal activity as a function of increasing absolute reward magnitude compared to adolescents. Additionally, reduced ventral striatal sensitivity to absolute anticipated reward (i.e., the difference in activity to medium versus small rewards) correlated with higher levels of trait Impulsivity. Thus, ventral striatal activity in anticipation of absolute and relative rewards develops with age. Absolute reward processing is also linked to individual differences in Impulsivity.     

Food Reward-Induced Neurotransmitter Changes in Cognitive Brain Regions

Recent evidence indicates that mechanisms involved in reward and mechanisms involved in learning interact, in that reward includes learning processes and learning includes reward processes. In spite of such interactions, reward and learning represent distinct functions. In the present study, as part of an examination of the differences in learning and reward mechanisms, it was assumed that food principally affects reward mechanisms. After a brief period of fasting, we assayed the release of three neurotransmitters and their associated metabolites in eight brain areas associated with learning and memory as a response to feeding. Using microdialysis for the assay, we found changes in the hippocampus, cortex, amygdala, and the thalamic nucleus, (considered cognitive areas), in addition to those in the nucleus accumbens and ventral tegmental area (considered reward areas). Extracellular dopamine levels increased in the nucleus accumbens, ventral tegmental area, amygdala, and thalamic nucleus, while they decreased in the hippocampus and prefrontal cortex. Dopamine metabolites increased in all areas tested (except the dorsal hippocampus); changes in norepinephrine varied with decreases in the accumbens, dorsal hippocampus, amygdala, and thalamic nucleus, and increases in the prefrontal cortex; serotonin levels decreased in all the areas tested; although its metabolite 5HIAA increased in two regions (the medial temporal cortex, and thalamic nucleus). Our assays indicate that in reward activities such as feeding, in addition to areas usually associated with reward such as the mesolimbic dopamine system, other areas associated with cognition also participate. Results also indicate that several transmitter systems play a part, with several neurotransmitters and several receptors involved in the response to food in a number of brain structures, and the changes in transmitter levels may be affected by metabolism and transport in addition to changes in release in a regionally heterogeneous manner. Food reward represents a complex pattern of changes in the brain that involve cognitive processes. Although food reward elements overlap with other reward systems sharing some neurotransmitter compounds, it significantly differs indicating a specific reward to process for food consumption. Like in other rewards, both learning and cognitive areas play a significant part in food reward. Special issue dedicated to Dr. Moussa Youdim.     

Convergent processing of both positive and negative motivational signals by the VTA dopamine neuronal populations

Dopamine neurons in the ventral tegmental area (VTA) have been traditionally studied for their roles in reward-related motivation or drug addiction. Here we study how the VTA dopamine neuron population may process fearful and negative experiences as well as reward information in freely behaving mice. Using multi-tetrode recording, we find that up to 89% of the putative dopamine neurons in the VTA exhibit significant activation in response to the conditioned tone that predict food reward, while the same dopamine neuron population also respond to the fearful experiences such as free fall and shake events. The majority of these VTA putative dopamine neurons exhibit suppression and offset-rebound excitation, whereas ～25% of the recorded putative dopamine neurons show excitation by the fearful events. Importantly, VTA putative dopamine neurons exhibit parametric encoding properties: their firing change durations are proportional to the fearful event durations. In addition, we demonstrate that the contextual information is crucial for these neurons to respectively elicit positive or negative motivational responses by the same conditioned tone. Taken together, our findings suggest that VTA dopamine neurons may employ the convergent encoding strategy for processing both positive and negative experiences, intimately integrating with cues and environmental context.     

Led into Temptation? Rewarding Brand Logos Bias the Neural Encoding of Incidental Economic Decisions

Human decision-making is driven by subjective values assigned to alternative choice options. These valuations are based on reward cues. It is unknown, however, whether complex reward cues, such as brand logos, may bias the neural encoding of subjective value in unrelated decisions. In this functional magnetic resonance imaging (fMRI) study, we subliminally presented brand logos preceding intertemporal choices. We demonstrated that priming biased participants' preferences towards more immediate rewards in the subsequent temporal discounting task. This was associated with modulations of the neural encoding of subjective values of choice options in a network of brain regions, including but not restricted to medial prefrontal cortex. Our findings demonstrate the general susceptibility of the human decision making system to apparently incidental contextual information. We conclude that the brain incorporates seemingly unrelated value information that modifies decision making outside the decision-maker's awareness.     

Dissociation between Dorsal and Ventral Posterior Parietal Cortical Responses to Incidental Changes in Natural Scenes

Background

The posterior parietal cortex (PPC) is thought to interact with the medial temporal lobe (MTL) to support spatial cognition and topographical memory. While the response of medial temporal lobe regions to topographical stimuli has been intensively studied, much less research has focused on the role of PPC and its functional connectivity with the medial temporal lobe.

Methodology/Principle Findings

Here we report a dissociation between dorsal and ventral regions of PPC in response to different types of change in natural scenes using an fMRI adaptation paradigm. During scanning subjects performed an incidental target detection task whilst viewing trial unique sequentially presented pairs of natural scenes, each containing a single prominent object. We observed a dissociation between the superior parietal gyrus and the angular gyrus, with the former showing greater sensitivity to spatial change, and the latter showing greater sensitivity to scene novelty. In addition, we observed that the parahippocampal cortex has increased functional connectivity with the angular gyrus, but not superior parietal gyrus, when subjects view change to the scene content.

Conclusions/Significance

Our findings provide support for proposed dissociations between dorsal and ventral regions of PPC and suggest that the dorsal PPC may support the spatial coding of the visual environment even when this information is incidental to the task at hand. Further, through revealing the differential functional interactions of the SPG and AG with the MTL our results help advance our understanding of how the MTL and PPC cooperate to update representations of the world around us.     

Primary and Secondary Rewards Differentially Modulate Neural Activity Dynamics during Working Memory

Background

Cognitive control and working memory processes have been found to be influenced by changes in motivational state. Nevertheless, the impact of different motivational variables on behavior and brain activity remains unclear.

Methodology/Principal Findings

The current study examined the impact of incentive category by varying on a within-subjects basis whether performance during a working memory task was reinforced with either secondary (monetary) or primary (liquid) rewards. The temporal dynamics of motivation-cognition interactions were investigated by employing an experimental design that enabled isolation of sustained and transient effects. Performance was dramatically and equivalently enhanced in each incentive condition, whereas neural activity dynamics differed between incentive categories. The monetary reward condition was associated with a tonic activation increase in primarily right-lateralized cognitive control regions including anterior prefrontal cortex (PFC), dorsolateral PFC, and parietal cortex. In the liquid condition, the identical regions instead showed a shift in transient activation from a reactive control pattern (primary probe-based activation) during no-incentive trials to proactive control (primary cue-based activation) during rewarded trials. Additionally, liquid-specific tonic activation increases were found in subcortical regions (amygdala, dorsal striatum, nucleus accumbens), indicating an anatomical double dissociation in the locus of sustained activation.

Conclusions/Significance

These different activation patterns suggest that primary and secondary rewards may produce similar behavioral changes through distinct neural mechanisms of reinforcement. Further, our results provide new evidence for the flexibility of cognitive control, in terms of the temporal dynamics of activation.     

Adolescents,Adults and Rewards: Comparing Motivational Neurocircuitry Recruitment Using fMRI

Background

Adolescent risk-taking, including behaviors resulting in injury or death, has been attributed in part to maturational differences in mesolimbic incentive-motivational neurocircuitry, including ostensible oversensitivity of the nucleus accumbens (NAcc) to rewards.

Methodology/Principal Findings

To test whether adolescents showed increased NAcc activation by cues for rewards, or by delivery of rewards, we scanned 24 adolescents (age 12–17) and 24 adults age (22–42) with functional magnetic resonance imaging while they performed a monetary incentive delay (MID) task. The MID task was configured to temporally disentangle potential reward or potential loss anticipation-related brain signal from reward or loss notification-related signal. Subjects saw cues signaling opportunities to win or avoid losing $0, $.50, or $5 for responding quickly to a subsequent target. Subjects then viewed feedback of their trial success after a variable interval from cue presentation of between 6 to17 s. Adolescents showed reduced NAcc recruitment by reward-predictive cues compared to adult controls in a linear contrast with non-incentive cues, and in a volume-of-interest analysis of signal change in the NAcc. In contrast, adolescents showed little difference in striatal and frontocortical responsiveness to reward deliveries compared to adults.

Conclusions/Significance

In light of divergent developmental difference findings between neuroimaging incentive paradigms (as well as at different stages within the same task), these data suggest that maturational differences in incentive-motivational neurocircuitry: 1) may be sensitive to nuances of incentive tasks or stimuli, such as behavioral or learning contingencies, and 2) may be specific to the component of the instrumental behavior (such as anticipation versus notification).     

A 4 Hz oscillation adaptively synchronizes prefrontal, VTA, and hippocampal activities

Network oscillations support transient communication across brain structures. We show here, in rats, that task-related neuronal activity in the medial prefrontal cortex (PFC), the hippocampus, and the ventral tegmental area (VTA), regions critical for working memory, is coordinated by a 4 Hz oscillation. A prominent increase of power and coherence of the 4 Hz oscillation in the PFC and the VTA and its phase modulation of gamma power in both structures was present in the working memory part of the task. Subsets of both PFC and hippocampal neurons predicted the turn choices of the rat. The goal-predicting PFC pyramidal neurons were more strongly phase locked to both 4 Hz and hippocampal theta oscillations than nonpredicting cells. The 4 Hz and theta oscillations were phase coupled and jointly modulated both gamma waves and neuronal spikes in the PFC, the VTA, and the hippocampus. Thus, multiplexed timing mechanisms in the PFC-VTA-hippocampus axis may support processing of information, including working memory.     

Effects of intra-VTA injection of neurotensin on local cerebral glucose utilization in freely moving rats

The [14C]2-deoxyglucose method was applied to measure the effects of the injection of neurotensin (7 microg) in the ventral tegmental area on local cerebral glucose utilization in the rat. Injection of neurotensin produced significant increases of glucose utilization in the shell of the nucleus accumbens and in the olfactory tubercle. These results indicate that stimulation of neurotensin receptors in the ventral tegmental area produces functional changes that are confined to the regions receiving mesolimbic projections within the rostral extended amygdaloid complex. These findings extend our understanding on the effects of neurotensin in the limbic system, with particular regard to reward pathways.     

Impact of Early Life Adversity on Reward Processing in Young Adults: EEG-fMRI Results from a Prospective Study over 25 Years

Several lines of evidence have implicated the mesolimbic dopamine reward pathway in altered brain function resulting from exposure to early adversity. The present study examined the impact of early life adversity on different stages of neuronal reward processing later in life and their association with a related behavioral phenotype, i.e. attention deficit/hyperactivity disorder (ADHD). 162 healthy young adults (mean age = 24.4 years; 58% female) from an epidemiological cohort study followed since birth participated in a simultaneous EEG-fMRI study using a monetary incentive delay task. Early life adversity according to an early family adversity index (EFA) and lifetime ADHD symptoms were assessed using standardized parent interviews conducted at the offspring''s age of 3 months and between 2 and 15 years, respectively. fMRI region-of-interest analysis revealed a significant effect of EFA during reward anticipation in reward-related areas (i.e. ventral striatum, putamen, thalamus), indicating decreased activation when EFA increased. EEG analysis demonstrated a similar effect for the contingent negative variation (CNV), with the CNV decreasing with the level of EFA. In contrast, during reward delivery, activation of the bilateral insula, right pallidum and bilateral putamen increased with EFA. There was a significant association of lifetime ADHD symptoms with lower activation in the left ventral striatum during reward anticipation and higher activation in the right insula during reward delivery. The present findings indicate a differential long-term impact of early life adversity on reward processing, implicating hyporesponsiveness during reward anticipation and hyperresponsiveness when receiving a reward. Moreover, a similar activation pattern related to lifetime ADHD suggests that the impact of early life stress on ADHD may possibly be mediated by a dysfunctional reward pathway.     

The hippocampal-VTA loop: controlling the entry of information into long-term memory

In this article we develop the concept that the hippocampus and the midbrain dopaminergic neurons of the ventral tegmental area (VTA) form a functional loop. Activation of the loop begins when the hippocampus detects newly arrived information that is not already stored in its long-term memory. The resulting novelty signal is conveyed through the subiculum, accumbens, and ventral pallidum to the VTA where it contributes (along with salience and goal information) to the novelty-dependent firing of these cells. In the upward arm of the loop, dopamine (DA) is released within the hippocampus; this produces an enhancement of LTP and learning. These findings support a model whereby the hippocampal-VTA loop regulates the entry of information into long-term memory.     

Pavlovian reward prediction and receipt in schizophrenia: relationship to anhedonia

Reward processing abnormalities have been implicated in the pathophysiology of negative symptoms such as anhedonia and avolition in schizophrenia. However, studies examining neural responses to reward anticipation and receipt have largely relied on instrumental tasks, which may confound reward processing abnormalities with deficits in response selection and execution. 25 chronic, medicated outpatients with schizophrenia and 20 healthy controls underwent functional magnetic resonance imaging using a pavlovian reward prediction paradigm with no response requirements. Subjects passively viewed cues that predicted subsequent receipt of monetary reward or non-reward, and blood-oxygen-level-dependent signal was measured at the time of cue presentation and receipt. At the group level, neural responses to both reward anticipation and receipt were largely similar between groups. At the time of cue presentation, striatal anticipatory responses did not differ between patients and controls. Right anterior insula demonstrated greater activation for nonreward than reward cues in controls, and for reward than nonreward cues in patients. At the time of receipt, robust responses to receipt of reward vs. nonreward were seen in striatum, midbrain, and frontal cortex in both groups. Furthermore, both groups demonstrated responses to unexpected versus expected outcomes in cortical areas including bilateral dorsolateral prefrontal cortex. Individual difference analyses in patients revealed an association between physical anhedonia and activity in ventral striatum and ventromedial prefrontal cortex during anticipation of reward, in which greater anhedonia severity was associated with reduced activation to money versus no-money cues. In ventromedial prefrontal cortex, this relationship held among both controls and patients, suggesting a relationship between anticipatory activity and anhedonia irrespective of diagnosis. These findings suggest that in the absence of response requirements, brain responses to reward receipt are largely intact in medicated individuals with chronic schizophrenia, while reward anticipation responses in left ventral striatum are reduced in those patients with greater anhedonia severity.     

Falling Out of Time: Enhanced Memory for Scenes Presented at Behaviorally Irrelevant Points in Time in Posttraumatic Stress Disorder (PTSD)

Spontaneous encoding of the visual environment depends on the behavioral relevance of the task performed simultaneously. If participants identify target letters or auditory tones while viewing a series of briefly presented natural and urban scenes, they demonstrate effective scene recognition only when a target, but not a behaviorally irrelevant distractor, appears together with the scene. Here, we show that individuals with posttraumatic stress disorder (PTSD), who witnessed the red sludge disaster in Hungary, show the opposite pattern of performance: enhanced recognition of scenes presented together with distractors and deficient recognition of scenes presented with targets. The recognition of trauma-related and neutral scenes was not different in individuals with PTSD. We found a positive correlation between memory for scenes presented with auditory distractors and re-experiencing symptoms (memory intrusions and flashbacks). These results suggest that abnormal encoding of visual scenes at behaviorally irrelevant events might be associated with intrusive experiences by disrupting the flow of time.     

Enhanced Memory for Scenes Presented at Behaviorally Relevant Points in Time

The ability to remember a briefly presented scene depends on a number of factors, such as its saliency, novelty, degree of threat, or behavioral relevance to a task. Here, however, we show that the encoding of a scene into memory may depend not only on what the scene contains but also when it occurs. Participants performed an attentionally demanding target detection task at fixation while also viewing a rapid sequence of full-field photographs of urban and natural scenes. Participants were then tested on whether they recognized a specific scene from the previous sequence. We found that scenes were recognized reliably only when presented concurrently with a target at fixation. This is evidence of a mechanism where traces of a visual scene are automatically encoded into memory at behaviorally relevant points in time regardless of the spatial focus of attention.     

Nicotinic modulation of synaptic transmission and plasticity in cortico-limbic circuits

Nicotine is the principle addictive agent delivered via cigarette smoking. The addictive activity of nicotine is due to potent interactions with nicotinic acetylcholine receptors (nAChRs) on neurons in the reinforcement and reward circuits of the brain. Beyond its addictive actions, nicotine is thought to have positive effects on performance in working memory and short-term attention-related tasks. The brain areas involved in such behaviors are part of an extensive cortico-limbic network that includes relays between prefrontal cortex (PFC) and cingulate cortex (CC), hippocampus, amygdala, ventral tegmental area (VTA) and the nucleus accumbens (nAcc). Nicotine activates a broad array of nAChRs subtypes that can be targeted to pre- as well as peri- and post-synaptic locations in these areas. Thereby, nicotine not only excites different types of neurons, but it also perturbs baseline neuronal communication, alters synaptic properties and modulates synaptic plasticity.In this review we focus on recent findings on nicotinic modulation of cortical circuits and their targets fields, which show that acute and transient activation of nicotinic receptors in cortico-limbic circuits triggers a series of events that affects cognitive performance in a long lasting manner. Understanding how nicotine induces long-term changes in synapses and alters plasticity in the cortico-limbic circuits is essential to determining how these areas interact in decoding fundamental aspects of cognition and reward.