Effects of L-dopa during Auditory Instrumental Learning in Humans

The dopaminergic neurotransmitter system is critically involved in promoting plasticity in auditory cortex. We combined functional magnetic resonance imaging (fMRI) and a pharmacological manipulation to investigate dopaminergic modulation of neural activity in auditory cortex during instrumental learning. Volunteers either received 100 mg L-dopa (Madopar) or placebo in an appetitive, differential instrumental conditioning paradigm, which involved learning that a specific category of frequency modulated tones predicts a monetary reward when fast responses were made in a subsequent reaction time task. The other category of frequency modulated tones was not related to a reward. Our behavioral data provides evidence that dopaminergic stimulation differentially impacts on the speed of instrumental responding in rewarded and unrewarded trials. L-dopa increased neural BOLD activity in left auditory cortex to tones in rewarded and unrewarded trials. This increase was related to plasma L-dopa levels and learning rate. Our data thus provides evidence for dopaminergic modulation of neural activity in auditory cortex, which occurs for both auditory stimuli related to a later reward and those not related to a reward.     

Reward and Novelty Enhance Imagination of Future Events in a Motivational-Episodic Network

Thinking about personal future events is a fundamental cognitive process that helps us make choices in daily life. We investigated how the imagination of episodic future events is influenced by implicit motivational factors known to guide decision making. In a two-day functional magnetic resonance imaging (fMRI) study, we controlled learned reward association and stimulus novelty by pre-familiarizing participants with two sets of words in a reward learning task. Words were repeatedly presented and consistently followed by monetary reward or no monetary outcome. One day later, participants imagined personal future events based on previously rewarded, unrewarded and novel words. Reward association enhanced the perceived vividness of the imagined scenes. Reward and novelty-based construction of future events were associated with higher activation of the motivational system (striatum and substantia nigra/ ventral tegmental area) and hippocampus, and functional connectivity between these areas increased during imagination of events based on reward-associated and novel words. These data indicate that implicit past motivational experience contributes to our expectation of what the future holds in store.     

To sleep, to strive, or both: how best to optimize memory

While numerous studies have shown that a night of sleep profits memory relative to wake, we still have little understanding about what factors mediate this effect of sleep. A clear understanding of the dynamics of this effect of sleep beyond the initial night of sleep is also lacking. Here, we examined the effect of extrinsic rewards on sleep-dependent declarative memory processing across 12 and 24 hr training-retest intervals. Subjects were either paid based on their performance at retest ($1 for each correct answer), or received a flat fee for participation. After a 12 hr interval we observed pronounced benefits of both sleep and reward on memory. Over an extended 24 hr interval we found 1) that an initial night of sleep partially protects memories from subsequent deterioration during wake, and 2) that sleep blocks further deterioration, and may even have a restorative effect on memory, when it follows a full day of wake. Interestingly, the benefit imparted to rewarded (relative to unrewarded) stimuli was equal for sleep and wake subjects, suggesting that the sleeping brain may not differentially process rewarded information, relative to wake. However, looking at the overall impact of sleep relative to reward in this protocol, it was apparent that sleep both imparted a stronger mnemonic boost than reward, and provided a benefit to memory regardless of whether it occurred in the first or the second 12 hrs following task training.     

The influence of barley food rewards on sheep movement through a handling system

The use of barley food rewards to improve sheep handling was evaluated in two experiments. In Experiment 1, six groups of sheep were trained to run through a race system and into a sheep-handling machine. Three groups were rewarded and three were not. After 10 days of training over 1 month, rewarded and unrewarded groups were either not handled, clamped in the machine, or clamped and inverted in the machine, and the efficiency of handling was measured. Food rewards significantly reduced the amount of labour required for sheep handling. Less time was spent pushing up rewarded sheep than unrewarded sheep for all handling treatments, and the race system was more than 90% efficient in 9 out of 15 tests with rewarded sheep compared with 1 out of 15 for unrewarded sheep.In Experiment 2, long-term memory was assessed by re-testing the same sheep 1 year later. Similar results to the first test were obtained for control (not handled) and clamped groups, but for inverted sheep there was no significant difference in push-up time between rewarded and unrewarded groups. These results suggest that sheep have excellent long-term memories of sheep handling procedures, both when rewarded and unrewarded. Although the effectiveness of food rewards was diminished as the severity of the handling treatment increased, the results indicate that rewarding sheep could lead to long-term improvement in sheep handling efficiency, especially when severe treatments are performed rarely.     

Dynamics of pre- and post-choice behaviour: rats approximate optimal strategy in a discrete-trial decision task

We simulate two types of environments to investigate how closely rats approximate optimal foraging. Rats initiated a trial where they chose between two spouts for sucrose, which was delivered at distinct probabilities. The discrete trial procedure used allowed us to observe the relationship between choice proportions, response latencies and obtained rewards. Our results show that rats approximate the optimal strategy across a range of environments that differ in the average probability of reward as well as the dynamics of the depletion-renewal cycle. We found that the constituent components of a single choice differentially reflect environmental contingencies. Post-choice behaviour, measured as the duration of time rats spent licking at the spouts on unrewarded trials, was the most sensitive index of environmental variables, adjusting most rapidly to changes in the environment. These findings have implications for the role of confidence in choice outcomes for guiding future choices.     

Visual associative learning in restrained honey bees with intact antennae

A restrained honey bee can be trained to extend its proboscis in response to the pairing of an odor with a sucrose reward, a form of olfactory associative learning referred to as the proboscis extension response (PER). Although the ability of flying honey bees to respond to visual cues is well-established, associative visual learning in restrained honey bees has been challenging to demonstrate. Those few groups that have documented vision-based PER have reported that removing the antennae prior to training is a prerequisite for learning. Here we report, for a simple visual learning task, the first successful performance by restrained honey bees with intact antennae. Honey bee foragers were trained on a differential visual association task by pairing the presentation of a blue light with a sucrose reward and leaving the presentation of a green light unrewarded. A negative correlation was found between age of foragers and their performance in the visual PER task. Using the adaptations to the traditional PER task outlined here, future studies can exploit pharmacological and physiological techniques to explore the neural circuit basis of visual learning in the honey bee.     

Trait Anticipatory Pleasure Predicts Effort Expenditure for Reward

Research in motivation and emotion has been increasingly influenced by the perspective that processes underpinning the motivated approach of rewarding goals are distinct from those underpinning enjoyment during reward consummation. This distinction recently inspired the construction of the Temporal Experience of Pleasure Scale (TEPS), a self-report measure that distinguishes trait anticipatory pleasure (pre-reward feelings of desire) from consummatory pleasure (feelings of enjoyment and gratification upon reward attainment). In a university community sample (N = 97), we examined the TEPS subscales as predictors of (1) the willingness to expend effort for monetary rewards, and (2) affective responses to a pleasant mood induction procedure. Results showed that both anticipatory pleasure and a well-known trait measure of reward motivation predicted effort-expenditure for rewards when the probability of being rewarded was relatively low. Against expectations, consummatory pleasure was unrelated to induced pleasant affect. Taken together, our findings provide support for the validity of the TEPS anticipatory pleasure scale, but not the consummatory pleasure scale.     

Modulation of caudate activity by action contingency

Research has increasingly implicated the striatum in the processing of reward-related information in both animals and humans. However, it is unclear whether human striatal activation is driven solely by the hedonic properties of rewards or whether such activation is reliant on other factors, such as anticipation of upcoming reward or performance of an action to earn a reward. We used event-related functional magnetic resonance imaging to investigate hemodynamic responses to monetary rewards and punishments in three experiments that made use of an oddball paradigm. We presented reward and punishment displays randomly in time, following an anticipatory cue, or following a button press response. Robust and differential activation of the caudate nucleus occurred only when a perception of contingency existed between the button press response and the outcome. This finding suggests that the caudate is involved in reinforcement of action potentially leading to reward, rather than in processing reward per se.     

The date-delay framing effect in temporal discounting depends on substance abuse

In the present study, individuals with substance use disorders (n=30) and non-addicted controls (n=30) were presented with a delay-discounting task with time being described either as dates or as temporal intervals. Three main results were obtained. First, in both groups reward size had a large impact on discounting future rewards, with discount rates becoming larger with smaller reward sizes. Second, participants discounted future rewards less strongly when their time of delivery was presented as a date instead of a temporal distance. Third, whereas discount rates of individuals with substance use disorders varied substantially with regard to the presentation of time in the task, the controls changed their choices depending on time presentation only slightly.     

Functional polymorphism of the mu-opioid receptor gene (OPRM1) influences reinforcement learning in humans

Previous reports on the functional effects (i.e., gain or loss of function), and phenotypic outcomes (e.g., changes in addiction vulnerability and stress response) of a commonly occurring functional single nucleotide polymorphism (SNP) of the mu-opioid receptor (OPRM1 A118G) have been inconsistent. Here we examine the effect of this polymorphism on implicit reward learning. We used a probabilistic signal detection task to determine whether this polymorphism impacts response bias to monetary reward in 63 healthy adult subjects: 51 AA homozygotes and 12 G allele carriers. OPRM1 AA homozygotes exhibited typical responding to the rewarded response--that is, their bias to the rewarded stimulus increased over time. However, OPRM1 G allele carriers exhibited a decline in response to the rewarded stimulus compared to the AA homozygotes. These results extend previous reports on the heritability of performance on this task by implicating a specific polymorphism. Through comparison with other studies using this task, we suggest a possible mechanism by which the OPRM1 polymorphism may confer reduced response to natural reward through a dopamine-mediated decrease during positive reinforcement learning.     

Competing demands of prosociality and equity in monkeys

Prosocial decisions may lead to unequal payoffs among group members. Although an aversion to inequity has been found in empirical studies of both human and nonhuman primates, the contexts previously studied typically do not involve a trade-off between prosociality and inequity. Here we investigate the apparent coexistence of these two factors, specifically the competing demands of prosociality and equity. We directly compare the responses of brown capuchin monkeys (Cebus apella) among situations where prosocial preferences conflict with equality, using a paradigm comparable to other studies of cooperation and inequity in this species. By choosing to pull a tray towards themselves, subjects rewarded themselves and/or another in conditions in which the partner either received the same or different rewards, or the subject received no reward. In unequal payoff conditions, subjects could obtain equality by choosing not to pull in the tray, so that neither individual was rewarded. The monkeys showed prosocial preferences even in situations of moderate disadvantageous inequity, preferring to pull in the tray more often when a partner was present than absent. However, when the discrepancy between rewards increased, prosocial behavior ceased.     

Associative Learning in Response to Color in the Parasitoid Wasp Nasonia vitripennis (Hymenoptera: Pteromalidae)

A parasitoid that can learn cues associated with the host microenvironment should have an increased chance of future host location and thereby increase its reproductive success. This study examines associative learning in response to simultaneous exposure to the colors yellow and blue in mated females of the parasitoid wasp Nasonia vitripennis. Preference was measured as the proportion of time spent on a color. When trained with one color rewarded with hosts and honey and the other unrewarded, females showed an increase in preference for the rewarded color with increasing number of training days (1, 3, and 7 days). Hosts and honey together produced a slightly greater preference toward the rewarded color than just hosts, which produced a greater preference than just honey. When trained with a variable reward on one color and a constant reward on the other, females preferred the color associated with the variable reward when it was yellow, but not when it was blue. Thus, relative to no reward, the presence of a variable reward decreased the strength of preference toward the constantly rewarded color. Finally, females trained with regular hosts on one color and used hosts on the other preferred the color associated with the regular hosts when that color was blue but showed no preference in the reverse situation. The presence of used hosts instead of no reward did not increase the strength of preference for the color associated with the regular hosts.     

Chimpanzees (Pan troglodytes) tolerate some degree of inequity while cooperating but refuse to donate effort for nothing

In cooperative hunting, a carcass cannot be divided equally, and hunts may be unsuccessful. We studied how chimpanzees respond to these two variables, working for unequal rewards and no rewards, which have been rarely included in experimental cooperative tasks. We presented chimpanzees with a task requiring three chimpanzees to work together and varied the reward structure in two separate experiments. In Experiment 1, two individuals received more rewards than the third, making the outcome unequal. We wanted to know if cooperation would continue or break down, and what mechanisms might maintain performance. Experiment 2 used equal rewards, but this time one or more locations were left unbaited on a proportion of trials. Thus, there was a chance of individuals working to receive nothing. In Experiment 1, the chimpanzees worked at a high rate, tolerating the unequal outcomes, with rank appearing to determine who got access to the higher-value locations. However, equal outcomes (used as a control) enhanced cooperative performance, most likely through motivational processes rather than the absence of inequity aversion. In Experiment 2, performance dropped off dramatically when the chimpanzees were not rewarded on every trial. Their strategy was irrational as donating effort would have led to more rewards in the long run for each individual. Our results lead to a hierarchy of performances by condition with equity > inequity > donating effort. Chimpanzees therefore tolerate mild inequity, but cannot tolerate receiving nothing when others are rewarded.     

The Role of Serotonin in the Regulation of Patience and Impulsivity

Classic theories suggest that central serotonergic neurons are involved in the behavioral inhibition that is associated with the prediction of negative rewards or punishment. Failed behavioral inhibition can cause impulsive behaviors. However, the behavioral inhibition that results from predicting punishment is not sufficient to explain some forms of impulsive behavior. In this article, we propose that the forebrain serotonergic system is involved in “waiting to avoid punishment” for future punishments and “waiting to obtain reward” for future rewards. Recently, we have found that serotonergic neurons increase their tonic firing rate when rats await food and water rewards and conditioned reinforcer tones. The rate of tonic firing during the delay period was significantly higher when rats were waiting for rewards than for tones, and rats were unable to wait as long for tones as for rewards. These results suggest that increased serotonergic neuronal firing facilitates waiting behavior when there is the prospect of a forthcoming reward and that serotonergic activation contributes to the patience that allows rats to wait longer. We propose a working hypothesis to explain how the serotonergic system regulates patience while waiting for future rewards.     

The value of success: acquiring gains, avoiding losses, and simply being successful

A large network of spatially contiguous, yet anatomically distinct regions in medial frontal cortex is involved in reward processing. Although it is clear these regions play a role in critical aspects of reward-related learning and decision-making, the individual contributions of each component remains unclear. We explored dissociations in reward processing throughout several key regions in the reward system and aimed to clarify the nature of previously observed outcome-related activity in a portion of anterior medial orbitofrontal cortex (mOFC). Specifically, we tested whether activity in anterior mOFC was related to processing successful actions, such that this region would respond similarly to rewards with and without tangible benefits, or whether this region instead encoded only quantifiable outcome values (e.g., money). Participants performed a task where they encountered monetary gains and losses (and non-gains and non-losses) during fMRI scanning. Critically, in addition to the outcomes with monetary consequences, the task included trials that provided outcomes without tangible benefits (participants were simply told that they were correct or incorrect). We found that anterior mOFC responded to all successful outcomes regardless of whether they carried tangible benefits (monetary gains and non-losses) or not (controls). These results support the hypothesis that anterior mOFC processes rewards in terms of a common currency and is capable of providing reward-based signals for everything we value, whether it be primary or secondary rewards or simply a successful experience without objectively quantifiable benefits.     

Neural Sensitivity to Absolute and Relative Anticipated Reward in Adolescents

Adolescence is associated with a dramatic increase in risky and impulsive behaviors that have been attributed to developmental differences in neural processing of rewards. In the present study, we sought to identify age differences in anticipation of absolute and relative rewards. To do so, we modified a commonly used monetary incentive delay (MID) task in order to examine brain activity to relative anticipated reward value (neural sensitivity to the value of a reward as a function of other available rewards). This design also made it possible to examine developmental differences in brain activation to absolute anticipated reward magnitude (the degree to which neural activity increases with increasing reward magnitude). While undergoing fMRI, 18 adolescents and 18 adult participants were presented with cues associated with different reward magnitudes. After the cue, participants responded to a target to win money on that trial. Presentation of cues was blocked such that two reward cues associated with $.20, $1.00, or $5.00 were in play on a given block. Thus, the relative value of the $1.00 reward varied depending on whether it was paired with a smaller or larger reward. Reflecting age differences in neural responses to relative anticipated reward (i.e., reference dependent processing), adults, but not adolescents, demonstrated greater activity to a $1 reward when it was the larger of the two available rewards. Adults also demonstrated a more linear increase in ventral striatal activity as a function of increasing absolute reward magnitude compared to adolescents. Additionally, reduced ventral striatal sensitivity to absolute anticipated reward (i.e., the difference in activity to medium versus small rewards) correlated with higher levels of trait Impulsivity. Thus, ventral striatal activity in anticipation of absolute and relative rewards develops with age. Absolute reward processing is also linked to individual differences in Impulsivity.     

Influence of Supraliminal Reward Information on Unconsciously Triggered Response Inhibition

Although executive functions (e.g., response inhibition) are often thought to interact consciously with reward, recent studies have demonstrated that they can also be triggered by unconscious stimuli. Further research has suggested a close relationship between consciously and unconsciously triggered response inhibition. To date, however, the effect of reward on unconsciously triggered response inhibition has not been explored. To address this issue, participants in this study performed runs of a modified Go/No-Go task during which they were exposed to both high and low value monetary rewards presented both supraliminally and subliminally. Participants were informed that they would earn the reward displayed if they responded correctly to each trial of the run. According to the results, when rewards were presented supraliminally, a greater unconsciously triggered response inhibition was observed for high-value rewards than for low-value rewards. In contrast, when rewards were presented subliminally, no enhanced unconsciously triggered response inhibition was observed. Results revealed that supraliminal and subliminal rewards have distinct effects on unconsciously triggered response inhibition. These findings have important implications for extending our understanding of the relationship between reward and response inhibition.     

Trial history effects in stroop task performance are independent of top-down control

In this study we sought to elucidate what mechanisms underlie the effects of trial history on information processing. We explicitly focused on the contribution of conflict control and S-R binding to sequential trial effects. Performance and brain activity were measured during two hours of continuous Stroop task performance. Mental fatigue, known to influence top-down processing, was used to elucidate separate effects via top-down and bottom-up mechanisms. Here we confirm that performance in the Stroop task is indeed strongly modulated by stimulus history. Performance was affected by the kind of advance information available; dependent on this information adjustments were made, resulting in differential effects of cognitive conflict, and S-R binding on subsequent performance. The influence of mental fatigue on information processing was mainly related to general effects on attention.     

Characterization of radiation-induced performance decrement using a two-lever shock-avoidance task

Rats were trained to perform a task involving responses on two levers. Responding on an avoidance lever delayed the onset of electrical footshock for 20 sec and responding on a warning lever turned on a light for 60 sec. When the light was on, the task on the avoidance lever was changed from unsignaled shock avoidance to signaled shock avoidance by preceding the shocks with 5-sec warning tones. The animals preferred the signaled avoidance condition. After 100 Gy of 60Co irradiation, the animals were less able to avoid shock, an effect from which the animals recovered somewhat over 90 min. The response rate on the avoidance lever remained at or above control rates, while the response rate on the warning lever showed an initial increase, followed by a decrease below baseline. The increase in responding on the avoidance lever occurred in bursts just after presentation of the shocks. The data suggest that under these experimental conditions a subject will not respond appropriately to avoid shock or acquire cues that can facilitate the avoidance of shock. The effects, however, do not reflect an inability to perform the required movements but instead appear to reflect some characteristic of the task associated with a particular lever.