Processing of social and monetary rewards in the human striatum

Despite an increasing focus on the neural basis of human decision making in neuroscience, relatively little attention has been paid to decision making in social settings. Moreover, although human social decision making has been explored in a social psychology context, few neural explanations for the observed findings have been considered. To bridge this gap and improve models of human social decision making, we investigated whether acquiring a good reputation, which is an important incentive in human social behaviors, activates the same reward circuitry as monetary rewards. In total, 19 subjects participated in functional magnetic resonance imaging (fMRI) experiments involving monetary and social rewards. The acquisition of one's good reputation robustly activated reward-related brain areas, notably the striatum, and these overlapped with the areas activated by monetary rewards. Our findings support the idea of a "common neural currency" for rewards and represent an important first step toward a neural explanation for complex human social behaviors.     

Effect of Social Influence on Effort-Allocation for Monetary Rewards

Though decades of research have shown that people are highly influenced by peers, few studies have directly assessed how the value of social conformity is weighed against other types of costs and benefits. Using an effort-based decision-making paradigm with a novel social influence manipulation, we measured how social influence affected individuals’ decisions to allocate effort for monetary rewards during trials with either high or low probability of receiving a reward. We found that information about the effort-allocation of peers modulated participant choices, specifically during conditions of low probability of obtaining a reward. This suggests that peer influence affects effort-based choices to obtain rewards especially under conditions of risk. This study provides evidence that people value social conformity in addition to other costs and benefits when allocating effort, and suggests that neuroeconomic studies that assess trade-offs between effort and reward should consider social environment as a factor that can influence decision-making.     

Neural Sensitivity to Absolute and Relative Anticipated Reward in Adolescents

Adolescence is associated with a dramatic increase in risky and impulsive behaviors that have been attributed to developmental differences in neural processing of rewards. In the present study, we sought to identify age differences in anticipation of absolute and relative rewards. To do so, we modified a commonly used monetary incentive delay (MID) task in order to examine brain activity to relative anticipated reward value (neural sensitivity to the value of a reward as a function of other available rewards). This design also made it possible to examine developmental differences in brain activation to absolute anticipated reward magnitude (the degree to which neural activity increases with increasing reward magnitude). While undergoing fMRI, 18 adolescents and 18 adult participants were presented with cues associated with different reward magnitudes. After the cue, participants responded to a target to win money on that trial. Presentation of cues was blocked such that two reward cues associated with $.20, $1.00, or $5.00 were in play on a given block. Thus, the relative value of the $1.00 reward varied depending on whether it was paired with a smaller or larger reward. Reflecting age differences in neural responses to relative anticipated reward (i.e., reference dependent processing), adults, but not adolescents, demonstrated greater activity to a $1 reward when it was the larger of the two available rewards. Adults also demonstrated a more linear increase in ventral striatal activity as a function of increasing absolute reward magnitude compared to adolescents. Additionally, reduced ventral striatal sensitivity to absolute anticipated reward (i.e., the difference in activity to medium versus small rewards) correlated with higher levels of trait Impulsivity. Thus, ventral striatal activity in anticipation of absolute and relative rewards develops with age. Absolute reward processing is also linked to individual differences in Impulsivity.     

Promising high monetary rewards for future task performance increases intermediate task performance

In everyday life contexts and work settings, monetary rewards are often contingent on future performance. Based on research showing that the anticipation of rewards causes improved task performance through enhanced task preparation, the present study tested the hypothesis that the promise of monetary rewards for future performance would not only increase future performance, but also performance on an unrewarded intermediate task. Participants performed an auditory Simon task in which they responded to two consecutive tones. While participants could earn high vs. low monetary rewards for fast responses to every second tone, their responses to the first tone were not rewarded. Moreover, we compared performance under conditions in which reward information could prompt strategic performance adjustments (i.e., when reward information was presented for a relatively long duration) to conditions preventing strategic performance adjustments (i.e., when reward information was presented very briefly). Results showed that high (vs. low) rewards sped up both rewarded and intermediate, unrewarded responses, and the effect was independent of the duration of reward presentation. Moreover, long presentation led to a speed-accuracy trade-off for both rewarded and unrewarded tones, whereas short presentation sped up responses to rewarded and unrewarded tones without this trade-off. These results suggest that high rewards for future performance boost intermediate performance due to enhanced task preparation, and they do so regardless whether people respond to rewards in a strategic or non-strategic manner.     

Ventromedial prefrontal cortex activation is associated with memory formation for predictable rewards

During reinforcement learning, dopamine release shifts from the moment of reward consumption to the time point when the reward can be predicted. Previous studies provide consistent evidence that reward-predicting cues enhance long-term memory (LTM) formation of these items via dopaminergic projections to the ventral striatum. However, it is less clear whether memory for items that do not precede a reward but are directly associated with reward consumption is also facilitated. Here, we investigated this question in an fMRI paradigm in which LTM for reward-predicting and neutral cues was compared to LTM for items presented during consumption of reliably predictable as compared to less predictable rewards. We observed activation of the ventral striatum and enhanced memory formation during reward anticipation. During processing of less predictable as compared to reliably predictable rewards, the ventral striatum was activated as well, but items associated with less predictable outcomes were remembered worse than items associated with reliably predictable outcomes. Processing of reliably predictable rewards activated the ventromedial prefrontal cortex (vmPFC), and vmPFC BOLD responses were associated with successful memory formation of these items. Taken together, these findings show that consumption of reliably predictable rewards facilitates LTM formation and is associated with activation of the vmPFC.     

Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation

We monitored single-neuron activity in the orbitofrontal cortex of rats performing a time-discounting task in which the spatial location of the reward predicted whether the delay preceding reward delivery would be short or long. We found that rewards delivered after a short delay elicited a stronger neuronal response than those delivered after a long delay in most neurons. Activity in these neurons was not influenced by reward size when delays were held constant. This was also true for a minority of neurons that exhibited sustained increases in firing in anticipation of delayed reward. Thus, encoding of time-discounted rewards in orbitofrontal cortex is independent of the encoding of absolute reward value. These results are contrary to the proposal that orbitofrontal neurons signal the value of delayed rewards in a common currency and instead suggest alternative proposals for the role this region plays in guiding responses for delayed versus immediate rewards.     

Social Influences on Inequity Aversion in Children

Adults and children are willing to sacrifice personal gain to avoid both disadvantageous and advantageous inequity. These two forms of inequity aversion follow different developmental trajectories, with disadvantageous inequity aversion emerging around 4 years and advantageous inequity aversion emerging around 8 years. Although inequity aversion is assumed to be specific to situations where resources are distributed among individuals, the role of social context has not been tested in children. Here, we investigated the influence of two aspects of social context on inequity aversion in 4- to 9-year-old children: (1) the role of the experimenter distributing rewards and (2) the presence of a peer with whom rewards could be shared. Experiment 1 showed that children rejected inequity at the same rate, regardless of whether the experimenter had control over reward allocations. This indicates that children’s decisions are based upon reward allocations between themselves and a peer and are not attempts to elicit more favorable distributions from the experimenter. Experiment 2 compared rejections of unequal reward allocations in children interacting with or without a peer partner. When faced with a disadvantageous distribution, children frequently rejected a smaller reward when a larger reward was visible, even if no partner would obtain the larger reward. This suggests that nonsocial factors partly explain disadvantageous inequity rejections. However, rejections of disadvantageous distributions were higher when the larger amount would go to a peer, indicating that social context enhances disadvantageous inequity aversion. By contrast, children rejected advantageous distributions almost exclusively in the social context. Therefore, advantageous inequity aversion appears to be genuinely social, highlighting its potential relevance for the development of fairness concerns. By comparing social and nonsocial factors, this study provides a detailed picture of the expression of inequity aversion in human ontogeny and raises questions about the function and evolution of inequity aversion in humans.     

Intense passionate love attenuates cigarette cue-reactivity in nicotine-deprived smokers: an FMRI study

Self-expanding experiences like falling in love or engaging in novel, exciting and interesting activities activate the same brain reward mechanism (mesolimbic dopamine pathway) that reinforces drug use and abuse, including tobacco smoking. This suggests the possibility that reward from smoking is substitutable by self-expansion (through competition with the same neural system), potentially aiding cessation efforts. Using a model of self-expansion in the context of romantic love, the present fMRI experiment examined whether, among nicotine-deprived smokers, relationship self-expansion is associated with deactivation of cigarette cue-reactivity regions. Results indicated that among participants who were experiencing moderate levels of craving, cigarette cue-reactivity regions (e.g., cuneus and posterior cingulate cortex) showed significantly less activation during self-expansion conditions compared with control conditions. These results provide evidence that rewards from one domain (self-expansion) can act as a substitute for reward from another domain (nicotine) to attenuate cigarette cue reactivity.     

草乌花蜜产量的梯度分布及熊蜂自下而上的访花行为

收益降低假说(declining reward hypothesis)认为熊蜂自下而上的访花顺序是对花蜜产量的直接响应,先访问下部花蜜产量高的花可以获得更多的收益;花开口方向假说认为自下而上访花是因为熊蜂更容易看见其上部的花朵。为了验证上述两个假说,我们于2008年8月在北京小龙门国家森林公园调查了红光熊蜂(Bombus ignitus)访问草乌(Aconitum kusnezoffii)直立和倒立顶生花序的访问顺序,测量了直立花序下部雌性阶段花和上部雄性阶段花花蜜的糖浓度、体积,计算了花蜜中的糖含量。结果表明,红光熊蜂在直立花序和倒立花序内均以向上运动为主,分别占总运动次数的62.77%和68.35%;直立花序下部雌性阶段花花蜜糖浓度比上部雄性阶段花低1.44%,但是花蜜体积和花蜜中的糖含量都显著高于雄性阶段的花。由于熊蜂访问倒立花序时先访问的是下部的低回报雄性阶段的花,然后再访问上部高回报的雌性阶段的花,这与收益降低假说矛盾,表明红光熊蜂自下而上访问草乌直立花序可能不是受到花蜜产量的调节。     

Sub-Optimal Allocation of Time in Sequential Movements

The allocation of limited resources such as time or energy is a core problem that organisms face when planning complex actions. Most previous research concerning planning of movement has focused on the planning of single, isolated movements. Here we investigated the allocation of time in a pointing task where human subjects attempted to touch two targets in a specified order to earn monetary rewards. Subjects were required to complete both movements within a limited time but could freely allocate the available time between the movements. The time constraint presents an allocation problem to the subjects: the more time spent on one movement, the less time is available for the other. In different conditions we assigned different rewards to the two tokens. How the subject allocated time between movements affected their expected gain on each trial. We also varied the angle between the first and second movements and the length of the second movement. Based on our results, we developed and tested a model of speed-accuracy tradeoff for sequential movements. Using this model we could predict the time allocation that would maximize the expected gain of each subject in each experimental condition. We compared human performance with predicted optimal performance. We found that all subjects allocated time sub-optimally, spending more time than they should on the first movement even when the reward of the second target was five times larger than the first. We conclude that the movement planning system fails to maximize expected reward in planning sequences of as few as two movements and discuss possible interpretations drawn from economic theory.     

Effects of multiple delayed rewards on delay discounting in an adjusting amount procedure

It has been suggested that when the delivery of several rewards is separated in time, e.g. one reward immediately and a second reward a few moments later, the value of an alternative that includes these "bundled" rewards will be the sum of the hyperbolic discount functions of the individual rewards. The current study examined this hypothesis using an adjusting amount procedure. In this procedure, rats chose between a delayed food alternative and an immediate food alternative, where the amount of immediate food altered according to each rat's choices. The size of the immediate reward when rats were indifferent between the delayed and immediate alternatives indexed the value of the delayed alternative. Discount functions describing the relationship between the indifference points and the delay to food were created for conditions in which the delay alternative consisted of a single reward (150mul of sucrose solution) delayed by 0, 2, 4, 8, or 16s following the reinforced response. These functions were used to predict the indifference points in other conditions for which an additional 150mul of sucrose solution was delivered at 0, 4, 8, or 16s following the reinforced response. The model fit the data well. However, there were systematic deviations that suggested animals were sensitive to the context within which delays were presented, in addition to the delays themselves. That is, preference for the delayed alternative was lower than predicted when the delay to the additional reward was long (8 or 16s) and higher than the predicted values when it was short (0 or 4s).     

网络游戏成瘾的奖赏加工缺陷及其神经机制

奖赏加工异常是网络游戏成瘾（internet gaming disorder，IGD）核心特征之一，近期研究结合认知神经科学技术对IGD的神经机制进行探讨发现，IGD与药物成瘾存在相似的神经基础，但仍然存在较多争议。本文从奖赏类型、奖赏加工阶段梳理了IGD人群奖赏加工的研究进展。IGD人群在游戏相关线索下，奖赏预期（reward anticipation）阶段的奖赏系统激活可能与注意偏向、情绪体验和渴求感增加有关。同时，IGD人群对自然奖赏表现出较为一致的低敏感性，该特征主要出现在结果评估（outcome evaluation）阶段。未来研究可以排除共病因素、结合生动的游戏奖赏刺激，进一步探究奖赏预期和结果评估异常如何推动IGD的发展。     

The neural basis of following advice

Learning by following explicit advice is fundamental for human cultural evolution, yet the neurobiology of adaptive social learning is largely unknown. Here, we used simulations to analyze the adaptive value of social learning mechanisms, computational modeling of behavioral data to describe cognitive mechanisms involved in social learning, and model-based functional magnetic resonance imaging (fMRI) to identify the neurobiological basis of following advice. One-time advice received before learning had a sustained influence on people's learning processes. This was best explained by social learning mechanisms implementing a more positive evaluation of the outcomes from recommended options. Computer simulations showed that this "outcome-bonus" accumulates more rewards than an alternative mechanism implementing higher initial reward expectation for recommended options. fMRI results revealed a neural outcome-bonus signal in the septal area and the left caudate. This neural signal coded rewards in the absence of advice, and crucially, it signaled greater positive rewards for positive and negative feedback after recommended rather than after non-recommended choices. Hence, our results indicate that following advice is intrinsically rewarding. A positive correlation between the model's outcome-bonus parameter and amygdala activity after positive feedback directly relates the computational model to brain activity. These results advance the understanding of social learning by providing a neurobiological account for adaptive learning from advice.     

Differential Reward Learning for Self and Others Predicts Self-Reported Altruism

In social environments, decisions not only determine rewards for oneself but also for others. However, individual differences in pro-social behaviors have been typically studied through self-report. We developed a decision-making paradigm in which participants chose from card decks with differing rewards for themselves and charity; some decks gave similar rewards to both, while others gave higher rewards for one or the other. We used a reinforcement-learning model that estimated each participant''s relative weighting of self versus charity reward. As shown both in choices and model parameters, individuals who showed relatively better learning of rewards for charity – compared to themselves – were more likely to engage in pro-social behavior outside of a laboratory setting indicated by self-report. Overall rates of reward learning, however, did not predict individual differences in pro-social tendencies. These results support the idea that biases toward learning about social rewards are associated with one''s altruistic tendencies.     

Influence of Supraliminal Reward Information on Unconsciously Triggered Response Inhibition

Although executive functions (e.g., response inhibition) are often thought to interact consciously with reward, recent studies have demonstrated that they can also be triggered by unconscious stimuli. Further research has suggested a close relationship between consciously and unconsciously triggered response inhibition. To date, however, the effect of reward on unconsciously triggered response inhibition has not been explored. To address this issue, participants in this study performed runs of a modified Go/No-Go task during which they were exposed to both high and low value monetary rewards presented both supraliminally and subliminally. Participants were informed that they would earn the reward displayed if they responded correctly to each trial of the run. According to the results, when rewards were presented supraliminally, a greater unconsciously triggered response inhibition was observed for high-value rewards than for low-value rewards. In contrast, when rewards were presented subliminally, no enhanced unconsciously triggered response inhibition was observed. Results revealed that supraliminal and subliminal rewards have distinct effects on unconsciously triggered response inhibition. These findings have important implications for extending our understanding of the relationship between reward and response inhibition.     

Responses to a simple barter task in chimpanzees, Pan troglodytes

Chimpanzees (Pan troglodytes) frequently participate in social exchange involving multiple goods and services of variable value, yet they have not been tested in a formalized situation to see whether they can barter using multiple tokens and rewards. We set up a simple barter economy with two tokens and two associated rewards and tested chimpanzees on their ability to obtain rewards by returning the matching token in situations in which their access to tokens was unlimited or limited. Chimpanzees easily learned to associate value with the tokens, as expected, and did barter, but followed a simple strategy of favoring the higher-value token, regardless of the reward proffered, instead of a more complex but more effective strategy of returning the token that matched the reward. This response is similar to that shown by capuchin monkeys in our previous study. We speculate that this response, while not ideal, may be sufficient to allow for stability of the social exchange system in these primates, and that the importance of social barter to both species may have led to this convergence of strategies.     

The ecology and evolution of patience in two New World monkeys

Decision making often involves choosing between small, short-term rewards and large, long-term rewards. All animals, humans included, discount future rewards--the present value of delayed rewards is viewed as less than the value of immediate rewards. Despite its ubiquity, there exists considerable but unexplained variation between species in their capacity to wait for rewards--that is, to exert patience or self-control. Using two closely related primates--common marmosets (Callithrix jacchus) and cotton-top tamarins (Saguinus oedipus)--we uncover a variable that may explain differences in how species discount future rewards. Both species faced a self-control paradigm in which individuals chose between taking an immediate small reward and waiting a variable amount of time for a large reward. Under these conditions, marmosets waited significantly longer for food than tamarins. This difference cannot be explained by life history, social behaviour or brain size. It can, however, be explained by feeding ecology: marmosets rely on gum, a food product acquired by waiting for exudate to flow from trees, whereas tamarins feed on insects, a food product requiring impulsive action. Foraging ecology, therefore, may provide a selective pressure for the evolution of self-control.     

The present values of delayed rewards are approximately additive

In two experiments, human subjects were asked to estimate their present values of single delayed rewards and their present values of temporal sequences of three rewards. Present values were solicited by asking subjects to indicate an amount of money v for which they would be indifferent between receiving v at the end of the session and receiving the delayed reward(s). A procedure was used for which responding the true value of v was the optimal strategy, and the actual payoff that each subject received was determined by one randomly selected trial. In Experiment 1 (n=29) each delayed reward was 9.90 dollars in cash. In Experiment 2 (n=19) the delayed rewards were dated 15 dollars gift certificates to a local restaurant. In both experiments, the present values of the sequences were approximately equal to the sums of the present values of their component rewards. The presence of outliers suggests that a few subjects may have valued sequences less than the sums of their single rewards. Effects of a preference for uniform sequences, if any, were too small to be detected. Discounting of sequences was well fit by a parallel hyperbolic discounting equation, consistent with Mazur's [Mazur, J.E., 1986. Choice between single and multiple delayed reinforcers. J. Exp. Anal. Behav. 46 (1), 67-77] results using multiple reinforcers.     

Patterns of Brain Activation when Mothers View Their Own Child and Dog: An fMRI Study

Neural substrates underlying the human-pet relationship are largely unknown. We examined fMRI brain activation patterns as mothers viewed images of their own child and dog and an unfamiliar child and dog. There was a common network of brain regions involved in emotion, reward, affiliation, visual processing and social cognition when mothers viewed images of both their child and dog. Viewing images of their child resulted in brain activity in the midbrain (ventral tegmental area/substantia nigra involved in reward/affiliation), while a more posterior cortical brain activation pattern involving fusiform gyrus (visual processing of faces and social cognition) characterized a mother''s response to her dog. Mothers also rated images of their child and dog as eliciting similar levels of excitement (arousal) and pleasantness (valence), although the difference in the own vs. unfamiliar child comparison was larger than the own vs. unfamiliar dog comparison for arousal. Valence ratings of their dog were also positively correlated with ratings of the attachment to their dog. Although there are similarities in the perceived emotional experience and brain function associated with the mother-child and mother-dog bond, there are also key differences that may reflect variance in the evolutionary course and function of these relationships.     

Neural responses during anticipation of a primary taste reward

The aim of this study was to determine the brain regions involved in anticipation of a primary taste reward and to compare these regions to those responding to the receipt of a taste reward. Using fMRI, we scanned human subjects who were presented with visual cues that signaled subsequent reinforcement with a pleasant sweet taste (1 M glucose), a moderately unpleasant salt taste (0.2 M saline), or a neutral taste. Expectation of a pleasant taste produced activation in dopaminergic midbrain, posterior dorsal amygdala, striatum, and orbitofrontal cortex (OFC). Apart from OFC, these regions were not activated by reward receipt. The findings indicate that when rewards are predictable, brain regions recruited during expectation are, in part, dissociable from areas responding to reward receipt.