Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value

Risky decision-making is altered in humans and animals with damage to the orbitofrontal cortex. However, the cellular function of the intact orbitofrontal cortex in processing information relevant for risky decisions is unknown. We recorded responses of single orbitofrontal neurons while monkeys viewed visual cues representing the key decision parameters, reward risk and value. Risk was defined as the mathematical variance of binary symmetric probability distributions of reward magnitudes; value was defined as non-risky reward magnitude. Monkeys displayed graded behavioral preferences for risky outcomes, as they did for value. A population of orbitofrontal neurons showed a distinctive risk signal: their cues and reward responses covaried monotonically with the variance of the different reward distributions without monotonically coding reward value. Furthermore, a small but statistically significant fraction of risk responses also coded reward value. These risk signals may provide physiological correlates for the role of the orbitofrontal cortex in risk processing.     

Neural differentiation of expected reward and risk in human subcortical structures

In decision-making under uncertainty, economic studies emphasize the importance of risk in addition to expected reward. Studies in neuroscience focus on expected reward and learning rather than risk. We combined functional imaging with a simple gambling task to vary expected reward and risk simultaneously and in an uncorrelated manner. Drawing on financial decision theory, we modeled expected reward as mathematical expectation of reward, and risk as reward variance. Activations in dopaminoceptive structures correlated with both mathematical parameters. These activations differentiated spatially and temporally. Temporally, the activation related to expected reward was immediate, while the activation related to risk was delayed. Analyses confirmed that our paradigm minimized confounds from learning, motivation, and salience. These results suggest that the primary task of the dopaminergic system is to convey signals of upcoming stochastic rewards, such as expected reward and risk, beyond its role in learning, motivation, and salience.     

The rodent orbitofrontal cortex gets time and direction

The orbitofrontal cortex (OFC) helps direct decision making through its flexible coding of reward and economic value. In this issue of Neuron, papers by Roesch et al. and Feierstein et al. demonstrate the importance of temporal and spatial features to processing in the rodent OFC.     

Neural responses during anticipation of a primary taste reward

The aim of this study was to determine the brain regions involved in anticipation of a primary taste reward and to compare these regions to those responding to the receipt of a taste reward. Using fMRI, we scanned human subjects who were presented with visual cues that signaled subsequent reinforcement with a pleasant sweet taste (1 M glucose), a moderately unpleasant salt taste (0.2 M saline), or a neutral taste. Expectation of a pleasant taste produced activation in dopaminergic midbrain, posterior dorsal amygdala, striatum, and orbitofrontal cortex (OFC). Apart from OFC, these regions were not activated by reward receipt. The findings indicate that when rewards are predictable, brain regions recruited during expectation are, in part, dissociable from areas responding to reward receipt.     

Neural signatures of economic preferences for risk and ambiguity

People often prefer the known over the unknown, sometimes sacrificing potential rewards for the sake of surety. Overcoming impulsive preferences for certainty in order to exploit uncertain but potentially lucrative options may require specialized neural mechanisms. Here, we demonstrate by functional magnetic resonance imaging (fMRI) that individuals' preferences for risk (uncertainty with known probabilities) and ambiguity (uncertainty with unknown probabilities) predict brain activation associated with decision making. Activation within the lateral prefrontal cortex was predicted by ambiguity preference and was also negatively correlated with an independent clinical measure of behavioral impulsiveness, suggesting that this region implements contextual analysis and inhibits impulsive responses. In contrast, activation of the posterior parietal cortex was predicted by risk preference. Together, this novel double dissociation indicates that decision making under ambiguity does not represent a special, more complex case of risky decision making; instead, these two forms of uncertainty are supported by distinct mechanisms.     

Getting formal with dopamine and reward

Recent neurophysiological studies reveal that neurons in certain brain structures carry specific signals about past and future rewards. Dopamine neurons display a short-latency, phasic reward signal indicating the difference between actual and predicted rewards. The signal is useful for enhancing neuronal processing and learning behavioral reactions. It is distinctly different from dopamine's tonic enabling of numerous behavioral processes. Neurons in the striatum, frontal cortex, and amygdala also process reward information but provide more differentiated information for identifying and anticipating rewards and organizing goal-directed behavior. The different reward signals have complementary functions, and the optimal use of rewards in voluntary behavior would benefit from interactions between the signals. Addictive psychostimulant drugs may exert their action by amplifying the dopamine reward signal.     

A functional difference in information processing between orbitofrontal cortex and ventral striatum during decision-making behaviour

Both orbitofrontal cortex (OFC) and ventral striatum (vStr) have been identified as key structures that represent information about value in decision-making tasks. However, the dynamics of how this information is processed are not yet understood. We recorded ensembles of cells from OFC and vStr in rats engaged in the spatial adjusting delay-discounting task, a decision-making task that involves a trade-off between delay to and magnitude of reward. Ventral striatal neural activity signalled information about reward before the rat''s decision, whereas such reward-related signals were absent in OFC until after the animal had committed to its decision. These data support models in which vStr is directly involved in action selection, but OFC processes decision-related information afterwards that can be used to compare the predicted and actual consequences of behaviour.     

Human processing of behaviorally relevant and irrelevant absence of expected rewards: a high-resolution ERP study

Acute lesions of the posterior medial orbitofrontal cortex (OFC) in humans may induce a state of reality confusion marked by confabulation, disorientation, and currently inappropriate actions. This clinical state is strongly associated with an inability to abandon previously valid anticipations, that is, extinction capacity. In healthy subjects, the filtering of memories according to their relation with ongoing reality is associated with activity in posterior medial OFC (area 13) and electrophysiologically expressed at 220-300 ms. These observations indicate that the human OFC also functions as a generic reality monitoring system. For this function, it is presumably more important for the OFC to evaluate the current behavioral appropriateness of anticipations rather than their hedonic value. In the present study, we put this hypothesis to the test. Participants performed a reversal learning task with intermittent absence of reward delivery. High-density evoked potential analysis showed that the omission of expected reward induced a specific electrocortical response in trials signaling the necessity to abandon the hitherto reward predicting choice, but not when omission of reward had no such connotation. This processing difference occurred at 200-300 ms. Source estimation using inverse solution analysis indicated that it emanated from the posterior medial OFC. We suggest that the human brain uses this signal from the OFC to keep thought and behavior in phase with reality.     

Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology

Neurons in a small number of brain structures detect rewards and reward-predicting stimuli and are active during the expectation of predictable food and liquid rewards. These neurons code the reward information according to basic terms of various behavioural theories that seek to explain reward-directed learning, approach behaviour and decision-making. The involved brain structures include groups of dopamine neurons, the striatum including the nucleus accumbens, the orbitofrontal cortex and the amygdala. The reward information is fed to brain structures involved in decision-making and organisation of behaviour, such as the dorsolateral prefrontal cortex and possibly the parietal cortex. The neural coding of basic reward terms derived from formal theories puts the neurophysiological investigation of reward mechanisms on firm conceptual grounds and provides neural correlates for the function of rewards in learning, approach behaviour and decision-making.     

Potential vulnerabilities of neuronal reward, risk, and decision mechanisms to addictive drugs

How do addictive drugs hijack the brain's reward system? This review speculates how normal, physiological reward processes may be affected by addictive drugs. Addictive drugs affect acute responses and plasticity in dopamine neurons and postsynaptic structures. These effects reduce reward discrimination, increase the effects of reward prediction error signals, and enhance neuronal responses to reward-predicting stimuli, which may contribute to compulsion. Addictive drugs steepen neuronal temporal reward discounting and create temporal myopia that impairs the control of drug taking. Tonically enhanced dopamine levels may disturb working memory mechanisms necessary for assessing background rewards and thus may generate inaccurate neuronal reward predictions. Drug-induced working memory deficits may impair neuronal risk signaling, promote risky behaviors, and facilitate preaddictive drug use. Malfunctioning adaptive reward coding may lead to overvaluation of drug rewards. Many of these malfunctions may result in inadequate neuronal decision mechanisms and lead to choices biased toward drug rewards.     

Multiple reward signals in the brain

The fundamental biological importance of rewards has created an increasing interest in the neuronal processing of reward information. The suggestion that the mechanisms underlying drug addiction might involve natural reward systems has also stimulated interest. This article focuses on recent neurophysiological studies in primates that have revealed that neurons in a limited number of brain structures carry specific signals about past and future rewards. This research provides the first step towards an understanding of how rewards influence behaviour before they are received and how the brain might use reward information to control learning and goal-directed behaviour.     

Relief as a reward: hedonic and neural responses to safety from pain

Relief fits the definition of a reward. Unlike other reward types the pleasantness of relief depends on the violation of a negative expectation, yet this has not been investigated using neuroimaging approaches. We hypothesized that the degree of negative expectation depends on state (dread) and trait (pessimism) sensitivity. Of the brain regions that are involved in mediating pleasure, the nucleus accumbens also signals unexpected reward and positive prediction error. We hypothesized that accumbens activity reflects the level of negative expectation and subsequent pleasant relief. Using fMRI and two purpose-made tasks, we compared hedonic and BOLD responses to relief with responses during an appetitive reward task in 18 healthy volunteers. We expected some similarities in task responses, reflecting common neural substrates implicated across reward types. However, we also hypothesized that relief responses would differ from appetitive rewards in the nucleus accumbens, since only relief pleasantness depends on negative expectations. The results confirmed these hypotheses. Relief and appetitive reward task activity converged in the ventromedial prefrontal cortex, which also correlated with appetitive reward pleasantness ratings. In contrast, dread and pessimism scores correlated with relief but not with appetitive reward hedonics. Moreover, only relief pleasantness covaried with accumbens activation. Importantly, the accumbens signal appeared to specifically reflect individual differences in anticipation of the adverse event (dread, pessimism) but was uncorrelated to appetitive reward hedonics. In conclusion, relief differs from appetitive rewards due to its reliance on negative expectations, the violation of which is reflected in relief-related accumbens activation.     

Risky Decisions and Their Consequences: Neural Processing by Boys with Antisocial Substance Disorder

Background

Adolescents with conduct and substance problems (“Antisocial Substance Disorder” (ASD)) repeatedly engage in risky antisocial and drug-using behaviors. We hypothesized that, during processing of risky decisions and resulting rewards and punishments, brain activation would differ between abstinent ASD boys and comparison boys.

Methodology/Principal Findings

We compared 20 abstinent adolescent male patients in treatment for ASD with 20 community controls, examining rapid event-related blood-oxygen-level-dependent (BOLD) responses during functional magnetic resonance imaging. In 90 decision trials participants chose to make either a cautious response that earned one cent, or a risky response that would either gain 5 cents or lose 10 cents; odds of losing increased as the game progressed. We also examined those times when subjects experienced wins, or separately losses, from their risky choices. We contrasted decision trials against very similar comparison trials requiring no decisions, using whole-brain BOLD-response analyses of group differences, corrected for multiple comparisons. During decision-making ASD boys showed hypoactivation in numerous brain regions robustly activated by controls, including orbitofrontal and dorsolateral prefrontal cortices, anterior cingulate, basal ganglia, insula, amygdala, hippocampus, and cerebellum. While experiencing wins, ASD boys had significantly less activity than controls in anterior cingulate, temporal regions, and cerebellum, with more activity nowhere. During losses ASD boys had significantly more activity than controls in orbitofrontal cortex, dorsolateral prefrontal cortex, brain stem, and cerebellum, with less activity nowhere.

Conclusions/Significance

Adolescent boys with ASD had extensive neural hypoactivity during risky decision-making, coupled with decreased activity during reward and increased activity during loss. These neural patterns may underlie the dangerous, excessive, sustained risk-taking of such boys. The findings suggest that the dysphoria, reward insensitivity, and suppressed neural activity observed among older addicted persons also characterize youths early in the development of substance use disorders.     

Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation

We monitored single-neuron activity in the orbitofrontal cortex of rats performing a time-discounting task in which the spatial location of the reward predicted whether the delay preceding reward delivery would be short or long. We found that rewards delivered after a short delay elicited a stronger neuronal response than those delivered after a long delay in most neurons. Activity in these neurons was not influenced by reward size when delays were held constant. This was also true for a minority of neurons that exhibited sustained increases in firing in anticipation of delayed reward. Thus, encoding of time-discounted rewards in orbitofrontal cortex is independent of the encoding of absolute reward value. These results are contrary to the proposal that orbitofrontal neurons signal the value of delayed rewards in a common currency and instead suggest alternative proposals for the role this region plays in guiding responses for delayed versus immediate rewards.     

What Do I Want and When Do I Want It: Brain Correlates of Decisions Made for Self and Other

A number of recent functional Magnetic Resonance Imaging (fMRI) studies on intertemporal choice behavior have demonstrated that so-called emotion- and reward-related brain areas are preferentially activated by decisions involving immediately available (but smaller) rewards as compared to (larger) delayed rewards. This pattern of activation was not seen, however, when intertemporal choices were made for another (unknown) individual, which speaks to that activation having been triggered by self-relatedness. In the present fMRI study, we investigated the brain correlates of individuals who passively observed intertemporal choices being made either for themselves or for an unknown person. We found higher activation within the ventral striatum, medial prefrontal and orbitofrontal cortex, pregenual anterior cingulate cortex, and posterior cingulate cortex when an immediate reward was possible for the observer herself, which is in line with findings from studies in which individuals actively chose immediately available rewards. Additionally, activation in the dorsal anterior cingulate cortex, posterior cingulate cortex, and precuneus was higher for choices that included immediate options than for choices that offered only delayed options, irrespective of who was to be the beneficiary. These results indicate that (1) the activations found in active intertemporal decision making are also present when the same decisions are merely observed, thus supporting the assumption that a robust brain network is engaged in immediate gratification; and (2) with immediate rewards, certain brain areas are activated irrespective of whether the observer or another person is the beneficiary of a decision, suggesting that immediacy plays a more general role for neural activation. An explorative analysis of participants’ brain activation corresponding to chosen rewards, further indicates that activation in the aforementioned brain areas depends on the mere presence, availability, or actual reception of immediate rewards.     

Toxoplasma gondii infection and testosterone congruently increase tolerance of male rats for risk of reward forfeiture

Decision making under risk involves balancing the potential of gaining rewards with the possibility of loss and/or punishment. Tolerance to risk varies between individuals. Understanding the biological basis of risk tolerance is pertinent because excessive tolerance contributes to adverse health and safety outcomes. Yet, not much is known about biological factors mediating inter-individual variability in this regard. We investigate if latent Toxoplasma gondii infection can cause risk tolerance. Using a rodent model of the balloon analogous risk task, we show that latent T. gondii infection leads to a greater tolerance of reward forfeiture. Furthermore, effects of the infection on risk can be recapitulated with testosterone supplementation alone, demonstrating that greater testosterone synthesis by the host post-infection is sufficient to change risk tolerance. T. gondii is a frequent parasite of humans and animals. Thus, the infection status can potentially explain some of the inter-individual variability in the risky decision making.     

Directional Influence between the Human Amygdala and Orbitofrontal Cortex at the Time of Decision-Making

There is a growing consensus that the brain makes simple choices, such as choosing between an apple and an orange, by assigning value to the options under consideration, and comparing those values to make a choice. There is also a consensus that value signals computed in orbitofrontal cortex (OFC) and amygdala play a critical role in the choice process. However, the nature of the flow of information between OFC and amygdala at the time of decision is still unknown. In order to study this question, simultaneous local field potentials were recorded from OFC and amygdala in human patients while they performed a simple food choice task. Although the interaction of these circuits has been studied in animals, this study examines the effective connectivity directly in the human brain on a moment-by-moment basis. A spectral conditional Granger causality analysis was performed in order to test if the modulation of activity goes mainly from OFC-to-amygdala, from amygdala-to-OFC, or if it is bi-directional. Influence from amygdala-to-OFC was dominant prior to the revealed choice, with a small but significant OFC influence on the amygdala earlier in the trial. Alpha oscillation amplitudes analyzed with the Hilbert-Huang transform revealed differences in choice valence coincident with temporally specific amygdala influence on the OFC.     

食物奖赏和渴求行为相关的大鼠左侧眶额叶皮质Delta频段脑电活动(英文）

眶额叶皮质与中脑边缘多巴胺奖赏系统有着复杂的相互纤维联系。先前的研究探讨了药物成瘾过程中眶额叶皮质的脑电活动。在本实验中,将探讨食物奖赏和渴求过程中该皮质的脑电活动。实验采用了两个环境:对照环境和食物刺激相关的环境。首先,训练大鼠在食物刺激相关的环境中吃巧克力花生豆,而后在该环境中设置两种不同的刺激方式:能看到和闻到但不能吃到(渴求实验),或者仍旧可以吃到巧克力花生豆(奖赏实验);同时进行左侧眶额叶皮质的脑电记录。结果发现,在食物刺激相关的环境中大鼠 Delta 频段(2-4 Hz)的脑电活动与食物刺激显著相关,此外,与在对照环境中相比,其相对功率在食物渴求时下降而在食物奖赏时升高。本实验表明,食物相关的奖励可以改变大鼠眶额叶皮质的脑电活动,而且,Delta 频段的脑电活动能够作为监测该奖励的一个指标。     

Evolution of honest reward signal in flowers

Some flowering plants signal the abundance of their rewards by changing their flower colour, scent or other floral traits as rewards are depleted. These floral trait changes can be regarded as honest signals of reward states for pollinators. Previous studies have hypothesized that these signals are used to maintain plant-level attractiveness to pollinators, but the evolutionary conditions leading to the development of honest signals have not been well investigated from a theoretical basis. We examined conditions leading to the evolution of honest reward signals in flowers by applying a theoretical model that included pollinator response and signal accuracy. We assumed that pollinators learn floral traits and plant locations in association with reward states and use this information to decide which flowers to visit. While manipulating the level of associative learning, we investigated optimal flower longevity, the proportion of reward and rewardless flowers, and honest- and dishonest-signalling strategies. We found that honest signals are evolutionarily stable only when flowers are visited by pollinators with both high and low learning abilities. These findings imply that behavioural variation in learning within a pollinator community can lead to the evolution of an honest signal even when there is no contribution of rewardless flowers to pollinator attractiveness.     

Reward and Uncertainty Favor Risky Decision-Making in Pilots: Evidence from Cardiovascular and Oculometric Measurements

In this paper we examined plan continuation error (PCE), a well known error made by pilots consisting in continuing the flight plan despite adverse meteorological conditions. Our hypothesis is that a large range of strong negative emotional consequences, including those induced by economic pressure, are associated with the decision to revise the flight plan and favor PCE. We investigated the economic hypothesis with a simplified landing task (reproduction of a real aircraft instrument) in which uncertainty and reward were manipulated. Heart rate (HR), heart rate variability (HRV) and eye tracking measurements were performed to get objective clues both on the cognitive and emotional state of the volunteers. Results showed that volunteers made more risky decisions under the influence of the financial incentive, in particular when uncertainty was high. Psychophysiological examination showed that HR increased and total HRV decreased in response to the cognitive load generated by the task. In addition, HR also increased in response to the financially motivated condition. Eventually, fixation times increased when uncertainty was high, confirming the difficulty in obtaining/interpreting information from the instrument in this condition. These results support the assumption that risky-decision making observed in pilots can be, at least partially, explained by a shift from cold to hot (emotional) decision-making in response to economic constraints and uncertainty.