Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice

In choosing between different rewards expected after unequal delays, humans and animals often prefer the smaller but more immediate reward, indicating that the subjective value or utility of reward is depreciated according to its delay. Here, we show that neurons in the primate caudate nucleus and ventral striatum modulate their activity according to temporally discounted values of rewards with a similar time course. However, neurons in the caudate nucleus encoded the difference in the temporally discounted values of the two alternative targets more reliably than neurons in the ventral striatum. In contrast, neurons in the ventral striatum largely encoded the sum of the temporally discounted values, and therefore, the overall goodness of available options. These results suggest a more pivotal role for the dorsal striatum in action selection during intertemporal choice.     

Study of the preference of high probability of attaining the goal or its subjective value

The authors used testimonies of students on the dominant motives of joining a higher education institution as the behavioral model that prefers a high probability of the attainment of a goal or its subjective value. They compared the features of the motivation sphere related to the choice of profession with the psychophysiological data of 38 students of a sports higher education institution. The motivation of this choice was determined using a test questionnaire that permitted the authors to measure the subjective value (importance) and accessibility (probability of attainment) of the selected profession. The psychophysiological features were evaluated on two levels, i.e., formal dynamic (temperamental characteristics) and physiological (frequency and amplitude characteristics of EEG). The athlete students’ motivating factor of the choice of profession was shown to form under the influence of individual and typological features, which manifest themselves in psychomotor activity and total ergicity (working capacity), while accessibility experiences a negative influence of the temperamental characteristics in the intellectual sphere, as well as the total ergicity level. The background EEG was more activated in those who chose sports as their profession because of deeming it more valuable. In students who were attracted by the accessibility of this field, a lower cerebral activation level was observed, especially in the frontal zones of the left hemisphere.     

Predictive neural coding of reward preference involves dissociable responses in human ventral midbrain and ventral striatum

Food preferences are acquired through experience and can exert strong influence on choice behavior. In order to choose which food to consume, it is necessary to maintain a predictive representation of the subjective value of the associated food stimulus. Here, we explore the neural mechanisms by which such predictive representations are learned through classical conditioning. Human subjects were scanned using fMRI while learning associations between arbitrary visual stimuli and subsequent delivery of one of five different food flavors. Using a temporal difference algorithm to model learning, we found predictive responses in the ventral midbrain and a part of ventral striatum (ventral putamen) that were related directly to subjects' actual behavioral preferences. These brain structures demonstrated divergent response profiles, with the ventral midbrain showing a linear response profile with preference, and the ventral striatum a bivalent response. These results provide insight into the neural mechanisms underlying human preference behavior.     

Aversive Event Anticipation Affects Connectivity between the Ventral Striatum and the Orbitofrontal Cortex in an fMRI Avoidance Task

Ability to anticipate aversive events is important for avoiding dangerous or unpleasant situations. The motivation to avoid an event is influenced by the incentive salience of an event-predicting cue. In an avoidance fMRI task we used tone intensities to manipulate salience in order to study the involvement of the orbitofrontal cortex in processing of incentive salience. In the task, cues predicting either aversive or neutral avoidable tones were presented. Ventral striatum, amygdala and anterior insula activations were significantly stronger during presentation of cues for aversive than neutral tones. A psychophysiological interaction analysis showed stronger connectivity between the ventral striatum and the orbitofrontal cortex during aversive than neutral conditions. The present study shows an interaction between the ventral striatum, a structure previously linked to negative incentive salience, and the orbitofrontal cortex supporting a role for this region in processing salience. In addition, this study replicates previous findings suggesting that the task is robust.     

Value representations in the primate striatum during matching behavior

Choosing the most valuable course of action requires knowing the outcomes associated with the available alternatives. The striatum may be important for representing the values of actions. We examined this in monkeys performing an oculomotor choice task. The activity of phasically active neurons (PANs) in the striatum covaried with two classes of information: action-values and chosen-values. Action-value PANs were correlated with value estimates for one of the available actions, and these signals were frequently observed before movement execution. Chosen-value PANs were correlated with the value of the action that had been chosen, and these signals were primarily observed later in the task, immediately before or persistently after movement execution. These populations may serve distinct functions mediated by the striatum: some PANs may participate in choice by encoding the values of the available actions, while other PANs may participate in evaluative updating by encoding the reward value of chosen actions.     

Subjective socioeconomic status predicts human ventral striatal responses to social status information

The enormous influence of hierarchical rank on social interactions [1] suggests that neural mechanisms exist to process status-related information [2] and ascribe value to it. The ventral striatum is prominently implicated in processing value and salience, independent of hedonic properties [3, 4], and a functional magnetic resonance imaging (fMRI) study of social status perception in humans demonstrated that viewing higher-ranked compared to lower-ranked individuals evokes a ventral striatal response [5], indicative of a greater assignment of value/salience to higher status. Consistent with this interpretation, nonhuman primates value information associated with higher-ranked conspecifics more than lower-ranked, as illustrated using a choice paradigm in which monkeys preferentially take the opportunity to view high-status monkeys [6]. Interestingly, this status-related value assignment in nonhuman primates is influenced by one's own hierarchical rank: high-status monkeys preferentially attend to conspecifics of high status, whereas low-status monkeys will also attend to other low-status monkeys [7]. Complementary to these findings, using fMRI and a social status judgment task in humans, we suggest a neurobiological mechanism by which one's own relative hierarchical rank influences the value attributed to particular social status information by demonstrating that one's subjective socioeconomic status differentially influences ventral striatal activity during processing of status-related information.     

Dopaminergic Lesions of the Dorsolateral Striatum in Rats Increase Delay Discounting in an Impulsive Choice Task

Dysregulated dopamine transmission in striatal circuitry is associated with impulsivity. The current study evaluated the influence of dopaminergic inputs to the dorsolateral striatum on impulsive choice, one aspect of impulsive behavior. We implemented an operant task that measures impulsive choice in rats via delay discounting wherein intracranial self-stimulation (ICSS) was used as the positive reinforcer. To do so, rats were anesthetized to allow implanting of a stimulating electrode within the lateral hypothalamus of one hemisphere and bilateral dorsal striatal injections of the dopaminergic toxin, 6-OHDA (lesioned) or its vehicle (sham). Following recovery, rats were trained in a delay discounting task wherein they selected between a small ICSS current presented immediately after lever pressing, and a large ICSS current presented following a 0 to 15s delay upon pressing the alternate lever. Task acquisition and reinforcer discrimination were similar for lesioned and sham rats. All rats exhibited an initial preference for the large reinforcer, and as the delay was increased, preference for the large reinforcer was decreased indicating that the subjective value of the large reinforcer was discounted as a function of delay time. However, this discounting effect was significantly enhanced in lesioned rats for the longer delays. These data reveal a contribution of dopaminergic inputs to the dorsolateral striatum on impulsive choice behavior, and provide new insights into neural substrates underlying discounting behaviors.     

Food reward in the absence of taste receptor signaling

Food palatability and hedonic value play central roles in nutrient intake. However, postingestive effects can influence food preferences independently of palatability, although the neurobiological bases of such mechanisms remain poorly understood. Of central interest is whether the same brain reward circuitry that is responsive to palatable rewards also encodes metabolic value independently of taste signaling. Here we show that trpm5-/- mice, which lack the cellular machinery required for sweet taste transduction, can develop a robust preference for sucrose solutions based solely on caloric content. Sucrose intake induced dopamine release in the ventral striatum of these sweet-blind mice, a pattern usually associated with receipt of palatable rewards. Furthermore, single neurons in this same ventral striatal region showed increased sensitivity to caloric intake even in the absence of gustatory inputs. Our findings suggest that calorie-rich nutrients can directly influence brain reward circuits that control food intake independently of palatability or functional taste transduction.     

Neurocomputational mechanism of controllability inference under a multi-agent setting

Controllability perception significantly influences motivated behavior and emotion and requires an estimation of one’s influence on an environment. Previous studies have shown that an agent can infer controllability by observing contingency between one’s own action and outcome if there are no other outcome-relevant agents in an environment. However, if there are multiple agents who can influence the outcome, estimation of one’s genuine controllability requires exclusion of other agents’ possible influence. Here, we first investigated a computational and neural mechanism of controllability inference in a multi-agent setting. Our novel multi-agent Bayesian controllability inference model showed that other people’s action-outcome contingency information is integrated with one’s own action-outcome contingency to infer controllability, which can be explained as a Bayesian inference. Model-based functional MRI analyses showed that multi-agent Bayesian controllability inference recruits the temporoparietal junction (TPJ) and striatum. Then, this inferred controllability information was leveraged to increase motivated behavior in the vmPFC. These results generalize the previously known role of the striatum and vmPFC in single-agent controllability to multi-agent controllability, and this generalized role requires the TPJ in addition to the striatum of single-agent controllability to integrate both self- and other-related information. Finally, we identified an innate positive bias toward the self during the multi-agent controllability inference, which facilitated behavioral adaptation under volatile controllability. Furthermore, low positive bias and high negative bias were associated with increased daily feelings of guilt. Our results provide a mechanism of how our sense of controllability fluctuates due to other people in our lives, which might be related to social learned helplessness and depression.     

'Good genes as heterozygosity': the major histocompatibility complex and mate choice in Atlantic salmon (Salmo salar).

According to the theory of mate choice based on heterozygosity, mates should choose each other in order to increase the heterozygosity of their offspring. In this study, we tested the 'good genes as heterozygosity' hypothesis of mate choice by documenting the mating patterns of wild Atlantic salmon (Salmo salar) using both major histocompatibility complex (MHC) and microsatellite loci. Specifically, we tested the null hypotheses that mate choice in Atlantic salmon is not dependent on the relatedness between potential partners or on the MHC similarity between mates. Three parameters were assessed: (i) the number of shared alleles between partners (x and y) at the MHC (M(xy)), (ii) the MHC amino-acid genotypic distance between mates' genotypes (AA(xy)), and (iii) genetic relatedness between mates (r(xy)). We found that Atlantic salmon choose their mates in order to increase the heterozygosity of their offspring at the MHC and, more specifically, at the peptide-binding region, presumably in order to provide them with better defence against parasites and pathogens. This was supported by a significant difference between the observed and expected AA(xy) (p = 0.0486). Furthermore, mate choice was not a mechanism of overall inbreeding avoidance as genetic relatedness supported a random mating scheme (p = 0.445). This study provides the first evidence that MHC genes influence mate choice in fish.     

Prefrontal coding of temporally discounted values during intertemporal choice

Reward from a particular action is seldom immediate, and the influence of such delayed outcome on choice decreases with delay. It has been postulated that when faced with immediate and delayed rewards, decision makers choose the option with maximum temporally discounted value. We examined the preference of monkeys for delayed reward in an intertemporal choice task and the neural basis for real-time computation of temporally discounted values in the dorsolateral prefrontal cortex. During this task, the locations of the targets associated with small or large rewards and their corresponding delays were randomly varied. We found that prefrontal neurons often encoded the temporally discounted value of reward expected from a particular option. Furthermore, activity tended to increase with [corrected] discounted values for targets [corrected] presented in the neuron's preferred direction, suggesting that activity related to temporally discounted values in the prefrontal cortex might determine the animal's behavior during intertemporal choice.     

Neurobiological and psychophysiological correlates of emotional dysregulation in ADHD patients

Emotional dysregulation in ADHD patients has become an important issue in recent years. Dysfunctions especially in brain regions related to emotional processing as well as psychophysiological reactions have been reported. This review includes functional MRI, event-related potentials as well as emotion-modulated startle studies in ADHD patients. It reflects a selective review of the authors with no claim of completeness. Changes in the processing of positive stimuli, with reduced brain activity in the amygdala, in the ventral striatum and reduced EEG potentials were found. The relevance of the observed dysregulation in emotional processing and psychophysiological reactions with regard to theoretical constructs of ADHD-specific emotional dysregulation versus comorbidity will be discussed.     

Parallel Representation of Value-Based and Finite State-Based Strategies in the Ventral and Dorsal Striatum

Previous theoretical studies of animal and human behavioral learning have focused on the dichotomy of the value-based strategy using action value functions to predict rewards and the model-based strategy using internal models to predict environmental states. However, animals and humans often take simple procedural behaviors, such as the “win-stay, lose-switch” strategy without explicit prediction of rewards or states. Here we consider another strategy, the finite state-based strategy, in which a subject selects an action depending on its discrete internal state and updates the state depending on the action chosen and the reward outcome. By analyzing choice behavior of rats in a free-choice task, we found that the finite state-based strategy fitted their behavioral choices more accurately than value-based and model-based strategies did. When fitted models were run autonomously with the same task, only the finite state-based strategy could reproduce the key feature of choice sequences. Analyses of neural activity recorded from the dorsolateral striatum (DLS), the dorsomedial striatum (DMS), and the ventral striatum (VS) identified significant fractions of neurons in all three subareas for which activities were correlated with individual states of the finite state-based strategy. The signal of internal states at the time of choice was found in DMS, and for clusters of states was found in VS. In addition, action values and state values of the value-based strategy were encoded in DMS and VS, respectively. These results suggest that both the value-based strategy and the finite state-based strategy are implemented in the striatum.     

Ventral Striatal Activity Correlates with Memory Confidence for Old- and New-Responses in a Difficult Recognition Test

Activity in the ventral striatum has frequently been associated with retrieval success, i.e., it is higher for hits than correct rejections. Based on the prominent role of the ventral striatum in the reward circuit, its activity has been interpreted to reflect the higher subjective value of hits compared to correct rejections in standard recognition tests. This hypothesis was supported by a recent study showing that ventral striatal activity is higher for correct rejections than hits when the value of rejections is increased by external incentives. These findings imply that the striatal response during recognition is context-sensitive and modulated by the adaptive significance of “oldness” or “newness” to the current goals. The present study is based on the idea that not only external incentives, but also other deviations from standard recognition tests which affect the subjective value of specific response types should modulate striatal activity. Therefore, we explored ventral striatal activity in an unusually difficult recognition test that was characterized by low levels of confidence and accuracy. Based on the human uncertainty aversion, in such a recognition context, the subjective value of all high confident decisions is expected to be higher than usual, i.e., also rejecting items with high certainty is deemed rewarding. In an accompanying behavioural experiment, participants rated the pleasantness of each recognition response. As hypothesized, ventral striatal activity correlated in the current unusually difficult recognition test not only with retrieval success, but also with confidence. Moreover, participants indicated that they were more satisfied by higher confidence in addition to perceived oldness of an item. Taken together, the results are in line with the hypothesis that ventral striatal activity during recognition codes the subjective value of different response types that is modulated by the context of the recognition test.     

Energetic state during learning affects foraging choices in starlings

We investigated the influence of energetic state at the timeof acquaintance with a new food source on preference for thatsource on later encounters, using wild-caught European starlingsas subjects. Twelve birds learned to obtain food rewards bypecking at either of two keys identified by color. The keyswere encountered in different sessions, while the subjects werefood deprived or prefed. Food rewards from both sources werealways identical. After an equal number of reinforced trialswith each source, the birds were presented with choices betweenthem. The birds significantly preferred the source that hadpreviously delivered food under higher deprivation. We relatethese results to findings reported elsewhere of preferencesfor options previously associated with greater effort. We hypothesizethat subjects may attribute value to an option according tothe marginal fitness gain associated with this option in thepast. Although this process may be adaptive under many circumstances,it violates the assumptions of normative models of choice thatimply mechanisms of valuation sensitive to the absolute propertiesof a payoff or to expected absolute changes in state.     

Striatal contributions to declarative memory retrieval

Declarative memory is known to depend on the medial temporal lobe memory system. Recently, there has been renewed focus on the relationship between the basal ganglia and declarative memory, including the involvement of striatum. However, the contribution of striatum to declarative memory retrieval remains unknown. Here, we review neuroimaging and neuropsychological evidence for the involvement of the striatum in declarative memory retrieval. From this review, we propose that, along with the prefrontal cortex (PFC), the striatum primarily supports cognitive control of memory retrieval. We conclude by proposing three hypotheses for the specific role of striatum in retrieval: (1) striatum modulates the re-encoding of retrieved items in accord with their expected utility (adaptive encoding), (2) striatum selectively admits information into working memory that is expected to increase the likelihood of successful retrieval (adaptive gating), and (3) striatum enacts adjustments in cognitive control based on the outcome of retrieval (reinforcement learning).     

Signatures of Value Comparison in Ventral Striatum Neurons

The ventral striatum (VS), like its cortical afferents, is closely associated with processing of rewards, but the relative contributions of striatal and cortical reward systems remains unclear. Most theories posit distinct roles for these structures, despite their similarities. We compared responses of VS neurons to those of ventromedial prefrontal cortex (vmPFC) Area 14 neurons, recorded in a risky choice task. Five major response patterns observed in vmPFC were also observed in VS: (1) offer value encoding, (2) value difference encoding, (3) preferential encoding of chosen relative to unchosen value, (4) a correlation between residual variance in responses and choices, and (5) prominent encoding of outcomes. We did observe some differences as well; in particular, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, the close match between vmPFC and VS suggests that cortex and its striatal targets make overlapping contributions to economic choice.     

How Do Females’ Genetic Characteristics Influence Male Mate Preference in the Terrestrial Isopod Armadillidium vulgare?

Genetic diversity is a key factor that can influence mate choice in many species. We experimentally determined the influence of this factor on mate preference in the crustacean terrestrial isopod Armadillidium vulgare. This biological model is gregarious which could increase the risk of inbreeding by mating with closely related partners. Mechanisms of inbreeding avoidance during mate choice can thus be expected. Moreover, previous studies predict that males would be the choosy sex. We performed Y‐choice tests giving males the choice between two females presenting different levels of heterozygosity and genetic similarity to the male. Our results show potential inbreeding avoidance according to the genetic characteristics of females presented to males. The higher the variation in genetic similarity to the male between females is, the higher the preference of the male towards the most dissimilar female is. Hence, male preferences may only be detectable when the difference between females’ genetic characteristics is large enough. If heterozygosity is associated with fitness in A. vulgare (as in many organisms), the patterns of mate preference we observe may be adaptive.     

Pertussis toxin treatment modifies opiate action in the rat brain striatum

In this report we present evidence that a guanine nucleotide regulatory protein, Gi, mediates opiate action in the rat brain striatum. Opiates inhibit basal adenylate cyclase activity in rat brain striatum. This effect on adenylate cyclase is dose-dependently attenuated by pretreatment of membranes with pertussis toxin, which ADP-ribosylates a protein with a molecular mass of 41,000 daltons. This protein co-migrates with the GTP-binding subunit of Gi, which mediates inhibition of adenylate cyclase. Several brain regions were compared for the extent of radiolabeling and effects on adenylate cyclase activity. Although Gi was found in each region examined, opiate inhibition of adenylate cyclase is clearly seen only in the striatum.     

Matching-to-sample by pigeons: the dissociation of comparison choice frequency from the probability of reinforcement

It has been proposed that comparison choice in matching-to-sample should depend on two factors, the relative probability of reinforcement associated with each of the comparison stimuli and the conditional probability of each comparison stimulus being correct given presentation of one of the samples. DiGian and Zentall [DiGian, K.A., Zentall, T.R., 2007. Matching-to-sample in pigeons: in the absence of sample memory, sample frequency is a better predictor of comparison choice than the probability of reinforcement for comparison choice. Learn Behav. 35, 242-261] have shown that sample frequency together with the probability of choosing each of the comparison stimuli in training can influence comparison choice when delays are introduced, when the number of reinforcements associated with each of the comparison stimuli is equated. Furthermore, Zentall and Clement [Zentall, T.R., Clement, T.S., 2002. Memory mechanisms in pigeons: Evidence of base-rate neglect. J. Exp. Psych.: Anim. Behav. Proc. 28, 111-115] have found that sample frequency can affect comparison choice when delays are introduced independently of the number of choices of each of the comparison stimuli in training and the number of reinforcements associated with each of the comparison stimuli is equated. In the present experiment we found that the probability of choosing each of the comparison stimuli in training can affect comparison choice when delays are introduced, independently of sample frequency and when the number of reinforcements associated with each of the comparison stimuli is equated. Together, these experiments suggest that when the sample is not available, there is a partial dissociation between comparison choice and the probability of reinforcement associated with each of the comparison stimuli.