Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice

In choosing between different rewards expected after unequal delays, humans and animals often prefer the smaller but more immediate reward, indicating that the subjective value or utility of reward is depreciated according to its delay. Here, we show that neurons in the primate caudate nucleus and ventral striatum modulate their activity according to temporally discounted values of rewards with a similar time course. However, neurons in the caudate nucleus encoded the difference in the temporally discounted values of the two alternative targets more reliably than neurons in the ventral striatum. In contrast, neurons in the ventral striatum largely encoded the sum of the temporally discounted values, and therefore, the overall goodness of available options. These results suggest a more pivotal role for the dorsal striatum in action selection during intertemporal choice.     

Intangibility in intertemporal choice

Since the advent of the discounted utility (DU) model, economists have thought about intertemporal choice in very specific terms. DU assumes that people make explicit trade-offs between costs and benefits occurring at different points in time. While this explicit trade-off perspective is simple and tractable, and has stimulated productive research, it does not provide a very realistic representation of a wide range of the most important intertemporal trade-offs that people face in daily life. If one considers the most important and commonly discussed examples of intertemporal choices, a striking pattern emerges: in almost all cases, early outcomes tend to be concrete (e.g. purchasing this latte), but later outcomes tend to be much less tangible (e.g. the unknown item that could have been purchased later with the money spent on this latte). We propose that people rely on anticipatory emotions as a proxy for intangible outcomes when trade-offs are implicit. This paper reviews neuroeconomic evidence that has begun to elucidate the role of anticipatory emotions in decisions involving intangible outcomes. Although most progress has been made in the domain of spending and saving, we discuss how the existing neuroeconomic research could be extended to other domains where trade-offs are ill defined.     

Evolutionary pressures on primate intertemporal choice

From finding food to choosing mates, animals must make intertemporal choices that involve fitness benefits available at different times. Species vary dramatically in their willingness to wait for delayed rewards. Why does this variation across species exist? An adaptive approach to intertemporal choice suggests that time preferences should reflect the temporal problems faced in a species''s environment. Here, I use phylogenetic regression to test whether allometric factors relating to body size, relative brain size and social group size predict how long 13 primate species will wait in laboratory intertemporal choice tasks. Controlling for phylogeny, a composite allometric factor that includes body mass, absolute brain size, lifespan and home range size predicted waiting times, but relative brain size and social group size did not. These findings support the notion that selective pressures have sculpted intertemporal choices to solve adaptive problems faced by animals. Collecting these types of data across a large number of species can provide key insights into the evolution of decision making and cognition.     

Nicotine and the behavioral mechanisms of intertemporal choice

Nicotine has been found to produce dose-dependent increases in impulsive choice (preference for smaller, sooner reinforcers relative to larger, later reinforcers) in rats. Such increases could be produced by either of two behavioral mechanisms: (1) an increase in delay discounting (i.e., exacerbating the impact of differences in reinforcer delays) which would increase the value of a sooner reinforcer relative to a later one, or (2) a decrease in magnitude sensitivity (i.e., diminishing the impact of differences in reinforcer magnitudes) which would increase the value of a smaller reinforcer relative to a larger one. To isolate which of these two behavioral mechanisms was likely responsible for nicotine's effect on impulsive choice, we manipulated reinforcer delay and magnitude using a concurrent, variable interval (VI 30 s, VI 30 s) schedule of reinforcement with 2 groups of Long-Evans rats (n = 6 per group). For one group, choices were made between a 1-s delay and a 9-s delay to 2 food pellets. For a second group, choices were made between 1 pellet and 3 pellets. Nicotine (vehicle, 0.03, 0.1, 0.3, 0.56 and 0.74 mg/kg) produced dose-dependent decreases in preference for large versus small magnitude reinforcers and had no consistent effect on preference for short versus long delays. This suggests that nicotine decreases sensitivity to reinforcer magnitude.     

Motor response vigour and visual fixation patterns reflect subjective valuation during intertemporal choice

Value-based decision-making is of central interest in cognitive neuroscience and psychology, as well as in the context of neuropsychiatric disorders characterised by decision-making impairments. Studies examining (neuro-)computational mechanisms underlying choice behaviour typically focus on participants’ decisions. However, there is increasing evidence that option valuation might also be reflected in motor response vigour and eye movements, implicit measures of subjective utility. To examine motor response vigour and visual fixation correlates of option valuation in intertemporal choice, we set up a task where the participants selected an option by pressing a grip force transducer, simultaneously tracking fixation shifts between options. As outlined in our preregistration (https://osf.io/k6jct), we used hierarchical Bayesian parameter estimation to model the choices assuming hyperbolic discounting, compared variants of the softmax and drift diffusion model, and assessed the relationship between response vigour and the estimated model parameters. The behavioural data were best explained by a drift diffusion model specifying a non-linear scaling of the drift rate by the subjective value differences. Replicating previous findings, we found a magnitude effect for temporal discounting, such that higher rewards were discounted less. This magnitude effect was further reflected in motor response vigour, such that stronger forces were exerted in the high vs. the low magnitude condition. Bayesian hierarchical linear regression further revealed higher grip forces, faster response times and a lower number of fixation shifts for trials with higher subjective value differences. An exploratory analysis revealed that subjective value sums across options showed an even more pronounced association with trial-wise grip force amplitudes. Our data suggest that subjective utility or implicit valuation is reflected in motor response vigour and visual fixation patterns during intertemporal choice. Taking into account response vigour might thus provide deeper insight into decision-making, reward valuation and maladaptive changes in these processes, e.g. in the context of neuropsychiatric disorders.     

Prefrontal neurons coding suppression of specific saccades

The prefrontal cortex has been implicated in the suppression of unwanted behavior, based upon observations of humans and monkeys with prefrontal lesions. Despite this, there has been little direct neurophysiological evidence for a mechanism that suppresses specific behavior. In this study, we used an oculomotor delayed match/nonmatch-to-sample task in which monkeys had to remember a stimulus location either as a marker of where to look or as a marker of where not to look. We found a group of neurons in both the frontal eye field and the caudal prefrontal cortex that carried signals selective for the forbidden stimulus. The activity of these "don't look" neurons correlated with the monkeys' success or failure on the task. These results demonstrate a frontal signal that is related to the active suppression of one action while the subject performs another.     

Population coding and behavioral choice

Many individual behavioral acts are produced by the combined activity of large populations of broadly tuned neurons, and the neuronal populations for different behaviors can overlap. Recent experiments monitoring and manipulating neuronal activity during behavioral decisions have begun to shed light on the mechanisms that enable overlapping populations of neurons to generate choices between categorically distinct behaviors.     

Cross-country relationships between life expectancy,intertemporal choice and age at first birth

Humans, like other animals, typically discount the value of delayed rewards relative to those available in the present. From an evolutionary perspective, prioritising immediate rewards is a predictable response to high local mortality rates, as is an acceleration of reproductive scheduling. In a sample of 46 countries, we explored the cross-country relationships between average life expectancy, intertemporal choice, and women's age at first birth. We find that, across countries, lower life expectancy is associated with both a smaller percentage of people willing to wait for a larger but delayed reward, as well as a younger age at first birth. These results, which hold when controlling for region and economic pressure (GDP-per capita), dovetail with findings at the individual level to suggest that life expectancy is an important ecological predictor of both intertemporal and reproductive decision-making.     

Prefrontal synaptic activation during hippocampal memory reactivation

1. Download : Download high-res image (188KB)
2. Download : Download full-size image

     

Influence of escalating alternative reinforcer values on cigarette choice

The availability of alternative reinforcers reduces drug taking. Escalating alternative reinforcer values have been used to initiate and maintain abstinence from drug use. A reset in reinforcer value has been added to the schedule of alternative reinforcer presentation to discourage relapse. The purpose of this preliminary study was to test the influence of escalating and escalating and resetting alternative reinforcer value on cigarette choice in the human laboratory. Fourteen daily cigarette smokers completed this experiment, which required one practice and three experimental sessions. During each experimental session, subjects made six choices between smoking a cigarette and receiving money, available under Constant, Escalating or Escalating and Resetting conditions. The total number of cigarettes chosen and puffs taken, but not the maximum consecutive number of cigarettes choices, was decreased in the Escalating condition relative to the Constant condition. The maximum number of consecutive cigarettes chosen was decreased in the Escalating and Resetting condition relative to the Constant condition. The proportion of money earned was increased in the Escalating condition relative to the Constant and Escalating and Resetting conditions. These initial findings indicate that whereas an escalating alternative reinforcer schedule reduces cigarette smoking overall, an escalating and resetting alternative reinforcer schedule may reduce repeated cigarette smoking (i.e., relapse).     

Prefrontal cortex guides context-appropriate responding during language production

Although language processing is thought to frequently require cognitive control, little is known about the cognitive and neural basis of the control of language. Here, we demonstrate that processing of context by the PFC plays an important role in the control of language comprehension and production. Using a missing letter paradigm and fMRI, we found that increased activation in the PFC (but not in posterior regions), while encoding and maintaining context information, predicted context-appropriate responses. Furthermore, greater selection demands increased activity during responding in the same regions engaged during the encoding and maintenance of context. Overall, as in other cognitive task domains, these results suggest that PFC context processing plays an important role in the control of language.     

The role of temporally coarse form processing during binocular rivalry

Presenting the eyes with spatially mismatched images causes a phenomenon known as binocular rivalry-a fluctuation of awareness whereby each eye's image alternately determines perception. Binocular rivalry is used to study interocular conflict resolution and the formation of conscious awareness from retinal images. Although the spatial determinants of rivalry have been well-characterized, the temporal determinants are still largely unstudied. We confirm a previous observation that conflicting images do not need to be presented continuously or simultaneously to elicit binocular rivalry. This process has a temporal limit of about 350 ms, which is an order of magnitude larger than the visual system's temporal resolution. We characterize this temporal limit of binocular rivalry by showing that it is independent of low-level information such as interocular timing differences, contrast-reversals, stimulus energy, and eye-of-origin information. This suggests the temporal factors maintaining rivalry relate more to higher-level form information, than to low-level visual information. Systematically comparing the role of form and motion-the processing of which may be assigned to ventral and dorsal visual pathways, respectively-reveals that this temporal limit is determined by form conflict rather than motion conflict. Together, our findings demonstrate that binocular conflict resolution depends on temporally coarse form-based processing, possibly originating in the ventral visual pathway.     

Prefrontal neurons predict choices during an auditory same-different task

The detection of stimuli is critical for an animal's survival [1]. However, it is not adaptive for an animal to respond automatically to every stimulus that is present in the environment [2-5]. Given that the prefrontal cortex (PFC) plays a key role in executive function [6-8], we hypothesized that PFC activity should be involved in context-dependent responses to uncommon stimuli. As a test of this hypothesis, monkeys participated in a same-different task, a variant of an oddball task [2]. During this task, a monkey heard multiple presentations of a "reference" stimulus that were followed by a "test" stimulus and reported whether these stimuli were the same or different. While they participated in this task, we recorded from neurons in the ventrolateral prefrontal cortex (vPFC; a cortical area involved in aspects of nonspatial auditory processing [9, 10]). We found that vPFC activity was correlated with the monkeys' choices. This finding demonstrates a direct link between single neurons and behavioral choices in the PFC on a nonspatial auditory task.     

Prostaglandins temporally regulate cytoplasmic actin bundle formation during Drosophila oogenesis

Prostaglandins (PGs)—lipid signals produced downstream of cyclooxygenase (COX) enzymes—regulate actin dynamics in cell culture and platelets, but their roles during development are largely unknown. Here we define a new role for Pxt, the Drosophila COX-like enzyme, in regulating the actin cytoskeleton—temporal restriction of actin remodeling during oogenesis. PGs are required for actin filament bundle formation during stage 10B (S10B). In addition, loss of Pxt results in extensive early actin remodeling, including actin filaments and aggregates, within the posterior nurse cells of S9 follicles; wild-type follicles exhibit similar structures at a low frequency. Hu li tai shao (Hts-RC) and Villin (Quail), an actin bundler, localize to all early actin structures, whereas Enabled (Ena), an actin elongation factor, preferentially localizes to those in pxt mutants. Reduced Ena levels strongly suppress early actin remodeling in pxt mutants. Furthermore, loss of Pxt results in reduced Ena localization to the sites of bundle formation during S10B. Together these data lead to a model in which PGs temporally regulate actin remodeling during Drosophila oogenesis by controlling Ena localization/activity, such that in S9, PG signaling inhibits, whereas at S10B, it promotes Ena-dependent actin remodeling.     

Spatially and temporally restricted expression of Pax2 during murine neurogenesis

The expression of the murine paired-box-containing gene, Pax2, is examined in the developing central nervous system by in situ hybridization. Pax2 expression is detected along the boundaries of primary divisions of the neural tube. Initially, Pax2 is expressed in the ventricular zone in two compartments of cells on either side of the sulcus limitans and along the entire rhombencephalon and spinal cord. At later times, Pax2 is restricted to progeny cells that have migrated to specific regions of the intermediate zone. In the eye, Pax2 expression is restricted to the ventral half of the optic cup and stalk and later to the optic disc and nerve. In the ear, expression is restricted to regions of the otic vesicle that form neuronal components. The transient and restricted nature of Pax2 expression suggests that this murine segmentation gene homologue may also establish compartmental boundaries and contribute to the specification of neuronal identity, as do certain Drosophila segmentation genes.     

Rapid Plasticity in the Prefrontal Cortex during Affective Associative Learning

MultiCS conditioning is an affective associative learning paradigm, in which affective categories consist of many similar and complex stimuli. Comparing visual processing before and after learning, recent MultiCS conditioning studies using time-sensitive magnetoencephalography (MEG) revealed enhanced activation of prefrontal cortex (PFC) regions towards emotionally paired versus neutral stimuli already during short-latency processing stages (i.e., 50 to 80 ms after stimulus onset). The present study aimed at showing that this rapid differential activation develops as a function of the acquisition and not the extinction of the emotional meaning associated with affectively paired stimuli. MEG data of a MultiCS conditioning study were analyzed with respect to rapid changes in PFC activation towards aversively (electric shock) paired and unpaired faces that occurred during the learning of stimulus-reinforcer contingencies. Analyses revealed an increased PFC activation towards paired stimuli during 50 to 80 ms already during the acquisition of contingencies, which emerged after a single pairing with the electric shock. Corresponding changes in stimulus valence could be observed in ratings of hedonic valence, although participants did not seem to be aware of contingencies. These results suggest rapid formation and access of emotional stimulus meaning in the PFC as well as a great capacity for adaptive and highly resolving learning in the brain under challenging circumstances.     

Dissonant Representations of Visual Space in Prefrontal Cortex during Eye Movements

1. Download high-res image (225KB)
2. Download full-size image