Will travel for food: spatial discounting in two new world monkeys

Nonhuman animals steeply discount the future, showing a preference for small, immediate over large, delayed rewards. Currently unclear is whether discounting functions depend on context. Here, we examine the effects of spatial context on discounting in cotton-top tamarins (Saguinus oedipus) and common marmosets (Callithrix jacchus), species known to differ in temporal discounting. We presented subjects with a choice between small, nearby rewards and large, distant rewards. Tamarins traveled farther for the large reward than marmosets, attending to the ratio of reward differences rather than their absolute values. This species difference contrasts with performance on a temporal task in which marmosets waited longer than tamarins for the large reward. These comparative data indicate that context influences choice behavior, with the strongest effect seen in marmosets who discounted more steeply over space than over time. These findings parallel details of each species' feeding ecology. Tamarins range over large distances and feed primarily on insects, which requires using quick, impulsive action. Marmosets range over shorter distances than tamarins and feed primarily on tree exudates, a clumped resource that requires patience to wait for sap to exude. These results show that discounting functions are context specific, shaped by a history of ecological pressures.     

Discounting of sequences of delayed rewards of different amounts

The main purpose of this study was to determine whether the magnitude effect is present in cases where delayed sequences of rewards are discounted. The magnitude effect refers to the inverse relationship between the amount of a reward and the steepness of temporal discounting. This study was conducted with a computer program to estimate the indifference points, which served as indicators of the present subjective value of delayed sequences of small and large rewards. In the indifference point the subjective value of a single, immediate reward was equal to the subjective value of the delayed sequence (or to the value of a single delayed reward). As a control condition, we added an experimental task involving choices between single immediate and single delayed rewards. The experiment showed that the sequences of large rewards are discounted less steeply than are the sequences of small rewards. This finding suggests that the magnitude effect is present within the delayed sequences of rewards. In addition, when outcomes are relatively large, the results suggest that a single reward is discounted less steeply than the sequence of a total nominal value equal to this single reward. However, for relatively small rewards, the difference is not statistically significant. The less steep discounting of sequences of large rewards may explain the reward-bundling effect, which refers to less steep discounting of longer sequences than of shorter ones: longer sequences usually have greater overall nominal value. The present study was conducted on hypothetical rewards, and the results should be validated using real rewards.     

Measurement of delay discounting using trial-by-trial consequences

DELAY DISCOUNTING IN HUMANS WAS INVESTIGATED USING THREE DIFFERENT PROCEDURES: a frequently used discounting procedure with hypothetical rewards and delays; a procedure with hypothetical rewards and delays compressed down to much smaller values; and a contingent procedure in which each choice had a direct consequence. In the contingent procedure, on every trial, participants actually experienced the delay and obtained the reward amount associated with their choice. Each participant was exposed to all three procedures. Orderly temporal discounting patterns were obtained in all three procedures and described well by a hyperbolic model. Comparisons of the data revealed patterns unique to each procedure. The distributions of the discounting measures differed across the three procedures. In the contingent procedure, several subjects showed no discounting, e.g. complete self-control. Procedural factors in studies of impulsivity are discussed, and suggestions are offered for experiments in which the contingent-discounting procedure may prove useful.     

The present values of delayed rewards are approximately additive

In two experiments, human subjects were asked to estimate their present values of single delayed rewards and their present values of temporal sequences of three rewards. Present values were solicited by asking subjects to indicate an amount of money v for which they would be indifferent between receiving v at the end of the session and receiving the delayed reward(s). A procedure was used for which responding the true value of v was the optimal strategy, and the actual payoff that each subject received was determined by one randomly selected trial. In Experiment 1 (n=29) each delayed reward was 9.90 dollars in cash. In Experiment 2 (n=19) the delayed rewards were dated 15 dollars gift certificates to a local restaurant. In both experiments, the present values of the sequences were approximately equal to the sums of the present values of their component rewards. The presence of outliers suggests that a few subjects may have valued sequences less than the sums of their single rewards. Effects of a preference for uniform sequences, if any, were too small to be detected. Discounting of sequences was well fit by a parallel hyperbolic discounting equation, consistent with Mazur's [Mazur, J.E., 1986. Choice between single and multiple delayed reinforcers. J. Exp. Anal. Behav. 46 (1), 67-77] results using multiple reinforcers.     

The date-delay framing effect in temporal discounting depends on substance abuse

In the present study, individuals with substance use disorders (n=30) and non-addicted controls (n=30) were presented with a delay-discounting task with time being described either as dates or as temporal intervals. Three main results were obtained. First, in both groups reward size had a large impact on discounting future rewards, with discount rates becoming larger with smaller reward sizes. Second, participants discounted future rewards less strongly when their time of delivery was presented as a date instead of a temporal distance. Third, whereas discount rates of individuals with substance use disorders varied substantially with regard to the presentation of time in the task, the controls changed their choices depending on time presentation only slightly.     

The evolution of social discounting in hierarchically clustered populations

The expression of a social behaviour may affect the fitness of actors and recipients living in the present and in the future of the population. When there is a risk that a future reward will not be experienced in such a context, the value of that reward should be discounted; but by how much? Here, we evaluate social discount rates for delayed fitness rewards to group of recipients living at different positions in both space and time than the actor in a hierarchically clustered population. This is a population where individuals are grouped into families, families into villages, villages into clans, and so on, possibly ad infinitum. The group-wide fitness effects are assumed to either increase or decrease the fecundity or the survival of recipients and can be arbitrarily extended in space and time. We find that actions changing the survival of individuals living in the future are generally more strongly discounted than fecundity-changing actions for all future times and that the value of future rewards increases as individuals live longer. We also find that delayed fitness effects may not only be discounted by a constant factor per unit delay (exponential discounting), but that, as soon as there is localized dispersal in a population, discounting per unit delay is likely to fall rapidly for small delays and then slowly for longer delays (hyperbolic discounting). As dispersal tends to be localized in natural populations, our results suggest that evolution is likely to favour individuals that express present-biased behaviours and that may be time-inconsistent with respect to their group-wide effects.     

The combined effects of delay and probability in discounting

Human discounting studies have frequently observed hyperbolic discounting of rewards that are delayed or probabilistic. However, no studies have systematically combined delay and probability in a single discounting procedure. Indifference points of hypothetical money rewards that are both delayed and probabilistic were determined. Probabilities were converted into comparable delays according to the h/k constant of proportionality determined by , and discounting rates were calculated. These data provided a very good fit to the hyperbolic model of discounting, suggesting that delay and probability can be combined into a single metric in studies of discounting. The inclusion of a magnitude condition found the Magnitude Effect commonly found in studies of temporal discounting. A temporal resolution of uncertainty condition found no effect. The present paper offers a novel statistical method, within an established framework, for the analysis of data from studies of discounting that combine delay and probability.     

Single units in the pigeon brain integrate reward amount and time-to-reward in an impulsive choice task

BACKGROUND: Animals prefer small over large rewards when the delays preceding large rewards exceed an individual tolerance limit. Such impulsive choice behavior occurs even in situations in which alternative strategies would yield more optimal outcomes. Behavioral research has shown that an animal's choice is guided by the alternative rewards' subjective values, which are a function of reward amount and time-to-reward. Despite increasing knowledge about the pharmacology and anatomy underlying impulsivity, it is still unknown how the brain combines reward amount and time-to-reward information to represent subjective reward value. RESULTS: We trained pigeons to choose between small, immediate rewards and large rewards delivered after gradually increasing delays. Single-cell recordings in the avian Nidopallium caudolaterale, the presumed functional analog of the mammalian prefrontal cortex, revealed that neural delay activation decreased with increasing delay length but also covaried with the expected reward amount. This integrated neural response was modulated by reward amount and delay, as predicted by a hyperbolical equation, of subjective reward value derived from behavioral studies. Furthermore, the neural activation pattern reflected the current reward preference and the time point of the shift from large to small rewards. CONCLUSIONS: The reported activity was modulated by the temporal devaluation of the anticipated reward in addition to reward amount. Our findings contribute to the understanding of neuropathologies such as drug addiction, pathological gambling, frontal lobe syndrome, and attention-deficit disorders, which are characterized by inappropriate temporal discounting and increased impulsiveness.     

Domain independence and stability in young and older adults' discounting of delayed rewards

Individual discounting rates for different types of delayed reward are typically assumed to reflect a single, underlying trait of impulsivity. Recently, we showed that discounting rates are orders of magnitude steeper for directly consumable liquid rewards than for monetary rewards (Jimura et al., 2009), raising the question of whether discounting rates for different types of reward covary at the individual level. Accordingly, the present study examined the relation between discounting of hypothetical money and real liquid rewards in young adults (Experiment 1) and older adults (Experiment 2). At the group level, young adults discounted monetary rewards more steeply than the older adults, but there was no significant age difference with respect to liquid rewards. At the individual level, the rates at which young and older participants discounted each reward type were stable over a two- to fifteen-week interval (rs > 70), but there was no significant correlation between the rates at which they discounted the two reward types. These results suggest that although similar decision-making processes may underlie the discounting of different types of rewards, the rates at which individuals discount money and directly consumable rewards may reflect separate, stable traits, rather than a single trait of impulsivity.     

Timing in reward and decision processes

Sensitivity to time, including the time of reward, guides the behaviour of all organisms. Recent research suggests that all major reward structures of the brain process the time of reward occurrence, including midbrain dopamine neurons, striatum, frontal cortex and amygdala. Neuronal reward responses in dopamine neurons, striatum and frontal cortex show temporal discounting of reward value. The prediction error signal of dopamine neurons includes the predicted time of rewards. Neurons in the striatum, frontal cortex and amygdala show responses to reward delivery and activities anticipating rewards that are sensitive to the predicted time of reward and the instantaneous reward probability. Together these data suggest that internal timing processes have several well characterized effects on neuronal reward processing.     

Pathological gambling severity is associated with impulsivity in a delay discounting procedure

Research and clinical expertise indicates that impulsivity is an underlying feature of pathological gambling. This study examined the extent to which impulsive behavior, defined by the rate of discounting delayed monetary rewards, varies with pathological gambling severity, assessed by the South Oaks Gambling Screen (SOGS). Sixty-two pathological gamblers completed a delay discounting task, the SOGS, the Eysenck impulsivity scale, the Addiction Severity Index (ASI), and questions about gambling and substance use at intake to outpatient treatment for pathological gambling. In the delay discounting task, participants chose between a large delayed reward (US $1000) and smaller more immediate rewards (US $1-$999) across a range of delays (6h to 25 years). The rate at which the delayed reward was discounted (k value) was derived for each participant and linear regression was used to identify the variables that predicted k values. Age, gender, years of education, substance abuse treatment history, and cigarette smoking history failed to significantly predict k values. Scores on the Eysenck impulsivity scale and the SOGS both accounted for a significant proportion of the variance in k values. The predictive value of the SOGS was 1.4 times that of the Eysenck scale. These results indicate that of the measures tested, gambling severity was the best single predictor of impulsive behavior in a delay discounting task in this sample of pathological gamblers.     

Thinking Outside the Euclidean Box: Riemannian Geometry and Inter-Temporal Decision-Making

Inter-temporal decisions involves assigning values to various payoffs occurring at different temporal distances. Past research has used different approaches to study these decisions made by humans and animals. For instance, considering that people discount future payoffs at a constant rate (e.g., exponential discounting) or at variable rate (e.g., hyperbolic discounting). In this research, we question the widely assumed, but seldom questioned, notion across many of the existing approaches that the decision space, where the decision-maker perceives time and monetary payoffs, is a Euclidean space. By relaxing the rigid assumption of Euclidean space, we propose that the decision space is a more flexible Riemannian space of Constant Negative Curvature. We test our proposal by deriving a discount function, which uses the distance in the Negative Curvature space instead of Euclidean temporal distance. The distance function includes both perceived values of time as well as money, unlike past work which has considered just time. By doing so we are able to explain many of the empirical findings in inter-temporal decision-making literature. We provide converging evidence for our proposal by estimating the curvature of the decision space utilizing manifold learning algorithm and showing that the characteristics (i.e., metric properties) of the decision space resembles those of the Negative Curvature space rather than the Euclidean space. We conclude by presenting new theoretical predictions derived from our proposal and implications for how non-normative behavior is defined.     

Monkeys Are More Patient in a Foraging Task than in a Standard Intertemporal Choice Task

Studies of animal impulsivity generally find steep subjective devaluation, or discounting, of delayed rewards – often on the order of a 50% reduction in value in a few seconds. Because such steep discounting is highly disfavored in evolutionary models of time preference, we hypothesize that discounting tasks provide a poor measure of animals’ true time preferences. One prediction of this hypothesis is that estimates of time preferences based on these tasks will lack external validity, i.e. fail to predict time preferences in other contexts. We examined choices made by four rhesus monkeys in a computerized patch-leaving foraging task interleaved with a standard intertemporal choice task. Monkeys were significantly more patient in the foraging task than in the intertemporal choice task. Patch-leaving behavior was well fit by parameter-free optimal foraging equations but poorly fit by the hyperbolic discount parameter obtained from the intertemporal choice task. Day-to-day variation in time preferences across the two tasks was uncorrelated with each other. These data are consistent with the conjecture that seemingly impulsive behavior in animals is an artifact of their difficulty understanding the structure of intertemporal choice tasks, and support the idea that animals are more efficient rate maximizers in the multi-second range than intertemporal choice tasks would suggest.     

Insulin,Central Dopamine D2 Receptors,and Monetary Reward Discounting in Obesity

Animal research finds that insulin regulates dopamine signaling and reward behavior, but similar research in humans is lacking. We investigated whether individual differences in body mass index, percent body fat, pancreatic β-cell function, and dopamine D2 receptor binding were related to reward discounting in obese and non-obese adult men and women. Obese (n = 27; body mass index>30) and non-obese (n = 20; body mass index<30) adults were assessed for percent body fat with dual-energy X-ray absorptiometry and for β-cell function using disposition index. Choice of larger, but delayed or less certain, monetary rewards relative to immediate, certain smaller monetary rewards was measured using delayed and probabilistic reward discounting tasks. Positron emission tomography using a non-displaceable D2-specific radioligand, [¹¹C](N-methyl)benperidol quantified striatal D2 receptor binding. Groups differed in body mass index, percent body fat, and disposition index, but not in striatal D2 receptor specific binding or reward discounting. Higher percent body fat in non-obese women related to preference for a smaller, certain reward over a larger, less likely one (greater probabilistic discounting). Lower β-cell function in the total sample and lower insulin sensitivity in obese related to stronger preference for an immediate and smaller monetary reward over delayed receipt of a larger one (greater delay discounting). In obese adults, higher striatal D2 receptor binding related to greater delay discounting. Interestingly, striatal D2 receptor binding was not significantly related to body mass index, percent body fat, or β-cell function in either group. Our findings indicate that individual differences in percent body fat, β-cell function, and striatal D2 receptor binding may each contribute to altered reward discounting behavior in non-obese and obese individuals. These results raise interesting questions about whether and how striatal D2 receptor binding and metabolic factors, including β-cell function, interact to affect reward discounting in humans.     

Delay discounting of real and hypothetical rewards III: steady-state assessments, forced-choice trials, and all real rewards

Human research in delay discounting has omitted several procedures typical of animal studies: forced-choice trials, consequences following each response, and assessment of stable response patterns. The present study manipulated these procedures across two conditions in which real or hypothetical rewards were arranged. Six college students participated in daily sessions, in which steady-state discounting of hypothetical and real rewards was assessed. No systematic effects of repeated exposure to hypothetical rewards was detected when compared with first day assessments of discounting. Likewise, no systematic effect of reward type (real versus hypothetical) was detected. When combined with previous research failing to detect a difference between hypothetical and potentially real rewards, these findings suggest that assessing discounting of hypothetical rewards in single sessions is a practical and valid procedure in the study of delay discounting.     

Reduced sensitivity to immediate reward during decision-making in older than younger adults

We examined whether older adults differ from younger adults in the degree to which they favor immediate over delayed rewards during decision-making. To examine the neural correlates of age-related differences in delay discounting we acquired functional MR images while participants made decisions between smaller but sooner and larger but later monetary rewards. The behavioral results show age-related reductions in delay discounting. Less impulsive decision-making in older adults was associated with lower ventral striatal activations to immediate reward. Furthermore, older adults showed an overall higher percentage of delayed choices and reduced activity in the dorsal striatum than younger adults. This points to a reduced reward sensitivity of the dorsal striatum in older adults. Taken together, our findings indicate that less impulsive decision-making in older adults is due to a reduced sensitivity of striatal areas to reward. These age-related changes in reward sensitivity may result from transformations in dopaminergic neuromodulation with age.     

Led into Temptation? Rewarding Brand Logos Bias the Neural Encoding of Incidental Economic Decisions

Human decision-making is driven by subjective values assigned to alternative choice options. These valuations are based on reward cues. It is unknown, however, whether complex reward cues, such as brand logos, may bias the neural encoding of subjective value in unrelated decisions. In this functional magnetic resonance imaging (fMRI) study, we subliminally presented brand logos preceding intertemporal choices. We demonstrated that priming biased participants' preferences towards more immediate rewards in the subsequent temporal discounting task. This was associated with modulations of the neural encoding of subjective values of choice options in a network of brain regions, including but not restricted to medial prefrontal cortex. Our findings demonstrate the general susceptibility of the human decision making system to apparently incidental contextual information. We conclude that the brain incorporates seemingly unrelated value information that modifies decision making outside the decision-maker's awareness.     

Sub-Optimal Allocation of Time in Sequential Movements

The allocation of limited resources such as time or energy is a core problem that organisms face when planning complex actions. Most previous research concerning planning of movement has focused on the planning of single, isolated movements. Here we investigated the allocation of time in a pointing task where human subjects attempted to touch two targets in a specified order to earn monetary rewards. Subjects were required to complete both movements within a limited time but could freely allocate the available time between the movements. The time constraint presents an allocation problem to the subjects: the more time spent on one movement, the less time is available for the other. In different conditions we assigned different rewards to the two tokens. How the subject allocated time between movements affected their expected gain on each trial. We also varied the angle between the first and second movements and the length of the second movement. Based on our results, we developed and tested a model of speed-accuracy tradeoff for sequential movements. Using this model we could predict the time allocation that would maximize the expected gain of each subject in each experimental condition. We compared human performance with predicted optimal performance. We found that all subjects allocated time sub-optimally, spending more time than they should on the first movement even when the reward of the second target was five times larger than the first. We conclude that the movement planning system fails to maximize expected reward in planning sequences of as few as two movements and discuss possible interpretations drawn from economic theory.