Parameters of delay discounting assessment: number of trials, effort, and sequential effects.

Procedural variants in estimating delay discounting (DD) have been shown to yield significant within-subject differences in estimated degree of delay discounting as well as variations in the patterns of choice. The purpose of this study was to evaluate the effect of subject control over the number of trials in a delay discounting task, on degree of delay discounting. Participants were assessed with two computerized DD assessments: the full-length method presented participants with a fixed set of 240 trials, and the abbreviated task, where once participants had shown indifference between the immediate and delayed rewards, the remaining trials for that delay value were omitted. While the full-length and abbreviated methods did not differentially affect patterns of choice or estimated delay discounting, the order of presentation (ascending or descending) of immediate rewards produced differences in each measure: rate of delay discounting was significantly lower when estimated with the descending sequence; a larger proportion of area under the discounting curve was concentrated around the indifference point trial with the descending sequence; and a lower correlation was observed between estimates obtained across methods with the descending sequence.     

Strain differences in delay discounting using inbred rats

A heightened aversion to delayed rewards is associated with substance abuse and numerous other neuropsychiatric disorders. Many of these disorders are heritable, raising the possibility that delay aversion may also have a significant genetic or heritable component. To examine this possibility, we compared delay discounting in six inbred strains of rats (Brown Norway, Copenhagen, Lewis, Fischer, Noble and Wistar Furth) using the adjusting amount procedure, which provides a measure of the subjective value of delayed rewards. The subjective value of rewards decreased as the delay to receipt increased for all strains. However, a main effect of strain and a strain × delay interaction indicated that some strains were more sensitive to the imposition of delays than others. Fitting a hyperbolic discount equation showed significant strain differences in sensitivity to delay ( k ). These data indicate that there are significant strain differences in delay discounting. All strains strongly preferred the 10% sucrose solution (the reinforcer in the delay discounting task) over water and the amount of sucrose consumed was correlated with sensitivity to delay. Locomotor activity was not correlated with delay discounting behavior. Additional research will be required to disentangle genetic influences from maternal effects and to determine how these factors influence the underlying association between heightened delay discounting and neuropsychiatric disorders.     

Dopaminergic Lesions of the Dorsolateral Striatum in Rats Increase Delay Discounting in an Impulsive Choice Task

Dysregulated dopamine transmission in striatal circuitry is associated with impulsivity. The current study evaluated the influence of dopaminergic inputs to the dorsolateral striatum on impulsive choice, one aspect of impulsive behavior. We implemented an operant task that measures impulsive choice in rats via delay discounting wherein intracranial self-stimulation (ICSS) was used as the positive reinforcer. To do so, rats were anesthetized to allow implanting of a stimulating electrode within the lateral hypothalamus of one hemisphere and bilateral dorsal striatal injections of the dopaminergic toxin, 6-OHDA (lesioned) or its vehicle (sham). Following recovery, rats were trained in a delay discounting task wherein they selected between a small ICSS current presented immediately after lever pressing, and a large ICSS current presented following a 0 to 15s delay upon pressing the alternate lever. Task acquisition and reinforcer discrimination were similar for lesioned and sham rats. All rats exhibited an initial preference for the large reinforcer, and as the delay was increased, preference for the large reinforcer was decreased indicating that the subjective value of the large reinforcer was discounted as a function of delay time. However, this discounting effect was significantly enhanced in lesioned rats for the longer delays. These data reveal a contribution of dopaminergic inputs to the dorsolateral striatum on impulsive choice behavior, and provide new insights into neural substrates underlying discounting behaviors.     

Single units in the pigeon brain integrate reward amount and time-to-reward in an impulsive choice task

BACKGROUND: Animals prefer small over large rewards when the delays preceding large rewards exceed an individual tolerance limit. Such impulsive choice behavior occurs even in situations in which alternative strategies would yield more optimal outcomes. Behavioral research has shown that an animal's choice is guided by the alternative rewards' subjective values, which are a function of reward amount and time-to-reward. Despite increasing knowledge about the pharmacology and anatomy underlying impulsivity, it is still unknown how the brain combines reward amount and time-to-reward information to represent subjective reward value. RESULTS: We trained pigeons to choose between small, immediate rewards and large rewards delivered after gradually increasing delays. Single-cell recordings in the avian Nidopallium caudolaterale, the presumed functional analog of the mammalian prefrontal cortex, revealed that neural delay activation decreased with increasing delay length but also covaried with the expected reward amount. This integrated neural response was modulated by reward amount and delay, as predicted by a hyperbolical equation, of subjective reward value derived from behavioral studies. Furthermore, the neural activation pattern reflected the current reward preference and the time point of the shift from large to small rewards. CONCLUSIONS: The reported activity was modulated by the temporal devaluation of the anticipated reward in addition to reward amount. Our findings contribute to the understanding of neuropathologies such as drug addiction, pathological gambling, frontal lobe syndrome, and attention-deficit disorders, which are characterized by inappropriate temporal discounting and increased impulsiveness.     

Nicotine and the behavioral mechanisms of intertemporal choice

Nicotine has been found to produce dose-dependent increases in impulsive choice (preference for smaller, sooner reinforcers relative to larger, later reinforcers) in rats. Such increases could be produced by either of two behavioral mechanisms: (1) an increase in delay discounting (i.e., exacerbating the impact of differences in reinforcer delays) which would increase the value of a sooner reinforcer relative to a later one, or (2) a decrease in magnitude sensitivity (i.e., diminishing the impact of differences in reinforcer magnitudes) which would increase the value of a smaller reinforcer relative to a larger one. To isolate which of these two behavioral mechanisms was likely responsible for nicotine's effect on impulsive choice, we manipulated reinforcer delay and magnitude using a concurrent, variable interval (VI 30 s, VI 30 s) schedule of reinforcement with 2 groups of Long-Evans rats (n = 6 per group). For one group, choices were made between a 1-s delay and a 9-s delay to 2 food pellets. For a second group, choices were made between 1 pellet and 3 pellets. Nicotine (vehicle, 0.03, 0.1, 0.3, 0.56 and 0.74 mg/kg) produced dose-dependent decreases in preference for large versus small magnitude reinforcers and had no consistent effect on preference for short versus long delays. This suggests that nicotine decreases sensitivity to reinforcer magnitude.     

Relations between delay discounting and low to moderate gambling, cannabis, and alcohol problems among university students

Research has generally demonstrated that the discounting of delayed rewards is associated with severity of addictive behaviour. Less clear, however, is the relative strength of the relation for specific addictive behaviours. University students (N = 218) completed a computerized delay discounting task for hypothetical monetary rewards, and gambling, cannabis, and alcohol problem severity was assessed. A multiple regression analysis revealed that while the overall model was significant, only gambling problem severity accounted for delay discounting scores above and beyond cannabis and alcohol problem severity. The results support the hypothesis that delay discounting of hypothetical monetary rewards is more associated with gambling than other addictive behaviour problems, including substance use problems.     

Pathological gambling severity is associated with impulsivity in a delay discounting procedure

Research and clinical expertise indicates that impulsivity is an underlying feature of pathological gambling. This study examined the extent to which impulsive behavior, defined by the rate of discounting delayed monetary rewards, varies with pathological gambling severity, assessed by the South Oaks Gambling Screen (SOGS). Sixty-two pathological gamblers completed a delay discounting task, the SOGS, the Eysenck impulsivity scale, the Addiction Severity Index (ASI), and questions about gambling and substance use at intake to outpatient treatment for pathological gambling. In the delay discounting task, participants chose between a large delayed reward (US $1000) and smaller more immediate rewards (US $1-$999) across a range of delays (6h to 25 years). The rate at which the delayed reward was discounted (k value) was derived for each participant and linear regression was used to identify the variables that predicted k values. Age, gender, years of education, substance abuse treatment history, and cigarette smoking history failed to significantly predict k values. Scores on the Eysenck impulsivity scale and the SOGS both accounted for a significant proportion of the variance in k values. The predictive value of the SOGS was 1.4 times that of the Eysenck scale. These results indicate that of the measures tested, gambling severity was the best single predictor of impulsive behavior in a delay discounting task in this sample of pathological gamblers.     

Probability and delay discounting of hypothetical sexual outcomes

The present study used the discounting procedure to characterize choice behaviors regarding hypothetical sexual outcomes. Eighty-six adult undergraduate students completed computerized delay and probability discounting tasks concerning hypothetical money and hypothetical sexual activity. Consistent with other discounting findings, hyperbolic and hyperbola-like decay models described individual and group median indifference point data well. These findings contribute to a growing literature on the relevance of the discounting procedure to decision-making processes and suggest that the discounting procedure may be useful for understanding the processes that underlie social problem behaviors associated with impulsive sexual decisions.     

Impulsive choice in inbred strains of mice

When humans or non-humans are given a choice between receiving a sooner-smaller (SS) reinforcer, or a later-larger (LL) reinforcer, the choice of a SS reinforcer represents impulsive choice whereas the choice of a LL reinforcer represents self-controlled choice. It has been suggested that both biological and genetic factors influence impulsive/self-control choice in this paradigm. In the present study, the inbred strains of BALB/c, C57BL/6, and DBA/2 mice were given a choice to press one of two levers in an operant chamber. Depending on their choice, mice received either a smaller reinforcer (one pellet delivered after 6s) sooner (SS) or a larger reinforcer (two pellets after 6, 9, 12, 18, or 30s) later (LL). Mice preference for the larger reinforcer decreased with longer delays. More importantly, the BALB/c mice chose the SS reinforcer more often than the C57BL/6 mice under the 9- and 12-s delays, and more often than the DBA/2 mice under the 9-s delay. This indicates that the choice pattern of the BALB/c strain is more "impulsive" than the other strains and suggests that specific gene configurations influence impulsive choice in mice.     

Role of the orbital prefrontal cortex in choice between delayed and uncertain reinforcers: a quantitative analysis

'Inter-temporal choice' refers to choice between two or more outcomes that differ with respect to their sizes, delays, and/or probabilities of occurrence. According to the multiplicative hyperbolic model of inter-temporal choice, the value of a reinforcer increases as a hyperbolic function of its size, and decreases as a hyperbolic function of its delay and the odds against its occurrence. These functions, each of which contains a single discounting parameter, are assumed to combine multiplicatively to determine the overall value of the reinforcer. The model gives rise to a quantitative methodology for analysing inter-temporal choice, based on a family of linear null equations which describe performance under conditions of indifference, when the values of the reinforcers are assumed to be equal. This approach was used to examine the effect of lesions of the orbital prefrontal cortex (OPFC) on inter-temporal choice in rats. Under halothane anaesthesia, rats received injections of the excitotoxin quinolinate into the OPFC or sham lesions. They were trained to press two levers (A and B) for food-pellet reinforcers in discrete-trials schedules. In free-choice trials, a press on A resulted in delivery of a pellet after a delay d(A) with a probability P=0.5; a press on B resulted in delivery of a pellet with a probability P=1 after a delay d(B). d(B) was increased progressively across successive blocks of six trials in each session, while d(A) was manipulated systematically across phases of the experiment. The indifference delays, d(B(50)) (value of d(B) corresponding to 50% choice of B) was estimated for each rat in each phase. Linear functions of d(B(50)) versus d(A) were derived, and the parameters of the function compared between the groups. In both groups, d(B(50)) increased linearly with d(A). The slope of the linear function was significantly steeper in the lesioned group than in the sham-lesioned group, whereas the intercept did not differ significantly between the groups. Analysis based on the relevant null equation indicated that the lesion of the OPFC increased the rate of both delay and odds discounting. Possible implications of the results for interpreting the effects of OPFC lesions on inter-temporal choice behaviour in man are discussed.     

Functional parameters of delay discounting assessment tasks: order of presentation

Several variants in the methods used to estimate delay discounting (DD) have been associated with within-subject differences in degree of DD. This study compared variants in the order of presentation of reward and delay values during assessment of DD of hypothetical cash. Participants were randomly assigned to one of two groups. For one group, the immediate reward values were presented in Ascending order and for the other, they were presented in Descending order. In addition, all participants completed a DD task with the reward values presented in a Random order. Degree of DD, calculated as area under the curve (AUC), was similar between the Ascending and Descending procedures, and was significantly higher with the Random procedure. Within-subjects AUC were positively correlated. Reaction times within choice trials changed systematically as a function of order of presentation of the immediate rewards, and distance to the indifference point within each delay value. Reaction times appear to parallel the effort involved in making the individual choices. Some procedural variants in the assessment of DD yield differences in behavior during the assessment task that affect the magnitude of the estimated delay discounting. Thus, the absolute magnitude of DD may not be directly comparable between methods.     

Amount-dependent temporal discounting?

Amount-dependent temporal discounting has been demonstrated for human choice between outcomes differing in amount and delay. In the only study to date with non-humans, Grace reported no evidence for amount-dependent temporal discounting with pigeons in a concurrent-chains procedure. The present experiments repeated Grace's procedure but with modifications to enhance the discrimination between small and large magnitude outcomes. In Experiment 1, sensitivity of pigeons' initial-link choice to the terminal link delay ratio was greater with large reinforcer durations in the terminal links than with small reinforcer durations. This result is consistent with a greater rate of temporal discounting for larger reinforcers (the reverse of the result for humans), but can also be explained as enhanced discrimination of delay ratios with larger reinforcer durations. The results of a second experiment supported Grace's conclusion that amount-dependent temporal discounting does not characterize pigeons' choice in concurrent chains. Because reinforcer amount was held constant between choice alternatives in the present experiments and that of Grace, but varied in the human studies, our results question whether prior demonstrations of amount-dependent discounting reflect the effects of reinforcer delay or of reinforcer amount. Differences in the procedures used to study discounting in humans (titration procedures) and non-humans (concurrent chains) may contribute to the divergent results across species.     

Delay discounting of saccharin in rhesus monkeys

The value of a reinforcer decreases as the time until its receipt increases, a phenomenon referred to as delay discounting. Although delay discounting of non-drug reinforcers has been studied extensively in a number of species, our knowledge of discounting in non-human primates is limited. In the present study, rhesus monkeys were allowed to choose in discrete trials between 0.05% saccharin delivered in different amounts and with different delays. Indifference points were calculated and discounting functions were established. Discounting functions for saccharin were well described by a hyperbolic function. Moreover, the discounting rates for saccharin in all six monkeys were comparable to those of other non-human animals responding for non-drug reinforcers. Also consistent with other studies of non-human animals, changing the amount of a saccharin reinforcer available after a 10-s delay did not affect its relative subjective value. Discounting functions for saccharin were steeper than we found in a previous study with cocaine, raising the possibility that drugs such as cocaine may be discounted less steeply than non-drug reinforcers.     

Response bias is unaffected by delay length in a delay discounting paradigm

This study examined the contribution of response bias to measures of delay discounting in Long-Evans rats (n = 8) using the adjusting amount procedure. Under this procedure, we assessed preference for 150 μl of 10% sucrose solution delivered following a delay over a variable-amount alternative delivered immediately. Bias was calculated based on relative preference when reinforcers were delivered immediately from both alternatives. We extended this assessment procedure to examine preference when rewards from both alternatives were equally delayed (2, 4, 8, or 16 s) in addition to assessing a traditional delay discounting function. Relative preference was similar across delays and slightly larger than 150 μl. These results indicate that response bias was stable and suggests a relative aversion for the adjusting alternative, which may be due to the variability in reward size associated with that alternative.     

When is it adaptive to be patient? A general framework for evaluating delayed rewards

The tendency of animals to seek instant gratification instead of waiting for greater long-term benefits has been described as impatient, impulsive or lacking in self-control. How can we explain the evolution of such seemingly irrational behaviour? Here we analyse optimal behaviour in a variety of simple choice situations involving delayed rewards. We show that preferences for more immediate rewards should depend on a variety of factors, including whether the choice is a one-off or is likely to be repeated, the information the animal has about the continuing availability of the rewards and the opportunity to gain rewards through alternative activities. In contrast to the common assertion that rational animals should devalue delayed rewards exponentially, we find that this pattern of discounting is optimal only under restricted circumstances. We predict preference reversal whenever waiting for delayed rewards entails loss of opportunities elsewhere, but the direction of this reversal depends on whether the animal will face the same choice repeatedly. Finally, we question the ecological relevance of standard laboratory tests for impulsive behaviour, arguing that animals rarely face situations analogous to the self-control paradigm in their natural environment. To understand the evolution of impulsiveness, a more promising strategy would be to identify decision rules that are adaptive in a realistic ecological setting, and examine how these rules determine patterns of behaviour in simultaneous choice tests.     

Measurement of delay discounting using trial-by-trial consequences

DELAY DISCOUNTING IN HUMANS WAS INVESTIGATED USING THREE DIFFERENT PROCEDURES: a frequently used discounting procedure with hypothetical rewards and delays; a procedure with hypothetical rewards and delays compressed down to much smaller values; and a contingent procedure in which each choice had a direct consequence. In the contingent procedure, on every trial, participants actually experienced the delay and obtained the reward amount associated with their choice. Each participant was exposed to all three procedures. Orderly temporal discounting patterns were obtained in all three procedures and described well by a hyperbolic model. Comparisons of the data revealed patterns unique to each procedure. The distributions of the discounting measures differed across the three procedures. In the contingent procedure, several subjects showed no discounting, e.g. complete self-control. Procedural factors in studies of impulsivity are discussed, and suggestions are offered for experiments in which the contingent-discounting procedure may prove useful.     

Future altruism: Social discounting of delayed rewards

Social discounting assesses an individual's willingness to forgo an outcome for the self in lieu of a larger outcome for someone else. The purpose of the present research was to examine the effect of adding a common delay to outcomes in a binary choice, social discounting procedure. Based on the premise that both social and temporal distances are dimensions of psychological distance, we hypothesized that social discounting should decrease as a function of delay to the outcomes. Across two within-subject experiments, participants indicated preference between a hypothetical money reward for the self or for someone else. The outcomes were associated with no, short, and long delays. Both studies confirmed our hypothesis that adding any delay to the receipt of outcomes decreases social discounting, though no significant differences were observed between short and long delays. These results are discussed in the context of some existing literature on altruism.     

Temporal discounting and heart rate reactivity to stress

Temporal discounting is the reduction of the value of a reinforcer as a function of increasing delay to its presentation. Impulsive individuals discount delayed consequences more rapidly than self-controlled individuals, and impulsivity has been related to substance abuse, gambling, and other problem behaviors. A growing body of literature has identified biological correlates of impulsivity, though little research to date has examined relations between delay discounting and markers of poor health (e.g., cardiovascular reactivity to stress). We evaluated the relation between one aspect of impulsivity, measured using a computerized temporal discounting task, and heart rate reactivity, measured as a change in heart rate from rest during a serial subtraction task. A linear regression showed that individuals who were more reactive to stress responded more impulsively (i.e., discounted delayed reinforcers more rapidly). When results were stratified by gender, the effect was observed for females, but not for males. This finding supports previous research on gender differences in cardiovascular reactivity and suggests that this type of reactivity may be an important correlate of impulsive behavior.     

Within-subject comparison of degree of delay discounting using titrating and fixed sequence procedures

Different procedures are often used across experiments to estimate the degree of delay discounting, a common measure of impulsivity. In all procedures, participants indicate their choice between a reward available immediately and one available after a delay. The present experiment determined whether there are differences in the degree of discounting for a hypothetical $100 produced by a procedure that titrates the immediate amount (titrating sequence procedure) versus a procedure that presents a fixed sequence of immediate amounts (fixed sequence procedure) using a within-subject design. The adult human participants showed no significant differences in degree of discounting between procedures as assessed by a hyperboloid model and the Area Under the Curve. Furthermore, the Area Under the Curve values from the two procedures showed a strong positive correlation. These findings suggest there may be no systematic difference between the degree of delay discounting as estimated by the titrating sequence and fixed sequence procedures. Given the apparent similarities in the results, it appears researchers may be justified in basing their choice of which procedure to use on convenience.     

The combined effects of delay and probability in discounting

Human discounting studies have frequently observed hyperbolic discounting of rewards that are delayed or probabilistic. However, no studies have systematically combined delay and probability in a single discounting procedure. Indifference points of hypothetical money rewards that are both delayed and probabilistic were determined. Probabilities were converted into comparable delays according to the h/k constant of proportionality determined by , and discounting rates were calculated. These data provided a very good fit to the hyperbolic model of discounting, suggesting that delay and probability can be combined into a single metric in studies of discounting. The inclusion of a magnitude condition found the Magnitude Effect commonly found in studies of temporal discounting. A temporal resolution of uncertainty condition found no effect. The present paper offers a novel statistical method, within an established framework, for the analysis of data from studies of discounting that combine delay and probability.