Response bias is unaffected by delay length in a delay discounting paradigm

This study examined the contribution of response bias to measures of delay discounting in Long-Evans rats (n = 8) using the adjusting amount procedure. Under this procedure, we assessed preference for 150 μl of 10% sucrose solution delivered following a delay over a variable-amount alternative delivered immediately. Bias was calculated based on relative preference when reinforcers were delivered immediately from both alternatives. We extended this assessment procedure to examine preference when rewards from both alternatives were equally delayed (2, 4, 8, or 16 s) in addition to assessing a traditional delay discounting function. Relative preference was similar across delays and slightly larger than 150 μl. These results indicate that response bias was stable and suggests a relative aversion for the adjusting alternative, which may be due to the variability in reward size associated with that alternative.     

Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation

We monitored single-neuron activity in the orbitofrontal cortex of rats performing a time-discounting task in which the spatial location of the reward predicted whether the delay preceding reward delivery would be short or long. We found that rewards delivered after a short delay elicited a stronger neuronal response than those delivered after a long delay in most neurons. Activity in these neurons was not influenced by reward size when delays were held constant. This was also true for a minority of neurons that exhibited sustained increases in firing in anticipation of delayed reward. Thus, encoding of time-discounted rewards in orbitofrontal cortex is independent of the encoding of absolute reward value. These results are contrary to the proposal that orbitofrontal neurons signal the value of delayed rewards in a common currency and instead suggest alternative proposals for the role this region plays in guiding responses for delayed versus immediate rewards.     

Delay of gratification and delay discounting in rats

Delay discounting (DD) and delay of gratification (DG) are two measures of impulsive behavior often viewed as reflecting the same or equivalent processes. However, there are some key differences in the contingencies of reinforcement between the procedures that may have implications for understanding impulsivity. This study used DD and DG procedures to determine if differences in contingencies of reinforcement specified by DD and DG alters how much organisms discount the value of delayed reinforcers. Twenty-four water-deprived rats performed one of two Adjusting Amount procedures, which consisted of repeated choices between a fixed amount of water (250 &mgr;l) delivered after a delay (0, 4, 8, 16, or 32 s) and an adjusting, usually lesser amount delivered immediately. Half of the rats (n=12) performed a DD procedure designed to assess preference for immediate over delayed reinforcers in which they had discrete choices between the immediate and delayed amounts of water. A DG procedure was used for the other half of the rats (n=12). In the DG procedure rats also selected between immediate and delayed alternatives, but if they chose the delayed alternative they could switch to and receive the immediate alternative at any time during the delay to the larger reward. In the DD procedure switching responses were not reinforced but were still recorded and used for analyses. The DD functions of the two groups did not differ significantly. However, at the longer delays, the DG group made significantly fewer switching responses than the DD group. A possible role of response inhibition in the DG procedure is discussed.     

Discounting of sequences of delayed rewards of different amounts

The main purpose of this study was to determine whether the magnitude effect is present in cases where delayed sequences of rewards are discounted. The magnitude effect refers to the inverse relationship between the amount of a reward and the steepness of temporal discounting. This study was conducted with a computer program to estimate the indifference points, which served as indicators of the present subjective value of delayed sequences of small and large rewards. In the indifference point the subjective value of a single, immediate reward was equal to the subjective value of the delayed sequence (or to the value of a single delayed reward). As a control condition, we added an experimental task involving choices between single immediate and single delayed rewards. The experiment showed that the sequences of large rewards are discounted less steeply than are the sequences of small rewards. This finding suggests that the magnitude effect is present within the delayed sequences of rewards. In addition, when outcomes are relatively large, the results suggest that a single reward is discounted less steeply than the sequence of a total nominal value equal to this single reward. However, for relatively small rewards, the difference is not statistically significant. The less steep discounting of sequences of large rewards may explain the reward-bundling effect, which refers to less steep discounting of longer sequences than of shorter ones: longer sequences usually have greater overall nominal value. The present study was conducted on hypothetical rewards, and the results should be validated using real rewards.     

Prefrontal coding of temporally discounted values during intertemporal choice

Reward from a particular action is seldom immediate, and the influence of such delayed outcome on choice decreases with delay. It has been postulated that when faced with immediate and delayed rewards, decision makers choose the option with maximum temporally discounted value. We examined the preference of monkeys for delayed reward in an intertemporal choice task and the neural basis for real-time computation of temporally discounted values in the dorsolateral prefrontal cortex. During this task, the locations of the targets associated with small or large rewards and their corresponding delays were randomly varied. We found that prefrontal neurons often encoded the temporally discounted value of reward expected from a particular option. Furthermore, activity tended to increase with [corrected] discounted values for targets [corrected] presented in the neuron's preferred direction, suggesting that activity related to temporally discounted values in the prefrontal cortex might determine the animal's behavior during intertemporal choice.     

Single units in the pigeon brain integrate reward amount and time-to-reward in an impulsive choice task

BACKGROUND: Animals prefer small over large rewards when the delays preceding large rewards exceed an individual tolerance limit. Such impulsive choice behavior occurs even in situations in which alternative strategies would yield more optimal outcomes. Behavioral research has shown that an animal's choice is guided by the alternative rewards' subjective values, which are a function of reward amount and time-to-reward. Despite increasing knowledge about the pharmacology and anatomy underlying impulsivity, it is still unknown how the brain combines reward amount and time-to-reward information to represent subjective reward value. RESULTS: We trained pigeons to choose between small, immediate rewards and large rewards delivered after gradually increasing delays. Single-cell recordings in the avian Nidopallium caudolaterale, the presumed functional analog of the mammalian prefrontal cortex, revealed that neural delay activation decreased with increasing delay length but also covaried with the expected reward amount. This integrated neural response was modulated by reward amount and delay, as predicted by a hyperbolical equation, of subjective reward value derived from behavioral studies. Furthermore, the neural activation pattern reflected the current reward preference and the time point of the shift from large to small rewards. CONCLUSIONS: The reported activity was modulated by the temporal devaluation of the anticipated reward in addition to reward amount. Our findings contribute to the understanding of neuropathologies such as drug addiction, pathological gambling, frontal lobe syndrome, and attention-deficit disorders, which are characterized by inappropriate temporal discounting and increased impulsiveness.     

Floral reward production is timed by an insect pollinator

Interval timing--sensitivity to elapsing durations--has recently been found to occur in an invertebrate pollinator, the bumble-bee (Bombus impatiens). Here, bumble-bees were required to time the interval between the start of foraging in a patch of low-quality artificial flowers providing 25% sucrose and the availability of a high-quality flower (HQF) that provided 50% sucrose after a fixed delay. The delay changed after every 20 foraging bouts in the order 30-150-30 s. Bees visited the HQF sooner when the delay was 30 s than when it was 150 s, and visits to the HQF peaked near the end of both delays. When the delay changed to 150 s, bees appeared to time both the previous 30 s delay and the new delay. To examine whether bees also learned what kind of reward was provided at the HQF, its usual reward was replaced with 25% sucrose in a final foraging bout. Bumble-bees rejected the HQF on the reward-replacement test. These results show that bumble-bees remembered both when reward was produced by the HQF and what type of reward was produced. These findings indicate that bumble-bees can learn both the timing and content of reward production.     

Parameters of delay discounting assessment: number of trials, effort, and sequential effects.

Procedural variants in estimating delay discounting (DD) have been shown to yield significant within-subject differences in estimated degree of delay discounting as well as variations in the patterns of choice. The purpose of this study was to evaluate the effect of subject control over the number of trials in a delay discounting task, on degree of delay discounting. Participants were assessed with two computerized DD assessments: the full-length method presented participants with a fixed set of 240 trials, and the abbreviated task, where once participants had shown indifference between the immediate and delayed rewards, the remaining trials for that delay value were omitted. While the full-length and abbreviated methods did not differentially affect patterns of choice or estimated delay discounting, the order of presentation (ascending or descending) of immediate rewards produced differences in each measure: rate of delay discounting was significantly lower when estimated with the descending sequence; a larger proportion of area under the discounting curve was concentrated around the indifference point trial with the descending sequence; and a lower correlation was observed between estimates obtained across methods with the descending sequence.     

Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice

In choosing between different rewards expected after unequal delays, humans and animals often prefer the smaller but more immediate reward, indicating that the subjective value or utility of reward is depreciated according to its delay. Here, we show that neurons in the primate caudate nucleus and ventral striatum modulate their activity according to temporally discounted values of rewards with a similar time course. However, neurons in the caudate nucleus encoded the difference in the temporally discounted values of the two alternative targets more reliably than neurons in the ventral striatum. In contrast, neurons in the ventral striatum largely encoded the sum of the temporally discounted values, and therefore, the overall goodness of available options. These results suggest a more pivotal role for the dorsal striatum in action selection during intertemporal choice.     

Reduced sensitivity to immediate reward during decision-making in older than younger adults

We examined whether older adults differ from younger adults in the degree to which they favor immediate over delayed rewards during decision-making. To examine the neural correlates of age-related differences in delay discounting we acquired functional MR images while participants made decisions between smaller but sooner and larger but later monetary rewards. The behavioral results show age-related reductions in delay discounting. Less impulsive decision-making in older adults was associated with lower ventral striatal activations to immediate reward. Furthermore, older adults showed an overall higher percentage of delayed choices and reduced activity in the dorsal striatum than younger adults. This points to a reduced reward sensitivity of the dorsal striatum in older adults. Taken together, our findings indicate that less impulsive decision-making in older adults is due to a reduced sensitivity of striatal areas to reward. These age-related changes in reward sensitivity may result from transformations in dopaminergic neuromodulation with age.     

Delay discounting and gambling

Delay discounting describes the decline in the value of a reinforcer as the delay to that reinforcer increases. A review of the available studies revealed that steep delay discounting is positively correlated with problem or pathological gambling. One hypothesis regarding this correlation derives from the discounting equation proposed by Mazur (1989). According to the equation, steeper discounting renders the difference between fixed-delayed rewards and gambling-like variable-delayed rewards larger; with the latter being more valuable. The present study was designed to test this prediction by first assessing rats’ impulsive choices across four delays to a larger-later reinforcer. A second condition quantified strength of preference for mixed- over fixed-delays, with the duration of the latter adjusted between sessions to achieve indifference. Strength of preference for the mixed-delay alternative is given by the fixed delay at indifference (lower fixed-delay values reflect stronger preferences). Percent impulsive choice was not correlated with the value of the fixed delay at indifference and, therefore, the prediction of the hyperbolic model of gambling was not supported. A follow-up assessment revealed a significant decrease in impulsive choice after the second condition. This shift in impulsive choice could underlie the failure to observe the predicted correlation between impulsive choice and degree of preference for mixed- over fixed delays.     

The discounting-by-interruptions hypothesis: model and experiment

Experimental animals often prefer small but immediate rewardseven when larger-delayed rewards provide a higher rate of intake.This impulsivity has important implications for models of foragingand cooperation. Behavioral ecologists have hypothesized thatanimals discount delayed rewards because delay imposes a collectionrisk. According to this long-standing hypothesis, delay reducesvalue because an interruption that occurs while an animal iswaiting may prevent it from collecting the delayed reward. Althoughthere have been many experimental demonstrations of animal preferencesfor immediacy, none have included any interruptions. This paperdevelops a simple model of discounting by interruptions andthen tests this model experimentally. The model considers theeffects of interruption rate and duration on choice behavior.The experiment tests the effects of interruptions on the choicebehavior of captive blue jays (Cyanocitta cristata) using afactorial design that manipulates the rate and duration of interruptions.The results do not support the discounting-by-interruptionshypothesis. This represents one of several lines of evidencesuggesting that investigators should seek alternative explanationsof the animal impulsivity.     

Delay of gratification by chimpanzees (Pan troglodytes) in working and waiting situations

We tested four chimpanzees in a self-control task in which food rewards accumulated as long as they were not eaten. In one condition, the chimpanzees had to perform a computer task that directly led to the delivery of the food rewards. In another condition, working on the computerized task was not required and any such work was not linked to the delivery of rewards. The third condition offered no computerized task (chimpanzees simply waited for food rewards to be delivered). Three of four chimpanzees showed no effect of the work scenario on delay of gratification. The one chimpanzee that showed an influence of work scenario on self-control was the overall poorest performing animal. This animal delayed gratification the longest, however, when work was required and reward delivery was directly linked to that work. Therefore, although there is little evidence linking delay of gratification to work requirements in chimpanzees, chimpanzees with lower overall self-control might benefit from having some work available if reward accumulation is contingent on performing that work.     

Pathological gambling severity is associated with impulsivity in a delay discounting procedure

Research and clinical expertise indicates that impulsivity is an underlying feature of pathological gambling. This study examined the extent to which impulsive behavior, defined by the rate of discounting delayed monetary rewards, varies with pathological gambling severity, assessed by the South Oaks Gambling Screen (SOGS). Sixty-two pathological gamblers completed a delay discounting task, the SOGS, the Eysenck impulsivity scale, the Addiction Severity Index (ASI), and questions about gambling and substance use at intake to outpatient treatment for pathological gambling. In the delay discounting task, participants chose between a large delayed reward (US $1000) and smaller more immediate rewards (US $1-$999) across a range of delays (6h to 25 years). The rate at which the delayed reward was discounted (k value) was derived for each participant and linear regression was used to identify the variables that predicted k values. Age, gender, years of education, substance abuse treatment history, and cigarette smoking history failed to significantly predict k values. Scores on the Eysenck impulsivity scale and the SOGS both accounted for a significant proportion of the variance in k values. The predictive value of the SOGS was 1.4 times that of the Eysenck scale. These results indicate that of the measures tested, gambling severity was the best single predictor of impulsive behavior in a delay discounting task in this sample of pathological gamblers.     

The ecology and evolution of patience in two New World monkeys

Decision making often involves choosing between small, short-term rewards and large, long-term rewards. All animals, humans included, discount future rewards--the present value of delayed rewards is viewed as less than the value of immediate rewards. Despite its ubiquity, there exists considerable but unexplained variation between species in their capacity to wait for rewards--that is, to exert patience or self-control. Using two closely related primates--common marmosets (Callithrix jacchus) and cotton-top tamarins (Saguinus oedipus)--we uncover a variable that may explain differences in how species discount future rewards. Both species faced a self-control paradigm in which individuals chose between taking an immediate small reward and waiting a variable amount of time for a large reward. Under these conditions, marmosets waited significantly longer for food than tamarins. This difference cannot be explained by life history, social behaviour or brain size. It can, however, be explained by feeding ecology: marmosets rely on gum, a food product acquired by waiting for exudate to flow from trees, whereas tamarins feed on insects, a food product requiring impulsive action. Foraging ecology, therefore, may provide a selective pressure for the evolution of self-control.     

Get It While It’s Hot: A Peak-First Bias in Self-Generated Choice Order in Rhesus Macaques

Animals typically must make a number of successive choices to achieve a goal: e.g., eating multiple food items before becoming satiated. However, it is unclear whether choosing the best first or saving the best for last represents the best choice strategy to maximize overall reward. Specifically, since outcomes can be evaluated prospectively (with future rewards discounted and more immediate rewards preferred) or retrospectively (with prior rewards discounted and more recent rewards preferred), the conditions under which each are used remains unclear. On the one hand, humans and non-human animals clearly discount future reward, preferring immediate rewards to delayed ones, suggesting prospective evaluation; on the other hand, it has also been shown that a sequence that ends well, i.e., with the best event or item last, is often preferred, suggesting retrospective evaluation. Here we hypothesized that when individuals are allowed to build the sequence themselves they are more likely to evaluate each item individually and therefore build a sequence using prospective evaluation. We examined the relationship between self-generated choice order and preference in rhesus monkeys in two experiments in which the distinctiveness of options were relatively high and low, respectively. We observed a positive linear relationship between choice order and preference among highly distinct options, indicating that the rhesus monkeys chose their preferred food first: i.e., a peak-first order preference. Overall, choice order depended on the degree of relative preference among alternatives and a peak-first bias, providing evidence for prospective evaluation when choice order is self-generated.     

Ability to delay gratification and BMI in preadolescence

Delay of gratification tasks require an individual to forgo an immediate reward and wait for a more desirable delayed reward. This study used an ecologically valid measure of delayed gratification to test the hypothesis that preadolescents with higher BMI would be less likely to delay gratification. Healthy Hawks is a 12-week educational/behavioral obesity intervention at the University of Kansas Medical Center. Each week, children earn a point if they complete their goals worksheet. They can spend that point immediately on a small toy prize or save points to use on a larger prize. We retrospectively calculated the percentage of points saved over the 12 weeks for 59 children (28 females) ages 8-12 years old (mean = 10.29 ± 1.39). Spearman correlation revealed that higher BMI percentile was associated with reduced point savings (r = 0.33, P = 0.01). Similarly, obese preadolescents saved significantly fewer points than healthy weight (HW) and overweight preadolescents (t (57) = 3.14, P < 0.01). Results from our ecologically valid measure support the theory that obese children are less likely to delay gratification than overweight and HW children. Even for nonfood rewards, preadolescent children with higher BMIs prefer the immediate reward over a delayed, larger reward. This has implications for developing specific strategies within obesity treatments aimed at improving delayed gratification.     

Strain differences in delay discounting using inbred rats

A heightened aversion to delayed rewards is associated with substance abuse and numerous other neuropsychiatric disorders. Many of these disorders are heritable, raising the possibility that delay aversion may also have a significant genetic or heritable component. To examine this possibility, we compared delay discounting in six inbred strains of rats (Brown Norway, Copenhagen, Lewis, Fischer, Noble and Wistar Furth) using the adjusting amount procedure, which provides a measure of the subjective value of delayed rewards. The subjective value of rewards decreased as the delay to receipt increased for all strains. However, a main effect of strain and a strain × delay interaction indicated that some strains were more sensitive to the imposition of delays than others. Fitting a hyperbolic discount equation showed significant strain differences in sensitivity to delay ( k ). These data indicate that there are significant strain differences in delay discounting. All strains strongly preferred the 10% sucrose solution (the reinforcer in the delay discounting task) over water and the amount of sucrose consumed was correlated with sensitivity to delay. Locomotor activity was not correlated with delay discounting behavior. Additional research will be required to disentangle genetic influences from maternal effects and to determine how these factors influence the underlying association between heightened delay discounting and neuropsychiatric disorders.     

Effects of reinforcer magnitude on an animal model of impulsive behavior

The effects of altering sucrose solution concentration on discounting of delayed rewards in rats were examined. Five different delays were used (0, 1, 2, 4, and 8s) and three different sucrose solution concentrations (3, 10, and 30%). It was hypothesized that high value sucrose solution concentrations would be discounted less than low value sucrose solution concentrations. The results indicated that the rats discounted the 30% sucrose solution concentration at a higher rate than the 3 or 10% sucrose solution concentration, a finding that apparently contradicted the hypothesis that higher value sucrose solution concentrations would be discounted less than lower value sucrose solution concentrations. However, a follow up experiment indicated that the 3 and 10% sucrose solution concentrations were preferred over the 30% concentration. Thus the results of Experiment 1 can be interpreted as supporting the hypothesis that high valued sucrose solution concentrations are discounted less than lower valued sucrose solution concentrations.     

Functional parameters of delay discounting assessment tasks: order of presentation

Several variants in the methods used to estimate delay discounting (DD) have been associated with within-subject differences in degree of DD. This study compared variants in the order of presentation of reward and delay values during assessment of DD of hypothetical cash. Participants were randomly assigned to one of two groups. For one group, the immediate reward values were presented in Ascending order and for the other, they were presented in Descending order. In addition, all participants completed a DD task with the reward values presented in a Random order. Degree of DD, calculated as area under the curve (AUC), was similar between the Ascending and Descending procedures, and was significantly higher with the Random procedure. Within-subjects AUC were positively correlated. Reaction times within choice trials changed systematically as a function of order of presentation of the immediate rewards, and distance to the indifference point within each delay value. Reaction times appear to parallel the effort involved in making the individual choices. Some procedural variants in the assessment of DD yield differences in behavior during the assessment task that affect the magnitude of the estimated delay discounting. Thus, the absolute magnitude of DD may not be directly comparable between methods.