On the scaling interpretation of exponents in hyperboloid models of delay and probability discounting

Previously, we (McKerchar et al., 2009) showed that two-parameter hyperboloid models ( [Green and Myerson, 2004] and [Rachlin, 2006]) provide significantly better fits to delay discounting data than simple, one-parameter hyperbolic and exponential models. Here, we extend this effort by comparing fits of the two-parameter hyperboloid models to data from a larger sample of participants (N = 171) who discounted probabilistic as well as delayed rewards. In particular, we examined the effects of amount on the exponents in the two hyperboloid models of delay and probability discounting in order to evaluate key theoretical predictions of the standard psychophysical scaling interpretation of these exponents. Both the Rachlin model and the Green and Myerson model provided very good fits to delay and probability discounting of both small and large amounts at both the group and individual levels (all R²s > .97 at the group level; all median R²s > .92 at the individual level). For delay discounting, the exponent in both models did not vary as a function of delayed amount, consistent with the psychophysical scaling interpretation. For probability discounting, however, the exponent in both models increased as the probabilistic amount increased—a finding inconsistent with the scaling interpretation.     

Rats bred for high alcohol drinking are more sensitive to delayed and probabilistic outcomes

Alcoholics and heavy drinkers score higher on measures of impulsivity than nonalcoholics and light drinkers. This may be because of factors that predate drug exposure (e.g. genetics). This study examined the role of genetics by comparing impulsivity measures in ethanol-naive rats selectively bred based on their high [high alcohol drinking (HAD)] or low [low alcohol drinking (LAD)] consumption of ethanol. Replicates 1 and 2 of the HAD and LAD rats, developed by the University of Indiana Alcohol Research Center, completed two different discounting tasks. Delay discounting examines sensitivity to rewards that are delayed in time and is commonly used to assess 'choice' impulsivity. Probability discounting examines sensitivity to the uncertain delivery of rewards and has been used to assess risk taking and risk assessment. High alcohol drinking rats discounted delayed and probabilistic rewards more steeply than LAD rats. Discount rates associated with probabilistic and delayed rewards were weakly correlated, while bias was strongly correlated with discount rate in both delay and probability discounting. The results suggest that selective breeding for high alcohol consumption selects for animals that are more sensitive to delayed and probabilistic outcomes. Sensitivity to delayed or probabilistic outcomes may be predictive of future drinking in genetically predisposed individuals.     

Delay discounting and gambling

Delay discounting describes the decline in the value of a reinforcer as the delay to that reinforcer increases. A review of the available studies revealed that steep delay discounting is positively correlated with problem or pathological gambling. One hypothesis regarding this correlation derives from the discounting equation proposed by Mazur (1989). According to the equation, steeper discounting renders the difference between fixed-delayed rewards and gambling-like variable-delayed rewards larger; with the latter being more valuable. The present study was designed to test this prediction by first assessing rats’ impulsive choices across four delays to a larger-later reinforcer. A second condition quantified strength of preference for mixed- over fixed-delays, with the duration of the latter adjusted between sessions to achieve indifference. Strength of preference for the mixed-delay alternative is given by the fixed delay at indifference (lower fixed-delay values reflect stronger preferences). Percent impulsive choice was not correlated with the value of the fixed delay at indifference and, therefore, the prediction of the hyperbolic model of gambling was not supported. A follow-up assessment revealed a significant decrease in impulsive choice after the second condition. This shift in impulsive choice could underlie the failure to observe the predicted correlation between impulsive choice and degree of preference for mixed- over fixed delays.     

Strain differences in delay discounting using inbred rats

A heightened aversion to delayed rewards is associated with substance abuse and numerous other neuropsychiatric disorders. Many of these disorders are heritable, raising the possibility that delay aversion may also have a significant genetic or heritable component. To examine this possibility, we compared delay discounting in six inbred strains of rats (Brown Norway, Copenhagen, Lewis, Fischer, Noble and Wistar Furth) using the adjusting amount procedure, which provides a measure of the subjective value of delayed rewards. The subjective value of rewards decreased as the delay to receipt increased for all strains. However, a main effect of strain and a strain × delay interaction indicated that some strains were more sensitive to the imposition of delays than others. Fitting a hyperbolic discount equation showed significant strain differences in sensitivity to delay ( k ). These data indicate that there are significant strain differences in delay discounting. All strains strongly preferred the 10% sucrose solution (the reinforcer in the delay discounting task) over water and the amount of sucrose consumed was correlated with sensitivity to delay. Locomotor activity was not correlated with delay discounting behavior. Additional research will be required to disentangle genetic influences from maternal effects and to determine how these factors influence the underlying association between heightened delay discounting and neuropsychiatric disorders.     

Relations between delay discounting and low to moderate gambling, cannabis, and alcohol problems among university students

Research has generally demonstrated that the discounting of delayed rewards is associated with severity of addictive behaviour. Less clear, however, is the relative strength of the relation for specific addictive behaviours. University students (N = 218) completed a computerized delay discounting task for hypothetical monetary rewards, and gambling, cannabis, and alcohol problem severity was assessed. A multiple regression analysis revealed that while the overall model was significant, only gambling problem severity accounted for delay discounting scores above and beyond cannabis and alcohol problem severity. The results support the hypothesis that delay discounting of hypothetical monetary rewards is more associated with gambling than other addictive behaviour problems, including substance use problems.     

The present values of delayed rewards are approximately additive

In two experiments, human subjects were asked to estimate their present values of single delayed rewards and their present values of temporal sequences of three rewards. Present values were solicited by asking subjects to indicate an amount of money v for which they would be indifferent between receiving v at the end of the session and receiving the delayed reward(s). A procedure was used for which responding the true value of v was the optimal strategy, and the actual payoff that each subject received was determined by one randomly selected trial. In Experiment 1 (n=29) each delayed reward was 9.90 dollars in cash. In Experiment 2 (n=19) the delayed rewards were dated 15 dollars gift certificates to a local restaurant. In both experiments, the present values of the sequences were approximately equal to the sums of the present values of their component rewards. The presence of outliers suggests that a few subjects may have valued sequences less than the sums of their single rewards. Effects of a preference for uniform sequences, if any, were too small to be detected. Discounting of sequences was well fit by a parallel hyperbolic discounting equation, consistent with Mazur's [Mazur, J.E., 1986. Choice between single and multiple delayed reinforcers. J. Exp. Anal. Behav. 46 (1), 67-77] results using multiple reinforcers.     

Parameters of delay discounting assessment: number of trials, effort, and sequential effects.

Procedural variants in estimating delay discounting (DD) have been shown to yield significant within-subject differences in estimated degree of delay discounting as well as variations in the patterns of choice. The purpose of this study was to evaluate the effect of subject control over the number of trials in a delay discounting task, on degree of delay discounting. Participants were assessed with two computerized DD assessments: the full-length method presented participants with a fixed set of 240 trials, and the abbreviated task, where once participants had shown indifference between the immediate and delayed rewards, the remaining trials for that delay value were omitted. While the full-length and abbreviated methods did not differentially affect patterns of choice or estimated delay discounting, the order of presentation (ascending or descending) of immediate rewards produced differences in each measure: rate of delay discounting was significantly lower when estimated with the descending sequence; a larger proportion of area under the discounting curve was concentrated around the indifference point trial with the descending sequence; and a lower correlation was observed between estimates obtained across methods with the descending sequence.     

Domain independence and stability in young and older adults' discounting of delayed rewards

Individual discounting rates for different types of delayed reward are typically assumed to reflect a single, underlying trait of impulsivity. Recently, we showed that discounting rates are orders of magnitude steeper for directly consumable liquid rewards than for monetary rewards (Jimura et al., 2009), raising the question of whether discounting rates for different types of reward covary at the individual level. Accordingly, the present study examined the relation between discounting of hypothetical money and real liquid rewards in young adults (Experiment 1) and older adults (Experiment 2). At the group level, young adults discounted monetary rewards more steeply than the older adults, but there was no significant age difference with respect to liquid rewards. At the individual level, the rates at which young and older participants discounted each reward type were stable over a two- to fifteen-week interval (rs > 70), but there was no significant correlation between the rates at which they discounted the two reward types. These results suggest that although similar decision-making processes may underlie the discounting of different types of rewards, the rates at which individuals discount money and directly consumable rewards may reflect separate, stable traits, rather than a single trait of impulsivity.     

The subjective value of delayed and probabilistic outcomes: Outcome size matters for gains but not for losses

The subjective value of a reward (gain) is related to factors such as its size, the delay to its receipt and the probability of its receipt. We examined whether the subjective value of losses was similarly affected by these factors in 128 adults. Participants chose between immediate/certain gains or losses and larger delayed/probabilistic gains or losses. Rewards of $100 were devalued as a function of their delay (“discounted”) relatively less than $10 gains while probabilistic $100 rewards were discounted relatively more than $10 rewards. However, there was no effect of outcome size on discounting of delayed or probabilistic losses. For delayed outcomes of each size, the degree to which gains were discounted was positively correlated with the degree to which losses were discounted, whereas for probabilistic outcomes, no such correlation was observed. These results suggest that the processes underlying the subjective valuation of losses are different from those underlying the subjective valuation of gains.     

Using crowdsourcing to examine relations between delay and probability discounting

Although the extensive lines of research on delay and/or probability discounting have greatly expanded our understanding of human decision-making processes, the relation between these two phenomena remains unclear. For example, some studies have reported robust associations between delay and probability discounting, whereas others have failed to demonstrate a consistent relation between the two. The current study sought to clarify this relation by examining the relation between delay and probability discounting in a large sample of internet users (n = 904) using the Amazon Mechanical Turk (AMT) crowdsourcing service. Because AMT is a novel data collection platform, the findings were validated through the replication of a number of previously established relations (e.g., relations between delay discounting and cigarette smoking status). A small but highly significant positive correlation between delay and probability discounting rates was obtained, and principal component analysis suggested that two (rather than one) components were preferable to account for the variance in both delay and probability discounting. Taken together, these findings suggest that delay and probability discounting may be related, but are not manifestations of a single construct (e.g., impulsivity).     

Impulsivity, risk taking, and timing

This study examined the relations among measures of impulsivity and timing. Impulsivity was assessed using delay and probability discounting, and self-report impulsivity (as measured by the Barratt Impulsiveness Scale; BIS-11). Timing was assessed using temporal perception as measured on a temporal bisection task and time perspective (as measured by the Zimbardo Time Perspective Inventory). One hundred and forty three college students completed these measures in a computer laboratory. The degree of delay discounting was positively correlated with the mean and range of the temporal bisection procedure. The degree of delay and probability discounting were also positively correlated. Self-reported motor impulsiveness on the BIS-11 was positively correlated with present hedonism and negatively correlated with future orientation on the ZTPI. Self-reported non-planning on the BIS-11 was positively correlated with fatalism on the ZTPI. These results show that people who overestimate the passage of time (perceive time as passing more quickly) hold less value in delayed rewards. They also confirm previous results regarding the relation between delay and probability discounting, as well as highlight similarities in self-report measures of impulsivity and time perspective.     

The discounting-by-interruptions hypothesis: model and experiment

Experimental animals often prefer small but immediate rewardseven when larger-delayed rewards provide a higher rate of intake.This impulsivity has important implications for models of foragingand cooperation. Behavioral ecologists have hypothesized thatanimals discount delayed rewards because delay imposes a collectionrisk. According to this long-standing hypothesis, delay reducesvalue because an interruption that occurs while an animal iswaiting may prevent it from collecting the delayed reward. Althoughthere have been many experimental demonstrations of animal preferencesfor immediacy, none have included any interruptions. This paperdevelops a simple model of discounting by interruptions andthen tests this model experimentally. The model considers theeffects of interruption rate and duration on choice behavior.The experiment tests the effects of interruptions on the choicebehavior of captive blue jays (Cyanocitta cristata) using afactorial design that manipulates the rate and duration of interruptions.The results do not support the discounting-by-interruptionshypothesis. This represents one of several lines of evidencesuggesting that investigators should seek alternative explanationsof the animal impulsivity.     

Delay discounting: trait variable?

Delay discounting refers to the tendency for outcomes that are remote in time to have less value than more immediate outcomes. Steep discounting of delayed outcomes is associated with a variety of social maladies. The degree of sensitivity to delayed outcomes may be a stable and pervasive individual characteristic. In analyses of archival data, the present study found positive correlations between the degree of delay discounting for one outcome (as measured by the Area Under the Curve), and the degree of discounting for other outcomes. Along with additional evidence reviewed, these data suggest that delay discounting may be considered a personality trait. Recent research in epigenetics, neuroscience, and behavior suggests delay discounting may prove to be a beneficial target for therapeutic attempts to produce global reductions in impulsivity related to delay discounting.     

Within-subject differences in degree of delay discounting as a function of order of presentation of hypothetical cash rewards

Procedural variants in estimating delay discounting (DD) have been shown to yield significant differences in estimated degree of DD as well as variations in individual patterns of choice. For example, a recent study found significantly different degrees of DD between groups assessed using either an ascending or descending order of presentation of the immediately available rewards. The purpose of this study was to test for within-subject effects of order of presentation of the immediate rewards in a DD task. In a single session, college students (N = 29) were asked to complete two DD tasks, one with the immediate rewards presented in ascending order and one in descending order. Consistent with previous results, significantly larger mean area under the discounting curve (AUC) was observed when the descending sequence was used compared to the ascending order of presentation; and the correlation between both measurements was moderate. These results suggest that some DD assessment tasks may be sensitive to contextual variables such as order and range of the reward and delay values.     

Estimating the subjective value of future rewards: comparison of adjusting-amount and adjusting-delay procedures

The present study examined whether equivalent discounting of delayed rewards is observed with different experimental procedures. If the underlying decision-making process is the same, then similar patterns of results should be observed regardless of procedure, and similar estimates of the subjective value of future rewards (i.e., indifference points) should be obtained. Two experiments compared discounting on three types of procedure: adjusting-delay (AD), adjusting-immediate-amount (AIA), and adjusting-delayed-amount (ADA). For the two procedures for which discounting functions can be established (i.e., AD and AIA), a hyperboloid provided good fits to the data at both the group and individual levels, and individuals' discounting on one procedure tended to be correlated with their discounting on the other. Notably, the AIA procedure produced the more consistent estimates of the degree of discounting, and in particular, discounting on the AIA procedure was unaffected by the order in which choices were presented. Regardless of which of the three procedures was used, however, similar patterns of results were obtained: Participants systematically discounted the value of delayed rewards, and robust magnitude effects were observed. Although each procedure may have its own advantages and disadvantages, use of all three types of procedure in the present study provided converging evidence for common decision-making processes underlying the discounting of delayed rewards.     

Discounting of sequences of delayed rewards of different amounts

The main purpose of this study was to determine whether the magnitude effect is present in cases where delayed sequences of rewards are discounted. The magnitude effect refers to the inverse relationship between the amount of a reward and the steepness of temporal discounting. This study was conducted with a computer program to estimate the indifference points, which served as indicators of the present subjective value of delayed sequences of small and large rewards. In the indifference point the subjective value of a single, immediate reward was equal to the subjective value of the delayed sequence (or to the value of a single delayed reward). As a control condition, we added an experimental task involving choices between single immediate and single delayed rewards. The experiment showed that the sequences of large rewards are discounted less steeply than are the sequences of small rewards. This finding suggests that the magnitude effect is present within the delayed sequences of rewards. In addition, when outcomes are relatively large, the results suggest that a single reward is discounted less steeply than the sequence of a total nominal value equal to this single reward. However, for relatively small rewards, the difference is not statistically significant. The less steep discounting of sequences of large rewards may explain the reward-bundling effect, which refers to less steep discounting of longer sequences than of shorter ones: longer sequences usually have greater overall nominal value. The present study was conducted on hypothetical rewards, and the results should be validated using real rewards.     

Response bias is unaffected by delay length in a delay discounting paradigm

This study examined the contribution of response bias to measures of delay discounting in Long-Evans rats (n = 8) using the adjusting amount procedure. Under this procedure, we assessed preference for 150 μl of 10% sucrose solution delivered following a delay over a variable-amount alternative delivered immediately. Bias was calculated based on relative preference when reinforcers were delivered immediately from both alternatives. We extended this assessment procedure to examine preference when rewards from both alternatives were equally delayed (2, 4, 8, or 16 s) in addition to assessing a traditional delay discounting function. Relative preference was similar across delays and slightly larger than 150 μl. These results indicate that response bias was stable and suggests a relative aversion for the adjusting alternative, which may be due to the variability in reward size associated with that alternative.     

The combined effects of delay and probability in discounting

Human discounting studies have frequently observed hyperbolic discounting of rewards that are delayed or probabilistic. However, no studies have systematically combined delay and probability in a single discounting procedure. Indifference points of hypothetical money rewards that are both delayed and probabilistic were determined. Probabilities were converted into comparable delays according to the h/k constant of proportionality determined by , and discounting rates were calculated. These data provided a very good fit to the hyperbolic model of discounting, suggesting that delay and probability can be combined into a single metric in studies of discounting. The inclusion of a magnitude condition found the Magnitude Effect commonly found in studies of temporal discounting. A temporal resolution of uncertainty condition found no effect. The present paper offers a novel statistical method, within an established framework, for the analysis of data from studies of discounting that combine delay and probability.     

Delay discounting and probability discounting as related to cigarette smoking status in adults

This study examined relations between adult smokers and non-smokers and the devaluation of monetary rewards as a function of delay (delay discounting, DD) or probability (probability discounting, PD). The extent to which individuals discount value, either as a function of a reward being delayed or probabilistic, has been taken to reflect individual differences in impulsivity. Those who discount most are considered most impulsive. Previous research has shown that adult smokers discount the value of delayed rewards more than adult non-smokers. However, in the one published study that examined probability discounting in adult smokers and non-smokers, the smokers did not discount the value of probabilistic rewards more than the non-smoker controls. From this past research, it was hypothesized that measures of delay discounting would differentiate between smokers and non-smokers but that probability discounting would not. Participants were 54 (25 female) adult smokers (n = 25) and non-smokers (n = 29). The smokers all reported smoking at least 20 cigarettes per day, and the non-smokers reported having never smoked. The results indicated that the smokers discounted significantly more than the non-smokers by both delay and probability. Unlike past findings, these results suggest that both delay and probability discounting are related to adult cigarette smoking; however, it also was determined that DD was a significantly stronger predictor of smoking than PD.     

Effect of delay and social distance on the perceived value of social interaction

Temporal discounting is a process by which the perceived value of an outcome decreases as delay increases. Social discounting is a process by which the perceived value of an outcome decreases as the social distance between the decision-maker and recipient increases. Both temporal and social discounting are well established for monetary outcomes. However, little is known regarding the effect of delay-to-occurrence or increased social distance on the perceived value of social interactions. The current study demonstrates that the decrease in perceived value for delayed social interactions is well described by the same hyperbola-like function used for monetary outcomes and that the rate of discounting decreases (indicating increased preference for larger, more delayed outcomes) as social distance increases. Conceptualizing the current discounting procedure as entailing both potential gains and losses, suggest that under certain circumstances, higher discount rates may not reflect increased preference.