Delay and Probability Discounting of Sexual and Monetary Outcomes in Individuals with Cocaine Use Disorders and Matched Controls

Individuals with cocaine use disorders are disproportionately affected by HIV/AIDS, partly due to higher rates of unprotected sex. Recent research suggests delay discounting of condom use is a factor in sexual HIV risk. Delay discounting is a behavioral economic concept describing how delaying an event reduces that event’s value or impact on behavior. Probability discounting is a related concept describing how the uncertainty of an event decreases its impact on behavior. Individuals with cocaine use disorders (n = 23) and matched non-cocaine-using controls (n = 24) were compared in decision-making tasks involving hypothetical outcomes: delay discounting of condom-protected sex (Sexual Delay Discounting Task), delay discounting of money, the effect of sexually transmitted infection (STI) risk on likelihood of condom use (Sexual Probability Discounting Task), and probability discounting of money. The Cocaine group discounted delayed condom-protected sex (i.e., were more likely to have unprotected sex vs. wait for a condom) significantly more than controls in two of four Sexual Delay Discounting Task partner conditions. The Cocaine group also discounted delayed money (i.e., preferred smaller immediate amounts over larger delayed amounts) significantly more than controls. In the Sexual Probability Discounting Task, both groups showed sensitivity to STI risk, however the groups did not differ. The Cocaine group did not consistently discount probabilistic money more or less than controls. Steeper discounting of delayed, but not probabilistic, sexual outcomes may contribute to greater rates of sexual HIV risk among individuals with cocaine use disorders. Probability discounting of sexual outcomes may contribute to risk of unprotected sex in both groups. Correlations showed sexual and monetary results were unrelated, for both delay and probability discounting. The results highlight the importance of studying specific behavioral processes (e.g., delay and probability discounting) with respect to specific outcomes (e.g., monetary and sexual) to understand decision making in problematic behavior.     

Nicotine and the behavioral mechanisms of intertemporal choice

Nicotine has been found to produce dose-dependent increases in impulsive choice (preference for smaller, sooner reinforcers relative to larger, later reinforcers) in rats. Such increases could be produced by either of two behavioral mechanisms: (1) an increase in delay discounting (i.e., exacerbating the impact of differences in reinforcer delays) which would increase the value of a sooner reinforcer relative to a later one, or (2) a decrease in magnitude sensitivity (i.e., diminishing the impact of differences in reinforcer magnitudes) which would increase the value of a smaller reinforcer relative to a larger one. To isolate which of these two behavioral mechanisms was likely responsible for nicotine's effect on impulsive choice, we manipulated reinforcer delay and magnitude using a concurrent, variable interval (VI 30 s, VI 30 s) schedule of reinforcement with 2 groups of Long-Evans rats (n = 6 per group). For one group, choices were made between a 1-s delay and a 9-s delay to 2 food pellets. For a second group, choices were made between 1 pellet and 3 pellets. Nicotine (vehicle, 0.03, 0.1, 0.3, 0.56 and 0.74 mg/kg) produced dose-dependent decreases in preference for large versus small magnitude reinforcers and had no consistent effect on preference for short versus long delays. This suggests that nicotine decreases sensitivity to reinforcer magnitude.     

Amount-dependent temporal discounting?

Amount-dependent temporal discounting has been demonstrated for human choice between outcomes differing in amount and delay. In the only study to date with non-humans, Grace reported no evidence for amount-dependent temporal discounting with pigeons in a concurrent-chains procedure. The present experiments repeated Grace's procedure but with modifications to enhance the discrimination between small and large magnitude outcomes. In Experiment 1, sensitivity of pigeons' initial-link choice to the terminal link delay ratio was greater with large reinforcer durations in the terminal links than with small reinforcer durations. This result is consistent with a greater rate of temporal discounting for larger reinforcers (the reverse of the result for humans), but can also be explained as enhanced discrimination of delay ratios with larger reinforcer durations. The results of a second experiment supported Grace's conclusion that amount-dependent temporal discounting does not characterize pigeons' choice in concurrent chains. Because reinforcer amount was held constant between choice alternatives in the present experiments and that of Grace, but varied in the human studies, our results question whether prior demonstrations of amount-dependent discounting reflect the effects of reinforcer delay or of reinforcer amount. Differences in the procedures used to study discounting in humans (titration procedures) and non-humans (concurrent chains) may contribute to the divergent results across species.     

Temporal discounting and heart rate reactivity to stress

Temporal discounting is the reduction of the value of a reinforcer as a function of increasing delay to its presentation. Impulsive individuals discount delayed consequences more rapidly than self-controlled individuals, and impulsivity has been related to substance abuse, gambling, and other problem behaviors. A growing body of literature has identified biological correlates of impulsivity, though little research to date has examined relations between delay discounting and markers of poor health (e.g., cardiovascular reactivity to stress). We evaluated the relation between one aspect of impulsivity, measured using a computerized temporal discounting task, and heart rate reactivity, measured as a change in heart rate from rest during a serial subtraction task. A linear regression showed that individuals who were more reactive to stress responded more impulsively (i.e., discounted delayed reinforcers more rapidly). When results were stratified by gender, the effect was observed for females, but not for males. This finding supports previous research on gender differences in cardiovascular reactivity and suggests that this type of reactivity may be an important correlate of impulsive behavior.     

Role of the orbital prefrontal cortex in choice between delayed and uncertain reinforcers: a quantitative analysis

'Inter-temporal choice' refers to choice between two or more outcomes that differ with respect to their sizes, delays, and/or probabilities of occurrence. According to the multiplicative hyperbolic model of inter-temporal choice, the value of a reinforcer increases as a hyperbolic function of its size, and decreases as a hyperbolic function of its delay and the odds against its occurrence. These functions, each of which contains a single discounting parameter, are assumed to combine multiplicatively to determine the overall value of the reinforcer. The model gives rise to a quantitative methodology for analysing inter-temporal choice, based on a family of linear null equations which describe performance under conditions of indifference, when the values of the reinforcers are assumed to be equal. This approach was used to examine the effect of lesions of the orbital prefrontal cortex (OPFC) on inter-temporal choice in rats. Under halothane anaesthesia, rats received injections of the excitotoxin quinolinate into the OPFC or sham lesions. They were trained to press two levers (A and B) for food-pellet reinforcers in discrete-trials schedules. In free-choice trials, a press on A resulted in delivery of a pellet after a delay d(A) with a probability P=0.5; a press on B resulted in delivery of a pellet with a probability P=1 after a delay d(B). d(B) was increased progressively across successive blocks of six trials in each session, while d(A) was manipulated systematically across phases of the experiment. The indifference delays, d(B(50)) (value of d(B) corresponding to 50% choice of B) was estimated for each rat in each phase. Linear functions of d(B(50)) versus d(A) were derived, and the parameters of the function compared between the groups. In both groups, d(B(50)) increased linearly with d(A). The slope of the linear function was significantly steeper in the lesioned group than in the sham-lesioned group, whereas the intercept did not differ significantly between the groups. Analysis based on the relevant null equation indicated that the lesion of the OPFC increased the rate of both delay and odds discounting. Possible implications of the results for interpreting the effects of OPFC lesions on inter-temporal choice behaviour in man are discussed.     

Dopaminergic Lesions of the Dorsolateral Striatum in Rats Increase Delay Discounting in an Impulsive Choice Task

Dysregulated dopamine transmission in striatal circuitry is associated with impulsivity. The current study evaluated the influence of dopaminergic inputs to the dorsolateral striatum on impulsive choice, one aspect of impulsive behavior. We implemented an operant task that measures impulsive choice in rats via delay discounting wherein intracranial self-stimulation (ICSS) was used as the positive reinforcer. To do so, rats were anesthetized to allow implanting of a stimulating electrode within the lateral hypothalamus of one hemisphere and bilateral dorsal striatal injections of the dopaminergic toxin, 6-OHDA (lesioned) or its vehicle (sham). Following recovery, rats were trained in a delay discounting task wherein they selected between a small ICSS current presented immediately after lever pressing, and a large ICSS current presented following a 0 to 15s delay upon pressing the alternate lever. Task acquisition and reinforcer discrimination were similar for lesioned and sham rats. All rats exhibited an initial preference for the large reinforcer, and as the delay was increased, preference for the large reinforcer was decreased indicating that the subjective value of the large reinforcer was discounted as a function of delay time. However, this discounting effect was significantly enhanced in lesioned rats for the longer delays. These data reveal a contribution of dopaminergic inputs to the dorsolateral striatum on impulsive choice behavior, and provide new insights into neural substrates underlying discounting behaviors.     

Delay discounting and gambling

Delay discounting describes the decline in the value of a reinforcer as the delay to that reinforcer increases. A review of the available studies revealed that steep delay discounting is positively correlated with problem or pathological gambling. One hypothesis regarding this correlation derives from the discounting equation proposed by Mazur (1989). According to the equation, steeper discounting renders the difference between fixed-delayed rewards and gambling-like variable-delayed rewards larger; with the latter being more valuable. The present study was designed to test this prediction by first assessing rats’ impulsive choices across four delays to a larger-later reinforcer. A second condition quantified strength of preference for mixed- over fixed-delays, with the duration of the latter adjusted between sessions to achieve indifference. Strength of preference for the mixed-delay alternative is given by the fixed delay at indifference (lower fixed-delay values reflect stronger preferences). Percent impulsive choice was not correlated with the value of the fixed delay at indifference and, therefore, the prediction of the hyperbolic model of gambling was not supported. A follow-up assessment revealed a significant decrease in impulsive choice after the second condition. This shift in impulsive choice could underlie the failure to observe the predicted correlation between impulsive choice and degree of preference for mixed- over fixed delays.     

Strain differences in delay discounting using inbred rats

A heightened aversion to delayed rewards is associated with substance abuse and numerous other neuropsychiatric disorders. Many of these disorders are heritable, raising the possibility that delay aversion may also have a significant genetic or heritable component. To examine this possibility, we compared delay discounting in six inbred strains of rats (Brown Norway, Copenhagen, Lewis, Fischer, Noble and Wistar Furth) using the adjusting amount procedure, which provides a measure of the subjective value of delayed rewards. The subjective value of rewards decreased as the delay to receipt increased for all strains. However, a main effect of strain and a strain × delay interaction indicated that some strains were more sensitive to the imposition of delays than others. Fitting a hyperbolic discount equation showed significant strain differences in sensitivity to delay ( k ). These data indicate that there are significant strain differences in delay discounting. All strains strongly preferred the 10% sucrose solution (the reinforcer in the delay discounting task) over water and the amount of sucrose consumed was correlated with sensitivity to delay. Locomotor activity was not correlated with delay discounting behavior. Additional research will be required to disentangle genetic influences from maternal effects and to determine how these factors influence the underlying association between heightened delay discounting and neuropsychiatric disorders.     

Discounting of delayed hypothetical money, alcohol, and food

For drug-dependent individuals, drugs of abuse that are delayed in time are discounted more steeply than money delayed in time in a hypothetical choice task. The reasons for this finding are not clear. This study examined whether steep discounting of drugs relative to money might be related to the function of drugs as primary/consumable reinforcers and money as a conditioned/non-consumable reinforcer. Twenty adults with no self-reported problems with money, alcohol, or food participated. They indicated their preferences for three hypothetical outcome types: delayed versus immediate money, delayed versus immediate food, and delayed versus immediate alcohol. Both the hyperbolic decay model and area under the curve analysis showed that money was discounted less steeply than alcohol or food, but that alcohol and food were discounted similarly. This finding replicates previous results showing that people without drug abuse problems show steep discounting of alcohol. Furthermore, this finding suggests that alcohol may be steeply discounted as part of a general process involving primary/consumable reinforcers, not necessarily because it is a drug.     

Impulsiveness without discounting: the ecological rationality hypothesis

Observed animal impulsiveness challenges ideas from foraging theory about the fitness value of food rewards, and may play a role in important behavioural phenomena such as cooperation and addiction. Behavioural ecologists usually invoke temporal discounting to explain the evolution of animal impulsiveness. According to the discounting hypothesis, delay reduces the fitness value of the delayed food. We develop an alternative model for the evolution of impulsiveness that does not require discounting. We show that impulsive or short-sighted rules can maximize long-term rates of food intake. The advantages of impulsive rules come from two sources. First, naturally occurring choices have a foreground-background structure that reduces the long-term cost of impulsiveness. Second, impulsive rules have a discrimination advantage because they tend to compare smaller quantities. Discounting contributes little to this result. Although we find that impulsive rules are optimal in a simple foreground-background choice situation in the absence of discounting, in contrast we do not find comparable impulsiveness in binary choice situations even when there is strong discounting.     

The effects of changes in consequences on hens' performance in delayed-matching-to-sample tasks

A delayed-matching-to-sample (DMTS) task was used to investigate remembering with domestic hens. In Conditions 1 and 3 of Experiment 1, six hens responded under a mixed-delay procedure with delays of 0.25, 2, and 8 s. In Condition 2, the reinforcer for correct responding was delayed for 6 s after each correct matching response on 2-s delay trials. In Condition 1, discrimination performance decreased monotonically over the three delays. With the delay to the reinforcer, the decreases were non-monotonic as a result of the considerable drop in the accuracy of discrimination on the 2-s delay trials. Performance at the 2-s delay did not recover completely in Condition 3. In Conditions 1 and 3 of Experiment 2, five hens responded under a mixed-delay procedure with delays of 0, 4, and 16 s. In Condition 2 no reinforcers were provided for correct responding on 0- and 16-s delay trials. When reinforcers were available on all trials discrimination performance decreased monotonically with delay. There were non-monotonic changes in discrimination with delay when there was extinction at two delays resulting mainly from a large drop in discrimination performance at 0 s. In addition, response latencies increased markedly at the two delays associated with extinction. Performance recovered completely in Condition 3. The data support the ideas that remembering involves a temporal discrimination that the effects of delaying reinforcement and removing reinforcement may differ, and that the measurement of response latencies may be a useful tool in DMTS procedures.     

Quantitative analyses of methamphetamine's effects on self-control choices: implications for elucidating behavioral mechanisms of drug action

The purpose of the present research was to utilize quantitative methods to identify behavioral mechanisms involved in the effects of stimulant drugs on choice in a self-control procedure. A logarithmic equation based upon a combination of the matching law and hyperbolic discounting was used to separate drug-induced changes in sensitivity to reinforcement delay from drug-induced changes in sensitivity to reinforcement amount. Pigeons responded under a concurrent-chains schedule. In the initial link, two keys were illuminated simultaneously and access to the terminal link was controlled by a single random-interval (RI) schedule; pecks on one or the other key lead to its terminal link with a 0.5 probability. In the terminal links, one alternative provided 1-s access to food (the smaller reinforcer) and the other alternative provided 4-s access to food (the larger reinforcer). The signaled delay to the smaller reinforcer always was 2s, whereas the signaled delay to the larger reinforcer increased from 2 to 40s within each session, across 10-min blocks. In general, intermediate doses of methamphetamine increased preference for the larger more delayed reinforcer. Quantitative analyses indicated that, in most cases, methamphetamine decreased sensitivity to reinforcement delay. In a few instances, concomitant decreases in sensitivity to reinforcement amount also occurred. These results suggest that a reduced sensitivity to reinforcement delay may be important behavioral mechanism of the effects of stimulants on self-control choices, and that this effect sometimes can be accompanied by a decreased sensitivity to reinforcement amount.     

Delay of gratification and delay discounting in rats

Delay discounting (DD) and delay of gratification (DG) are two measures of impulsive behavior often viewed as reflecting the same or equivalent processes. However, there are some key differences in the contingencies of reinforcement between the procedures that may have implications for understanding impulsivity. This study used DD and DG procedures to determine if differences in contingencies of reinforcement specified by DD and DG alters how much organisms discount the value of delayed reinforcers. Twenty-four water-deprived rats performed one of two Adjusting Amount procedures, which consisted of repeated choices between a fixed amount of water (250 &mgr;l) delivered after a delay (0, 4, 8, 16, or 32 s) and an adjusting, usually lesser amount delivered immediately. Half of the rats (n=12) performed a DD procedure designed to assess preference for immediate over delayed reinforcers in which they had discrete choices between the immediate and delayed amounts of water. A DG procedure was used for the other half of the rats (n=12). In the DG procedure rats also selected between immediate and delayed alternatives, but if they chose the delayed alternative they could switch to and receive the immediate alternative at any time during the delay to the larger reward. In the DD procedure switching responses were not reinforced but were still recorded and used for analyses. The DD functions of the two groups did not differ significantly. However, at the longer delays, the DG group made significantly fewer switching responses than the DD group. A possible role of response inhibition in the DG procedure is discussed.     

Acquisition of and withdrawal from cocaine self-administration regulates 5-HT1B mRNA expression in rat striatum

This study investigated how different stages of cocaine self-administration in rats affect the expression of two serotonin receptors in dorsal and ventral striatum, the 5-HT_1B and 5-HT₆ subtypes, which have both been implicated in mediating some aspects of cocaine-related behaviors. In the first experiment, rats were trained to work for saccharin (oral) or cocaine (i.v.) reinforcers. We found that continuous access to cocaine for 23 days did not change the level of 5-HT_1B mRNA expression compared to control animals receiving saccharin. However, a single cocaine session, given either by self-administration or non-contingently, increased 5-HT_1B mRNA in dorsal striatum, whereas forced abstinence for two weeks after cocaine reduced 5-HT_1B mRNA expression in the same subregion. 5-HT₆ mRNA was not changed by any of these treatments. A follow-up experiment investigated the effects of limited versus extended access to cocaine as well as forced abstinence, and we found that 14 days of forced abstinence significantly reduced 5-HT_1B mRNA throughout the dorsal and ventral striatum compared to no withdrawal. These results suggest that the influence of 5-HT_1B receptors in striatal projection neurons may be increased during cocaine acquisition and reduced after forced abstinence and may therefore be targets for pharmacological intervention in addiction.     

Tradeoffs among delay, rate, and amount of reinforcement

In Experiment 1, an adjusting-delay procedure was used to measure pigeons' choices between a single delayed reinforcer and a range of different variable-time schedules. Indifference points showed an inverse relation between rate of reinforcement and delay that was well described by a hyperbolic equation. An adjusting-amount procedure was used in Experiment 2, in which pigeons chose between an adjusting amount of food delivered after a 0.5-s delay and 3 s of food delivered after a range of different delays, and the effects of delay were similar to those found in Experiment 1. The results from both experiments indicated that, for pigeons, the strength of a reinforcer decreased rapidly with increasing delay. Estimates of a decay rate parameter in the hyperbolic equation were similar to those found in other studies with pigeons, but the rates of temporal discounting were three or four times faster than those found in studies with rats, suggesting a possible species difference.     

The present values of delayed rewards are approximately additive

In two experiments, human subjects were asked to estimate their present values of single delayed rewards and their present values of temporal sequences of three rewards. Present values were solicited by asking subjects to indicate an amount of money v for which they would be indifferent between receiving v at the end of the session and receiving the delayed reward(s). A procedure was used for which responding the true value of v was the optimal strategy, and the actual payoff that each subject received was determined by one randomly selected trial. In Experiment 1 (n=29) each delayed reward was 9.90 dollars in cash. In Experiment 2 (n=19) the delayed rewards were dated 15 dollars gift certificates to a local restaurant. In both experiments, the present values of the sequences were approximately equal to the sums of the present values of their component rewards. The presence of outliers suggests that a few subjects may have valued sequences less than the sums of their single rewards. Effects of a preference for uniform sequences, if any, were too small to be detected. Discounting of sequences was well fit by a parallel hyperbolic discounting equation, consistent with Mazur's [Mazur, J.E., 1986. Choice between single and multiple delayed reinforcers. J. Exp. Anal. Behav. 46 (1), 67-77] results using multiple reinforcers.     

Is Working More Costly than Waiting in Monkeys?

We studied how value for instrumental action is discounted by predicted effort and delay. The monkeys were trained to perform instrumental trials that required a bar release when a visual target changed from red-to-green. There were two trial conditions. In delay trials, after the monkeys performed one instrumental trial correctly a reward was delivered 0–7 seconds later. In work trials, the monkeys had to perform 0, 1, or 2 additional instrumental trials to obtain a reward. The lengths of trials in delay matched the time it took to complete work trials. The length of delay or number of trials was indicated by a visual cue presented throughout the trial. Our hypothesis was that the monkeys would all show temporal discounting of reward in the delay trials, and that in the work trials the monkeys’ performance might reflect an additional cost due to working. The error rate increased linearly as remaining cost increased for all 8 monkeys. For 4 monkeys the error rate was significantly larger in work trials than in delay trials (effort sensitive monkeys). For the other 4 monkeys there was no significant difference in error rate (effort insensitive monkeys). Since the error rate has an inverse relation with value for action, these results suggest that value is discounted hyperbolically by effort as well as by delay. Error rates generally increased as the testing sessions progressed and the total reward accumulated (i.e., effect of reward devaluation). The relative impact of delay and effort on error rates was reasonably stable within subjects. Thus, within the monkey population there seems to be a significant dichotomy in the sensitivity governing whether working is more costly than waiting, possibly arising from a constitutional or genetic trait.     

Commodity specific rates of temporal discounting: Does metabolic function underlie differences in rates of discounting?

Discounting rates vary as a function of commodity type. Previous studies suggest five potential characteristics of the commodity that could explain these differences: type of reinforcer (primary or secondary), if the commodity is perishable, if the commodity is satiable, if the commodity can be directly consumed, and immediacy of consumption. This paper suggests that these characteristics may best be viewed as related to a more fundamental characteristic: metabolic processing. In order to explore the possibility that metabolic processing underlies changes in discount rates, the difference in discounting between food, money, music CDs, DVDs, and books are compared. Music CDs, DVDs, and books share many characteristics in common with food, including gaining value through a physiological process, but are not directly metabolized. Results are consistent with previous findings of commodity specific discount rates and show that metabolic function plays a role in determining discount rates with those commodities that are metabolized being discounted at a higher rate. These results are interpreted as evidence that the discount rate for different commodities lies along a continuum with those that serve an exchange function rather than a direct function (money) anchoring the low end and those that serve a direct metabolic function capping the high end (food, alcohol, drugs).     

Response bias is unaffected by delay length in a delay discounting paradigm

This study examined the contribution of response bias to measures of delay discounting in Long-Evans rats (n = 8) using the adjusting amount procedure. Under this procedure, we assessed preference for 150 μl of 10% sucrose solution delivered following a delay over a variable-amount alternative delivered immediately. Bias was calculated based on relative preference when reinforcers were delivered immediately from both alternatives. We extended this assessment procedure to examine preference when rewards from both alternatives were equally delayed (2, 4, 8, or 16 s) in addition to assessing a traditional delay discounting function. Relative preference was similar across delays and slightly larger than 150 μl. These results indicate that response bias was stable and suggests a relative aversion for the adjusting alternative, which may be due to the variability in reward size associated with that alternative.     

Human performance on a two-alternative rapid-acquisition choice task

Davison and Baum [Davison, M., Baum, W. M., 2000. Choice in a variable environment: every reinforcer counts. Journal of the Experimental Analysis of Behavior 74, 1-24.] developed a concurrent-schedule procedure where, within each session, different reinforcer ratios were arranged across components separated by brief black-outs. Behaviour adapted quickly to the reinforcer ratios and reinforcers also had local effects on responding. This procedure has been used with pigeons and rats. In the present experiment, we adapted the Davison and Baum procedure to study the effects of reinforcement on human choice behaviour. Eighteen participants were presented with four different reinforcer ratios within a single 50-minute session. Mean sensitivity to the reinforcer ratios increased within components, and preference was greater for the just-reinforced response alternative immediately following reinforcer delivery, similar to the results from non-human experiments. Although there were limitations to the current procedure, the local time scale analyses are a novel way of examining human operant behaviour.