The Role of Serotonin in the Regulation of Patience and Impulsivity

Classic theories suggest that central serotonergic neurons are involved in the behavioral inhibition that is associated with the prediction of negative rewards or punishment. Failed behavioral inhibition can cause impulsive behaviors. However, the behavioral inhibition that results from predicting punishment is not sufficient to explain some forms of impulsive behavior. In this article, we propose that the forebrain serotonergic system is involved in “waiting to avoid punishment” for future punishments and “waiting to obtain reward” for future rewards. Recently, we have found that serotonergic neurons increase their tonic firing rate when rats await food and water rewards and conditioned reinforcer tones. The rate of tonic firing during the delay period was significantly higher when rats were waiting for rewards than for tones, and rats were unable to wait as long for tones as for rewards. These results suggest that increased serotonergic neuronal firing facilitates waiting behavior when there is the prospect of a forthcoming reward and that serotonergic activation contributes to the patience that allows rats to wait longer. We propose a working hypothesis to explain how the serotonergic system regulates patience while waiting for future rewards.     

Delay discounting and gambling

Delay discounting describes the decline in the value of a reinforcer as the delay to that reinforcer increases. A review of the available studies revealed that steep delay discounting is positively correlated with problem or pathological gambling. One hypothesis regarding this correlation derives from the discounting equation proposed by Mazur (1989). According to the equation, steeper discounting renders the difference between fixed-delayed rewards and gambling-like variable-delayed rewards larger; with the latter being more valuable. The present study was designed to test this prediction by first assessing rats’ impulsive choices across four delays to a larger-later reinforcer. A second condition quantified strength of preference for mixed- over fixed-delays, with the duration of the latter adjusted between sessions to achieve indifference. Strength of preference for the mixed-delay alternative is given by the fixed delay at indifference (lower fixed-delay values reflect stronger preferences). Percent impulsive choice was not correlated with the value of the fixed delay at indifference and, therefore, the prediction of the hyperbolic model of gambling was not supported. A follow-up assessment revealed a significant decrease in impulsive choice after the second condition. This shift in impulsive choice could underlie the failure to observe the predicted correlation between impulsive choice and degree of preference for mixed- over fixed delays.     

Dynamics of pre- and post-choice behaviour: rats approximate optimal strategy in a discrete-trial decision task

We simulate two types of environments to investigate how closely rats approximate optimal foraging. Rats initiated a trial where they chose between two spouts for sucrose, which was delivered at distinct probabilities. The discrete trial procedure used allowed us to observe the relationship between choice proportions, response latencies and obtained rewards. Our results show that rats approximate the optimal strategy across a range of environments that differ in the average probability of reward as well as the dynamics of the depletion-renewal cycle. We found that the constituent components of a single choice differentially reflect environmental contingencies. Post-choice behaviour, measured as the duration of time rats spent licking at the spouts on unrewarded trials, was the most sensitive index of environmental variables, adjusting most rapidly to changes in the environment. These findings have implications for the role of confidence in choice outcomes for guiding future choices.     

Estrogen Enhances Performance of Female Rats during Acquisition of a Radial Arm Maze

Estrogen can influence the expression of behaviors not associated directly with reproduction, including learning and memory. However, the effects of estrogen on learning and memory in mammals are complex, dependent on a variety of factors. The radial arm maze is a traditional experimental task that takes advantage of the natural foraging strategy of rats and provides an appropriate measure for studying the effects of estrogen on working memory in this species. In the experiments reported here, ovariectomized rats were implanted subcutaneously with 5-mm Silastic capsules containing 25% estradiol diluted with cholesterol. Control females received 5-mm Silastic capsules containing 100% cholesterol. Results of three separate experiments demonstrated that estradiol administered by Silastic implants for 30 days prior to eight-arm radial maze training, during the 24 days of maze training, or both significantly improved working memory performance compared to females treated with cholesterol alone, as indicated by improved arm choice accuracy over trials. The positive effect of estradiol exposure prior to training suggests that estrogen may induce neuronal changes that persist beyond the period of exposure with functional consequences for behavior.     

An assessment of executive function in two different rat models of attention-deficit hyperactivity disorder: Spontaneously hypertensive versus Lphn3 knockout rats

Attention-deficit/hyperactivity disorder (ADHD) a common neurodevelopmental disorder of childhood and often comorbid with other externalizing disorders (EDs). There is evidence that externalizing behaviors share a common genetic etiology. Recently, a genome-wide, multigenerational sample linked variants in the Lphn3 gene to ADHD and other externalizing behaviors. Likewise, limited research in animal models has provided converging evidence that Lphn3 plays a role in EDs. This study examined the impact of Lphn3 deletion (i.e., Lphn3^−/−) in rats on measures of behavioral control associated with externalizing behavior. Impulsivity was assessed for 30 days via a differential reinforcement of low rates (DRL) task and working memory evaluated for 25 days using a delayed spatial alternation (DSA) task. Data from both tasks were averaged into 5-day testing blocks. We analyzed overall performance, as well as response patterns in just the first and last blocks to assess acquisition and steady-state performance, respectively. “Positive control” measures on the same tasks were measured in an accepted animal model of ADHD–the spontaneously hypertensive rat (SHR). Compared with wildtype controls, Lphn3^−/− rats exhibited deficits on both the DRL and DSA tasks, indicative of deficits in impulsive action and working memory, respectively. These deficits were less severe than those in the SHRs, who were profoundly impaired on both tasks compared with their control strain, Wistar-Kyoto rats. The results provide evidence supporting a role for Lphn3 in modulating inhibitory control and working memory, and suggest additional research evaluating the role of Lphn3 in the manifestation of EDs more broadly is warranted.     

The relationship between impulsive choice and impulsive action: a cross-species translational study

Maladaptive impulsivity is a core symptom in various psychiatric disorders. However, there is only limited evidence available on whether different measures of impulsivity represent largely unrelated aspects or a unitary construct. In a cross-species translational study, thirty rats were trained in impulsive choice (delayed reward task) and impulsive action (five-choice serial reaction time task) paradigms. The correlation between those measures was assessed during baseline performance and after pharmacological manipulations with the psychostimulant amphetamine and the norepinephrine reuptake inhibitor atomoxetine. In parallel, to validate the animal data, 101 human subjects performed analogous measures of impulsive choice (delay discounting task, DDT) and impulsive action (immediate and delayed memory task, IMT/DMT). Moreover, all subjects completed the Stop Signal Task (SST, as an additional measure of impulsive action) and filled out the Barratt impulsiveness scale (BIS-11). Correlations between DDT and IMT/DMT were determined and a principal component analysis was performed on all human measures of impulsivity. In both rats and humans measures of impulsive choice and impulsive action did not correlate. In rats the within-subject pharmacological effects of amphetamine and atomoxetine did not correlate between tasks, suggesting distinct underlying neural correlates. Furthermore, in humans, principal component analysis identified three independent factors: (1) self-reported impulsivity (BIS-11); (2) impulsive action (IMT/DMT and SST); (3) impulsive choice (DDT). This is the first study directly comparing aspects of impulsivity using a cross-species translational approach. The present data reveal the non-unitary nature of impulsivity on a behavioral and pharmacological level. Collectively, this warrants a stronger focus on the relative contribution of distinct forms of impulsivity in psychopathology.     

Strain differences in delay discounting using inbred rats

A heightened aversion to delayed rewards is associated with substance abuse and numerous other neuropsychiatric disorders. Many of these disorders are heritable, raising the possibility that delay aversion may also have a significant genetic or heritable component. To examine this possibility, we compared delay discounting in six inbred strains of rats (Brown Norway, Copenhagen, Lewis, Fischer, Noble and Wistar Furth) using the adjusting amount procedure, which provides a measure of the subjective value of delayed rewards. The subjective value of rewards decreased as the delay to receipt increased for all strains. However, a main effect of strain and a strain × delay interaction indicated that some strains were more sensitive to the imposition of delays than others. Fitting a hyperbolic discount equation showed significant strain differences in sensitivity to delay ( k ). These data indicate that there are significant strain differences in delay discounting. All strains strongly preferred the 10% sucrose solution (the reinforcer in the delay discounting task) over water and the amount of sucrose consumed was correlated with sensitivity to delay. Locomotor activity was not correlated with delay discounting behavior. Additional research will be required to disentangle genetic influences from maternal effects and to determine how these factors influence the underlying association between heightened delay discounting and neuropsychiatric disorders.     

Progesterone attenuates impulsive action in a Go/No-Go task for sucrose pellets in female and male rats

Impulsivity, or a tendency to act without anticipation of future consequences, is associated with drug abuse. Impulsivity is typically separated into two main measures, impulsive action and impulsive choice. Given the association of impulsivity and drug abuse, treatments that reduce impulsivity have been proposed as an effective method for countering drug addiction. Progesterone has emerged as a promising treatment, as it is associated with decreased addiction-related behaviors and impulsive action. The goal of the present study was to determine the effects of progesterone (PRO) on impulsive action for food: a Go/No-Go task. Female and male rats responded for sucrose pellets during a Go component when lever pressing was reinforced on a variable-interval 30-s schedule. During the alternate No-Go component, withholding a lever press was reinforced on a differential reinforcement of other (DRO) behavior 30-s schedule, where a lever press reset the DRO timer. Impulsive action was operationally defined as the inability to withhold a response during the No-Go component (i.e. the number of DRO resets). Once Go/No-Go behavior was stable, responding between rats treated with PRO (0.5 mg/kg) or vehicle was examined. Progesterone significantly decreased the total number of DRO resets in both males and females, but it did not affect VI responding for sucrose pellets. This suggests that PRO decreases motor impulsivity for sucrose pellets without affecting motivation for food. Thus, PRO may reduce motor impulsivity, a behavior underlying drug addiction.     

When is it adaptive to be patient? A general framework for evaluating delayed rewards

The tendency of animals to seek instant gratification instead of waiting for greater long-term benefits has been described as impatient, impulsive or lacking in self-control. How can we explain the evolution of such seemingly irrational behaviour? Here we analyse optimal behaviour in a variety of simple choice situations involving delayed rewards. We show that preferences for more immediate rewards should depend on a variety of factors, including whether the choice is a one-off or is likely to be repeated, the information the animal has about the continuing availability of the rewards and the opportunity to gain rewards through alternative activities. In contrast to the common assertion that rational animals should devalue delayed rewards exponentially, we find that this pattern of discounting is optimal only under restricted circumstances. We predict preference reversal whenever waiting for delayed rewards entails loss of opportunities elsewhere, but the direction of this reversal depends on whether the animal will face the same choice repeatedly. Finally, we question the ecological relevance of standard laboratory tests for impulsive behaviour, arguing that animals rarely face situations analogous to the self-control paradigm in their natural environment. To understand the evolution of impulsiveness, a more promising strategy would be to identify decision rules that are adaptive in a realistic ecological setting, and examine how these rules determine patterns of behaviour in simultaneous choice tests.     

Dopaminergic Lesions of the Dorsolateral Striatum in Rats Increase Delay Discounting in an Impulsive Choice Task

Dysregulated dopamine transmission in striatal circuitry is associated with impulsivity. The current study evaluated the influence of dopaminergic inputs to the dorsolateral striatum on impulsive choice, one aspect of impulsive behavior. We implemented an operant task that measures impulsive choice in rats via delay discounting wherein intracranial self-stimulation (ICSS) was used as the positive reinforcer. To do so, rats were anesthetized to allow implanting of a stimulating electrode within the lateral hypothalamus of one hemisphere and bilateral dorsal striatal injections of the dopaminergic toxin, 6-OHDA (lesioned) or its vehicle (sham). Following recovery, rats were trained in a delay discounting task wherein they selected between a small ICSS current presented immediately after lever pressing, and a large ICSS current presented following a 0 to 15s delay upon pressing the alternate lever. Task acquisition and reinforcer discrimination were similar for lesioned and sham rats. All rats exhibited an initial preference for the large reinforcer, and as the delay was increased, preference for the large reinforcer was decreased indicating that the subjective value of the large reinforcer was discounted as a function of delay time. However, this discounting effect was significantly enhanced in lesioned rats for the longer delays. These data reveal a contribution of dopaminergic inputs to the dorsolateral striatum on impulsive choice behavior, and provide new insights into neural substrates underlying discounting behaviors.     

Choice and timing in concurrent chains: effects of initial-link duration

Theories of timing have been applied to choice between delayed rewards by assuming that delays are represented in memory and that subjects sample from memory when choosing between alternatives. To search for covariation in single-trial measures of performance that might confirm this assumption, we used a procedure that allowed for convergent measurement of choice and timing behavior. Four pigeons responded in a concurrent chains/peak procedure in which the terminal links were fixed-interval (FI) 8s and FI 16s, across conditions the duration of the initial-link schedule was either short or long, and one quarter of the terminal links lasted for 48 s and ended without reinforcer delivery. Preference for the FI 8-s alternative was stronger with shorter initial links, replicating the 'initial-link effect'. Responding on no-food trials was unaffected by initial-link duration, and aggregated across trials, was typical of the peak procedure: response distributions were approximately Gaussian, with modes near the FI schedule durations, and variance was greater for the FI 16-s terminal link. Analysis of local measures of initial-link performance (e.g., pause to begin responding, time spent responding, number and duration of visits to each alternative, etc.) found that the initial-link effect was associated with an increase in the number and duration of visits per cycle to the nonpreferred alternative. Regression analyses showed that local initial-link measures contributed relatively little additional variance in predicting performance on individual no-food trials beyond that accounted for by FI schedule. Our results provide no clear evidence that initial- and terminal-link responding in concurrent chains are mediated by a common representation of terminal-link delays.     

Mechanisms by which neonatal testosterone exposure mediates sex differences in impulsivity in prepubertal rats

Neonatal testosterone, either acting directly or through its conversion to estradiol, can exert organizational effects on the brain and behavior. The goal of the current study was to examine sex differences and determine the role of neonatal testosterone on prefrontal cortex-dependent impulsive choice behavior in prepubertal rats. Male and female prepubertal rats were tested on the delay-based impulsive choice task. Impulsive choice was defined as choosing an immediate small food reward over a delayed large reward. In a first experiment to examine sex differences, males made significantly more impulsive choices than did females. In a second experiment to examine the organizational effects of testosterone, females treated with neonatal testosterone made significantly more impulsive choices than did control females and their performance was indistinguishable from that of control males. In a third experiment to determine if the effect of testosterone on performance is due to the actions of androgens or estrogens through its conversion to estradiol, males treated neonatally with the aromatase inhibitor formestane, which blocks the conversion of testosterone to estradiol, females treated neonatally with the non-aromatizable androgen dihydrotestosterone, and females treated neonatally with estradiol made significantly more impulsive choices than did control females and their performance was indistinguishable from that of control males. Results indicate that male pubertal rats display increased impulsive choice behavior as compared to females, that this sex difference results from organizing actions of testosterone during the neonatal period, and that this effect can result from both androgenic and estrogenic actions.     

Neural Sensitivity to Absolute and Relative Anticipated Reward in Adolescents

Adolescence is associated with a dramatic increase in risky and impulsive behaviors that have been attributed to developmental differences in neural processing of rewards. In the present study, we sought to identify age differences in anticipation of absolute and relative rewards. To do so, we modified a commonly used monetary incentive delay (MID) task in order to examine brain activity to relative anticipated reward value (neural sensitivity to the value of a reward as a function of other available rewards). This design also made it possible to examine developmental differences in brain activation to absolute anticipated reward magnitude (the degree to which neural activity increases with increasing reward magnitude). While undergoing fMRI, 18 adolescents and 18 adult participants were presented with cues associated with different reward magnitudes. After the cue, participants responded to a target to win money on that trial. Presentation of cues was blocked such that two reward cues associated with $.20, $1.00, or $5.00 were in play on a given block. Thus, the relative value of the $1.00 reward varied depending on whether it was paired with a smaller or larger reward. Reflecting age differences in neural responses to relative anticipated reward (i.e., reference dependent processing), adults, but not adolescents, demonstrated greater activity to a $1 reward when it was the larger of the two available rewards. Adults also demonstrated a more linear increase in ventral striatal activity as a function of increasing absolute reward magnitude compared to adolescents. Additionally, reduced ventral striatal sensitivity to absolute anticipated reward (i.e., the difference in activity to medium versus small rewards) correlated with higher levels of trait Impulsivity. Thus, ventral striatal activity in anticipation of absolute and relative rewards develops with age. Absolute reward processing is also linked to individual differences in Impulsivity.     

Potential vulnerabilities of neuronal reward, risk, and decision mechanisms to addictive drugs

How do addictive drugs hijack the brain's reward system? This review speculates how normal, physiological reward processes may be affected by addictive drugs. Addictive drugs affect acute responses and plasticity in dopamine neurons and postsynaptic structures. These effects reduce reward discrimination, increase the effects of reward prediction error signals, and enhance neuronal responses to reward-predicting stimuli, which may contribute to compulsion. Addictive drugs steepen neuronal temporal reward discounting and create temporal myopia that impairs the control of drug taking. Tonically enhanced dopamine levels may disturb working memory mechanisms necessary for assessing background rewards and thus may generate inaccurate neuronal reward predictions. Drug-induced working memory deficits may impair neuronal risk signaling, promote risky behaviors, and facilitate preaddictive drug use. Malfunctioning adaptive reward coding may lead to overvaluation of drug rewards. Many of these malfunctions may result in inadequate neuronal decision mechanisms and lead to choices biased toward drug rewards.     

Spatial Memory in Spontaneously Hypertensive Rats (SHR)

The spontaneously hypertensive rat (SHR) is an established animal model of ADHD. It has been suggested that ADHD symptoms arise from deficits in executive functions such as working memory, attentional control and decision making. Both ADHD patients and SHRs show deficits in spatial working memory. However, the data on spatial working memory deficits in SHRs are not consistent. It has been suggested that the reported cognitive deficits of SHRs may be related to the SHRs’ locomotor activity. We have used a holeboard (COGITAT) to study both cognition and activity in order to evaluate the influence of the activity on the cognitive performance of SHRs. In comparison to Wistar-Kyoto (WKY) rats, SHRs did not have any impairment in spatial working memory and reference memory. When the rats’ locomotor activity was taken into account, the SHRs’ working memory and reference memory were significantly better than in WKY rats. The locomotor activity appears to be a confounding factor in spatial memory tasks and should therefore be controlled for in future studies. In the SHR model of ADHD, we were unable to demonstrate an impairment of working memory which has been reported in patients with ADHD.     

Working memory of action: a comparative study of ability to selecting response based on previous action in New World monkeys (Saimiri sciureus and Callithrix jacchus)

Working memory of the outcome of one's own action is important for organizing and learning appropriate behaviors in a given condition. To examine whether non-human primates with different neocortical sizes show different abilities regarding working memory for action, the performance of squirrel monkeys (Saimiri sciureus) and common marmosets (Callithrix jacchus) in a kind of delayed-response task was compared. In this task, subjects were required to select a response based on short-term memory of the outcome of their own prior action, which requires working memory of action. These monkeys have a similar phylogenetic status (i.e. Ceboidea), but the size of the neocortex relative to the rest of the brain (relative size of the neocortex) is quite different. We found that both two species could attend to and remember their own actions and could select a response based on that memory. However, the performance level was higher for squirrel monkeys than for marmosets. These results suggest that non-human primates with differentially developed neocortices have different abilities regarding working memory of action.     

Probability and delay discounting of hypothetical sexual outcomes

The present study used the discounting procedure to characterize choice behaviors regarding hypothetical sexual outcomes. Eighty-six adult undergraduate students completed computerized delay and probability discounting tasks concerning hypothetical money and hypothetical sexual activity. Consistent with other discounting findings, hyperbolic and hyperbola-like decay models described individual and group median indifference point data well. These findings contribute to a growing literature on the relevance of the discounting procedure to decision-making processes and suggest that the discounting procedure may be useful for understanding the processes that underlie social problem behaviors associated with impulsive sexual decisions.     

Will travel for food: spatial discounting in two new world monkeys

Nonhuman animals steeply discount the future, showing a preference for small, immediate over large, delayed rewards. Currently unclear is whether discounting functions depend on context. Here, we examine the effects of spatial context on discounting in cotton-top tamarins (Saguinus oedipus) and common marmosets (Callithrix jacchus), species known to differ in temporal discounting. We presented subjects with a choice between small, nearby rewards and large, distant rewards. Tamarins traveled farther for the large reward than marmosets, attending to the ratio of reward differences rather than their absolute values. This species difference contrasts with performance on a temporal task in which marmosets waited longer than tamarins for the large reward. These comparative data indicate that context influences choice behavior, with the strongest effect seen in marmosets who discounted more steeply over space than over time. These findings parallel details of each species' feeding ecology. Tamarins range over large distances and feed primarily on insects, which requires using quick, impulsive action. Marmosets range over shorter distances than tamarins and feed primarily on tree exudates, a clumped resource that requires patience to wait for sap to exude. These results show that discounting functions are context specific, shaped by a history of ecological pressures.     

Enriched environment, nitric oxide production and synaptic plasticity prevent the aging-dependent impairment of spatial cognition

In rodents, neuronal plasticity decreases and spatial learning and working memory deficits increase upon aging. Several authors have shown that rats reared in enriched environments have better cognitive performance in association with increased neuronal plasticity than animals reared in standard environments. We hypothesized that enriched environment could preserve animals from the age-associated neurological impairments, mainly through NO-dependent mechanisms of induction of neuronal plasticity. We present evidence that 27 months old rats from an enriched environment show a better performance in spatial working memory than standard reared rats of the same age. Both mtNOS and cytosolic nNOS activities were found significantly increased (73% and 155%, respectively) in female rats from enriched environment as compared with control animals kept in a standard environment. The enzymatic activity of complex I was 80% increased in rats from enriched environment as compared with control rats. We conclude that an extensively enriched environment prevents old rats from the aging-associated impairment of spatial cognition, synaptic plasticity and nitric oxide production.     

Effects of gonadectomy on performance in operant tasks measuring prefrontal cortical function in adult male rats

Previous studies have shown that gonadectomy in adult male rats influences the acquisition and performance of spatial and other working memory tasks that depend in part on the medial prefrontal cortex and its dopamine innervation. Stimulated by previous findings that gonadectomy alters dopamine axon density in not only medial but several other prefrontal fields, the present studies asked whether gonadectomy might also broadly impact dopamine-dependent prefrontal functions, and whether these effects bore any relation to hormone modulation of mesoprefrontal dopamine afferents. Specifically, control, gonadectomized, and gonadectomized rats given estradiol or testosterone propionate were tested on a series of operant tasks that together measured medial prefrontal functions of spatial working memory, impulsivity and extradimensional set shifting and orbital prefrontal functions of reversal learning/perseveration and motivation. Afterwards, animals were sacrificed, their bulbospongiosus muscles were removed and weighed, their brains were processed for immunocytochemistry for the dopamine-synthesizing enzyme tyrosine hydroxylase, and axon densities were measured in orbital and medial prefrontal fields. Statistical evaluations of group effects on behavior and regression analyses comparing individual performance with muscle weights and axon density measures revealed androgen-reversible effects of gonadectomy on acquisition of spatial working memory and extradimensional set shifting that were correlated with bulbospongiosus weight and medial prefrontal dopamine axon density, estrogen-sensitive influences of gonadectomy on motivation and response withholding that were correlated with bulbospongiosus weight but not with dopamine innervation, and still other prefrontal functions, i.e., impulsivity, reversal learning, that were insensitive to gonadectomy and unrelated to gonadectomy-induced changes in muscle weight or prefrontal dopamine innervation.