Resistance to extinction, generalization decrement, and conditioned reinforcement

This study investigated generalization decrement during an extinction resistance-to-change test for pigeon key pecking using a two-component multiple schedule with equal variable-interval 3-min schedules and different reinforcer amounts (one component presented 2-s access to reinforcement and the other 8s). After establishing baseline responding, subjects were assigned to one of the two extinction conditions: hopper stimuli (hopper and hopper light were activated but no food was available) or Control (inactive hopper and hopper light). Responding in the 8-s component was more resistant to extinction than responding in the 2-s component, the hopper stimuli group was more resistant to extinction compared to the Control group, and an interaction between amount of reinforcement, extinction condition, and session block was present. This finding supports generalization decrement as a factor that influences resistance to extinction. Hopper-time data (the amount of time subjects spent with their heads in the hopper) were compared to resistance-to-change data in an investigation of the role of conditioned reinforcement on resistance to change.     

Extinction deficits in male rhesus macaques with a history of self-injurious behavior

Self-injurious behavior (SIB) occurs in both human and nonhuman primate populations. Despite the potential for harm, SIB may persist in part because of an inability to inhibit behavior that results in wounding. A lever-pressing task was used to test the prediction that monkeys with SIB would show greater persistence in lever-pressing on extinction trials than monkeys without the disorder. The subjects were 15 individually-housed adult male rhesus macaques, 10 of which (the SIB group) had a veterinary record of self-inflicted wounding. All of the monkeys were trained to lever-press for food rewards to a criterion of 400 total responses. The test procedures consisted of five daily 30-min sessions divided into six 5-min intervals. On day 1, the subjects received continuous reinforcement. On days 2-4, testing consisted of alternating reinforced/unreinforced 5-min intervals, beginning with reinforcement. Reinforced intervals were cued with a buzzer. On day 5, the subjects received no reinforcement. The number of lever-presses and behavioral responses were recorded during each session. Saliva samples were collected for cortisol measurement before and after test sessions on days 1, 2, and 5. As predicted, monkeys with SIB lever-pressed more than controls during extinction intervals on days 2-4. There was no difference on day 1 or day 5. The frequency of scratching, yawning, and abnormal behavior increased when reinforcement was intermittent (days 2-4) or absent (day 5). Cortisol levels were highest with continuous reinforcement (day 1), and may reflect differential levels of food intake rather than stress. The presence of extinction deficits suggests that SIB may persist in some monkeys because they lack the ability to regulate the intensity of their biting behavior.     

Stimulus change dis-habituates operant responding supported by water reinforcers

The present experiment examined whether habituation contributes to within-session decreases in operant responding for water reinforcers. The experiment asked if this responding can be dishabituated, a fundamental property of habituated behavior. During baseline, rats’ lever pressing was reinforced by water on a variable interval 15-s schedule. During experimental conditions, rats responded on the same schedule and a new stimulus was introduced for 5 min at 15, 30 or 45 min into the 60-min session. The new stimulus was extinction, continuous reinforcement or flashing lights in different conditions. Rate of responding primarily decreased within the session during baseline. Introducing a new stimulus sometimes suppressed (extinction, continuous reinforcement) and sometimes increased (flashing lights) responding while it was in effect. The new stimulus increased responding after it ended and before it was presented in the session. The results are incompatible with the idea that non-habituation satiety factors (e.g., cellular hydration and blood volume) contributed to within-session changes in responding. These satiety factors should increase with increases in consumption, decrease with decreases in consumption and remain constant with constant consumption of water. Nevertheless, all stimulus changes increased operant responding for water. These results support the idea that habituation contributes to within-session decreases in responding for water reinforcers.     

Effect of Short-Term Intermittent Normobaric Hypoxia on the Regulation of External Respiration in Humans

Baseline external respiration and gas exchange values, as well as ventilatory thresholds and sensitivity to the O₂ and CO₂ stimuli in hypoxic and hypercapnic tests, were measured 1 h before and after a session of intermittent normobaric hypoxia (INH) (six repetitions with a 5-min inhalation of a gas mixture (10% O₂) alternating with a 3-min inhalation of atmospheric air). After an INH session, the background CO₂ level in the lungs increased by 10%. In the hypercapnic test, the actuation threshold of the ventilatory response did not change, whereas ventilatory sensitivity increased. The maximal pulmonary ventilation and the corresponding critical CO₂ level in the lungs also increased at the end of the test. In the hypoxic test, the ventilatory response occurred at a decreased level of blood oxygenation after an INH session, the pulmonary ventilation level being decreased and the CO₂ content in the lungs being increased at the end of the test. The data obtained evidence the maintenance of changed gas homeostasis for 1 h after an INH session. In this process, control of respiration was effected, with the hypoxic drive being weakened and the peripheral chemoreceptor sensitivity being decreased. The hypercapnic drive also increased, which may be determined by readjustment in the central mechanisms of respiratory regulation.     

The effect of second-half reinforcer type on responding for sucrose in the first half of the session

The present study investigated whether the sucrose-reinforced lever pressing of rats in the first half of a 50-min session would be sensitive to upcoming food-pellet reinforcement in the second half. In Experiment 1, the type of reinforcer in the first half of the session was always liquid sucrose and type of reinforcer in the second half (liquid sucrose or food pellets) varied across conditions. Sucrose concentration varied across groups (1, 5, or 25%). Results showed that rates and patterns of responding for 1%, and sometimes for 5%, sucrose reinforcers in the first half of the session were higher and steeper, respectively, when food-pellet, rather than sucrose, reinforcement occurred in the second half. Responding for 25% sucrose was not similarly affected. Experiment 2 replicated the results of Experiment 1 using a within-subjects design. Although the present results represent induction (i.e. the opposite of contrast), they are consistent with some results on consummatory contrast. They also further demonstrate that responding on interval schedules of reinforcement can be altered prospectively. By doing so, however, they pose potential problems for current theories for why operant response rates change within the session.     

Quantitative analyses of methamphetamine's effects on self-control choices: implications for elucidating behavioral mechanisms of drug action

The purpose of the present research was to utilize quantitative methods to identify behavioral mechanisms involved in the effects of stimulant drugs on choice in a self-control procedure. A logarithmic equation based upon a combination of the matching law and hyperbolic discounting was used to separate drug-induced changes in sensitivity to reinforcement delay from drug-induced changes in sensitivity to reinforcement amount. Pigeons responded under a concurrent-chains schedule. In the initial link, two keys were illuminated simultaneously and access to the terminal link was controlled by a single random-interval (RI) schedule; pecks on one or the other key lead to its terminal link with a 0.5 probability. In the terminal links, one alternative provided 1-s access to food (the smaller reinforcer) and the other alternative provided 4-s access to food (the larger reinforcer). The signaled delay to the smaller reinforcer always was 2s, whereas the signaled delay to the larger reinforcer increased from 2 to 40s within each session, across 10-min blocks. In general, intermediate doses of methamphetamine increased preference for the larger more delayed reinforcer. Quantitative analyses indicated that, in most cases, methamphetamine decreased sensitivity to reinforcement delay. In a few instances, concomitant decreases in sensitivity to reinforcement amount also occurred. These results suggest that a reduced sensitivity to reinforcement delay may be important behavioral mechanism of the effects of stimulants on self-control choices, and that this effect sometimes can be accompanied by a decreased sensitivity to reinforcement amount.     

The proportion of fixed interval trials to probe trials affects acquisition of the peak procedure fixed interval timing task

A common procedure for studying the ability of animals to time is the peak procedure. With the peak procedure, animals are first trained on a fixed interval schedule (i.e., 30s). After the animals have been well trained on the fixed interval schedule, probe trials are introduced. On probe trials, the stimulus is presented longer (i.e., 90s) and the animal does not receive reinforcement for responding. When animals are first presented with probe trials responding remains flat following the point that reinforcement normally occurs on fixed interval trials. The descending slope that eventually emerges is acquired with experience with probe trials. The present experiments manipulated the percentage of probe trials compared to FI trials across groups of rats. It was hypothesized that the descending limb of peak responding would be acquired more quickly when there were many probe trials per session as this might facilitate extinction of responding beyond the interval that reinforcement normally occurs. It was found, however, that acquisition of peak responding occurred best when there were few probe trials per session.     

Stimulus specificity in the acquisition and extinction of conditioned taste aversion

An experiment evaluated whether the acquisition and extinction of conditioned taste aversion in the rat is stimulus-specific by testing the degree of response transfer between sweet and salty tastes. Animals in the paired-same and paired-different groups received a presentation of a gustatory CS and a cyclophosphamide injection US. Nonconditioned control groups received unpaired CS /US presentations or the CS followed by a vehicle injection. Taste avoidance was evaluated in three nonreinforced test sessions. In the paired-same, unpaired and vehicle groups, all test sessions were conducted with the same flavor as originally used in training, whereas the paired-different group was tested with a novel flavor on the first and second sessions and with the originally trained flavor in last session. Stimulus specific acquisition was apparent in the first test session, when the animals in the group paired-same exhibited lower fluid intake than the other three groups. Evidence of specificity of extinction was apparent in the last test session, when animals in the group paired-different exhibited lower fluid intake than the other three groups. These results provide further evidence of stimulus specificity in acquisition and extinction of conditioned taste aversion, supporting the associative interpretation of these phenomena.     

Oscillations following periodic reinforcement

Three experiments examined behavior in extinction following periodic reinforcement. During the first phase of Experiment 1, four groups of pigeons were exposed to fixed interval (FI 16 s or FI 48 s) or variable interval (VI 16 s or VI 48 s) reinforcement schedules. Next, during the second phase, each session started with reinforcement trials and ended with an extinction segment. Experiment 2 was similar except that the extinction segment was considerably longer. Experiment 3 replaced the FI schedules with a peak procedure, with FI trials interspersed with non-food peak interval (PI) trials that were four times longer. One group of pigeons was exposed to FI 20 s PI 80 s trials, and another to FI 40 s PI 160 s trials. Results showed that, during the extinction segment, most pigeons trained with FI schedules, but not with VI schedules, displayed pause-peck oscillations with a period close to, but slightly greater than the FI parameter. These oscillations did not start immediately after the onset of extinction. Comparing the oscillations from Experiments 1 and 2 suggested that the alternation of reconditioning and re-extinction increases the reliability and earlier onset of the oscillations. In Experiment 3 the pigeons exhibited well-defined pause-peck cycles since the onset of extinction. These cycles had periods close to twice the value of the FI and lasted for long intervals of time. We discuss some hypotheses concerning the processes underlying behavioral oscillations following periodic reinforcement.     

Isolated effects of number of acquisition trials on extinction of rat conditioned approach behavior

Four conditioned approach experiments with rats assessed for effects of number of acquisition trials on extinction of conditioned responding, when number of acquisition sessions and total acquisition time were held constant. In Experiment 1, 32 trials per acquisition session led to more extinction responding than did 1 or 2 trials per session but less than did 4 trials per session. In Experiment 2, 2 trials per acquisition session led to more spontaneous recovery than did 32 trials per session. These latter findings are reminiscent of the overtraining extinction effect (OEE). Experiment 3 attempted to reduce the OEE with a preconditioning phase of partial reinforcement. Experiment 4 attempted to reduce the beneficial within-subject effects of increasing the number of acquisition trials on extinction observed by Gottlieb and Rescorla (2010) by extinguishing stimuli in different sessions. Overall, results suggest a procedural asymmetry: between-subject, increasing the number of trials between any pair of trials does not lead to greater persistence of responding during extinction; within-subject, it does. Results are discussed from an associative perspective, with a focus on explanations involving either frustration or comparator mechanisms, and from an information processing perspective, with a focus on Rate Estimation Theory.     

Intertrial interval as a contextual stimulus

Four experiments with rats investigated whether the time between appetitive conditioning trials can serve as a discriminative cue for responding during the next conditional stimulus (CS). In Experiment 1, rats that received extinction trials with a 4-min intertrial interval (ITI) showed spontaneous recovery after a retention interval of 16 min, whereas rats that received extinction with a 16-min ITI did not. Experiments 2 and 3 investigated more explicit discriminations between the 4- and 16-min ITIs. When a 16-min ITI signaled that the CS would be reinforced and a 4-min ITI signaled that it would not, the ITIs modulated responding to the CS. But when the 4-min ITI signaled reinforcement and the 16-min ITI did not, there was less evidence of modulation by the ITIs. This asymmetry was due at least partly to a difficulty in performance rather than learning. Experiment 4 investigated similar discriminations with 1- and 4-min ITIs. Here the results took a different form: time in the reinforced ITI elicited responding directly, but did not modulate responding to the CS. ITI can function as a contextual cue, and the results suggest new similarities between the processes behind interval timing and associative learning.     

Manipulating pre-feed, density of reinforcement, and extinction produces disruption in the Location variation of a temporal discrimination task in pigeons

When pharmacological and non-pharmacological agents are used to disrupt temporal discrimination, two major findings have emerged in the literature. One result reveals lateral shifts of the psychophysical curve for time due to disruptors, while the other is a decrease in accuracy for classifying short and long intervals and a flattening of the psychophysical curve. These results represent a discrepancy within the timing literature that requires clarification. The current study determined the effects of pre-feed, increased density of reinforcement during session, and extinction on the Location variation of a temporal discrimination procedure. The results showed that extinction and pre-feed (at higher levels), when presented in an acute fashion, led to right-ward shifts in the psychophysical curve. Our results, when compared to similar studies in the literature, suggest that lateral shifts are more likely to be found due to disruptors when the Location variation is being used and when procedures are less complicated.     

Habituation contributes to the decline in wheel running within wheel-running reinforcement periods

Habituation appears to play a role in the decline in wheel running within an interval. Aoyama and McSweeney [Aoyama, K., McSweeney, F.K., 2001. Habituation contributes to within-session changes in free wheel running. J. Exp. Anal. Behav. 76, 289-302] showed that when a novel stimulus was presented during a 30-min interval, wheel-running rates following the stimulus increased to levels approximating those earlier in the interval. The present study sought to assess the role of habituation in the decline in running that occurs over a briefer interval. In two experiments, rats responded on fixed-interval 30-s schedules for the opportunity to run for 45 s. Forty reinforcers were completed in each session. In the first experiment, the brake and chamber lights were repeatedly activated and inactivated after 25 s of a reinforcement interval had elapsed to assess the effect on running within the remaining 20 s. Presentations of the brake/light stimulus occurred during nine randomly determined reinforcement intervals in a session. In the second experiment, a 110 dB tone was emitted after 25 s of the reinforcement interval. In both experiments, presentation of the stimulus produced an immediate decline in running that dissipated over sessions. No increase in running following the stimulus was observed in the first experiment until the stimulus-induced decline dissipated. In the second experiment, increases in running were observed following the tone in the first session as well as when data were averaged over several sessions. In general, the results concur with the assertion that habituation plays a role in the decline in wheel running that occurs within both long and short intervals.     

Modulation of long-term memory and extinction responses induced by growth hormone (GH) and growth hormone releasing hormone (GHRH) in rats

The purpose of this work is to study the participation of growth hormone (GH) and growth hormone releasing hormone (GHRH) in the modulation of long-term memory and the extinction response of a passive avoidance task in rats. However, the effect on memory vary according to the age of the animals due to plasma levels of either hormone being modified during the aging process. Male Wistar rats were divided according to age into two experimental blocks (young rats 3 months old and aged rats 24 months old at the start of the experiment) where each block received the same treatment. Each experimental block was then divided randomly into three groups where two were experimental and the other served as control. The animals were then submitted to a one-trial passive avoidance conditioning and tested for memory retention 24 hrs after as well as twice a week until the extinction response occurred. The control group received an isotonic saline solution and the other two groups received 0.8 U.I. Of GH or 4 mcg of GHRH respectively. All substances were in a 0.08 ml volume and applied 24 hrs before training as well as 24 hrs before each retention session. The results indicate that GH and GHRH modulate longterm memory as well as the extinction response and in either case the response seems to vary with age. GH and GHRH facilitates long-term memory in young rats but not in aged rats. Finally, whereas GH delays the extinction response in both groups, GHRH retards the extinction only in aged rats.     

Contextual determinants of temporal control: Behavioral contrast in a free-operant psychophysical procedure

The question of how temporal control of responding might be influenced by contingency changes in other contexts was investigated. Each of three pigeons first was exposed to a two-component multiple schedule in which a two-key free-operant psychophysical procedure operated in one component and a variable-interval schedule operated in the other component. The variable-interval schedule then was changed to extinction while the free-operant psychophysical procedure remained unchanged. Finally, the variable-interval schedule was reintroduced. Response rates on the left key and the estimated temporal threshold under the free-operant psychophysical procedure increased for each pigeon when the alternate component schedule was changed to extinction and then decreased again when the variable-interval schedule was reintroduced. The results suggest one way that temporal control is affected by its context, and may be interpreted through the direct effects of overall reinforcement rate on temporal control mechanisms or the disruptive effects of alternative sources of reinforcement on temporally controlled behavior.     

Stimulus control over induction produced by upcoming food-pellet reinforcement

Research has demonstrated that rats increase their rate of lever pressing for sucrose reinforcement when food-pellet reinforcement will soon be available within the session. Recent results suggest that this increase occurs because stimuli in the session come to signal different levels of overall reinforcement. The present experiment tested this idea by having rats respond in two types of session. In one, they pressed lever A for 1% sucrose reinforcers in the first half of the session and lever B for food-pellet reinforcers in the second half (the opposing lever was retracted in the respective half). In the other, they pressed lever B for 1% sucrose reinforcers in the first half and lever A for 1% sucrose reinforcers in the second. Thus, the presence of lever A in the first half of the session was predictive of upcoming food-pellet reinforcement, but only lever B was ever used to obtain food pellets. Subjects responded at a higher rate on lever B in the first half of the session than on lever A, despite food-pellet reinforcement being unavailable in such sessions. Furthermore, they responded at a higher rate on lever B during probe sessions in which both levers were available. These results demonstrate that stimulus control over induction occurs when a stimulus becomes differentially associated with a heightened level of reinforcement. However, questions remain as to whether this association is Pavlovian or whether stimuli with "predictive value" may ever lead to induction.     

Extinction under a behavioral microscope: isolating the sources of decline in operant response rate

Extinction performance is often used to assess underlying psychological processes without the interference of reinforcement. For example, in the extinction/reinstatement paradigm, motivation to seek drug is assessed by measuring responding elicited by drug-associated cues without drug reinforcement. However, extinction performance is governed by several psychological processes that involve motivation, memory, learning, and motoric functions. These processes are confounded when overall response rate is used to measure performance. Based on evidence that operant responding occurs in bouts, this paper proposes an analytic procedure that separates extinction performance into several behavioral components: (1-3) the baseline bout initiation rate, within-bout response rate, and bout length at the onset of extinction; (4-6) their rates of decay during extinction; (7) the time between extinction onset and the decline of responding; (8) the asymptotic response rate at the end of extinction; (9) the refractory period after each response. Data that illustrate the goodness of fit of this analytic model are presented. This paper also describes procedures to isolate behavioral components contributing to extinction performance and make inferences about experimental effects on these components. This microscopic behavioral analysis allows the mapping of different psychological processes to distinct behavioral components implicated in extinction performance, which may further our understanding of the psychological effects of neurobiological treatments.     

Stimulus magnitude effects on the extinction of conditioned consummatory responses to flavored sucrose solution in mice

Paradoxical extinction effects in the conditioned consummatory behavior of rodents have remained largely elusive. Here, appetitive flavor conditioning was studied to determine if a paradoxical magnitude of reinforcement extinction effect (MREE) can occur in the consummatory behavior of mice. During acquisition training of two experiments with factorial design, animals received daily access to either 32% or 4% sucrose solution, and goal tracking time was measured in one-minute bins. In Experiment 1 the solutions were flavored with either 5% or 0.5% almond essence and in Experiment 2 with 2% almond essence, but combined with continuous or partial schedules of reinforcement. During extinction tests of Experiment 1 and 2, water flavored with 0.5% or 2% almond was presented, respectively. Consummatory performance decreased more abruptly during the initial portion of the extinction sessions after training with 32% as compared to 4% sucrose solution. Furthermore, when given a choice test after extinction training (Experiment 2), animals trained with 32% sucrose, preferred the flavored solution, but animals trained with 4% preferred the unflavored solution. These results are interpreted as indicative of the occurrence of a paradoxical MREE in conditioned consummatory behaviors.     

The Role of Density Regulation in Extinction Processes and Population Viability Analysis

We review the role of density dependence in the stochastic extinction of populations and the role density dependence has played in population viability analysis (PVA) case studies. In total, 32 approaches have been used to model density regulation in theoretical or applied extinction models, 29 of them are mathematical functions of density dependence, and one approach uses empirical relationships between density and survival, reproduction, or growth rates. In addition, quasi-extinction levels are sometimes applied as a substitute for density dependence at low population size. Density dependence further has been modelled via explicit individual spacing behaviour and/or dispersal. We briefly summarise the features of density dependence available in standard PVA software, provide summary statistics about the use of density dependence in PVA case studies, and discuss the effects of density dependence on extinction probability. The introduction of an upper limit for population size has the effect that the probability of ultimate extinction becomes 1. Mean time to extinction increases with carrying capacity if populations start at high density, but carrying capacity often does not have any effect if populations start at low numbers. In contrast, the Allee effect is usually strong when populations start at low densities but has only a limited influence on persistence when populations start at high numbers. Contrary to previous opinions, other forms of density dependence may lead to increased or decreased persistence, depending on the type and strength of density dependence, the degree of environmental variability, and the growth rate. Furthermore, effects may be reversed for different quasi-extinction levels, making the use of arbitrary quasi-extinction levels problematic. Few systematic comparisons of the effects on persistence between different models of density dependence are available. These effects can be strikingly different among models. Our understanding of the effects of density dependence on extinction of metapopulations is rudimentary, but even opposite effects of density dependence can occur when metapopulations and single populations are contrasted. We argue that spatially explicit models hold particular promise for analysing the effects of density dependence on population viability provided a good knowledge of the biology of the species under consideration exists. Since the results of PVAs may critically depend on the way density dependence is modelled, combined efforts to advance statistical methods, field sampling, and modelling are urgently needed to elucidate the relationships between density, vital rates, and extinction probability.     

Amygdala-prefrontal synchronization underlies resistance to extinction of aversive memories

Emotional memories can persist for a lifetime but can also undergo extinction. Although we know about the mechanisms involved in expression and extinction, we know very little about the mechanisms that determine whether a specific memory would persist or not. Here, we use partial reinforcement extinction effect (PREE) to explore the neural mechanisms that differentiate persistent from labile memories. We recorded the simultaneous activity of neurons in the amygdala and the dorsal anterior cingulate cortex (dACC) while monkeys engaged in tone-odor aversive conditioning. We report that under continuous reinforcement schedule (ConS), activity in the amygdala precedes behavioral response, whereas under partial schedule (ParS), dACC activity precedes it. Moreover, we find that ParS induced cross-regional pairwise correlations throughout the entire acquisition session, and their magnitude and precision predicted the later resistance to extinction. Our results suggest that memory persistence depends on distributed representations and, specifically, resistance to extinction of aversive memories is maintained by correlated amygdala-dACC activity.