Relapse processes after the extinction of instrumental learning: renewal, resurgence, and reacquisition

It is widely recognized that extinction (the procedure in which a Pavlovian conditioned stimulus or an instrumental action is repeatedly presented without its reinforcer) weakens behavior without erasing the original learning. Most of the experiments that support this claim have focused on several "relapse" effects that occur after Pavlovian extinction, which collectively suggest that the original learning is saved through extinction. However, although such effects do occur after instrumental extinction, they have not been explored there in as much detail. This article reviews recent research in our laboratory that has investigated three relapse effects that occur after the extinction of instrumental (operant) learning. In renewal, responding returns after extinction when the behavior is tested in a different context; in resurgence, responding recovers when a second response that has been reinforced during extinction of the first is itself put on extinction; and in rapid reacquisition, extinguished responding returns rapidly when the response is reinforced again. The results provide new insights into extinction and relapse, and are consistent with principles that have been developed to explain extinction and relapse as they occur after Pavlovian conditioning. Extinction of instrumental learning, like Pavlovian learning, involves new learning that is relatively dependent on the context for expression.     

Dissociable Effects of 5-HT2C Receptor Antagonism and Genetic Inactivation on Perseverance and Learned Non-Reward in an Egocentric Spatial Reversal Task

Cognitive flexibility can be assessed in reversal learning tests, which are sensitive to modulation of 5-HT_2C receptor (5-HT_2CR) function. Successful performance in these tests depends on at least two dissociable cognitive mechanisms which may separately dissipate associations of previous positive and negative valence. The first is opposed by perseverance and the second by learned non-reward. The current experiments explored the effect of reducing function of the 5-HT_2CR on the cognitive mechanisms underlying egocentric reversal learning in the mouse. Experiment 1 used the 5-HT_2CR antagonist SB242084 (0.5 mg/kg) in a between-groups serial design and Experiment 2 used 5-HT_2CR KO mice in a repeated measures design. Animals initially learned to discriminate between two egocentric turning directions, only one of which was food rewarded (denoted CS+, CS−), in a T- or Y-maze configuration. This was followed by three conditions; (1) Full reversal, where contingencies reversed; (2) Perseverance, where the previous CS+ became CS− and the previous CS− was replaced by a novel CS+; (3) Learned non-reward, where the previous CS− became CS+ and the previous CS+ was replaced by a novel CS-. SB242084 reduced perseverance, observed as a decrease in trials and incorrect responses to criterion, but increased learned non-reward, observed as an increase in trials to criterion. In contrast, 5-HT_2CR KO mice showed increased perseverance. 5-HT_2CR KO mice also showed retarded egocentric discrimination learning. Neither manipulation of 5-HT_2CR function affected performance in the full reversal test. These results are unlikely to be accounted for by increased novelty attraction, as SB242084 failed to affect performance in an unrewarded novelty task. In conclusion, acute 5-HT_2CR antagonism and constitutive loss of the 5-HT_2CR have opposing effects on perseverance in egocentric reversal learning in mice. It is likely that this difference reflects the broader impact of 5HT_2CR loss on the development and maintenance of cognitive function.     

Worrying affects associative fear learning: a startle fear conditioning study

A valuable experimental model for the pathogenesis of anxiety disorders is that they originate from a learned association between an intrinsically non-aversive event (Conditioned Stimulus, CS) and an anticipated disaster (Unconditioned Stimulus, UCS). Most anxiety disorders, however, do not evolve from a traumatic experience. Insights from neuroscience show that memory can be modified post-learning, which may elucidate how pathological fear can develop after relatively mild aversive events. Worrying--a process frequently observed in anxiety disorders--is a potential candidate to strengthen the formation of fear memory after learning. Here we tested in a discriminative fear conditioning procedure whether worry strengthens associative fear memory. Participants were randomly assigned to either a Worry (n = 23) or Control condition (n = 25). After fear acquisition, the participants in the Worry condition processed six worrisome questions regarding the personal aversive consequences of an electric stimulus (UCS), whereas the Control condition received difficult but neutral questions. Subsequently, extinction, reinstatement and re-extinction of fear were tested. Conditioned responding was measured by fear-potentiated startle (FPS), skin conductance (SCR) and UCS expectancy ratings. Our main results demonstrate that worrying resulted in increased fear responses (FPS) to both the feared stimulus (CS(+)) and the originally safe stimulus (CS(-)), whereas FPS remained unchanged in the Control condition. In addition, worrying impaired both extinction and re-extinction learning of UCS expectancy. The implication of our findings is that they show how worry may contribute to the development of anxiety disorders by affecting associative fear learning.     

Effect of amount of context extinction on revaluation of a target CS

In two experiments using rats in a conditioned lick suppression paradigm the effects of posttraining extinction of the conditioning context on responding to the target conditioned stimulus (CS) which had previously been paired with a footshock was examined in a neutral context. Experiment 1 replicated Marlin's [Learn. Motiv. 13 (1981) 526] observation that this treatment can reduce responding to the target CS. This finding is contrary to other reports that context extinction enhances responding to the target CS. Experiment 2 examined the amount of posttraining context extinction as a variable potentially controlling this effect. It found the mediated extinction effect following one, but not 10 h of context extinction.     

Interstimulus interval as a discriminative stimulus: Evidence of the generality of a novel asymmetry in temporal discrimination learning

Three appetitive conditioning experiments with rats examined temporal discrimination learning within Pavlovian conditioning trials. In all experiments, the duration of a feature white noise stimulus signaled whether or not a subsequent 10-s target tone would be reinforced. In Experiment 1, the feature durations were 4 and 1 min. For one group of rats (Group 4+/1−), 4 min of noise signaled that the tone would be reinforced and 1 min of noise signaled that the tone would not be reinforced. A second group (Group 1+/4−) was trained with the reverse contingency. The results showed a clear asymmetry in temporal discrimination learning: rats trained with 4+/1− (Long+/Short−) learned the discrimination readily (responding more in the tone on reinforced than on nonreinforced trials), whereas rats trained with 1+/4− (Short+/Long) did not. In Experiment 2, the feature durations were shortened to 60 and 15 s. Due to strong excitatory conditioning of the 15-s feature, the reverse asymmetry was observed, with the Short+/Long− discrimination learned more readily than the Long+/Short− discrimination. However, Experiment 3 demonstrated that the original Long+/Short− advantage could be recovered while using 60− and 15-s feature durations if the excitatory conditioning of the feature was reduced by including nonreinforced feature trials. The results support previous research involving the timing of intertrial intervals and are consistent with the temporal elements hypothesis which holds that the passage of time is encoded as a series of hypothetical stimulus elements.     

Orexin受体1对摄食条件反射的调控研究

目的:观察Orexin受体1对摄食条件反射的调控研究。方法:将40只大鼠随机分为四组,分别为NS/NS组,SB/SB组,NS/SB组,SB/NS组,每组10只大鼠,给予大鼠八组"条件刺激-无条件刺激(CS-US)"训练,给予无条件刺激后立即进行条件刺激,每组进行训练前30 min给予SB或生理盐水(NS)。获得性训练后,给予2组反射消失性训练,即给予8次条件刺激。条件刺激是给予大鼠10 s,2kHZ声音刺激,无条件刺激是直接给予大鼠食物。结果:给予条件刺激后,四组大鼠摄食行为均明显增加,摄食间隔均明显缩短,当条件刺激强度增加时,摄食行为也增加,而预先给予SB,与NS/NS组相比,其余组大鼠摄食行为相对减少(P0.05),摄食间隔增加。消失性训练中,与NS/NS,NS/SB组相比,SB/NS组和SB/SB组大鼠摄食行为明显减少(P0.05)。与其他三组大鼠相比,SB/NS组大鼠摄食间隔缩短。预先给予SB使后两次刺激后摄食间隔明显增加(P0.05)。结论:OX1R信号通路调控摄食反射的产生和消失。     

Effect of US identity on elimination and recovery of autoshaped responding with explicitly unpaired and degraded contingency extinction procedures

The depressive effects of noncontingent and explicitly unpaired food unconditioned stimuli (USs) and the recovery from those effects on autoshaped responding were examined in a series of experiments with pigeons. In Experiments 1 and 2, responding to a keylight conditioned stimulus (CS) previously paired with food was depressed equally by explicitly unpaired presentations of either that same food or a different food. Furthermore, responding recovered equally following removal of the explicitly unpaired foods. In contrast, Experiments 3 and 4 showed that noncontingent presentations of a food US depressed responding more to a keylight CS paired with that same food than to a keylight CS paired with a different food. Moreover, removal of the noncontingent foods led to complete recovery but more rapid extinction of responding to the same keylight relative to the different keylight. The implications of these results for understanding the mechanisms underlying depression and recovery of responding following degraded contingency and explicitly unpaired extinction procedures are discussed.     

Intertrial interval as a contextual stimulus

Four experiments with rats investigated whether the time between appetitive conditioning trials can serve as a discriminative cue for responding during the next conditional stimulus (CS). In Experiment 1, rats that received extinction trials with a 4-min intertrial interval (ITI) showed spontaneous recovery after a retention interval of 16 min, whereas rats that received extinction with a 16-min ITI did not. Experiments 2 and 3 investigated more explicit discriminations between the 4- and 16-min ITIs. When a 16-min ITI signaled that the CS would be reinforced and a 4-min ITI signaled that it would not, the ITIs modulated responding to the CS. But when the 4-min ITI signaled reinforcement and the 16-min ITI did not, there was less evidence of modulation by the ITIs. This asymmetry was due at least partly to a difficulty in performance rather than learning. Experiment 4 investigated similar discriminations with 1- and 4-min ITIs. Here the results took a different form: time in the reinforced ITI elicited responding directly, but did not modulate responding to the CS. ITI can function as a contextual cue, and the results suggest new similarities between the processes behind interval timing and associative learning.     

Effects of an extinguished CS on competition with another CS

Three experiments were conducted using a conditioned taste aversion procedure with rats to examine the effect of nonreinforced presentations of a conditioned stimulus (CS) on its ability to compete with a target stimulus for manifest conditioned responding. Two CSs (A and B) were presented in a serial compound and then paired with the unconditioned stimulus. CS A was first paired with the US and then presented without the US (i.e., extinction) prior to reinforced presentation of the AB compound. Experiment 1 showed that A was poor at competing with B for conditioned responding when given conditioning and extinction prior to reinforcement of AB relative to a group that received both A and B for the first time during compound conditioning. That is, an extinguished A stimulus allowed greater manifest acquisition to B. Experiment 2 found that extinction treatment produced a poor CR to the pretrained and extinguished CS itself following compound conditioning. Experiment 3 found that interposing a retention interval after extinction of A and prior to compound conditioning enhanced A's ability to compete with B. The results of these experiments are discussed with regard to different theories of extinction and associative competition.     

Genetic disruptions of Drosophila Pavlovian learning leave extinction learning intact

Individuals who experience traumatic events may develop persistent posttraumatic stress disorder. Patients with this disorder are commonly treated with exposure therapy, which has had limited long‐term success. In experimental neurobiology, fear extinction is a model for exposure therapy. In this behavioral paradigm, animals are repeatedly exposed in a safe environment to the fearful stimulus, which leads to greatly reduced fear. Studying animal models of extinction already has lead to better therapeutic strategies and development of new candidate drugs. Lack of a powerful genetic model of extinction, however, has limited progress in identifying underlying molecular and genetic factors. In this study, we established a robust behavioral paradigm to study the short‐term effect (acquisition) of extinction in Drosophila melanogaster. We focused on the extinction of olfactory aversive 1‐day memory with a task that has been the main workhorse for genetics of memory in flies. Using this paradigm, we show that extinction can inhibit each of two genetically distinct forms of consolidated memory. We then used a series of single‐gene mutants with known impact on associative learning to examine the effects on extinction. We find that extinction is intact in each of these mutants, suggesting that extinction learning relies on different molecular mechanisms than does Pavlovian learning.     

BI-stability of emotions and motivations: An evolutionary consequence of the open-ended capacity for learning

Species, endowed with an open-ended capacity for learning, which is one of the highest evolutionary achievements,will profit most from this ability, if they are urged one way or other to invest any surplus of energy in expanding and refining their behavioural repertoire and in adapting it to prevailing circumstances, while incurring as little risk and stress as possible. It is therefore argued that an open-ended capacity for learning is maximally adding to survival if paired to two distinct tendencies:

1. a tendency to seek high-arousal evoking situations whenever surplus energy is available, and
2. a tendency to seek arousal reducing situations as soon as the surplus energy is exhausted.

This suggests that a bi-stable “telic/paratelic” system of prefered levels of arousal, as described for humans by Apter & Smith's theory of motivational reversals, and as expressed in Kortmulder's model of the “centripetal/centrifugal” bi-polarity of animal behavioural tendencies, can be considered an Evolutionarily Stable Strategy (E.S.S.), as compared to homeostatic systems of arousal and motivation.     

The extinction context enables extinction performance after a change in context

One experiment with human participants determined the extent to which recovery of extinguished responding with a context switch was due to a failure to retrieve contextually controlled learning, or some other process such as participants learning that context changes signal reversals in the meaning of stimulus-outcome relationships. In a video game, participants learned to suppress mouse clicking in the presence of a stimulus that predicted an attack. Then, that stimulus underwent extinction in a different context (environment within the game). Following extinction, suppression was recovered and then extinguished again during testing in the conditioning context. In a final test, participants that were tested in the context where extinction first took place showed less of a recovery than those tested in a neutral context, but they showed a recovery of suppression nevertheless. A change in context tended to cause a change in the meaning of the stimulus, leading to recovery in both the neutral and extinction contexts. The extinction context attenuated that recovery, perhaps by enabling retrieval of the learning that took place in extinction. Recovery outside an extinction context is due to a failure of the context to enable the learning acquired during extinction, but only in part.     

Contextual fear conditioning differs for infant, adolescent, and adult rats

Contextual fear conditioning was tested in infant, adolescent, and adult rats in terms of Pavlovian-conditioned suppression. When a discrete auditory-conditioned stimulus (CS) was paired with footshock (unconditioned stimulus, US) within the largely olfactory context, infants and adolescents conditioned to the context with substantial effectiveness, but adult rats did not. When unpaired presentations of the CS and US occurred within the context, contextual fear conditioning was strong for adults, weak for infants, but about as strong for adolescents as when pairings of CS and US occurred in the context. Nonreinforced presentations of either the CS or context markedly reduced contextual fear conditioning in infants, but, in adolescents, CS extinction had no effect on contextual fear conditioning, although context extinction significantly reduced it. Neither CS extinction nor context extinction affected responding to the CS–context compound in infants, suggesting striking discrimination between the compound and its components. Female adolescents showed the same lack of effect of component extinction on response to the compound as infants, but CS extinction reduced responding to the compound in adolescent males, a sex difference seen also in adults. Theoretical implications are discussed for the development of perceptual-cognitive processing and hippocampus role.     

Classical conditioning and retention in normal and mutantDrosophila melanogaster

Summary By changing the conditioned discrimination paradigm of Quinn et al. (1974) from an instrumental procedure to a classical (Pavlovian) one, we have demonstrated strong learning in type flies. About 150 flies were sequestered in a closed chamber and trained by explosing them sequentially to two odors in air currents. Flies received twelve electric shock pulses in the presence of the first odor (CS+) but not in the presence of the second odor (CS–). To test for conditioned avoidance responses, flies were transported to a Tmaze choice point, between converging currents of the two odors. Typically, 95% of trained flies avoided the shock-associated odor (CS+).Acquisition of learning was a function of the number of shock pulses received during CS+ presentation and was asymptotic within one training cycle. Conditioned avoidance increased with increasing shock intensity or odor concentration and was very resistant to extinction. Learning was best when CS+ presentations overlap shock (delay conditioning) and then decreased with increasing CS-US interstimulus intervals. Shocking flies immediately before CS+ presentation (backward conditioning) produced no learning. Nonassociative control procedures (CS Alone, US Alone and Explicitly Unpaired) produced slight decreases in avoidance responses, but these affected both odors equally and did not alter our associative learning index (A).Memory in wild-type flies decayed gradually over the first seven hours after training and still was present 24 h later. The mutantsamnesiac, rutabaga anddunce showed appreciable learning acquisition, but their memories decayed very rapidly during the first 30 min. After this, the rates of decay slowed sharply; conditioned avoidance still was measurable at least three hours after training.Abbreviations OCT 3-octanol - MCH 4-methylcyclohexanol - C-S Canton-Special - CS conditioned stimulus - US unconditioned stimulus     

Relative temporal representations in Pavlovian conditioning

The transfer of relative temporal representations was assessed in a series of three experiments. In each experiment, rats (Rattus norvegicus) received one set of conditioned stimulus (CS) and intertrial interval (ITI) durations in Phase 1 and another set in Phase 2. The ratio between the CS and ITI intervals was either changed or maintained across phases. On the hypothesis that relative temporal representations are learned, groups receiving maintained temporal ratios across phases were expected to display greater change in responding upon encountering the new intervals. When the CS duration decreased across phases, maintaining the temporal ratio did lead to greater change in Day 1 of Phase 2 towards the final pattern of responding. However, when the CS increased across phases, maintaining the temporal ratio across phases did not facilitate adjustment to the new intervals, suggesting that extinction of previously reinforced times induced new learning. These results provide evidence that under some conditions, relative relationships in temporal maps may survive transformation-of-scale, like relative relationships in spatial maps.     

Allosteric communication in dihydrofolate reductase: signaling network and pathways for closed to occluded transition and back

Escherichia coli dihydrofolate reductase (DHFR) catalyzes the reduction of dihydrofolate to tetrahydrofolate. During the catalytic cycle, DHFR undergoes conformational transitions between the closed (CS) and occluded (OS) states that, respectively, describe whether the active site is closed or occluded by the Met20 loop. The CS→OS and the reverse transition may be viewed as allosteric transitions. Using a sequence-based approach, we identify a network of residues that represents the allostery wiring diagram. Many of the residues in the allostery wiring diagram, which are dispersed throughout the adenosine-binding domain as well as the loop domain, are not conserved. Several of the residues in the network have been previously shown by NMR experiments, mutational studies, and molecular dynamics simulations to be linked to equilibration conformational fluctuations of DHFR. To further probe the nature of events that occur during conformational fluctuations, we use a self-organized polymer model to monitor the kinetics of the CS→OS and the reverse transitions. During the CS→OS transition, coordinated changes in a number of residues in the loop domain enable the Met20 loop to slide along the α-helix in the adenosine-binding domain. Sliding is triggered by pulling of the Met20 loop by the βG-βH loop and the pushing action of the βG-βH loop. The residues that facilitate the Met20 loop motion are part of the network of residues that transmit allosteric signals during the CS→OS transition. Replacement of M16 and G121, whose C^α atoms are about 4.3 Å in the CS, by a disulfide cross-link impedes that CS→OS transition. The order of events in the OS→CS transition is not the reverse of the forward transition. The contact Glu18-Ser49 in the OS persists until the sliding of the Met20 loop is nearly complete. The ensemble of structures in the transition state in both the allosteric transitions is heterogeneous. The most probable transition-state structure resembles the OS (CS) in the CS→OS (OS→CS) transition, which is in accord with the Hammond postulate. Structures resembling the OS (CS) are present as minor (∼ 1-3%) components in equilibrated CS (OS) structures.     

A New Framework for Cortico-Striatal Plasticity: Behavioural Theory Meets In Vitro Data at the Reinforcement-Action Interface

Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s) coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem—action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface.     

Extinction under a behavioral microscope: isolating the sources of decline in operant response rate

Extinction performance is often used to assess underlying psychological processes without the interference of reinforcement. For example, in the extinction/reinstatement paradigm, motivation to seek drug is assessed by measuring responding elicited by drug-associated cues without drug reinforcement. However, extinction performance is governed by several psychological processes that involve motivation, memory, learning, and motoric functions. These processes are confounded when overall response rate is used to measure performance. Based on evidence that operant responding occurs in bouts, this paper proposes an analytic procedure that separates extinction performance into several behavioral components: (1-3) the baseline bout initiation rate, within-bout response rate, and bout length at the onset of extinction; (4-6) their rates of decay during extinction; (7) the time between extinction onset and the decline of responding; (8) the asymptotic response rate at the end of extinction; (9) the refractory period after each response. Data that illustrate the goodness of fit of this analytic model are presented. This paper also describes procedures to isolate behavioral components contributing to extinction performance and make inferences about experimental effects on these components. This microscopic behavioral analysis allows the mapping of different psychological processes to distinct behavioral components implicated in extinction performance, which may further our understanding of the psychological effects of neurobiological treatments.     

Resting Heart Rate Variability Predicts Safety Learning and Fear Extinction in an Interoceptive Fear Conditioning Paradigm

This study aimed to investigate whether interindividual differences in autonomic inhibitory control predict safety learning and fear extinction in an interoceptive fear conditioning paradigm. Data from a previously reported study (N = 40) were extended (N = 17) and re-analyzed to test whether healthy participants'' resting heart rate variability (HRV) - a proxy of cardiac vagal tone - predicts learning performance. The conditioned stimulus (CS) was a slight sensation of breathlessness induced by a flow resistor, the unconditioned stimulus (US) was an aversive short-lasting suffocation experience induced by a complete occlusion of the breathing circuitry. During acquisition, the paired group received 6 paired CS-US presentations; the control group received 6 explicitly unpaired CS-US presentations. In the extinction phase, both groups were exposed to 6 CS-only presentations. Measures included startle blink EMG, skin conductance responses (SCR) and US-expectancy ratings. Resting HRV significantly predicted the startle blink EMG learning curves both during acquisition and extinction. In the unpaired group, higher levels of HRV at rest predicted safety learning to the CS during acquisition. In the paired group, higher levels of HRV were associated with better extinction. Our findings suggest that the strength or integrity of prefrontal inhibitory mechanisms involved in safety- and extinction learning can be indexed by HRV at rest.