The role of contextual conditioning in the effect of reinforcer devaluation on instrumental performance by rats

Different groups of rats received different amounts of training to lever press for a food reinforcer before an aversion was conditioned to the food. This devaluation of the reinforcer reduced responding in both subsequent extinction and reinforced tests of responding to a degree that was independent of the amount of instrumental training. Moreover, interpolating context extinction between aversion conditioning and the extinction test reduced the magnitude of the devaluation effect, thereby indicating that Pavlovian contextual conditioning may play a role in the instrumental devaluation effect.     

Resistance to extinction, generalization decrement, and conditioned reinforcement

This study investigated generalization decrement during an extinction resistance-to-change test for pigeon key pecking using a two-component multiple schedule with equal variable-interval 3-min schedules and different reinforcer amounts (one component presented 2-s access to reinforcement and the other 8s). After establishing baseline responding, subjects were assigned to one of the two extinction conditions: hopper stimuli (hopper and hopper light were activated but no food was available) or Control (inactive hopper and hopper light). Responding in the 8-s component was more resistant to extinction than responding in the 2-s component, the hopper stimuli group was more resistant to extinction compared to the Control group, and an interaction between amount of reinforcement, extinction condition, and session block was present. This finding supports generalization decrement as a factor that influences resistance to extinction. Hopper-time data (the amount of time subjects spent with their heads in the hopper) were compared to resistance-to-change data in an investigation of the role of conditioned reinforcement on resistance to change.     

Ethanol seeking by long evans rats is not always a goal-directed behavior

Background

Two parallel and interacting processes are said to underlie animal behavior, whereby learning and performance of a behavior is at first via conscious and deliberate (goal-directed) processes, but after initial acquisition, the behavior can become automatic and stimulus-elicited (habitual). With respect to instrumental behaviors, animal learning studies suggest that the duration of training and the action-outcome contingency are two factors involved in the emergence of habitual seeking of “natural” reinforcers (e.g., sweet solutions, food or sucrose pellets). To rigorously test whether behaviors reinforced by abused substances such as ethanol, in particular, similarly become habitual was the primary aim of this study.

Methodology/Principal Findings

Male Long Evans rats underwent extended or limited operant lever press training with 10% sucrose/10% ethanol (10S10E) reinforcement (variable interval (VI) or (VR) ratio schedule of reinforcement), or with 10% sucrose (10S) reinforcement (VI schedule only). Once training and pretesting were complete, the impact of outcome devaluation on operant behavior was evaluated after lithium chloride injections were paired with the reinforcer, or unpaired 24 hours later. After limited, but not extended instrumental training, lever pressing by groups trained under VR with 10S10E and under VI with 10S was sensitive to outcome devaluation. In contrast, responding by both the extended and limited training 10S10E VI groups was not sensitive to ethanol devaluation during the test for habitual behavior.

Conclusions/Significance

Operant behavior by rats trained to self-administer an ethanol-sucrose solution showed variable sensitivity to a change in the value of ethanol, with relative insensitivity developing sooner in animals that received time-variable ethanol reinforcement during training sessions. One important implication, with respect to substance abuse in humans, is that initial learning about the relationship between instrumental actions and the opportunity to consume ethanol-containing drinks can influence the time course for the development or expression of habitual ethanol seeking behavior.     

Naloxone enhancement of punishment in the rat

Rats were trained to lever press on a variable interval one minute schedule of reinforcement for food pellets. Subsequently in punishment sessions, lever presses also produced electric grid shocks on a fixed ratio 20 schedule. With punishment, lever pressing levels were reduced by about 15%. When naloxone was administered prior to punishment sessions, shocks reduced lever pressing levels by about 50% following the 3 mg/kg dose and by aboaut 80% following the 10 mg/kg dose. This naloxone-induced enhancement of the effect of punishment was probably mediated through an antagonism of naturally-occurring brain peptides with opiate-like activities.     

Effect of short-term prefeeding and body weight on wheel running and responding reinforced by the opportunity to run in a wheel

A biobehavioural analysis of activity anorexia suggests that the motivation for physical activity is regulated by food supply and body weight. In the present experiment, food allocation was varied within subjects by prefeeding food-deprived rats 0, 5, 10 and 15 g of food before sessions of lever pressing for wheel-running reinforcement. The experiment assessed the effects of prefeeding on rates of wheel running, lever pressing, and postreinforcement pausing. Results showed that prefeeding animals 5 g of food had no effect. Prefeeding 10 g of food reduced lever pressing for wheel running and rates of wheel running without a significant change in body weight; the effect was, however, transitory. Prefeeding 15 g of food increased the animals' body weights, resulting in a sustained decrease of wheel running and lever pressing, and an increase in postreinforcement pausing. Overall the results indicate that the motivation for physical activity is regulated by changes in local food supply, but is sustained only when there is a concomitant change in body weight.     

Responding for sucrose and wheel-running reinforcement: effect of body weight manipulation

As body weight increases, the excitatory strength of a stimulus signaling an opportunity to run should weaken to a greater degree than that of a stimulus signaling an opportunity to eat. To test this hypothesis, six male albino Wistar rats were placed in running wheels and exposed to a fixed interval 30-s schedule that produced either a drop of 15% sucrose solution or the opportunity to run for 15s as reinforcing consequences for lever pressing. Each reinforcer type was signaled by a different stimulus. The effect of varying body weight on responding maintained by these two reinforcers was investigated by systematically increasing and decreasing post-session food amounts. The initial body weight was 335 g. Body weights were increased to approximately 445 g and subsequently returned to 335 g. As body weight increased, overall and local lever-pressing rates decreased while post-reinforcement pauses lengthened. Analysis of post-reinforcement pauses and local lever-pressing rates in terms of transitions between successive reinforcers revealed that local response rates in the presence of stimuli signaling upcoming wheel and sucrose reinforcers were similarly affected. However, pausing in the presence of the stimulus signaling a wheel-running reinforcer lengthened to a greater extent than did pausing in the presence of the stimulus signaling sucrose. This result suggests that as body weight approaches ad-lib levels, the likelihood of initiation of responding to obtain an opportunity to run approaches zero and the animal "rejects" the opportunity to run in a manner similar to the rejection of less preferred food items in studies of food selectivity.     

Reward-related behavioral paradigms for addiction research in the mouse: performance of common inbred strains

The mouse has emerged as a uniquely valuable species for studying the molecular and genetic basis of complex behaviors and modeling neuropsychiatric disease states. While valid and reliable preclinical assays for reward-related behaviors are critical to understanding addiction-related processes, and various behavioral procedures have been developed and characterized in rats and primates, there have been relatively few studies using operant-based addiction-relevant behavioral paradigms in the mouse. Here we describe the performance of the C57BL/6J inbred mouse strain on three major reward-related paradigms, and replicate the same procedures in two other commonly used inbred strains (DBA/2J, BALB/cJ). We examined Pavlovian-instrumental transfer (PIT) by measuring the ability of an auditory cue associated with food reward to promote an instrumental (lever press) response. In a separate experiment, we assessed the acquisition and extinction of a simple stimulus-reward instrumental behavior on a touch screen based task. Reinstatement of this behavior was then examined following either continuous exposure to cues (conditioned reinforcers, CRs) associated with reward, brief reward and CR exposure, or brief reward exposure followed by continuous CR exposure. The third paradigm examined sensitivity of an instrumental (lever press) response to devaluation of food reward (a probe for outcome insensitive, habitual behavior) by repeated pairing with malaise. Results showed that C57BL/6J mice displayed robust PIT, as well as clear extinction and reinstatement, but were insensitive to reinforcer devaluation. DBA/2J mice showed good PIT and (rewarded) reinstatement, but were slow to extinguish and did not show reinforcer devaluation or significant CR-reinstatement. BALB/cJ mice also displayed good PIT, extinction and reinstatement, and retained instrumental responding following devaluation, but, unlike the other strains, demonstrated reduced Pavlovian approach behavior (food magazine head entries). Overall, these assays provide robust paradigms for future studies using the mouse to elucidate the neural, molecular and genetic factors underpinning reward-related behaviors relevant to addiction research.     

Running response reinforcement by food-hoarding in the golden hamster

Partial reinforcement (PR) effects on animal locomotor behavior were studied in the golden hamster, using food-hoarding activity as a reinforcer. The first experiment demonstrated that hoarding reinforces a running response towards the goal section of a straight-alley runway, and that no such learning occurs when sated hamsters were not allowed to hoard food. However, a second experiment using various partial reinforcement schedules and a continuous reinforcement schedule did not give any evidence for the existence of a partial reinforcement acquisition effect (PRAE). The third experiment confirmed these results with an extended training procedure and showed a slight partial reinforcement extinction effect (PREE) mainly in the first sessions of the extinction phase.     

The role of contextual cues in operant responding in rats

Following lever-press training on a variable-interval 60-second schedule of food presentation, groups of rats either remained in their home cages or were exposed to the operant chamber, from which lever and food had been removed, for five sessions. The lever was replaced in the chamber and rats from Group 1 (exposure to chamber) and Group 3 (home cage) were returned to the variable-interval schedule. Although response rates in test sessions were somewhat lower than at the end of training, there was no statistically significant difference in rates for either group. Rats in Group 2 (exposure to chamber) and Group 4 (home cage) received two test sessions of extinction. During the first session, Group 2 rates of lever pressing were significantly higher than Group 4 rates. These findings do not support the view that associations between contextual cues and the reinforcer serve to energize instrumental behavior (Pearce & Hall, 1979), and provide only minimal support for the view that contextual cues control responses that compete with the operant (Mills, 1980).     

Effects of reinforcer magnitude on response acquisition with unsignaled delayed reinforcement

Sixteen rats received eight 1-h sessions of a tandem fixed-ratio 1 differential-reinforcement-of-other-behavior 30-s schedule with reinforcer magnitude at one or six food pellet(s) across groups of eight rats. The larger reinforcer magnitude established the lever press more effectively. First, mean schedule completions differed across groups, in terms of both their overall difference and in their increase across sessions. Second, whereas the larger reinforcer magnitude established reliable response acquisition in all eight rats by the end of the experiment, the smaller reinforcer magnitude only established reliable response acquisition in four of eight rats. The present results systematically replicate earlier findings obtaining more reliable response acquisition with unsignaled delayed reinforcement with greater food deprivation. Taken together, this work demonstrates the influence of motivational variables on response acquisition with unsignaled delayed reinforcement.     

Role of conditioned reinforcers in the initiation, maintenance and extinction of drug-seeking behavior.

The development of a secondary reinforcer as a result of associating a neutral stimulus (buzzer) with intravenous (IV) doses of morpine was studied in rats. Secondary reinforcement developed in the absence of physical dependence and followed the association of the stimulus with either response-contingent or non-contingent injections of morphine. Strength of the conditioned reinforcer, measured in terms of responding on a lever for the stimulus plus infusion of saline solution, was proportional to the unit dosage of morphine employed in pairings of buzzer and drug. When extinction of the lever-press response for IV morphine was conducted (by substituting saline for morphine solution) in the absence of the conditioned reinforcing stimulus, it was seen later that the stimulus could still elicit lever responses, until it too had been present for a sufficient interval of non-reinforced responding. Similarly, extinction of the response for morphine by blocking its action with naloxone in the absence of the stimulus did not eliminate the conditioned reinforcement. Another study showed that a passive, subcutaneous (SC) dose of morphine served to maintain lever-pressing on a contingency of buzzer plus saline infusion. Furthermore, the stimuli resulting from the presence of morphine (after a SC injection) were able to reinstate the lever-responding with only the buzzer-saline contingency when such responses had previously been extinguished. Moreover, it was shown that d-amphetamine could restore responding under the same conditions, and that morphine could also do so for rats in which the primary reinforcer had been d-amphetamine. It is suggested that animal data such as these show that procedures designed for the elimination of human drug-taking behavior must take into account secondary reinforcers as well as the primary reinforcer(s).     

Comparison of the instrumental reinforcer devaluation effect in two strains of rats (Wistar and Lister)

In this experiment, the effect of the reinforcer devaluation upon instrumental performance was analysed in two strains of rats (Wistar and Lister): Food deprived rats were trained to press a lever for sucrose pellets in a single session. Immediately after the fulfilment of this session, half of the Wistar and Lister rats received an injection of lithium chloride (LiCl), while the remaining animals were injected with a sodium chloride (NaCl) solution. A subsequent extinction test showed that the subjects who had received immediate LiCl did not press the lever as often as those injected with NaCl. No differences in response suppression were found between the two strains of rats. These results also show that a single devaluation experience is sufficient for an impact on instrumental performance.     

Extinction as discrimination: the molar view

The traditional molecular view of behavior explains extinction as the dissipation or inhibition of strength, formerly built up by contiguous reinforcement. In obstinate opposition to this explanation was the partial-reinforcement extinction effect: a partially reinforced response extinguishes more slowly than a continuously reinforced response. It suggests instead that extinction is discrimination. Four pigeons were exposed to daily sessions in which a variable period of food delivery, produced by pecking on a variable-interval schedule, was followed by extinction. The rate of food delivery was varied over a wide range across conditions. Varying the amount of food per delivery inversely with rate of delivery kept response rate from varying excessively. The results confirmed and extended the partial-reinforcement effect; persistence of pecking and time to extinction were inversely related to rate of obtaining food. The results support the molar view of extinction, not as loss of strength of a particular discrete response, but as a transition from one allocation of time among activities to another. Although molecular theories dismiss discrimination due to repeated training and extinction as an impurity or complication, repeated cycles of availability and privation are probably typical of the environment in which most vertebrate species evolved.     

Activation of Exchange Protein Activated by cAMP in the Rat Basolateral Amygdala Impairs Reconsolidation of a Memory Associated with Self-Administered Cocaine

The intracellular mechanisms underlying memory reconsolidation critically involve cAMP signaling. These events were originally attributed to PKA activation by cAMP, but the identification of Exchange Protein Activated by cAMP (Epac), as a distinct mediator of cAMP signaling, suggests that cAMP-regulated processes that subserve memory reconsolidation are more complex. Here we investigated how activation of Epac with 8-pCPT-cAMP (8-CPT) impacts reconsolidation of a memory that had been associated with cocaine self-administration. Rats were trained to lever press for cocaine on an FR-1 schedule, in which each cocaine delivery was paired with a tone+light cue. Lever pressing was then extinguished in the absence of cue presentations and cocaine delivery. Following the last day of extinction, rats were put in a novel context, in which the conditioned cue was presented to reactivate the cocaine-associated memory. Immediate bilateral infusions of 8-CPT into the basolateral amygdala (BLA) following reactivation disrupted subsequent cue-induced reinstatement in a dose-dependent manner, and modestly reduced responding for conditioned reinforcement. When 8-CPT infusions were delayed for 3 hours after the cue reactivation session or were given after a cue extinction session, no effect on cue-induced reinstatement was observed. Co-administration of 8-CPT and the PKA activator 6-Bnz-cAMP (10 nmol/side) rescued memory reconsolidation while 6-Bnz alone had no effect, suggesting an antagonizing interaction between the two cAMP signaling substrates. Taken together, these studies suggest that activation of Epac represents a parallel cAMP-dependent pathway that can inhibit reconsolidation of cocaine-cue memories and reduce the ability of the cue to produce reinstatement of cocaine-seeking behavior.     

A comparison of the effects of discriminative and Pavlovian inhibitors and excitors on instrumental responding

In two experiments, the effects of Pavlovian or discriminative conditioned inhibitors on operant responding were investigated in rats. Experiment 1 found that a Pavlovian conditioned inhibitor for food suppressed food-reinforced lever pressing more than a non-differentially trained control stimulus did. Experiment 2 demonstrated that an operant discriminative inhibitor produced greater suppression of lever pressing than a Pavlovian conditioned inhibitor. Experiment 2 also found that compounding an operant discriminative stimulus (SD) for food-reinforced responding with another SD for food-reinforced responding resulted in more additive summation than when an SD was compounded with a Pavlovian conditioned excitor for food. The results of these experiments support two-factor theories that postulate that incentive and response discriminative processes summate algebraically when the processes are inhibitory or excitatory.     

60 Hz electric fields and incandescent light as aversive stimuli controlling the behavior of rats responding under concurrent schedules of reinforcement

Several reports have shown that animals will sometimes engage in behaviors that reduce their exposure to a 60 Hz electric field (E-field). The field, therefore, can function as an aversive stimulus. In other studies, the E-field at equivalent strengths failed to function as an aversive stimulus. The present experiment, using rats, demonstrates how factors other than field strength can influence whether a subject engages in behavior that reduces field exposure. The general design consisted of giving the rat a choice between two alternatives, one of which sometimes included an added stimulus. Each subject was trained to press each of two levers to obtain food. Pressing one lever was reinforced intermittently under a variable interval 2 min schedule (VI 2); pressing the other lever was reinforced by a second VI 2 schedule operating independently of the first. Under this concurrent schedule the rat spent 50% of the daily 50 min session responding to each of the levers, indicating that they were equally “valued.” Next, while the schedules remained in effect, the first response to one of the levers turned on a 100 kV/m E-field which remained on until the rat pressed the other lever. The time spent responding under the schedule associated with the field was reduced by about 5–10%. When the procedure was changed so that no lever presses produced food, i.e., extinction, but the added stimulus contingency remained, the rats spent even less time in the presence of the field. Similar outcomes were observed during both the concurrent food or extinction schedules when incandescent light was used. Thus, both an E-field and incandescent light functioned as aversive stimuli, but the magnitude of the aversiveness was small. Aversiveness depended not only on stimulus intensity, but also on behavioral factors. Bioelectromagnetics 19:210–221, 1998. © 1998 Wiley-Liss, Inc.     

Dynamics of reorganization of instrumental food getting behavior of monkey under conditions of reinforcement delay

Six adult monkeys (Macaca fascicularis) were trained to switch on a lamp by pressing the lever, to hold the lever for not less than 1 s (the lamp being switched on), and get a portion of food. After reaching a learning criterion, the delay between the lever pressing and food reinforcement was increased to 2.5 s. The experimental procedure was repeated in three experimental sessions with 2-month between-session intervals. It was shown that the retraining process after the uneven change in the delay duration developed in three stages: (1) stage of emotional hyperreactivity that reflected a mismatch between the cation and absence of the expected result; (2) stage of intermediate stabilization, when the percent of efficient attempts was the same as under conditions of 1-s delay; (3) stage of purposive instrumental lever holding till the moment of reinforcement presentation.     

Do peptide-induced changes in feeding occur because of changes in motivation to eat?

The usual paradigm in which peptides are tested for their effect on food intake involves measuring intake of readily available food. In the lever press apparatus, the subjects must "work" to get food. Such work has traditionally been used as a means of measuring motivation. Mice were trained to press a lever for milk reinforcement. After achieving a stable level of performance, we tested the effects of gastrin-related peptide (GRP), bombesin (BBS) and cholecystokinin octapeptide (CCK-8) on lever pressing. All three peptides suppressed lever pressing for milk reinforcement. Prefeeding mice with milk increased the suppression of lever pressing to a greater extent in peptide-treated mice than in saline-treated mice. As the duration of prefeeding increased, lever pressing decreased. When mice were required to make more lever presses to obtain milk, both saline- and CCK-8-treated mice increased their lever pressing. However, saline-treated mice pressed at a higher rate than CCK-8-treated mice. Unlike the results obtained with saline and CCK-8, administration of a known gustatory adversant, lithium chloride, suppressed lever pressing to the same degree in mice fed or not fed prior to training. The results are consistent with the hypothesis that these peptides act as satiety agents.     

Excitatory backward conditioning in an appetitive conditioned reinforcement preparation with rats

Four experiments were conducted to examine appetitive backward conditioning in a conditioned reinforcement preparation. In all experiments, off-line classical conditioning was conducted following lever-press training on two levers. Presentations of a sucrose solution by a liquid dipper served as an unconditioned stimulus (US) and two auditory stimuli served as conditioned stimuli (CSs); one was paired with the US in either a forward (Experiment 1a) or a backward (Experiments 1b, 2, and 3) relationship, and the other served as a control CS, which was not paired with the US. In testing, each lever-press response produced a presentation of one of the CSs instead of appetitive reinforcers. The response to a lever was facilitated, compared to the response to another lever, when the response produced the backward CS presentation as well as when it produced the forward CS presentation; that is, the backward CS served as an excitatory conditioned reinforcer.     

Resistance to extinction and behavioral momentum

In the metaphor of behavioral momentum, reinforcement is assumed to strengthen discriminated operant behavior in the sense of increasing its resistance to disruption, and extinction is viewed as disruption by contingency termination and reinforcer omission. In multiple schedules of intermittent reinforcement, resistance to extinction is an increasing function of reinforcer rate, consistent with a model based on the momentum metaphor. The partial-reinforcement extinction effect, which opposes the effects of reinforcer rate, can be explained by the large disruptive effect of terminating continuous reinforcement despite its strengthening effect during training. Inclusion of a term for the context of reinforcement during training allows the model to account for a wide range of multiple-schedule extinction data and makes contact with other formulations. The relation between resistance to extinction and reinforcer rate on single schedules of intermittent reinforcement is exactly opposite to that for multiple schedules over the same range of reinforcer rates; however, the momentum model can give an account of resistance to extinction in single as well as multiple schedules. An alternative analysis based on the number of reinforcers omitted to an extinction criterion supports the conclusion that response strength is an increasing function of reinforcer rate during training.