Stimulus change dis-habituates operant responding supported by water reinforcers

The present experiment examined whether habituation contributes to within-session decreases in operant responding for water reinforcers. The experiment asked if this responding can be dishabituated, a fundamental property of habituated behavior. During baseline, rats’ lever pressing was reinforced by water on a variable interval 15-s schedule. During experimental conditions, rats responded on the same schedule and a new stimulus was introduced for 5 min at 15, 30 or 45 min into the 60-min session. The new stimulus was extinction, continuous reinforcement or flashing lights in different conditions. Rate of responding primarily decreased within the session during baseline. Introducing a new stimulus sometimes suppressed (extinction, continuous reinforcement) and sometimes increased (flashing lights) responding while it was in effect. The new stimulus increased responding after it ended and before it was presented in the session. The results are incompatible with the idea that non-habituation satiety factors (e.g., cellular hydration and blood volume) contributed to within-session changes in responding. These satiety factors should increase with increases in consumption, decrease with decreases in consumption and remain constant with constant consumption of water. Nevertheless, all stimulus changes increased operant responding for water. These results support the idea that habituation contributes to within-session decreases in responding for water reinforcers.     

Extending mathematical principles of reinforcement into the domain of behavioral pharmacology

Mathematical principles of reinforcement (MPR) attempts to integrate the empirical laws of reinforcement schedules that have accumulated over the decades. MPR is based on three principles: incentives excite behavior; there are temporal constraints on responding; and coupling of responses to reinforcers strengthens behavior [Behav. Brain Sci. 17 (1994) 105]. In the present paper MPR is extended into the domain of behavioral pharmacology, specifically to model the effects of D-amphetamine on operant behavior. In Experiment 1a, a five-component multiple fixed-ratio schedule was designed to generate behavioral baselines that were subsequently used to assess drug effect. In Experiments 1b and 1c, the quality and quantity of reinforcer were manipulated. The data generated by the three experiments were consistent with MPR. In Experiment 2, MPR was used to model the effects of D-amphetamine on rats responding under the five-component multiple fixed-ratio schedule. According to the model, the rate-decreasing effects of D-amphetamine were due primarily to motor disruption and secondarily to increased impulsivity; at the highest dosages, D-amphetamine also may have decreased the incentive value of food.     

Is extinction the hallmark of operant discrimination?: Reinforcement and S effects

Using a successive discrimination procedure with rats, three experiments investigated the contribution of reinforcement rate and amount of S(Delta) exposure on the acquisition of an operant discrimination. S(D) components and were always 2 min in length, while S(Delta) (extinction) components were either 1 min or 4 min in length; responses in S(D) were reinforced on one of four schedules. In Experiment 1, each of eight groups were exposed to one possible combination of rate of reinforcement and S(Delta) component length. At every level of reinforcement, the 4 min S(Delta) groups acquired the discrimination more quickly. However, within each level of reinforcement, the proportions of responding in S(D) as a function cumulative S(Delta) exposure were equivalent, regardless of the number of reinforcers earned in S(D), suggesting that extinction is the "hallmark" of discrimination. Experiment 2 sought to replicate these results in a within-subjects design, and although the 4 min S(Delta) conditions always produced superior discriminations, the lack of discriminated responding in some conditions suggested that stimulus disparity was reduced. Experiment 3 clarified those results and extended the finding that the acquisition of operant discrimination closely parallels extinction of responding in S(Delta). In sum, it appears that higher reinforcement rates and longer S(Delta) exposure facilitate the acquisition of discriminated operant responding.     

Dishabituation with component transitions may contribute to the interactions observed during multiple schedules

Rates of responding by rats were usually higher during the variable interval (VI) 30-s component of a multiple VI 30-s fixed interval (FI) 30-s schedule than during the same component of a multiple VI 30-s VI 30-s schedule (Experiment 1). Response rates were also usually higher during the FI 30-s component of a multiple VI 30-s FI 30-s schedule than during the same component of a multiple FI 30-s FI 30-s schedule (Experiment 2). The differences in response rates were not observed when the components provided VI or FI 120-s schedules. These results were predicted by the idea that differences in habituation to the reinforcer between multiple schedules contribute to behavioral interactions, such as behavioral contrast. However, differences in habituation were not apparent in the within-session patterns of responding. Finding differences in response rates in both experiments violates widely-held assumptions about behavioral interactions, including that behavioral contrast does not occur for rats and that improving the conditions of reinforcement decreases, rather than increases, response rate in the alternative component.     

Three tests of 'anticipatory responding' as an account for induction produced by upcoming food-pellet reinforcement

Previous research has demonstrated that rats' rates of lever pressing for low-concentration liquid-sucrose reinforcers are increased when food-pellet, rather than sucrose, reinforcement will be upcoming in the same session (i.e. induction). The present experiments were designed to determine whether this induction was the product of 'anticipatory responses' for the upcoming food pellets being added to the responses being made for the currently available sucrose reinforcement. Experiment 1 tested this idea by summing sucrose-reinforced responding and 'anticipatory responding' from different conditions and comparing the sum to responding from a third condition in which subjects responded for sucrose when food-pellet reinforcement was upcoming. The comparison yielded similar response rates. Experiment 2 employed a blackout, of different durations in different conditions, to delay the upcoming food-pellet reinforcement. Consistent with the anticipatory-responding account, the delay decreased the size of the induction. However, results from the blackouts were not entirely consistent with the anticipatory-responding explanation. Experiment 3 provided, in some conditions, sucrose and food-pellet reinforcement in the first and second halves of the session, respectively, for responding on separate levers. These conditions separated 'anticipatory responses' for the food pellets from responses for the sucrose reinforcers. However, induction in responding for sucrose was still present. Together, these experiments demonstrate that, although anticipatory responses may contribute to induction in some instances, they are not solely responsible for the effect.     

Reinforcer satiation and resistance to change of responding maintained by qualitatively different reinforcers

In previous research on resistance to change, differential disruption of operant behavior by satiation has been used to assess the relative strength of responding maintained by different rates or magnitudes of the same reinforcer in different stimulus contexts. The present experiment examined resistance to disruption by satiation of one reinforcer type when qualitatively different reinforcers were arranged in different contexts. Rats earned either food pellets or a 15% sucrose solution on variable-interval 60-s schedules of reinforcement in the two components of a multiple schedule. Resistance to satiation was assessed by providing free access either to food pellets or the sucrose solution prior to or during sessions. Responding systematically decreased more relative to baseline in the component associated with the satiated reinforcer. These findings suggest that when qualitatively different reinforcers maintain responding, relative resistance to change depends upon the relations between reinforcers and disrupter types.     

The effect of second-half reinforcer type on responding for sucrose in the first half of the session

The present study investigated whether the sucrose-reinforced lever pressing of rats in the first half of a 50-min session would be sensitive to upcoming food-pellet reinforcement in the second half. In Experiment 1, the type of reinforcer in the first half of the session was always liquid sucrose and type of reinforcer in the second half (liquid sucrose or food pellets) varied across conditions. Sucrose concentration varied across groups (1, 5, or 25%). Results showed that rates and patterns of responding for 1%, and sometimes for 5%, sucrose reinforcers in the first half of the session were higher and steeper, respectively, when food-pellet, rather than sucrose, reinforcement occurred in the second half. Responding for 25% sucrose was not similarly affected. Experiment 2 replicated the results of Experiment 1 using a within-subjects design. Although the present results represent induction (i.e. the opposite of contrast), they are consistent with some results on consummatory contrast. They also further demonstrate that responding on interval schedules of reinforcement can be altered prospectively. By doing so, however, they pose potential problems for current theories for why operant response rates change within the session.     

Response-cost punishment via token loss with pigeons

Two experiments were conducted to investigate punishment via response-contingent removal of conditioned token reinforcers (response cost) with pigeons. In Experiment 1, key pecking was maintained on a two-component multiple second-order schedule of token delivery, with light emitting diodes (LEDs) serving as token reinforcers. In both components, responding produced tokens according to a random-interval 20-s schedule and exchange periods according to a variable-ratio schedule. During exchange periods, each token was exchangeable for 2.5-s access to grain. In one component, responses were conjointly punished according to fixed-ratio schedules of token removal. Response rates in this punishment component decreased to low levels while response rates in the alternate (no-punishment) component were unaffected. Responding was eliminated when it produced neither tokens nor exchange periods (Extinction), but was maintained at moderate levels when it produced tokens in the signaled absence of food reinforcement, suggesting that tokens served as effective conditioned reinforcers. In Experiment 2, the effect of the response-cost punishment contingency was separated from changes in the density of food reinforcement. This was accomplished by yoking either the number of food deliveries per component (Yoked Food) or the temporal placement of all stimulus events (tokens, exchanges, food deliveries) (Yoked Complete), from the punishment to the no-punishment component. Response rates decreased in both components, but decreased more rapidly and were generally maintained at lower levels in the punishment component than in the yoked component. In showing that the response-cost contingency had a suppressive effect on responding in addition to that produced by reductions in reinforcement density, the present results suggest that response-cost punishment shares important features with other forms of punishment.     

A comparison of the effects of discriminative and Pavlovian inhibitors and excitors on instrumental responding

In two experiments, the effects of Pavlovian or discriminative conditioned inhibitors on operant responding were investigated in rats. Experiment 1 found that a Pavlovian conditioned inhibitor for food suppressed food-reinforced lever pressing more than a non-differentially trained control stimulus did. Experiment 2 demonstrated that an operant discriminative inhibitor produced greater suppression of lever pressing than a Pavlovian conditioned inhibitor. Experiment 2 also found that compounding an operant discriminative stimulus (SD) for food-reinforced responding with another SD for food-reinforced responding resulted in more additive summation than when an SD was compounded with a Pavlovian conditioned excitor for food. The results of these experiments support two-factor theories that postulate that incentive and response discriminative processes summate algebraically when the processes are inhibitory or excitatory.     

The role of place of reinforcer delivery in the appearance of positive induction when rats respond for 1% sucrose

Rats increase their rate of operant responding for 1% sucrose reinforcement in the first half of an experimental session if a high-valued reinforcer will be available in the second half. Previous research suggests that this induction effect occurs because the reinforcing value of the low-valued substance has increased. The present study investigated whether this increase may occur because of where the substances are delivered. Rats pressed a lever to earn 1% liquid-sucrose reinforcers in the first half of the session. In control conditions, they also pressed for 1% sucrose in the second half. In treatment conditions, they pressed for food-pellet (Experiment 1) or 32% sucrose (Experiment 2) reinforcers in the second half, with these reinforcers either being delivered to the same location as the 1% sucrose or to a different location. Upcoming food-pellet or 32% sucrose reinforcement increased rates of lever pressing for 1% sucrose in the first half of the session, with the largest increase observed when the high-valued reinforcer was delivered to the same location as the 1% sucrose. Qualitatively similar results were found with rates of consumption of 1% sucrose reinforcers in the first half of the session, which were measured in Experiment 2. The location to which reinforcers are delivered appears to be one of the factors that contributes to this induction effect. The present results may therefore identify one of the factors that determine whether differential conditions of reinforcement will lead to contrast or induction.     

How rats combine temporal cues

The procedures for classical and operant conditioning, and for many timing procedures, involve the delivery of reinforcers that may be related to the time of previous reinforcers and responses, and to the time of onsets and terminations of stimuli. The behavior resulting from such procedures can be described as bouts of responding that occur in some pattern at some rate. A packet theory of timing and conditioning is described that accounts for such behavior under a wide range of procedures. Applications include the food searching by rats in Skinner boxes under conditions of fixed and random reinforcement, brief and sustained stimuli, and several response-food contingencies. The approach is used to describe how multiple cues from reinforcers and stimuli combine to determine the rate and pattern of response bouts.     

Varying reinforcer duration produces behavioral interactions during multiple schedules

The experiments tested the idea that changes in habituation to the reinforcer contribute to behavioral interactions during multiple schedules. This idea predicts that changing an aspect of the reinforcer should disrupt habituation and produce an interaction. Pigeons and rats responded on multiple variable interval variable interval schedules. Introducing variability into the duration of reinforcers in one component increased response rates in both components when the schedules provided high, but not low, rates of reinforcement. The increases in constant-component response rates grew larger as the session progressed. Within-session decreases in responding were smaller when the other component provided variable-, rather than fixed-, duration reinforcers. These results are consistent with the idea that changes in habituation to the reinforcer contribute to behavioral interactions. They help to explain why interactions do not occur for some subjects under conditions that produce them for others. Finally, the results question the assumption that induction and behavioral contrast are always produced by different theoretical mechanisms.     

Operant performance following tail-pinch in the rat: effects of d-amphetamine

To extend the investigation of tail-pinch induced behavioral changes, rats performing on a differential reinforcement of low rates of 10 sec (DRL10), a fixed-interval of 60 sec (F160), and a fixed-ratio of 20 (FR20) schedules were exposed to a paper clip applied to the tail. While a 10 min tail-pinch conducted 1 hr before operant sessions significantly altered the DRL10 behavior, this stressor had little effect on either F160 or FR20 responding. Marked DRL10 behavior performance changes following tail-pinch included increases in the number of lever presses, decreases in the number of the reinforcers, and disruption in the frequency distribution of inter-response times (IRT). These DRL10 operant deficits were diminished when the subject received a tail-pinch pretreatment followed by d-amphetamine treatment (0.2 and 2.0 mg/kg). In combination with biochemical data from others, the present results suggest that catecholamine systems are involved in modulation of DRL10 behavior following tail-pinch.     

Habituation and the reinforcing effectiveness of visual stimuli

The term "sensory reinforcer" has been used to refer to sensory stimuli (e.g. light onset) that are primary reinforcers in order to differentiate them from other more biologically important primary reinforcers (e.g. food and water). Acquisition of snout poke responding for a visual stimulus (5s light onset) with fixed ratio 1 (FR 1), variable-interval 1min (VI 1min), or variable-interval 6min (VI 6min) schedules of reinforcement was tested in three groups of rats (n=8/group). The VI 6min schedule of reinforcement produced a higher response rate than the FR 1 or VI 1min schedules of visual stimulus reinforcement. One explanation for greater responding on the VI 6min schedule relative to the FR 1 and VI 1min schedules is that the reinforcing effectiveness of light onset habituated more rapidly in the FR 1 and VI 1min groups as compared to the VI 6min group. The inverse relationship between response rate and the rate of visual stimulus reinforcement is opposite to results from studies with biologically important reinforcers which indicate a positive relationship between response and reinforcement rate. Rapid habituation of reinforcing effectiveness may be a fundamental characteristic of sensory reinforcers that differentiates them from biologically important reinforcers, which are required to maintain homeostatic balance.     

Habituation contributes to the decline in wheel running within wheel-running reinforcement periods

Habituation appears to play a role in the decline in wheel running within an interval. Aoyama and McSweeney [Aoyama, K., McSweeney, F.K., 2001. Habituation contributes to within-session changes in free wheel running. J. Exp. Anal. Behav. 76, 289-302] showed that when a novel stimulus was presented during a 30-min interval, wheel-running rates following the stimulus increased to levels approximating those earlier in the interval. The present study sought to assess the role of habituation in the decline in running that occurs over a briefer interval. In two experiments, rats responded on fixed-interval 30-s schedules for the opportunity to run for 45 s. Forty reinforcers were completed in each session. In the first experiment, the brake and chamber lights were repeatedly activated and inactivated after 25 s of a reinforcement interval had elapsed to assess the effect on running within the remaining 20 s. Presentations of the brake/light stimulus occurred during nine randomly determined reinforcement intervals in a session. In the second experiment, a 110 dB tone was emitted after 25 s of the reinforcement interval. In both experiments, presentation of the stimulus produced an immediate decline in running that dissipated over sessions. No increase in running following the stimulus was observed in the first experiment until the stimulus-induced decline dissipated. In the second experiment, increases in running were observed following the tone in the first session as well as when data were averaged over several sessions. In general, the results concur with the assertion that habituation plays a role in the decline in wheel running that occurs within both long and short intervals.     

Prolonged microwave irradiation of rats: Effects on concurrent operant behavior

Two measures of performance were used to study the effects of pulse-modulated microwave radiation (PM MWR) on schedule-controlled operant behavior of rats: 1) cued (S^D), fixed-ratio (FR) bar pressing for food reinforcement; and 2) noncued (S^d) bar pressing in the absence of food reinforcement. The animals were irradiated and the behavioral data were obtained concurrently, during daily three-hour sessions, five days per week for six to nine weeks. Each experiment began with a two to three-week baseline interval of sham irradiation; a two to three-week interval of sham irradiation followed the irradiation phase. The irradiated animals were exposed to 1.3-Ghz PM MWR (pulse width of 1 microsecond at 600 pulses per second) at whole-body, average specific absorbed-dose rates of from 1.5–6.7 mW/g. Control and irradiated animals were tested in identical, cylindrical waveguide exposure/behavioral assemblies; different groups of irradiated and sham-irradiated animals were used for each dose rate. At 1.5 mW/g, the levels of S^D operant responding by control and irradiated animals were comparable, and showed similar progressive diminutions over the course of each daily session. S^d operant responding was more variable, but again comparable, with both groups showing similar, progressive declines in rate of responding during each session. At 3.6 mW/g, no specific effects on S^D operant response rates were observed. However, there was an initial and transient increase in the rate of extinction of S^d responding. At 6.7 mW/g, S^D response rates were slightly reduced, whereas there was a major reduction in noncued (S^d) operant responding followed by a sharp rebound during the first post-MWR week. This marked reduction in S^d operant responding at MWR onset was in contrast to the relative stability and persistence of FR responding for food reinforcement.     

Ethanol seeking by long evans rats is not always a goal-directed behavior

Background

Two parallel and interacting processes are said to underlie animal behavior, whereby learning and performance of a behavior is at first via conscious and deliberate (goal-directed) processes, but after initial acquisition, the behavior can become automatic and stimulus-elicited (habitual). With respect to instrumental behaviors, animal learning studies suggest that the duration of training and the action-outcome contingency are two factors involved in the emergence of habitual seeking of “natural” reinforcers (e.g., sweet solutions, food or sucrose pellets). To rigorously test whether behaviors reinforced by abused substances such as ethanol, in particular, similarly become habitual was the primary aim of this study.

Methodology/Principal Findings

Male Long Evans rats underwent extended or limited operant lever press training with 10% sucrose/10% ethanol (10S10E) reinforcement (variable interval (VI) or (VR) ratio schedule of reinforcement), or with 10% sucrose (10S) reinforcement (VI schedule only). Once training and pretesting were complete, the impact of outcome devaluation on operant behavior was evaluated after lithium chloride injections were paired with the reinforcer, or unpaired 24 hours later. After limited, but not extended instrumental training, lever pressing by groups trained under VR with 10S10E and under VI with 10S was sensitive to outcome devaluation. In contrast, responding by both the extended and limited training 10S10E VI groups was not sensitive to ethanol devaluation during the test for habitual behavior.

Conclusions/Significance

Operant behavior by rats trained to self-administer an ethanol-sucrose solution showed variable sensitivity to a change in the value of ethanol, with relative insensitivity developing sooner in animals that received time-variable ethanol reinforcement during training sessions. One important implication, with respect to substance abuse in humans, is that initial learning about the relationship between instrumental actions and the opportunity to consume ethanol-containing drinks can influence the time course for the development or expression of habitual ethanol seeking behavior.     

Operant Sensation Seeking in the Mouse

Operant methods are powerful behavioral tools for the study of motivated behavior. These ''self-administration'' methods have been used extensively in drug addiction research due to their high construct validity. Operant studies provide researchers a tool for preclinical investigation of several aspects of the addiction process. For example, mechanisms of acute reinforcement (both drug and non-drug) can be tested using pharmacological or genetic tools to determine the ability of a molecular target to influence self-administration behavior^1-6. Additionally, drug or food seeking behaviors can be studied in the absence of the primary reinforcer, and the ability of pharmacological compounds to disrupt this process is a preclinical model for discovery of molecular targets and compounds that may be useful for the treatment of addiction^3,7-9. One problem with performing intravenous drug self-administration studies in the mouse is the technical difficulty of maintaining catheter patency. Attrition rates in these experiments are high and can reach 40% or higher^10-15. Another general problem with drug self-administration is discerning which pharmacologically-induced effects of the reinforcer produce specific behaviors. For example, measurement of the reinforcing and neurological effects of psychostimulants can be confounded by their psychomotor effects. Operant methods using food reinforcement can avoid these pitfalls, although their utility in studying drug addiction is limited by the fact that some manipulations that alter drug self-administration have a minimal impact on food self-administration. For example, mesolimbic dopamine lesion or knockout of the D1 dopamine receptor reduce cocaine self-administration without having a significant impact on food self-administration ^12,16.Sensory stimuli have been described for their ability to support operant responding as primary reinforcers (i.e. not conditioned reinforcers)^17-22. Auditory and visual stimuli are self-administered by several species^18,21,23, although surprisingly little is known about the neural mechanisms underlying this reinforcement. The operant sensation seeking (OSS) model is a robust model for obtaining sensory self-administration in the mouse, allowing the study of neural mechanisms important in sensory reinforcement²⁴. An additional advantage of OSS is the ability to screen mutant mice for differences in operant behavior that may be relevant to addiction. We have reported that dopamine D1 receptor knockout mice, previously shown to be deficient in psychostimulant self-administration, also fail to acquire OSS²⁴. This is a unique finding in that these mice are capable of learning an operant task when food is used as a reinforcer. While operant studies using food reinforcement can be useful in the study of general motivated behavior and the mechanisms underlying food reinforcement, as mentioned above, these studies are limited in their application to studying molecular mechanisms of drug addiction. Thus, there may be similar neural substrates mediating sensory and psychostimulant reinforcement that are distinct from food reinforcement, which would make OSS a particularly attractive model for the study of drug addiction processes. The degree of overlap between other molecular targets of OSS and drug reinforcers is unclear, but is a topic that we are currently pursuing. While some aspects of addiction such as resistance to extinction may be observed with OSS, we have found that escalation ²⁵ is not observed in this model²⁴. Interestingly, escalation of intake and some other aspects of addiction are observed with self-administration of sucrose²⁶. Thus, when non-drug operant procedures are desired to study addiction-related processes, food or sensory reinforcers can be chosen to best fit the particular question being asked.In conclusion, both food self-administration and OSS in the mouse have the advantage of not requiring an intravenous catheter, which allows a higher throughput means to study the effects of pharmacological or genetic manipulation of neural targets involved in motivation. While operant testing using food as a reinforcer is particularly useful in the study of the regulation of food intake, OSS is particularly apt for studying reinforcement mechanisms of sensory stimuli and may have broad applicability to novelty seeking and addiction.Download video file.^{(54M, mov)}     

Suppression of operant responding for food and water reinforcement following intraventricular injections of Bombesin in the rat

The effects of injecting Bombesin (BBS) into the lateral cerebral ventricle on operant responding for food and water in the rat were investigated in two experiments. In the first experiment injections of BBS suppressed both operant responding for water and post-session water consumption. A combined treatment of water preloading and BBS injections produced greater suppression of post-session water consumption than either BBS injections or water preloading. This suggests that the peptide has a primary antidipsic effect. In the second experiment BBS produced a significant suppression of operant responding for food reward as well as lowered body temperature. This suggests that BBS may serve as a true satiety signal for food motivated behavior.