Resistance to change varies inversely with reinforcement context

We report two experiments which test whether resistance to prefeeding and satiation for a variable-interval (VI) schedule that delivers a constant rate of reinforcement varies inversely with the reinforcement rate for an alternative schedule. In Experiment 1, eight pigeons responded in a multiple schedule in which the red key was always associated with a VI 90-s schedule and the green key with either a richer (VI 18s) or leaner (VI 540s) schedule in different conditions. After baseline training in each condition, prefeeding test sessions were conducted in which 10g, 20g, 30g, 40g, and 50g food were provided one-hour prior to test. Additional baseline training was given between each test session. In Experiment 2, two groups of pigeons responded in a multiple schedule similar to Experiment 1. After baseline training, pigeons were exposed to a 5-h satiation test session in which the VI 90-s schedule was available continuously. Test sessions were conducted when pigeons were maintained at 85%, 95%, and 85% of their body weights in an ABA design. Results of both experiments showed that responding in the VI 90-s schedule that alternated with a leaner schedule during baseline was more resistant to prefeeding and satiation. These data rule out alternative explanations for results of previous studies, and confirm that resistance to change varies inversely with reinforcement context.     

Rapid acquisition of preference in concurrent schedules: Effects of d-amphetamine on sensitivity to reinforcement amount

In the present study, effects of d-amphetamine on sensitivity to reinforcement amount under concurrent schedules were examined using a rapid-acquisition procedure. Four pigeons key pecked under single concurrent variable-interval 30-s schedules of grain presentation. Two different reinforcer-amount ratios (7:1 and 1:7) changed across sessions according to a 31-step pseudo-random binary sequence (PRBS). After at least four times through the PRBS, response ratios generally tracked the session-to-session changes in amount ratios; estimates of sensitivity ranged from 0.26 to 0.31 across the four pigeons. Effects of a range of doses of d-amphetamine (0.3-5.6 mg/kg) then were determined. For 3 of 4 pigeons, at least one dose, which did not dramatically alter overall response output or bias, decreased sensitivity to reinforcement amount. These results suggest that reducing sensitivity of responding to reinforcement amount may be one behavioral mechanism of stimulants, which may have implications for interpreting drug effects on self-control.     

Segmentation and the pairing hypothesis

The effect of stimulus contiguity and response contingency on responding in chain schedules was examined in two experiments. In Experiment 1, four pigeons were trained on two simple three-link chain schedules that alternated within sessions. Initial links were correlated with a variable-interval 30s schedule, and middle and terminal links were correlated with interdependent variable-interval 30s variable-interval 30s schedules. The combined duration of the interdependent schedules summed to 60s. The two chains differed with respect to signaling of the schedule components: a two-stimulus chain had one stimulus paired with the initial link and one stimulus paired with both the middle and the terminal link, while a three-stimulus chain had a different stimulus paired with the each of the three links. The results showed that the two-stimulus chain maintained lower initial-link responding than the three-stimulus chain. In Experiment 2, four pigeons were exposed to three separate conditions, the two- and three-stimulus chains of Experiment 1 and a three-stimulus chain that had a 3s delay to terminal-link entry from the middle-link response that produced it. The two-stimulus chain maintained lower initial-link responding than the three-stimulus chain, as in Experiment 1, and a similar initial-link responding was maintained by the two-stimulus chain and the three-stimulus chain with the delay contingency. The results demonstrate that a stimulus noncontiguous with food can maintain responding that is sometimes greater than a stimulus contiguous with food, depending on the response contingency for terminal-link entry. The results are contrary to the pairing hypothesis of conditioned reinforcement.     

Differential resurgence and response elimination

Resurgence refers to the transient recovery of previously reinforced, but presently not reinforced, responding when more recently reinforced responding is extinguished. The primary purpose of our research was to determine how differential resurgence results from the procedures used to eliminate that responding. There were three conditions in each of five experiments. In Condition 1, key pecking by pigeons was maintained under a two-component multiple variable-interval (VI) 30-s VI 30-s schedule. In Condition 2, this pecking was eliminated in different ways across components. In Condition 3, extinction was in effect for all responses, and resurgence of key pecking was compared across components. These three conditions were repeated for most pigeons, and the procedures used to eliminate responding in Condition 2 varied across experiments. In Experiment 1, there was greater resurgence, and an earlier onset of it, after a differential-reinforcement-of-other-behavior (DRO) schedule than after a VI schedule was correlated with pecking an alternative key. Experiments 2 and 3 showed that the differential resurgence in Experiment 1 probably was not due to conditional stimulus control or the periodicity of food delivery, respectively. In Experiment 4, there was no systematic difference in resurgence after either a DRO schedule or a VI schedule correlated with treadle pressing. In Experiment 5, there was greater resurgence, and/or an earlier onset of it, after a VI schedule correlated with treadle pressing than after a VI schedule correlated with pecking an alternative key. Taken together, the results showed that the reinforcement of an alternative key-peck response was the most effective means of reducing subsequent key-peck resurgence. The relation of these results to an understanding of resurgence is discussed.     

Stimulus change dis-habituates operant responding supported by water reinforcers

The present experiment examined whether habituation contributes to within-session decreases in operant responding for water reinforcers. The experiment asked if this responding can be dishabituated, a fundamental property of habituated behavior. During baseline, rats’ lever pressing was reinforced by water on a variable interval 15-s schedule. During experimental conditions, rats responded on the same schedule and a new stimulus was introduced for 5 min at 15, 30 or 45 min into the 60-min session. The new stimulus was extinction, continuous reinforcement or flashing lights in different conditions. Rate of responding primarily decreased within the session during baseline. Introducing a new stimulus sometimes suppressed (extinction, continuous reinforcement) and sometimes increased (flashing lights) responding while it was in effect. The new stimulus increased responding after it ended and before it was presented in the session. The results are incompatible with the idea that non-habituation satiety factors (e.g., cellular hydration and blood volume) contributed to within-session changes in responding. These satiety factors should increase with increases in consumption, decrease with decreases in consumption and remain constant with constant consumption of water. Nevertheless, all stimulus changes increased operant responding for water. These results support the idea that habituation contributes to within-session decreases in responding for water reinforcers.     

Superstitious responding and reinforcement rate under concurrent variable-interval extinction schedules

To examine superstitious responding, four pigeons key pecked under multiple concurrent variable-interval 45 s variable-interval 90 s concurrent variable-interval 90 s variable-interval 180 s schedules in the absence of a changeover delay. The two variable-interval 90 s schedules then were replaced by extinction, and key-peck responding during extinction was examined as a function of the prevailing reinforcement rate. During the first several sessions, extinction-key responding was maintained closer to baseline levels in the presence of the higher reinforcement rate, and this effect dissipated or even reversed with continued exposure to extinction. Although extinction-key responding generally decreased to near-zero levels after several sessions, in a few instances, it continued for 30 and 45 sessions. These results demonstrate how concurrent variable-interval extinction schedules can be used to investigate what often has been labeled superstitious responding.     

Rats exhibit asymmetrical retention functions for hedonic samples

Rats were trained in a symbolic delayed matching-to-sample task to discriminate hedonic sample stimuli that consisted of food or no food. Retention functions decreased more rapidly on trials initiated by a food sample than on trials initiated by a no-food sample when retention intervals were manipulated within session. The asymmetrical functions could not be explained in terms of mediation of choice responding by magazine head-entry behavior during the retention interval. Unlike within-session changes in retention interval, between-session changes did not result in steeper forgetting functions for food samples. These results in rats are consistent with previous findings reported for the presence and absence of visual samples in pigeons (Wixted, 1993).     

Operant behavior in dwarf hamsters (Phodopus campbelli): Effects of rate of reinforcement and reinforcer flavor variety

We investigated operant behavior in a novel species, the dwarf hamster (Phodopus campbelli). In two experiments, hamsters were trained to lever-press for food reinforcement. In Experiment 1, rate of reinforcement was manipulated across conditions using four variable-interval schedules of reinforcement (delivering one to eight reinforcers per min). As predicted, within-session decreases in responding were steepest on the richest schedule. In Experiment 2, lever-pressing was reinforced by either a constant or a variety of flavored food pellets. Within-session decreases in responding were steeper when the reinforcer flavor remained constant than when it was varied within the session. In both experiments, subjects hoarded most reinforcers in their cheek pouches rather than consuming them in the operant chambers. These results are incompatible with post-ingestive satiety variables as explanations for within-session decreases in operant responding and suggest that habituation to repeatedly presented reinforcers best accounts for subjects’ response patterns. Additionally, a mathematical model that describes behavior undergoing habituation also described the present results, thus strengthening the conclusion that habituation mediates the reinforcing efficacy of food.     

Extinguished operant responding shows stimulus specificity

The experiment tested for stimulus specificity in extinguished operant responding. Eight pigeons pecked keys for food reinforcers delivered by a variable interval (VI) 60-s schedule. The key was illuminated with red light during some sessions and white light during others. Then, responding was placed on extinction. During some sessions of extinction, the color of the key light remained constant throughout the session (red or white). During other sessions the color changed at 30 min into the session (red to white or white to red). Response rate increased after the change of key color in extinction. If it is assumed that key color is part of the stimulus to which subjects habituate, then these results are consistent with McSweeney and Swindell's [J. Gen. Psychol. 129 (2002) 364] suggestion that responding declines in extinction partly because subjects habituate to the stimuli that support conditioned responding. Habituation is relatively specific to the exact nature of the stimulus presented. Therefore, changes in the stimulus violate stimulus specificity and restore habituated responding. The results are also consistent with other theories that attribute extinction to a reduction of stimulus control [e.g., Psychol. Bull. 114 (1993) 80; J. Exp. Psychol.: Anim. Behav. Process. 16 (1990) 235], but considerations such as parsimony and testability favor the habituation hypothesis over these theories.     

Rapid acquisition of preference in concurrent schedules: effects of reinforcement amount

Four pigeons were trained on concurrent variable-interval 30-s schedules. Relative reinforcer amounts arranged across the two alternatives were varied across sessions according to a pseudorandom binary sequence [cf., Hunter, I., Davison, M., 1985. Determination of a behavioral transfer function: white-noise analysis of session-to-session response-ratio dynamics on concurrent VI schedules. J. Exp. Anal. Behav. 43, 43-59]; the ratios (left/right) were either 1/7 or 7/1. Reinforcer amount was manipulated by varying the number of 1.2s hopper presentations. Sessions ended after 30 reinforcers (15 for each alternative). After approximately 30 sessions, response ratios for all pigeons began to track the changes in amount ratio (i.e., subjects' responding showed a moderate increase in sensitivity of responding to reinforcer amount). Characteristics of responding were similar to procedures in which reinforcer rate and immediacy have been manipulated, although sensitivity estimates for amount were lower than those previously obtained with rate and immediacy. This procedure may serve as a useful method for studying the effects of certain environmental manipulations (e.g., drug administration) on sensitivity to reinforcer amount.     

Effects of free-food deliveries and delays on choice under concurrent-chains schedules

Pigeons responded on a concurrent-chains schedule with two equal variable-interval (VI) schedules as initial links and delays to food of 3 and 12s as the two terminal links. In even-numbered sessions, no other reinforcement schedule was present, and all pigeons showed a strong preference for the response key that had the shorter, 3-s terminal-link delay. In odd-numbered sessions, the initial links were interrupted at random times by one of three different types of events. When the interruptions were immediate food deliveries, the response percentages increased on the key that had the 3-s delay. When the interruptions were 30-s delays followed by food, the response percentages remained approximately unchanged. When the interruptions were 30-s delays with no food, the response percentages decreased. The results were used to compare the predictions of different mathematical models of concurrent-chains performance. The results favored models that assume that preference is determined by the relative amount of advantage that is gained when a terminal link is entered.     

Effects of unpredictable changes in initial-link duration on choice and timing

Four pigeons responded in a concurrent-chains procedure in which terminal-link schedules were fixed-interval (FI) 10 s and FI 20 s. Across sessions, the location of the shorter terminal-link changed according to a pseudorandom binary sequence. Each session, the variable-interval initial-link schedule value was sampled from a uniform distribution that ranged from 0.01 to 30 s. On some terminal links, food was withheld to obtain measures of temporal control. Terminal-link delays determined choice (log initial-link response ratios) and timing (start and stop times on no-food trials) measures, which stabilized within the 1st half of each session. Preference for the shorter terminal-link delay was a monotonically decreasing function of initial-link duration. There was no evidence of control by initial-link durations from previous sessions.     

Within-session changes in responding during concurrent variable interval variable ratio schedules

Rats (Experiment 1) and pigeons (Experiment 2) responded on several concurrent variable interval (VI) variable ratio (VR) schedules. The rate of, but not the time spent, responding in each component usually changed within-sessions. The bias and sensitivity parameters of the generalized matching law (GML) did not change systematically within-sessions. The fit of the GML to the data did not change within-sessions for pigeons, but it was better in the middle than at the beginning or end of the session for some for rats. Both over- and under-matching occurred. These results imply that within-session changes in responding do not usually cause problems for assessing the validity of the GML when subjects respond on concurrent VI VR schedules. The results also suggest that under- and over-matching are not produced by different factors, but rather lie on a continuum.     

The effect of food deprivation on self-control

Two experiments examined the effect of food deprivation on choice in a discrete-trials self-control paradigm, choice between a larger, more-delayed reinforcer and a smaller, less-delayed reinforcer. In Experiment 1, four pigeons were each deprived to 65%, 80%, and 90% of their free-feeding weights, and the delay to the smaller reinforcer was varied. Deprivation level did not affect choice, but the rate of ineffective key pecks made during the reinforcer delays increased as deprivation increased. In Experiment 2, four pigeons were exposed to conditions in which they were fed up to their 80% free-feeding weights following experimental sessions, and in which they were given no postsession feedings. Both the pigeons' weights and their latencies to insert their heads into the food hopper when food was available were lower when the pigeons were not fed following experimental sessions. Choice showed no change. Deprivation level affects response rate and eating behavior in these procedures, but not choice.     

The role of contextual cues in operant responding in rats

Following lever-press training on a variable-interval 60-second schedule of food presentation, groups of rats either remained in their home cages or were exposed to the operant chamber, from which lever and food had been removed, for five sessions. The lever was replaced in the chamber and rats from Group 1 (exposure to chamber) and Group 3 (home cage) were returned to the variable-interval schedule. Although response rates in test sessions were somewhat lower than at the end of training, there was no statistically significant difference in rates for either group. Rats in Group 2 (exposure to chamber) and Group 4 (home cage) received two test sessions of extinction. During the first session, Group 2 rates of lever pressing were significantly higher than Group 4 rates. These findings do not support the view that associations between contextual cues and the reinforcer serve to energize instrumental behavior (Pearce & Hall, 1979), and provide only minimal support for the view that contextual cues control responses that compete with the operant (Mills, 1980).     

Resistance to extinction, generalization decrement, and conditioned reinforcement

This study investigated generalization decrement during an extinction resistance-to-change test for pigeon key pecking using a two-component multiple schedule with equal variable-interval 3-min schedules and different reinforcer amounts (one component presented 2-s access to reinforcement and the other 8s). After establishing baseline responding, subjects were assigned to one of the two extinction conditions: hopper stimuli (hopper and hopper light were activated but no food was available) or Control (inactive hopper and hopper light). Responding in the 8-s component was more resistant to extinction than responding in the 2-s component, the hopper stimuli group was more resistant to extinction compared to the Control group, and an interaction between amount of reinforcement, extinction condition, and session block was present. This finding supports generalization decrement as a factor that influences resistance to extinction. Hopper-time data (the amount of time subjects spent with their heads in the hopper) were compared to resistance-to-change data in an investigation of the role of conditioned reinforcement on resistance to change.     

Reinforcer satiation and resistance to change of responding maintained by qualitatively different reinforcers

In previous research on resistance to change, differential disruption of operant behavior by satiation has been used to assess the relative strength of responding maintained by different rates or magnitudes of the same reinforcer in different stimulus contexts. The present experiment examined resistance to disruption by satiation of one reinforcer type when qualitatively different reinforcers were arranged in different contexts. Rats earned either food pellets or a 15% sucrose solution on variable-interval 60-s schedules of reinforcement in the two components of a multiple schedule. Resistance to satiation was assessed by providing free access either to food pellets or the sucrose solution prior to or during sessions. Responding systematically decreased more relative to baseline in the component associated with the satiated reinforcer. These findings suggest that when qualitatively different reinforcers maintain responding, relative resistance to change depends upon the relations between reinforcers and disrupter types.     

Within-session modulation of timed anticipatory responding: When to start responding

The peak procedure is widely used in the study of interval timing with animals. Multiple timing measures can be derived from peak responding. These measures are typically presented as averages across many trials based on the implicit assumption that peak responding is stable throughout the session. We tested this assumption by examining whether peak responding changed over the course of the session in 45 mice that were trained on a fixed-interval 30-s schedule. All common measures of peak responding, except stop times, changed over the course of the session: start times increased, response rates and spreads decreased, and, although less reliably, peak times also shifted rightward. These results are congruent with a motivational interpretation, whereby increased satiety leads to the observed behavioral signature of within-session modulation of timed anticipatory responding.     

Reinforcement omission in concurrent fixed-interval and random-interval schedules

The present experiment examined overall and local effects of omission of reinforcers in a choice situation. Pigeons' key-pecking responses were reinforced under concurrent fixed-interval and random-interval schedules of food presentation. After some weeks of baseline sessions in which the probability of reinforcement was 1.00, approximately 25% of food presentations from the fixed-interval schedule were omitted and replaced by timeout periods. In such omission sessions, the overall relative rates of responding to the fixed-interval schedule became lower than those in the baseline sessions. On the other hand, when relative rates of responding to the fixed-interval schedule in the omission sessions were calculated separately for fixed-interval cycles preceded by timeout periods and those preceded by food presentations, the relative rates in the former type of fixed-interval cycles were higher than those in the latter type for three out of four pigeons. These results mean that relative rates of responding cannot always be regarded as reflecting a relative value of an alternative, and that the overall effect of the omission of fixed-interval reinforcers is not reducible to the local effect of omission.