Stimulus control over induction produced by upcoming food-pellet reinforcement

Research has demonstrated that rats increase their rate of lever pressing for sucrose reinforcement when food-pellet reinforcement will soon be available within the session. Recent results suggest that this increase occurs because stimuli in the session come to signal different levels of overall reinforcement. The present experiment tested this idea by having rats respond in two types of session. In one, they pressed lever A for 1% sucrose reinforcers in the first half of the session and lever B for food-pellet reinforcers in the second half (the opposing lever was retracted in the respective half). In the other, they pressed lever B for 1% sucrose reinforcers in the first half and lever A for 1% sucrose reinforcers in the second. Thus, the presence of lever A in the first half of the session was predictive of upcoming food-pellet reinforcement, but only lever B was ever used to obtain food pellets. Subjects responded at a higher rate on lever B in the first half of the session than on lever A, despite food-pellet reinforcement being unavailable in such sessions. Furthermore, they responded at a higher rate on lever B during probe sessions in which both levers were available. These results demonstrate that stimulus control over induction occurs when a stimulus becomes differentially associated with a heightened level of reinforcement. However, questions remain as to whether this association is Pavlovian or whether stimuli with "predictive value" may ever lead to induction.     

The effect of second-half reinforcer type on responding for sucrose in the first half of the session

The present study investigated whether the sucrose-reinforced lever pressing of rats in the first half of a 50-min session would be sensitive to upcoming food-pellet reinforcement in the second half. In Experiment 1, the type of reinforcer in the first half of the session was always liquid sucrose and type of reinforcer in the second half (liquid sucrose or food pellets) varied across conditions. Sucrose concentration varied across groups (1, 5, or 25%). Results showed that rates and patterns of responding for 1%, and sometimes for 5%, sucrose reinforcers in the first half of the session were higher and steeper, respectively, when food-pellet, rather than sucrose, reinforcement occurred in the second half. Responding for 25% sucrose was not similarly affected. Experiment 2 replicated the results of Experiment 1 using a within-subjects design. Although the present results represent induction (i.e. the opposite of contrast), they are consistent with some results on consummatory contrast. They also further demonstrate that responding on interval schedules of reinforcement can be altered prospectively. By doing so, however, they pose potential problems for current theories for why operant response rates change within the session.     

Duration, response, and location: the influence of upcoming 32% sucrose on rats' licking or lever pressing for 1% liquid sucrose

The present study investigated whether rats' rates of licking or pressing a lever for 1% liquid sucrose delivered by a continuous reinforcement schedule would decrease (contrast) or increase (induction) when the upcoming period would allow access to 32% sucrose and whether such changes would be influenced by how long each substance was available. In Experiment 1, different groups of rats licked a spout or pressed a lever for 1% sucrose in the first half of the session and, in different conditions, for 1% or 32% sucrose in the second half. Across conditions, halves of the session were 3, 6, 12, or 24 min long. Upcoming 32% sucrose significantly decreased rates of licking at each duration whereas it increased rates of lever pressing except when access duration was 3 min. Experiment 2 replicated Experiment 1 with the exception that rats that licked did so from the same spout in both halves of the session and rats that pressed a lever collected the sucrose reinforcers in the different halves at different locations. In these procedures, upcoming 32% sucrose significantly increased rates of licking. Significant, but small, increases in rates of lever pressing were still observed. The present results suggest that continuous reinforcement or duration of access to sucrose are not primary determinants of whether contrast or induction is observed. Rather, they suggest that the type of behavior (licking versus pressing a lever) and the location at which the substances are collected and consumed play a large role in which effect occurs.     

Upcoming food-pellet reinforcement alters rats' lever pressing for liquid sucrose delivered by a progressive-ratio schedule

The present study investigated whether rats' responding for liquid-sucrose reinforcement delivered by a progressive-ratio (PR) schedule would be altered by the addition of food-pellet reinforcement available subsequent to the PR schedule. In Experiment 1, six rats lever pressed for 1% sucrose reinforcers delivered by a PR 3 schedule. In Experiment 2, six rats lever pressed for 5% sucrose delivered by a PR 5 schedule. In both experiments, baseline sessions consisted of 40min of exposure to the PR schedule. In the first treatment condition, a 25-min period of food-pellet reinforcement, delivered by a random-interval 60-s schedule, immediately followed the initial 40min. In the second treatment condition, the 25-min period of food-pellet reinforcement became available when 10min elapsed without the subject completing a ratio on the PR schedule. Results from both experiments showed that upcoming food-pellet reinforcement increased the number of ratios subjects completed on the PR schedule. Portions of the present results represent a partial replication of results reported by Baron and Derenne [J. Exp. Anal. Behav. 73 (2000) 291], who used a similar procedure. They also augment a growing body of research on positive induction.     

Three tests of 'anticipatory responding' as an account for induction produced by upcoming food-pellet reinforcement

Previous research has demonstrated that rats' rates of lever pressing for low-concentration liquid-sucrose reinforcers are increased when food-pellet, rather than sucrose, reinforcement will be upcoming in the same session (i.e. induction). The present experiments were designed to determine whether this induction was the product of 'anticipatory responses' for the upcoming food pellets being added to the responses being made for the currently available sucrose reinforcement. Experiment 1 tested this idea by summing sucrose-reinforced responding and 'anticipatory responding' from different conditions and comparing the sum to responding from a third condition in which subjects responded for sucrose when food-pellet reinforcement was upcoming. The comparison yielded similar response rates. Experiment 2 employed a blackout, of different durations in different conditions, to delay the upcoming food-pellet reinforcement. Consistent with the anticipatory-responding account, the delay decreased the size of the induction. However, results from the blackouts were not entirely consistent with the anticipatory-responding explanation. Experiment 3 provided, in some conditions, sucrose and food-pellet reinforcement in the first and second halves of the session, respectively, for responding on separate levers. These conditions separated 'anticipatory responses' for the food pellets from responses for the sucrose reinforcers. However, induction in responding for sucrose was still present. Together, these experiments demonstrate that, although anticipatory responses may contribute to induction in some instances, they are not solely responsible for the effect.     

Reinforcer devaluation by extinction depends on the food restriction protocol

A common feature of reinforcer devaluation studies is that new learning induces the devaluation. The present study used extinction to induce new learning about the conditioned reinforcer in a heterogeneous chain schedule. Rats pressed a lever in a heterogeneous chain schedule to produce a conditioned reinforcer (light) associated with the opportunity to obtain an unconditioned reinforcer (food) by pulling a chain. The density of food reinforcement correlated with the conditioned reinforcer was varied in a comparison of continuous and variable-ratio reinforcement schedules of chain pulling; this had no noticeable effect on conditioned reinforcer devaluation produced by extinction of chain pulling. In contrast, how rats were deprived appeared to matter very much. Restricting meal duration to 1h daily produced more lever pressing during baseline training and a greater reductive effect of devaluation on lever pressing than restricting body weight to 80% of a control rat's weight, which eliminated the devaluation effect. Further analysis suggested that meal-duration restriction may have produced devaluation effects because it was more effective than weight restriction in reducing rats' body weights. Our results exposed an important limitation on the devaluation of conditioned reinforcers: slight differences in food restriction, using two commonly employed food-restriction procedures, can produce completely different interpretations of reinforcer devaluation while leaving reinforcer-based learning intact.     

Influence of budget and reinforcement location on risk-sensitive preference

Little is known about the effect that procedural variables have on risk-sensitive preference. This study assessed the effect of procedural variables on pigeons' choice between a fixed and variable amount of reinforcement (amount risk) and, in a separate condition, between a fixed and variable delay until reinforcement (delay risk). Experiment 1 investigated the impact of water reinforcement and risk dimension when pigeons were in a restrictive budget, where access to water was less than that necessary to maintain current body weight, and a condition where the pigeons had ample access to water. Pigeons exhibited a greater tendency to prefer the variable alternative for delay risk than for amount risk in both restrictive and ample budgets. Varying water budget had no effect on risk preference. Experiment 2 investigated the influence of water reinforcer location while in a restrictive budget, in which reinforcers were delivered to a single location, two distinct locations, or a randomly selected location. With amount risk, pigeons were risk averse when reinforcers were delivered in separate or random locations and were indifferent to risk when delivered to a single location. With delay risk, pigeons were generally risk prone with no effect from reinforcement location. The finding that pigeons were risk averse when reinforcers were delivered to separate locations and were indifferent to risk when delivered to a single location offers a methodological explanation to the inconsistent findings in the literature with amount risk.     

Reinforcer satiation and resistance to change of responding maintained by qualitatively different reinforcers

In previous research on resistance to change, differential disruption of operant behavior by satiation has been used to assess the relative strength of responding maintained by different rates or magnitudes of the same reinforcer in different stimulus contexts. The present experiment examined resistance to disruption by satiation of one reinforcer type when qualitatively different reinforcers were arranged in different contexts. Rats earned either food pellets or a 15% sucrose solution on variable-interval 60-s schedules of reinforcement in the two components of a multiple schedule. Resistance to satiation was assessed by providing free access either to food pellets or the sucrose solution prior to or during sessions. Responding systematically decreased more relative to baseline in the component associated with the satiated reinforcer. These findings suggest that when qualitatively different reinforcers maintain responding, relative resistance to change depends upon the relations between reinforcers and disrupter types.     

Species differences between rats and pigeons in choices with probabilistic and delayed reinforcers

An adjusting-delay procedure was used to study rats' choices with probabilistic and delayed reinforcers, and to compare them with previous results from pigeons. A left lever press led to a 5-s delay signaled by a light and a tone, followed by a food pellet on 50% of the trials. A right lever press led to an adjusting delay signaled by a light followed by a food pellet on 100% of the trials. In some conditions, the light and tone for the probabilistic reinforcer were present only on trials that delivered food. In other conditions, the light and tone were present on all trials that the left lever was chosen. Similar studies with pigeons [Mazur, J.E., 1989. Theories of probabilistic reinforcement. J. Exp. Anal. Behav. 51, 87-99; Mazur, J.E., 1991. Conditioned reinforcement and choice with delayed and uncertain primary reinforcers. J. Exp. Anal. Behav. 63, 139-150] found that choice of the probabilistic reinforcer increased dramatically when the delay-interval stimuli were omitted on no-food trials, but this study found no such effect with the rats. In other conditions, the probability of food was varied, and comparisons to previous studies with pigeons indicated that rats showed greater sensitivity to decreasing reinforcer probabilities. The results support the hypothesis that rats' choices in these situations depend on the total time between a choice response and a reinforcer, whereas pigeons' choices are strongly influenced by the presence of delay-interval stimuli.     

Operant behavior in dwarf hamsters (Phodopus campbelli): Effects of rate of reinforcement and reinforcer flavor variety

We investigated operant behavior in a novel species, the dwarf hamster (Phodopus campbelli). In two experiments, hamsters were trained to lever-press for food reinforcement. In Experiment 1, rate of reinforcement was manipulated across conditions using four variable-interval schedules of reinforcement (delivering one to eight reinforcers per min). As predicted, within-session decreases in responding were steepest on the richest schedule. In Experiment 2, lever-pressing was reinforced by either a constant or a variety of flavored food pellets. Within-session decreases in responding were steeper when the reinforcer flavor remained constant than when it was varied within the session. In both experiments, subjects hoarded most reinforcers in their cheek pouches rather than consuming them in the operant chambers. These results are incompatible with post-ingestive satiety variables as explanations for within-session decreases in operant responding and suggest that habituation to repeatedly presented reinforcers best accounts for subjects’ response patterns. Additionally, a mathematical model that describes behavior undergoing habituation also described the present results, thus strengthening the conclusion that habituation mediates the reinforcing efficacy of food.     

Excitatory backward conditioning in an appetitive conditioned reinforcement preparation with rats

Four experiments were conducted to examine appetitive backward conditioning in a conditioned reinforcement preparation. In all experiments, off-line classical conditioning was conducted following lever-press training on two levers. Presentations of a sucrose solution by a liquid dipper served as an unconditioned stimulus (US) and two auditory stimuli served as conditioned stimuli (CSs); one was paired with the US in either a forward (Experiment 1a) or a backward (Experiments 1b, 2, and 3) relationship, and the other served as a control CS, which was not paired with the US. In testing, each lever-press response produced a presentation of one of the CSs instead of appetitive reinforcers. The response to a lever was facilitated, compared to the response to another lever, when the response produced the backward CS presentation as well as when it produced the forward CS presentation; that is, the backward CS served as an excitatory conditioned reinforcer.     

Responding for sucrose and wheel-running reinforcement: effect of pre-running

Six male albino Wistar rats were placed in running wheels and exposed to a fixed interval 30-s schedule that produced either a drop of 15% sucrose solution or the opportunity to run for 15s as reinforcing consequences for lever pressing. Each reinforcer type was signaled by a different stimulus. To assess the effect of pre-running, animals were allowed to run for 1h prior to a session of responding for sucrose and running. Results showed that, after pre-running, response rates in the later segments of the 30-s schedule decreased in the presence of a wheel-running stimulus and increased in the presence of a sucrose stimulus. Wheel-running rates were not affected. Analysis of mean post-reinforcement pauses (PRP) broken down by transitions between successive reinforcers revealed that pre-running lengthened pausing in the presence of the stimulus signaling wheel running and shortened pauses in the presence of the stimulus signaling sucrose. No effect was observed on local response rates. Changes in pausing in the presence of stimuli signaling the two reinforcers were consistent with a decrease in the reinforcing efficacy of wheel running and an increase in the reinforcing efficacy of sucrose. Pre-running decreased motivation to respond for running, but increased motivation to work for food.     

Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement

Across two experiments, a peak procedure was used to assess the timing of the onset and offset of an opportunity to run as a reinforcer. The first experiment investigated the effect of reinforcer duration on temporal discrimination of the onset of the reinforcement interval. Three male Wistar rats were exposed to fixed-interval (FI) 30-s schedules of wheel-running reinforcement and the duration of the opportunity to run was varied across values of 15, 30, and 60s. Each session consisted of 50 reinforcers and 10 probe trials. Results showed that as reinforcer duration increased, the percentage of postreinforcement pauses longer than the 30-s schedule interval increased. On probe trials, peak response rates occurred near the time of reinforcer delivery and peak times varied with reinforcer duration. In a second experiment, seven female Long-Evans rats were exposed to FI 30-s schedules leading to 30-s opportunities to run. Timing of the onset and offset of the reinforcement period was assessed by probe trials during the schedule interval and during the reinforcement interval in separate conditions. The results provided evidence of timing of the onset, but not the offset of the wheel-running reinforcement period. Further research is required to assess if timing occurs during a wheel-running reinforcement period.     

Response-reinforcer relations and resistance to change

Behavioral momentum theory suggests that the relation between a response and a reinforcer (i.e., response-reinforcer relation) governs response rates and the relation between a stimulus and a reinforcer (i.e., stimulus-reinforcer relation) governs resistance to change. The present experiments compared the effects degrading response-reinforcer relations with response-independent or delayed reinforcers on resistance to change in conditions with equal stimulus-reinforcer relations. In Experiment 1, pigeons responded on equal variable-interval schedules of immediate reinforcement in three components of a multiple schedule. Additional response-independent reinforcers were available in one component and additional delayed reinforcers were available in another component. The results showed that resistance to disruption was greater in the components with added reinforcers than without them (i.e., better stimulus-reinforcer relations), but did not differ for the components with added response-independent and delayed reinforcement. In Experiment 2, a component presenting immediate reinforcement alternated with either a component that arranged equal rates of reinforcement with a proportion of those reinforcers being response independent or a component with a proportion of the reinforcers being delayed. Results showed that resistance to disruption tended to be either similar across components or slightly lower when response-reinforcer relations were degraded with either response-independent or delayed reinforcers. These findings suggest that degrading response-reinforcer relations can impact resistance to change, but that impact does not depend on the specific method and is small relative to the effects of the stimulus-reinforcer relation.     

Varying reinforcer duration produces behavioral interactions during multiple schedules

The experiments tested the idea that changes in habituation to the reinforcer contribute to behavioral interactions during multiple schedules. This idea predicts that changing an aspect of the reinforcer should disrupt habituation and produce an interaction. Pigeons and rats responded on multiple variable interval variable interval schedules. Introducing variability into the duration of reinforcers in one component increased response rates in both components when the schedules provided high, but not low, rates of reinforcement. The increases in constant-component response rates grew larger as the session progressed. Within-session decreases in responding were smaller when the other component provided variable-, rather than fixed-, duration reinforcers. These results are consistent with the idea that changes in habituation to the reinforcer contribute to behavioral interactions. They help to explain why interactions do not occur for some subjects under conditions that produce them for others. Finally, the results question the assumption that induction and behavioral contrast are always produced by different theoretical mechanisms.     

The effect of prefeeding on fixed-ratio pausing is jointly determined by past and upcoming reinforcer magnitudes

Pausing within multiple fixed-ratio schedules differing in reinforcer magnitude is jointly controlled by both past and upcoming conditions of reinforcement. Abrupt shifts from a just-received large reinforcer to a signaled upcoming small reinforcer (i.e., a negative incentive shift) produce marked disruptions in responding, as indexed by extended pausing. The purpose of this experiment was to determine if reducing the level of food deprivation via prefeeding enhanced these disruptive effects. Five Long Evans rats lever-pressed according to a fixed-ratio schedule. Half of the components ended in a relatively large reinforcer (three 45-mg food pellets) and half ended in a relatively small reinforcer (one pellet). Components alternated irregularly, yielding four transitions between reinforcers: small-small, small-large, large-small (the negative incentive shift), and large-large. During five, 1-session prefeeding probes, rats were given 12 g of food in their home cages 1 h prior to the start of the session. Under steady-state conditions, negative incentive shifts engendered the longest pausing. Prefeeding produced large absolute and relative increases in pausing during negative incentive shifts, and small increases in pausing in the other transitions. The results are interpreted within a resistance to change framework.     

Haloperidol, dynamics of choice, and the parameters of the matching law

The idea that dopamine mediates the reinforcing effects of stimuli persists in the field of neurosciences. The present study shows that haloperidol, a dopamine antagonist, does not eliminate the reinforcing value of food reinforcers. The ratio of reinforcers changed seven times across two levers within sessions, modeling a dynamic environment. The magnitude of the reinforcer was manipulated independently of the reinforcer ratio. Four doses of intraperitoneal haloperidol were assessed over periods of 12 daily sessions. Haloperidol did not impair the discrimination that the rats established between rich and lean levers; the response distributions favored the lever associated with the higher probability of reinforcement and the larger pellets. The parameters of the generalized matching law (bias and sensitivity) were used to estimate effects of haloperidol upon the motor system and upon the rats' motivation for food reinforcers.     

Comparison of the instrumental reinforcer devaluation effect in two strains of rats (Wistar and Lister)

In this experiment, the effect of the reinforcer devaluation upon instrumental performance was analysed in two strains of rats (Wistar and Lister): Food deprived rats were trained to press a lever for sucrose pellets in a single session. Immediately after the fulfilment of this session, half of the Wistar and Lister rats received an injection of lithium chloride (LiCl), while the remaining animals were injected with a sodium chloride (NaCl) solution. A subsequent extinction test showed that the subjects who had received immediate LiCl did not press the lever as often as those injected with NaCl. No differences in response suppression were found between the two strains of rats. These results also show that a single devaluation experience is sufficient for an impact on instrumental performance.     

Transitional choice behavior in concurrent-chain schedules

The choice responses of four pigeons were examined in 20 periods of transition in a concurrent-chain procedure with variable-interval schedules as initial links and fixed delays to reinforcement as terminal links. In some conditions, the delays to reinforcement were different for the two terminal links, and changes in preference were recorded after the delays for the two response keys were switched. In other conditions, the reinforcer delays were equal for the two keys, but which key delivered 80% of the reinforcers was periodically switched. Choice proportions changed more quickly after a switch in reinforcement percentages than after a switch in the delays, thereby contradicting the hypothesis that faster changes would occur when the switch in conditions was easier to discriminate. Analyses of response sequences showed that the effects of individual reinforcers were larger and lasted longer in conditions with changing reinforcement percentages than in conditions with changing terminal-link delays. Rates of change in choice behavior do not appear to be limited by the unpredictability of variable reinforcement schedules, because the changes in behavior were slow and gradual even when there was a large and sudden change in reinforcer delays.     

Choice in a variable environment: Effects of d-amphetamine on sensitivity to reinforcement

Four pigeons responded under a 7-component mixed schedule in which each component arranged a different left:right reinforcer ratio (27:1, 9:1, 3:1, 1:1, 1:3, 1:9, 1:27). Components were unsignaled, and the order within each session was randomly determined. After extensive exposure to these contingencies, effects of a range of doses of d-amphetamine (0.3-5.6 mg/kg) on estimates of sensitivity to reinforcement at several levels of analysis were assessed. Under non-drug conditions, the structure of choice was similar to that previously reported under this procedure. That is, responding adjusted within components to the reinforcer ratio in effect (i.e., sensitivity estimates were higher in the 2nd than in the 1st half of components), and individual reinforcers produced “preference pulses” (i.e., each food presentation produced an immediate, local, shift in preference toward the response that just produced food). Although there was a general tendency for d-amphetamine to reduce overall sensitivity to reinforcement, the size of this effect and its reliability varied across pigeons. Further analysis, however, revealed that intermediate d-amphetamine doses consistently reduced sensitivity immediately following reinforcer presentations; that is, these doses consistently attenuated preference pulses.