Responding for sucrose and wheel-running reinforcement: effect of pre-running

Six male albino Wistar rats were placed in running wheels and exposed to a fixed interval 30-s schedule that produced either a drop of 15% sucrose solution or the opportunity to run for 15s as reinforcing consequences for lever pressing. Each reinforcer type was signaled by a different stimulus. To assess the effect of pre-running, animals were allowed to run for 1h prior to a session of responding for sucrose and running. Results showed that, after pre-running, response rates in the later segments of the 30-s schedule decreased in the presence of a wheel-running stimulus and increased in the presence of a sucrose stimulus. Wheel-running rates were not affected. Analysis of mean post-reinforcement pauses (PRP) broken down by transitions between successive reinforcers revealed that pre-running lengthened pausing in the presence of the stimulus signaling wheel running and shortened pauses in the presence of the stimulus signaling sucrose. No effect was observed on local response rates. Changes in pausing in the presence of stimuli signaling the two reinforcers were consistent with a decrease in the reinforcing efficacy of wheel running and an increase in the reinforcing efficacy of sucrose. Pre-running decreased motivation to respond for running, but increased motivation to work for food.     

Rats do not respond differently in the presence of stimuli signaling wheel-running reinforcers of different durations

Rats were exposed to a fixed interval 30 s schedule that produced opportunities to run of equal or unequal durations to assess the effect of differences in duration on responding. Each duration was signaled by a different stimulus. Wheel-running reinforcer duration pairs were 30 s 30 s, 50 s 10 s, and 55 s 5 s. An analysis of median postreinforcement pause duration and mean local lever-pressing rates broken down by previous reinforcer duration and duration of signaled upcoming reinforcer showed that postreinforcement pause duration was affected by the duration of the previous reinforcer but not by the stimulus signaling the duration of the upcoming reinforcer. Local lever-pressing rates were not affected by either previous or upcoming reinforcer duration. In general, the results are consistent with indifference between these durations obtained using a concurrent choice procedure.     

Reinforcer satiation and resistance to change of responding maintained by qualitatively different reinforcers

In previous research on resistance to change, differential disruption of operant behavior by satiation has been used to assess the relative strength of responding maintained by different rates or magnitudes of the same reinforcer in different stimulus contexts. The present experiment examined resistance to disruption by satiation of one reinforcer type when qualitatively different reinforcers were arranged in different contexts. Rats earned either food pellets or a 15% sucrose solution on variable-interval 60-s schedules of reinforcement in the two components of a multiple schedule. Resistance to satiation was assessed by providing free access either to food pellets or the sucrose solution prior to or during sessions. Responding systematically decreased more relative to baseline in the component associated with the satiated reinforcer. These findings suggest that when qualitatively different reinforcers maintain responding, relative resistance to change depends upon the relations between reinforcers and disrupter types.     

The effect of prefeeding on fixed-ratio pausing is jointly determined by past and upcoming reinforcer magnitudes

Pausing within multiple fixed-ratio schedules differing in reinforcer magnitude is jointly controlled by both past and upcoming conditions of reinforcement. Abrupt shifts from a just-received large reinforcer to a signaled upcoming small reinforcer (i.e., a negative incentive shift) produce marked disruptions in responding, as indexed by extended pausing. The purpose of this experiment was to determine if reducing the level of food deprivation via prefeeding enhanced these disruptive effects. Five Long Evans rats lever-pressed according to a fixed-ratio schedule. Half of the components ended in a relatively large reinforcer (three 45-mg food pellets) and half ended in a relatively small reinforcer (one pellet). Components alternated irregularly, yielding four transitions between reinforcers: small-small, small-large, large-small (the negative incentive shift), and large-large. During five, 1-session prefeeding probes, rats were given 12 g of food in their home cages 1 h prior to the start of the session. Under steady-state conditions, negative incentive shifts engendered the longest pausing. Prefeeding produced large absolute and relative increases in pausing during negative incentive shifts, and small increases in pausing in the other transitions. The results are interpreted within a resistance to change framework.     

Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement

Across two experiments, a peak procedure was used to assess the timing of the onset and offset of an opportunity to run as a reinforcer. The first experiment investigated the effect of reinforcer duration on temporal discrimination of the onset of the reinforcement interval. Three male Wistar rats were exposed to fixed-interval (FI) 30-s schedules of wheel-running reinforcement and the duration of the opportunity to run was varied across values of 15, 30, and 60s. Each session consisted of 50 reinforcers and 10 probe trials. Results showed that as reinforcer duration increased, the percentage of postreinforcement pauses longer than the 30-s schedule interval increased. On probe trials, peak response rates occurred near the time of reinforcer delivery and peak times varied with reinforcer duration. In a second experiment, seven female Long-Evans rats were exposed to FI 30-s schedules leading to 30-s opportunities to run. Timing of the onset and offset of the reinforcement period was assessed by probe trials during the schedule interval and during the reinforcement interval in separate conditions. The results provided evidence of timing of the onset, but not the offset of the wheel-running reinforcement period. Further research is required to assess if timing occurs during a wheel-running reinforcement period.     

Operant behavior in dwarf hamsters (Phodopus campbelli): Effects of rate of reinforcement and reinforcer flavor variety

We investigated operant behavior in a novel species, the dwarf hamster (Phodopus campbelli). In two experiments, hamsters were trained to lever-press for food reinforcement. In Experiment 1, rate of reinforcement was manipulated across conditions using four variable-interval schedules of reinforcement (delivering one to eight reinforcers per min). As predicted, within-session decreases in responding were steepest on the richest schedule. In Experiment 2, lever-pressing was reinforced by either a constant or a variety of flavored food pellets. Within-session decreases in responding were steeper when the reinforcer flavor remained constant than when it was varied within the session. In both experiments, subjects hoarded most reinforcers in their cheek pouches rather than consuming them in the operant chambers. These results are incompatible with post-ingestive satiety variables as explanations for within-session decreases in operant responding and suggest that habituation to repeatedly presented reinforcers best accounts for subjects’ response patterns. Additionally, a mathematical model that describes behavior undergoing habituation also described the present results, thus strengthening the conclusion that habituation mediates the reinforcing efficacy of food.     

Role of conditioned reinforcers in the initiation, maintenance and extinction of drug-seeking behavior.

The development of a secondary reinforcer as a result of associating a neutral stimulus (buzzer) with intravenous (IV) doses of morpine was studied in rats. Secondary reinforcement developed in the absence of physical dependence and followed the association of the stimulus with either response-contingent or non-contingent injections of morphine. Strength of the conditioned reinforcer, measured in terms of responding on a lever for the stimulus plus infusion of saline solution, was proportional to the unit dosage of morphine employed in pairings of buzzer and drug. When extinction of the lever-press response for IV morphine was conducted (by substituting saline for morphine solution) in the absence of the conditioned reinforcing stimulus, it was seen later that the stimulus could still elicit lever responses, until it too had been present for a sufficient interval of non-reinforced responding. Similarly, extinction of the response for morphine by blocking its action with naloxone in the absence of the stimulus did not eliminate the conditioned reinforcement. Another study showed that a passive, subcutaneous (SC) dose of morphine served to maintain lever-pressing on a contingency of buzzer plus saline infusion. Furthermore, the stimuli resulting from the presence of morphine (after a SC injection) were able to reinstate the lever-responding with only the buzzer-saline contingency when such responses had previously been extinguished. Moreover, it was shown that d-amphetamine could restore responding under the same conditions, and that morphine could also do so for rats in which the primary reinforcer had been d-amphetamine. It is suggested that animal data such as these show that procedures designed for the elimination of human drug-taking behavior must take into account secondary reinforcers as well as the primary reinforcer(s).     

Small and large fixed ratios with the same unit price

Key pressing of rats was maintained under multiple and discrete-trial choice schedules with reinforcer units of 45 mg food pellets or 3.5 s dips of sucrose solution. Both smaller and larger fixed ratio (FR) schedules were associated with the same unit price in a manner, for example, that each of eight iterations of FR120 was associated with delivery of a single reinforcer unit and one instance of FR960 was associated with eight reinforcer units. FR requirement varied between 20 and 1560 per aggregate reinforcer and unit price varied between 20 and 240 per reinforcer unit. During multiple schedules with food reinforcers, rates and patterns of responding were comparable over nearly a 50-fold range of FR requirements (20-1380) when unit price was 20; over nearly a six-fold range of FR requirements (120-720) when unit price was 120; and was only marginally maintained when unit price was 240. Demand for food pellets was comparatively inelastic at FRs between 20 and 120, during which subjects did not receive supplemental feeding outside experimental sessions, but was elastic at FRs greater than 240, when subjects sometimes did receive supplemental feeding. In a discrete-trial choice procedure with a constant unit price of 120 for sucrose solution, subjects were indifferent between smaller FRs and alternative FRs as large as 480, but began switching away from larger FRs that were 600 or greater. Because responding had been comparably maintained under both FR120 and FRs as large as 960 in the multiple schedule, results from the choice procedure indicated that choice performance was influenced by variables other than FR requirement and unit price. Because aggregate reinforcers were the same for smaller and larger FRs, the most likely reason for preferring smaller FRs was the nearness in time to some reinforcer.     

The effect of second-half reinforcer type on responding for sucrose in the first half of the session

The present study investigated whether the sucrose-reinforced lever pressing of rats in the first half of a 50-min session would be sensitive to upcoming food-pellet reinforcement in the second half. In Experiment 1, the type of reinforcer in the first half of the session was always liquid sucrose and type of reinforcer in the second half (liquid sucrose or food pellets) varied across conditions. Sucrose concentration varied across groups (1, 5, or 25%). Results showed that rates and patterns of responding for 1%, and sometimes for 5%, sucrose reinforcers in the first half of the session were higher and steeper, respectively, when food-pellet, rather than sucrose, reinforcement occurred in the second half. Responding for 25% sucrose was not similarly affected. Experiment 2 replicated the results of Experiment 1 using a within-subjects design. Although the present results represent induction (i.e. the opposite of contrast), they are consistent with some results on consummatory contrast. They also further demonstrate that responding on interval schedules of reinforcement can be altered prospectively. By doing so, however, they pose potential problems for current theories for why operant response rates change within the session.     

Reinforcer devaluation by extinction depends on the food restriction protocol

A common feature of reinforcer devaluation studies is that new learning induces the devaluation. The present study used extinction to induce new learning about the conditioned reinforcer in a heterogeneous chain schedule. Rats pressed a lever in a heterogeneous chain schedule to produce a conditioned reinforcer (light) associated with the opportunity to obtain an unconditioned reinforcer (food) by pulling a chain. The density of food reinforcement correlated with the conditioned reinforcer was varied in a comparison of continuous and variable-ratio reinforcement schedules of chain pulling; this had no noticeable effect on conditioned reinforcer devaluation produced by extinction of chain pulling. In contrast, how rats were deprived appeared to matter very much. Restricting meal duration to 1h daily produced more lever pressing during baseline training and a greater reductive effect of devaluation on lever pressing than restricting body weight to 80% of a control rat's weight, which eliminated the devaluation effect. Further analysis suggested that meal-duration restriction may have produced devaluation effects because it was more effective than weight restriction in reducing rats' body weights. Our results exposed an important limitation on the devaluation of conditioned reinforcers: slight differences in food restriction, using two commonly employed food-restriction procedures, can produce completely different interpretations of reinforcer devaluation while leaving reinforcer-based learning intact.     

The role of place of reinforcer delivery in the appearance of positive induction when rats respond for 1% sucrose

Rats increase their rate of operant responding for 1% sucrose reinforcement in the first half of an experimental session if a high-valued reinforcer will be available in the second half. Previous research suggests that this induction effect occurs because the reinforcing value of the low-valued substance has increased. The present study investigated whether this increase may occur because of where the substances are delivered. Rats pressed a lever to earn 1% liquid-sucrose reinforcers in the first half of the session. In control conditions, they also pressed for 1% sucrose in the second half. In treatment conditions, they pressed for food-pellet (Experiment 1) or 32% sucrose (Experiment 2) reinforcers in the second half, with these reinforcers either being delivered to the same location as the 1% sucrose or to a different location. Upcoming food-pellet or 32% sucrose reinforcement increased rates of lever pressing for 1% sucrose in the first half of the session, with the largest increase observed when the high-valued reinforcer was delivered to the same location as the 1% sucrose. Qualitatively similar results were found with rates of consumption of 1% sucrose reinforcers in the first half of the session, which were measured in Experiment 2. The location to which reinforcers are delivered appears to be one of the factors that contributes to this induction effect. The present results may therefore identify one of the factors that determine whether differential conditions of reinforcement will lead to contrast or induction.     

Four-alternative choice violates the constant-ratio rule

Six pigeons responded on a four-key concurrent variable-interval schedule in which a 27:9:3:1 distribution of reinforcers between the keys changed every 10 reinforcers. Their behaviour quickly came under the control of this changing four-way reinforcer ratio. However, preference between a pair of keys depended not only on the relative reinforcer rates on those keys, but also on the absolute levels of those rates. This contradicts the constant-ratio rule that underpins the matching approach to choice, but is predicted by a contingency-discriminability model that assumes that organisms may occasionally misattribute reinforcers to a response that did not produce them. Reinforcers produced strong preference pulses, or transient increases in responding on the just-reinforced key. Despite accurate tracking of the reinforcer ratio, reinforcers obtained late in components and from leaner keys still produced strong pulses, suggesting both extended and local control of behaviour. Patterns of switching between keys were graded and similarly controlled by the reinforcer rates on each key. Whether considered in terms of switching, local preference pulses, or extended preference, behaviour was controlled by a rapidly changing four-way reinforcer ratio in a graduated, continuous manner that is unlikely to be explained by a simple heuristic such as fix-and-sample.     

The reinforcing property and the rewarding aftereffect of wheel running in rats: a combination of two paradigms

Wheel running reinforces the behavior that generates it and produces a preference for the context that follows it. The goal of the present study was to demonstrate both of these effects in the same animals. Twelve male Wistar rats were first exposed to a fixed-interval 30 s schedule of wheel-running reinforcement. The operant was lever-pressing and the reinforcer was the opportunity to run for 45 s. Following this phase, the method of place conditioning was used to test for a rewarding aftereffect following operant sessions. On alternating days, half the rats responded for wheel-running reinforcement while the other half remained in their home cage. Upon completion of the wheel-running reinforcement sessions, rats that ran and rats that remained in their home cages were placed into a chamber of a conditioned place preference (CPP) apparatus for 30 min. Each animal received six pairings of a distinctive context with wheel running and six pairings of a different context with their home cage. On the test day, animals were free to move between the chambers for 10 min. Results showed a conditioned place preference for the context associated with wheel running; however, time spent in the context associated with running was not related to wheel-running rate, lever-pressing rate, or post-reinforcement pause duration.     

Habituation and the reinforcing effectiveness of visual stimuli

The term "sensory reinforcer" has been used to refer to sensory stimuli (e.g. light onset) that are primary reinforcers in order to differentiate them from other more biologically important primary reinforcers (e.g. food and water). Acquisition of snout poke responding for a visual stimulus (5s light onset) with fixed ratio 1 (FR 1), variable-interval 1min (VI 1min), or variable-interval 6min (VI 6min) schedules of reinforcement was tested in three groups of rats (n=8/group). The VI 6min schedule of reinforcement produced a higher response rate than the FR 1 or VI 1min schedules of visual stimulus reinforcement. One explanation for greater responding on the VI 6min schedule relative to the FR 1 and VI 1min schedules is that the reinforcing effectiveness of light onset habituated more rapidly in the FR 1 and VI 1min groups as compared to the VI 6min group. The inverse relationship between response rate and the rate of visual stimulus reinforcement is opposite to results from studies with biologically important reinforcers which indicate a positive relationship between response and reinforcement rate. Rapid habituation of reinforcing effectiveness may be a fundamental characteristic of sensory reinforcers that differentiates them from biologically important reinforcers, which are required to maintain homeostatic balance.     

Three tests of 'anticipatory responding' as an account for induction produced by upcoming food-pellet reinforcement

Previous research has demonstrated that rats' rates of lever pressing for low-concentration liquid-sucrose reinforcers are increased when food-pellet, rather than sucrose, reinforcement will be upcoming in the same session (i.e. induction). The present experiments were designed to determine whether this induction was the product of 'anticipatory responses' for the upcoming food pellets being added to the responses being made for the currently available sucrose reinforcement. Experiment 1 tested this idea by summing sucrose-reinforced responding and 'anticipatory responding' from different conditions and comparing the sum to responding from a third condition in which subjects responded for sucrose when food-pellet reinforcement was upcoming. The comparison yielded similar response rates. Experiment 2 employed a blackout, of different durations in different conditions, to delay the upcoming food-pellet reinforcement. Consistent with the anticipatory-responding account, the delay decreased the size of the induction. However, results from the blackouts were not entirely consistent with the anticipatory-responding explanation. Experiment 3 provided, in some conditions, sucrose and food-pellet reinforcement in the first and second halves of the session, respectively, for responding on separate levers. These conditions separated 'anticipatory responses' for the food pellets from responses for the sucrose reinforcers. However, induction in responding for sucrose was still present. Together, these experiments demonstrate that, although anticipatory responses may contribute to induction in some instances, they are not solely responsible for the effect.     

Effect of short-term prefeeding and body weight on wheel running and responding reinforced by the opportunity to run in a wheel

A biobehavioural analysis of activity anorexia suggests that the motivation for physical activity is regulated by food supply and body weight. In the present experiment, food allocation was varied within subjects by prefeeding food-deprived rats 0, 5, 10 and 15 g of food before sessions of lever pressing for wheel-running reinforcement. The experiment assessed the effects of prefeeding on rates of wheel running, lever pressing, and postreinforcement pausing. Results showed that prefeeding animals 5 g of food had no effect. Prefeeding 10 g of food reduced lever pressing for wheel running and rates of wheel running without a significant change in body weight; the effect was, however, transitory. Prefeeding 15 g of food increased the animals' body weights, resulting in a sustained decrease of wheel running and lever pressing, and an increase in postreinforcement pausing. Overall the results indicate that the motivation for physical activity is regulated by changes in local food supply, but is sustained only when there is a concomitant change in body weight.     

Reinforcement omission in concurrent fixed-interval and random-interval schedules

The present experiment examined overall and local effects of omission of reinforcers in a choice situation. Pigeons' key-pecking responses were reinforced under concurrent fixed-interval and random-interval schedules of food presentation. After some weeks of baseline sessions in which the probability of reinforcement was 1.00, approximately 25% of food presentations from the fixed-interval schedule were omitted and replaced by timeout periods. In such omission sessions, the overall relative rates of responding to the fixed-interval schedule became lower than those in the baseline sessions. On the other hand, when relative rates of responding to the fixed-interval schedule in the omission sessions were calculated separately for fixed-interval cycles preceded by timeout periods and those preceded by food presentations, the relative rates in the former type of fixed-interval cycles were higher than those in the latter type for three out of four pigeons. These results mean that relative rates of responding cannot always be regarded as reflecting a relative value of an alternative, and that the overall effect of the omission of fixed-interval reinforcers is not reducible to the local effect of omission.     

Varying reinforcer duration produces behavioral interactions during multiple schedules

The experiments tested the idea that changes in habituation to the reinforcer contribute to behavioral interactions during multiple schedules. This idea predicts that changing an aspect of the reinforcer should disrupt habituation and produce an interaction. Pigeons and rats responded on multiple variable interval variable interval schedules. Introducing variability into the duration of reinforcers in one component increased response rates in both components when the schedules provided high, but not low, rates of reinforcement. The increases in constant-component response rates grew larger as the session progressed. Within-session decreases in responding were smaller when the other component provided variable-, rather than fixed-, duration reinforcers. These results are consistent with the idea that changes in habituation to the reinforcer contribute to behavioral interactions. They help to explain why interactions do not occur for some subjects under conditions that produce them for others. Finally, the results question the assumption that induction and behavioral contrast are always produced by different theoretical mechanisms.     

Demand for sucrose in the genetically obese Zucker (fa/fa) rat

Obese Zucker rats (fa/fa) eat more food than lean controls in free-feeding conditions, which strongly influences their phenotypic expression. Few studies, however, characterize their food consumption in environments that are more representative of foraging conditions, e.g., how effort plays a role in food procurement. This study examined the reinforcing efficacy of sucrose in obese Zucker rats by varying the responses required to obtain single sucrose pellets. Male Zucker rats (15 lean, 14 obese) lever-pressed under eight fixed ratio (FR) schedules of sucrose reinforcement, in which the number of lever-presses required to gain access to a single sucrose pellet varied from 1 to 300. Linear and exponential demand equations, which characterize the value of a reinforcer by its sensitivity to price (FR), were fit to the number of food reinforcers and responses made. Free food consumption was also examined. Obese Zuckers, compared to leans, consumed more food under free-feeding conditions. Moreover, they had higher levels of consumption and response output, but only at low FR values. Both groups were equally sensitive to price increases at higher FR values. This suggests that environmental conditions may interact with genes in the expression of food reinforcer efficacy.     

Response-reinforcer relations and resistance to change

Behavioral momentum theory suggests that the relation between a response and a reinforcer (i.e., response-reinforcer relation) governs response rates and the relation between a stimulus and a reinforcer (i.e., stimulus-reinforcer relation) governs resistance to change. The present experiments compared the effects degrading response-reinforcer relations with response-independent or delayed reinforcers on resistance to change in conditions with equal stimulus-reinforcer relations. In Experiment 1, pigeons responded on equal variable-interval schedules of immediate reinforcement in three components of a multiple schedule. Additional response-independent reinforcers were available in one component and additional delayed reinforcers were available in another component. The results showed that resistance to disruption was greater in the components with added reinforcers than without them (i.e., better stimulus-reinforcer relations), but did not differ for the components with added response-independent and delayed reinforcement. In Experiment 2, a component presenting immediate reinforcement alternated with either a component that arranged equal rates of reinforcement with a proportion of those reinforcers being response independent or a component with a proportion of the reinforcers being delayed. Results showed that resistance to disruption tended to be either similar across components or slightly lower when response-reinforcer relations were degraded with either response-independent or delayed reinforcers. These findings suggest that degrading response-reinforcer relations can impact resistance to change, but that impact does not depend on the specific method and is small relative to the effects of the stimulus-reinforcer relation.