Resistance to change varies inversely with reinforcement context

We report two experiments which test whether resistance to prefeeding and satiation for a variable-interval (VI) schedule that delivers a constant rate of reinforcement varies inversely with the reinforcement rate for an alternative schedule. In Experiment 1, eight pigeons responded in a multiple schedule in which the red key was always associated with a VI 90-s schedule and the green key with either a richer (VI 18s) or leaner (VI 540s) schedule in different conditions. After baseline training in each condition, prefeeding test sessions were conducted in which 10g, 20g, 30g, 40g, and 50g food were provided one-hour prior to test. Additional baseline training was given between each test session. In Experiment 2, two groups of pigeons responded in a multiple schedule similar to Experiment 1. After baseline training, pigeons were exposed to a 5-h satiation test session in which the VI 90-s schedule was available continuously. Test sessions were conducted when pigeons were maintained at 85%, 95%, and 85% of their body weights in an ABA design. Results of both experiments showed that responding in the VI 90-s schedule that alternated with a leaner schedule during baseline was more resistant to prefeeding and satiation. These data rule out alternative explanations for results of previous studies, and confirm that resistance to change varies inversely with reinforcement context.     

Dishabituation with component transitions may contribute to the interactions observed during multiple schedules

Rates of responding by rats were usually higher during the variable interval (VI) 30-s component of a multiple VI 30-s fixed interval (FI) 30-s schedule than during the same component of a multiple VI 30-s VI 30-s schedule (Experiment 1). Response rates were also usually higher during the FI 30-s component of a multiple VI 30-s FI 30-s schedule than during the same component of a multiple FI 30-s FI 30-s schedule (Experiment 2). The differences in response rates were not observed when the components provided VI or FI 120-s schedules. These results were predicted by the idea that differences in habituation to the reinforcer between multiple schedules contribute to behavioral interactions, such as behavioral contrast. However, differences in habituation were not apparent in the within-session patterns of responding. Finding differences in response rates in both experiments violates widely-held assumptions about behavioral interactions, including that behavioral contrast does not occur for rats and that improving the conditions of reinforcement decreases, rather than increases, response rate in the alternative component.     

Differential resurgence and response elimination

Resurgence refers to the transient recovery of previously reinforced, but presently not reinforced, responding when more recently reinforced responding is extinguished. The primary purpose of our research was to determine how differential resurgence results from the procedures used to eliminate that responding. There were three conditions in each of five experiments. In Condition 1, key pecking by pigeons was maintained under a two-component multiple variable-interval (VI) 30-s VI 30-s schedule. In Condition 2, this pecking was eliminated in different ways across components. In Condition 3, extinction was in effect for all responses, and resurgence of key pecking was compared across components. These three conditions were repeated for most pigeons, and the procedures used to eliminate responding in Condition 2 varied across experiments. In Experiment 1, there was greater resurgence, and an earlier onset of it, after a differential-reinforcement-of-other-behavior (DRO) schedule than after a VI schedule was correlated with pecking an alternative key. Experiments 2 and 3 showed that the differential resurgence in Experiment 1 probably was not due to conditional stimulus control or the periodicity of food delivery, respectively. In Experiment 4, there was no systematic difference in resurgence after either a DRO schedule or a VI schedule correlated with treadle pressing. In Experiment 5, there was greater resurgence, and/or an earlier onset of it, after a VI schedule correlated with treadle pressing than after a VI schedule correlated with pecking an alternative key. Taken together, the results showed that the reinforcement of an alternative key-peck response was the most effective means of reducing subsequent key-peck resurgence. The relation of these results to an understanding of resurgence is discussed.     

Effect of signaling reinforcement on resistance to change in a multiple schedule

This study evaluated the effect of a signal on resistance to change using a multiple schedule of reinforcement. Experiment 1 presented pigeons with three schedules: a signaled delay to reinforcement schedule (a two-link chain schedule with a variable-interval 120-s initial link followed by a 5-s fixed-time schedule), an unsignaled delay schedule (a comparable two-link tandem schedule), and an immediate, zero-delay variable-interval 125-s schedule. Two separate disruption procedures assessed resistance to change: extinction and adding a variable-time 20-s schedule of reinforcement to the inter-component interval. Resistance to change tests were conducted twice, once with the signal stimulus (the terminal link of the chain schedule) present and once with it absent. Results from both disruption procedures showed that signal absence reduced resistance to change for the pre-signal stimulus. In probe choice tests subjects strongly preferred the signal stimulus over the unsignaled stimulus and exhibited no reliable preference when given a choice between the signal stimulus and immediate stimulus. Experiment 2 presented two equal signaled schedules where, during resistance to change tests, the signal remained for one schedule and was removed for the second. Resistance to change was consistently lower when the signal was absent.     

Dynamics of temporal control in rats: the effects of a brief transition in interval duration

Five rats lever-pressed for liquid reinforcers delivered according to a fixed-interval (FI) reinforcement schedule, where the interval requirement changed at an unpredictable point within a session. In a short square wave (SSW) condition, eight 30-s intervals were intercalated in a series of 120-s intervals so that the intervals changed from 120 to 30 s then back to 120 s. In a long square wave (LSW) condition the intervals changed from 120 to 480 s then back to 120 s. We observed rapid temporal control of post-reinforcement wait time duration by the IFI duration in the SSW condition only: Wait times decreased significantly during a transition to shorter (30 s) intervals; whereas wait times did not reliably increase during a transition to longer (480 s) intervals. Furthermore, in the SSW condition, wait time in post-transition intervals was shorter than that observed during pre-transition intervals. The results show that rats' wait times are sensitive to moment-to-moment changes in interval duration and that the dynamics depend on the direction in which the intervals change.     

Pigeons' memory for sequences of light flashes: reliance on temporal properties and evidence for delay interval/gap confusion

In Experiment 1, pigeons were trained to discriminate between sequences of two and four light flashes (illumination of the feeder). Retention functions obtained with dark delays exhibited a choose-many bias at a 1-s delay and a choose-few bias at delays of 4 and 8s. Retention functions obtained with illuminated delays only displayed a slight choose-few bias. In Experiment 2, additional birds were trained with the same sample sequences. However, the intertrial interval was illuminated by the houselight for Group Light, and it was dark for Group Dark. The acquisition data suggested that multiple temporal features of the light flash sequences controlled choice responding. For both groups, the retention functions were similar to those obtained in Experiment 1. In Experiment 3, baseline training with a 1-s dark delay eliminated the choose-many bias, but a significant choose-few bias at extended dark delays was still observed. Pigeons discriminate light flash sequences by relying on temporal properties of the sequence rather than using an event switch to count flashes. The biased-forgetting effects obtained in these studies appear to be primarily due to confusion between the delay interval and the gap between light flashes.     

Effects of retention interval on performance in a numerical reproduction task

The retention interval (RI) between the sample and production phase in a numerical reproduction task was varied to determine whether a "produce-small" effect would be obtained with increased delays. Four pigeons were trained with a retention interval of 2s, and then tested with intervals of 0.5s and 8s. Results showed a number-dependent, "produce-large" effect-response number increased when RI was increased-analyses of average response number and accuracy suggested RI affected responding most on the 2-flash trials with an 8-s RI. Additionally, discrimination between trial types decreased as RI increased. Existing explanations for the "choose-short/small" effect appear unable to account for these results; however the "produce-large" effect may be attributed to a disruption in stimulus control over responding.     

Effect of contingent auditory stimuli on concurrent schedule performance: an alternative punisher to electric shock

This study explored whether load auditory stimuli could be used as functional punishing stimuli in place of electric shock. Three experiments examined the effect of a loud auditory stimulus on rats’ responding maintained by a concurrent reinforcement schedule. In Experiment 1, overall response rate decreased when a concurrent 1.5 s tone presentation schedule was superimposed on the concurrent variable interval (VI) 180-s, VI 180-s reinforcement schedule. On the contrary, response rate increased when a click presentation schedule was added. In Experiment 2, the extent of the response suppression with a 1.5 s tone presentation varied as a function of the frequency of the reinforcement schedule maintaining responses; the leaner the schedule employed, the greater the response suppression. In Experiment 3, response suppression was observed to be inversely related to the duration of the tone; response facilitation was observed when a 3.0-s tone was used. In Experiments 1 and 2, a preference shift towards the alternative with richer reinforcement was observed when the tone schedule was added. In contrast, the preference shifted towards the leaner alternative when the click or longer duration stimulus was used. These results imply that both the type and duration of a loud auditory stimulus, as well as the reinforcement schedule maintaining responses, have a critical role in determining the effect of the stimuli on responding. They also suggest that a loud auditory stimulus can be used as a positive punisher in a choice situation for rats, when the duration of the tone is brief, and the reinforcement schedule maintaining responses is lean.     

Response-cost punishment via token loss with pigeons

Two experiments were conducted to investigate punishment via response-contingent removal of conditioned token reinforcers (response cost) with pigeons. In Experiment 1, key pecking was maintained on a two-component multiple second-order schedule of token delivery, with light emitting diodes (LEDs) serving as token reinforcers. In both components, responding produced tokens according to a random-interval 20-s schedule and exchange periods according to a variable-ratio schedule. During exchange periods, each token was exchangeable for 2.5-s access to grain. In one component, responses were conjointly punished according to fixed-ratio schedules of token removal. Response rates in this punishment component decreased to low levels while response rates in the alternate (no-punishment) component were unaffected. Responding was eliminated when it produced neither tokens nor exchange periods (Extinction), but was maintained at moderate levels when it produced tokens in the signaled absence of food reinforcement, suggesting that tokens served as effective conditioned reinforcers. In Experiment 2, the effect of the response-cost punishment contingency was separated from changes in the density of food reinforcement. This was accomplished by yoking either the number of food deliveries per component (Yoked Food) or the temporal placement of all stimulus events (tokens, exchanges, food deliveries) (Yoked Complete), from the punishment to the no-punishment component. Response rates decreased in both components, but decreased more rapidly and were generally maintained at lower levels in the punishment component than in the yoked component. In showing that the response-cost contingency had a suppressive effect on responding in addition to that produced by reductions in reinforcement density, the present results suggest that response-cost punishment shares important features with other forms of punishment.     

Effects of d-amphetamine on behavior maintained by signaled and unsignaled delays to reinforcement

Four pigeons responded under a progressive-delay procedure. In a signaled-delay condition, a chained variable interval (VI) 30-s progressive time (PT) 4-s schedule was arranged; in an unsignaled-delay condition, a tandem VI 30-s PT 4-s schedule was arranged. Two pigeons experienced a signaled-unsignaled-signaled sequence; whereas, two pigeons experienced an unsignaled-signaled-unsignaled sequence. Effects of saline and d-amphetamine were determined under each condition. At intermediate doses (1.0 and 1.78 m/kg) delay functions were shallower, area under the curve was increased, and, when possible, break points were increased compared to saline; these effects were not systematically related to signaling conditions. These effects on control by delay often were accompanied by decreased response rates at 0 s. These results suggest that stimulus conditions associated with the delay may not play a crucial role in effects of d-amphetamine and other stimulants on behavior controlled by reinforcement delay.     

Segmentation and the pairing hypothesis

The effect of stimulus contiguity and response contingency on responding in chain schedules was examined in two experiments. In Experiment 1, four pigeons were trained on two simple three-link chain schedules that alternated within sessions. Initial links were correlated with a variable-interval 30s schedule, and middle and terminal links were correlated with interdependent variable-interval 30s variable-interval 30s schedules. The combined duration of the interdependent schedules summed to 60s. The two chains differed with respect to signaling of the schedule components: a two-stimulus chain had one stimulus paired with the initial link and one stimulus paired with both the middle and the terminal link, while a three-stimulus chain had a different stimulus paired with the each of the three links. The results showed that the two-stimulus chain maintained lower initial-link responding than the three-stimulus chain. In Experiment 2, four pigeons were exposed to three separate conditions, the two- and three-stimulus chains of Experiment 1 and a three-stimulus chain that had a 3s delay to terminal-link entry from the middle-link response that produced it. The two-stimulus chain maintained lower initial-link responding than the three-stimulus chain, as in Experiment 1, and a similar initial-link responding was maintained by the two-stimulus chain and the three-stimulus chain with the delay contingency. The results demonstrate that a stimulus noncontiguous with food can maintain responding that is sometimes greater than a stimulus contiguous with food, depending on the response contingency for terminal-link entry. The results are contrary to the pairing hypothesis of conditioned reinforcement.     

Effect of unsignaled delays between stimuli in a chain schedule on responding and resistance to change

Behavioral momentum theory is an evolving theoretical account of the strength of behavior. One challenge for the theory is determining the role of signal stimuli in determining response strength. This study evaluated the effect of an unsignaled delay between the initial link and terminal link of a two-link chain schedule on resistance to change using a multiple schedule of reinforcement. Pigeons were presented two different signaled delay to reinforcement schedules. Both schedules employed a two-link chain schedule with a variable interval 120-s initial link followed by a 5-s fixed time terminal link schedule. One of the schedules included a 5-s unsignaled delay between the initial link and the terminal link. Resistance to change was assessed with two separate disruption procedures: extinction and adding a variable time 20-s schedule of reinforcement to the inter-component interval. Baseline responding was lower in the schedule with the unsignaled delay but resistance to change for the initial link was unaffected by the unsignaled delay. The results suggest that not all unsignaled delays are equal in their effect on resistance to change.     

Representation of time intervals in a double bisection task: Relative or absolute?

We examined if the representation of time intervals in a temporal discrimination task is based not only on their absolute but also on their relative durations. Six pigeons learned two temporal discriminations. In the first, red and green choices were correct following 2-s and 8-s samples, respectively. In the second, vertical and horizontal bar choices were correct following 4-s and 16-s samples, respectively. In a previous study [Zentall, T.R., Weaver, J.E., Clement, T.S., 2004. Pigeons group time intervals according to their relative duration. Psychon. Bull. Rev. 11, 113-117.], tests with 4-s samples and red/green comparisons revealed a bias for red, whereas tests with 8-s samples with vertical/horizontal comparisons revealed a bias for horizontal. These results were interpreted in terms of relative encoding of sample durations. We attempted to replicate this finding but instead of testing with only 4-s or 8-s samples, we tested with several other sample durations to obtain a psychometric function. Results were inconsistent with the relative encoding hypothesis.     

Oscillations following periodic reinforcement

Three experiments examined behavior in extinction following periodic reinforcement. During the first phase of Experiment 1, four groups of pigeons were exposed to fixed interval (FI 16 s or FI 48 s) or variable interval (VI 16 s or VI 48 s) reinforcement schedules. Next, during the second phase, each session started with reinforcement trials and ended with an extinction segment. Experiment 2 was similar except that the extinction segment was considerably longer. Experiment 3 replaced the FI schedules with a peak procedure, with FI trials interspersed with non-food peak interval (PI) trials that were four times longer. One group of pigeons was exposed to FI 20 s PI 80 s trials, and another to FI 40 s PI 160 s trials. Results showed that, during the extinction segment, most pigeons trained with FI schedules, but not with VI schedules, displayed pause-peck oscillations with a period close to, but slightly greater than the FI parameter. These oscillations did not start immediately after the onset of extinction. Comparing the oscillations from Experiments 1 and 2 suggested that the alternation of reconditioning and re-extinction increases the reliability and earlier onset of the oscillations. In Experiment 3 the pigeons exhibited well-defined pause-peck cycles since the onset of extinction. These cycles had periods close to twice the value of the FI and lasted for long intervals of time. We discuss some hypotheses concerning the processes underlying behavioral oscillations following periodic reinforcement.     

Responding for sucrose and wheel-running reinforcement: effect of pre-running

Six male albino Wistar rats were placed in running wheels and exposed to a fixed interval 30-s schedule that produced either a drop of 15% sucrose solution or the opportunity to run for 15s as reinforcing consequences for lever pressing. Each reinforcer type was signaled by a different stimulus. To assess the effect of pre-running, animals were allowed to run for 1h prior to a session of responding for sucrose and running. Results showed that, after pre-running, response rates in the later segments of the 30-s schedule decreased in the presence of a wheel-running stimulus and increased in the presence of a sucrose stimulus. Wheel-running rates were not affected. Analysis of mean post-reinforcement pauses (PRP) broken down by transitions between successive reinforcers revealed that pre-running lengthened pausing in the presence of the stimulus signaling wheel running and shortened pauses in the presence of the stimulus signaling sucrose. No effect was observed on local response rates. Changes in pausing in the presence of stimuli signaling the two reinforcers were consistent with a decrease in the reinforcing efficacy of wheel running and an increase in the reinforcing efficacy of sucrose. Pre-running decreased motivation to respond for running, but increased motivation to work for food.     

Effects of unsignaled delays of reinforcement on fixed-interval schedule performance

Key pecking of pigeons was maintained by a fixed-interval (FI) 61-s schedule. The effects of resetting and nonresetting unsignaled delays of reinforcement then were examined. The resetting delay was programmed as a differential-reinforcement-of-other-behavior schedule, and the nonresetting delay as a fixed-time schedule. Three delay durations (0.5, 1 and 10 s) were examined. Overall response rates were decreased by one and 10-s delays and increased by 0.5-s delays. Response patterns changed from positively accelerated to more linear when resetting or nonresetting 10-s delays were imposed, but remained predominantly positively accelerated when resetting and nonresetting 0.5- and 1-s delays were in effect. In general, temporal control, as measured by quarter-life values, changed less than overall response rates when delays of reinforcement were in effect. The response patterns controlled by FI schedules are more resilient to the nominally disruptive effects of delays of reinforcement than are corresponding overall response rates.     

Reward magnitude and timing in pigeons

We investigated the interaction of motivation and timing by manipulating the expected reward magnitude during a peak procedure. Four pigeons were tested with three different reward magnitudes, operationalized as duration of food access. Each stimulus predicted a different reward magnitude on a 5 s fixed-interval schedule. Trials with different reward magnitudes were randomly intermingled in a session. Most pigeons responded less often and started responding later on peak trials when a smaller reward was expected, but showed no differences in response termination or peak times. Reward magnitude was independently corroborated through unreinforced choice trials, when pigeons chose between the three stimuli presented simultaneously. These results contribute to a growing body of evidence that the expected reward magnitude influences the decision to start anticipatory responding in tasks where the reward becomes available after a fixed interval, but does not alter peak times, nor the decision to stop responding on peak trials.     

Choice and timing in concurrent chains: effects of initial-link duration

Theories of timing have been applied to choice between delayed rewards by assuming that delays are represented in memory and that subjects sample from memory when choosing between alternatives. To search for covariation in single-trial measures of performance that might confirm this assumption, we used a procedure that allowed for convergent measurement of choice and timing behavior. Four pigeons responded in a concurrent chains/peak procedure in which the terminal links were fixed-interval (FI) 8s and FI 16s, across conditions the duration of the initial-link schedule was either short or long, and one quarter of the terminal links lasted for 48 s and ended without reinforcer delivery. Preference for the FI 8-s alternative was stronger with shorter initial links, replicating the 'initial-link effect'. Responding on no-food trials was unaffected by initial-link duration, and aggregated across trials, was typical of the peak procedure: response distributions were approximately Gaussian, with modes near the FI schedule durations, and variance was greater for the FI 16-s terminal link. Analysis of local measures of initial-link performance (e.g., pause to begin responding, time spent responding, number and duration of visits to each alternative, etc.) found that the initial-link effect was associated with an increase in the number and duration of visits per cycle to the nonpreferred alternative. Regression analyses showed that local initial-link measures contributed relatively little additional variance in predicting performance on individual no-food trials beyond that accounted for by FI schedule. Our results provide no clear evidence that initial- and terminal-link responding in concurrent chains are mediated by a common representation of terminal-link delays.     

Psychological distance to reward in monkeys

A concurrent-chains schedule was employed to examine crab-eating macaques' choice between segmented and unsegmented fixed-interval 30-s schedules. A stimulus change occurred during one of the fixed-interval schedules (segmented schedule), while no stimulus change occurred during the other (unsegmented schedule). The subjects were exposed to three consecutive stimulus-change conditions, in an attempt to explore whether correlation of the stimulus changes with a fixed interval of nonreinforcement affects the preference between the alternatives. When the stimulus change always occurred at a half point of the segmented schedule (the first and third conditions), all subjects preferred the fixed-interval schedule without stimulus changes. This replicated previous findings with pigeons and humans. On the other hand, when the intervals before and after the stimulus change varied on every entry to the segmented schedule with the sum of them being constant (the second condition), the subjects were indifferent to the alternatives. These results are consistent with the view that the preference for unsegmented schedules over segmented ones might be due to aversiveness which the initial component in the segmented schedule possesses as a result of its correlation with a fixed interval of nonreinforcement.     

Tradeoffs among delay, rate, and amount of reinforcement

In Experiment 1, an adjusting-delay procedure was used to measure pigeons' choices between a single delayed reinforcer and a range of different variable-time schedules. Indifference points showed an inverse relation between rate of reinforcement and delay that was well described by a hyperbolic equation. An adjusting-amount procedure was used in Experiment 2, in which pigeons chose between an adjusting amount of food delivered after a 0.5-s delay and 3 s of food delivered after a range of different delays, and the effects of delay were similar to those found in Experiment 1. The results from both experiments indicated that, for pigeons, the strength of a reinforcer decreased rapidly with increasing delay. Estimates of a decay rate parameter in the hyperbolic equation were similar to those found in other studies with pigeons, but the rates of temporal discounting were three or four times faster than those found in studies with rats, suggesting a possible species difference.