Effects of unsignaled delays of reinforcement on fixed-interval schedule performance

Key pecking of pigeons was maintained by a fixed-interval (FI) 61-s schedule. The effects of resetting and nonresetting unsignaled delays of reinforcement then were examined. The resetting delay was programmed as a differential-reinforcement-of-other-behavior schedule, and the nonresetting delay as a fixed-time schedule. Three delay durations (0.5, 1 and 10 s) were examined. Overall response rates were decreased by one and 10-s delays and increased by 0.5-s delays. Response patterns changed from positively accelerated to more linear when resetting or nonresetting 10-s delays were imposed, but remained predominantly positively accelerated when resetting and nonresetting 0.5- and 1-s delays were in effect. In general, temporal control, as measured by quarter-life values, changed less than overall response rates when delays of reinforcement were in effect. The response patterns controlled by FI schedules are more resilient to the nominally disruptive effects of delays of reinforcement than are corresponding overall response rates.     

When to respond? And how much? : Temporal control and response output on mixed-fixed-interval schedules with unequally probable components

Rats were trained on mixed-fixed-interval (FI) schedules, with component FIs of 30 and 60s. The probability of reinforcement according to FI 30s varied between conditions, across values of 0.1, 0.3, 0.5, 0.7 and 0.9. When response rate in the 60s intervals was measured, separate response peaks, one close to 30s, the other at 60s, could be identified when the probability of reinforcement at 30s was 0.3 or greater. Nonlinear regression found that the location of the earlier peak was always close to 30s, that the coefficient of variation of the response functions at 30 and 60s were unaffected by reinforcement probability, but that the 30s component appeared to be timed slightly more precisely than the 60s one. Response rate at around 30s increased with increasing probability of reinforcement according to FI 30s, but responding at 60s was unaffected by reinforcement probability. The data are discussed with respect to a number of contemporary models of animal timing (scalar expectancy theory, the Behavioural Theory of Timing and the Learning to Time model), and a recent account of response output on FI-like schedules.     

Dishabituation with component transitions may contribute to the interactions observed during multiple schedules

Rates of responding by rats were usually higher during the variable interval (VI) 30-s component of a multiple VI 30-s fixed interval (FI) 30-s schedule than during the same component of a multiple VI 30-s VI 30-s schedule (Experiment 1). Response rates were also usually higher during the FI 30-s component of a multiple VI 30-s FI 30-s schedule than during the same component of a multiple FI 30-s FI 30-s schedule (Experiment 2). The differences in response rates were not observed when the components provided VI or FI 120-s schedules. These results were predicted by the idea that differences in habituation to the reinforcer between multiple schedules contribute to behavioral interactions, such as behavioral contrast. However, differences in habituation were not apparent in the within-session patterns of responding. Finding differences in response rates in both experiments violates widely-held assumptions about behavioral interactions, including that behavioral contrast does not occur for rats and that improving the conditions of reinforcement decreases, rather than increases, response rate in the alternative component.     

Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement

Across two experiments, a peak procedure was used to assess the timing of the onset and offset of an opportunity to run as a reinforcer. The first experiment investigated the effect of reinforcer duration on temporal discrimination of the onset of the reinforcement interval. Three male Wistar rats were exposed to fixed-interval (FI) 30-s schedules of wheel-running reinforcement and the duration of the opportunity to run was varied across values of 15, 30, and 60s. Each session consisted of 50 reinforcers and 10 probe trials. Results showed that as reinforcer duration increased, the percentage of postreinforcement pauses longer than the 30-s schedule interval increased. On probe trials, peak response rates occurred near the time of reinforcer delivery and peak times varied with reinforcer duration. In a second experiment, seven female Long-Evans rats were exposed to FI 30-s schedules leading to 30-s opportunities to run. Timing of the onset and offset of the reinforcement period was assessed by probe trials during the schedule interval and during the reinforcement interval in separate conditions. The results provided evidence of timing of the onset, but not the offset of the wheel-running reinforcement period. Further research is required to assess if timing occurs during a wheel-running reinforcement period.     

An Assessment of Fixed Interval Timing in Free-Flying Honey Bees (Apis mellifera ligustica): An Analysis of Individual Performance

Interval timing is a key element of foraging theory, models of predator avoidance, and competitive interactions. Although interval timing is well documented in vertebrate species, it is virtually unstudied in invertebrates. In the present experiment, we used free-flying honey bees (Apis mellifera ligustica) as a model for timing behaviors. Subjects were trained to enter a hole in an automated artificial flower to receive a nectar reinforcer (i.e. reward). Responses were continuously reinforced prior to exposure to either a fixed interval (FI) 15-sec, FI 30-sec, FI 60-sec, or FI 120-sec reinforcement schedule. We measured response rate and post-reinforcement pause within each fixed interval trial between reinforcers. Honey bees responded at higher frequencies earlier in the fixed interval suggesting subject responding did not come under traditional forms of temporal control. Response rates were lower during FI conditions compared to performance on continuous reinforcement schedules, and responding was more resistant to extinction when previously reinforced on FI schedules. However, no “scalloped” or “break-and-run” patterns of group or individual responses reinforced on FI schedules were observed; no traditional evidence of temporal control was found. Finally, longer FI schedules eventually caused all subjects to cease returning to the operant chamber indicating subjects did not tolerate the longer FI schedules.     

EMG Biofeedback: The Effects of CRF, FR, VR, FI, and VI Schedules of Reinforcement on the Acquisition and Extinction of Increases in Forearm Muscle Tension

Biofeedback was used to increase forearm-muscle tension. Feedback was delivered under continuous reinforcement (CRF), variable interval (VI), fixed interval (FI), variable ratio (VR), and fixed ratio (FR) schedules of reinforcement when college students increased their muscle tension (electromyograph, EMG) above a high threshold. There were three daily sessions of feedback, and Session 3 was immediately followed by a session without feedback (extinction). The CRF schedule resulted in the highest EMG, closely followed by the FR and VR schedules, and the lowest EMG scores were produced by the FI and VI schedules. Similarly, the CRF schedule resulted in the greatest amount of time-above-threshold and the VI and FI schedules produced the lowest time-above-threshold. The highest response rates were generated by the FR schedule, followed by the VR schedule. The CRF schedule produced relatively low response rates, comparable to the rates under the VI and FI schedules. Some of the data are consistent with the partial-reinforcement-extinction effect. The present data suggest that different schedules of feedback should be considered in muscle-strengthening contexts such as during the rehabilitation of muscles following brain damage or peripheral nervous-system injury.     

Fixed interval and fixed time treadle pressing in the pigeon: A comparison with FI and FT keypecking

Experiment I used non-naive pigeons having previously performed on both keypecking and treadlepressing Fixed Interval schedules. In condition IT, treadlepressing was reinforced on successive Fixed Interval 60 seconds, Fixed Time 60 seconds and Fixed Interval 60 seconds schedules. Subsequently (condition IK), the same subjects pecked a key on an identical schedule sequence (FI60, FT60, FI60). In Experiment II, separate groups of naïve subjects were assigned either to treadlepressing (condition IIT) or keypecking (condition IIK) and to the same schedule sequence (FI60, FT60, FI60). Treadle pressing and keypecking decreased greatly in Fixed Time schedules. Curvature indices, pauses and running rates were less sensitive than response rates to the switching from one schedule to the other. Experiments I and II yielded similar results, experimental history accounting only for minor differences. The results were discussed in relation to interspecies differences in the temporal regulation of behavior and operant versus respondent control of the response and schedule-induced behaviour.     

Oscillations following periodic reinforcement

Three experiments examined behavior in extinction following periodic reinforcement. During the first phase of Experiment 1, four groups of pigeons were exposed to fixed interval (FI 16 s or FI 48 s) or variable interval (VI 16 s or VI 48 s) reinforcement schedules. Next, during the second phase, each session started with reinforcement trials and ended with an extinction segment. Experiment 2 was similar except that the extinction segment was considerably longer. Experiment 3 replaced the FI schedules with a peak procedure, with FI trials interspersed with non-food peak interval (PI) trials that were four times longer. One group of pigeons was exposed to FI 20 s PI 80 s trials, and another to FI 40 s PI 160 s trials. Results showed that, during the extinction segment, most pigeons trained with FI schedules, but not with VI schedules, displayed pause-peck oscillations with a period close to, but slightly greater than the FI parameter. These oscillations did not start immediately after the onset of extinction. Comparing the oscillations from Experiments 1 and 2 suggested that the alternation of reconditioning and re-extinction increases the reliability and earlier onset of the oscillations. In Experiment 3 the pigeons exhibited well-defined pause-peck cycles since the onset of extinction. These cycles had periods close to twice the value of the FI and lasted for long intervals of time. We discuss some hypotheses concerning the processes underlying behavioral oscillations following periodic reinforcement.     

Influence of the reinforcement magnitude on omission effects

Reinforcement Omission Effects (ROEs), indicated by higher rate of responses after nonreinforced trials in a partial reinforcement schedule, have been interpreted as behavioral transient facilitation after nonreinforcement induced by primary frustration, and/or behavioral transient inhibition after reinforcement induced by demotivation or temporal control. The size of the ROEs should depend directly on the reinforcement magnitude. The present experiment aimed to clarify the relationship between reinforcement magnitude and the omission effects manipulating the magnitude linked to discriminative stimuli in a partial reinforcement FI schedule. The results showed that response rates were higher after omission than after reinforcement delivery. Besides, response rates were highest immediately after the reinforcement omission of a larger magnitude than of a smaller magnitude. These data are interpreted in terms of ROEs multiple process behavioral facilitation after nonreinforcement, and behavioral transient inhibition after reinforcement.     

Interval timing in Siamese fighting fish (Betta splendens)

The present study evaluated the temporal performance of Siamese fighting fish (Betta splendens) given short-term exposure to four fixed interval (FI) schedules of reinforcement, FI 30, 60, 120, and 240 s, during which a reinforcer (mirror image) was given for the first response (swimming through a hoop) after the interval requirement had elapsed. Response levels were generally low early in an interval and increased as the interval elapsed; wait times and break points in an interval increased with increases in the FI requirement. The results were similar to that obtained with other species and different types of responses and reinforcers, and demonstrate that the procedure is a feasible method for studying interval timing in fish.     

Psychological distance to reward: equating the number of stimulus and response segments

Psychological distance to reward, or the segmentation effect, refers to the preference for a terminal link of a concurrent-chains schedule consisting of a simple reinforcement schedule (e.g. fixed interval [FI] 30s) relative to its chained-schedule counterpart (e.g. chained FI 15s FI 15s). This experiment was conducted to examine whether the segmentation effect is due to the number of terminal-link stimulus and response segments per se. Three pigeons pecked under a concurrent-chains schedule in which identical variable-interval (VI) schedules operated in the initial links. In each session, half the terminal-link entries followed one initial-link key and the other half followed the other initial-link key. The initial-link keys correlated with the different terminal links were manipulated across conditions. In the first three conditions, each terminal link contained a chained fixed-time (FT) FT schedule, and in the final three conditions, each terminal link contained a chained FI FI schedule. In each condition, in one terminal link (alternating), the order of two key colors correlated with the different schedule segments alternated across terminal-link entries, whereas in the other terminal link (constant), the order of two other key colors was identical for each entry. With the chained FT FT schedule terminal links, there was indifference between the alternating and constant terminal links within and across pigeons, as indexed by initial-link choice proportions. In addition, terminal-link response rates were relatively low. With the chained FI FI schedule terminal links, for each pigeon, there was relatively more preference for the alternating terminal link and terminal-link response rates increased relative to conditions with the chained FT FT schedule terminal links. These data suggest that the segmentation effect is not due simply to the number of terminal-link stimulus or response segments per se, but rather to a required period of responding during a stimulus segment that never is paired with reinforcement.     

Transference effects of prior non-contingent reinforcement on the acquisition of temporal control on fixed-interval schedules

In two experiments we examined the influence of response and time factors on the speed of acquisition of temporal control on FI schedules. In Experiment 1, prior exposure to FT accelerated the development of temporal control on FI schedules of the same temporal value. It was also found that the slower acquisition on FI with prior RT was similar to that of rats with prior standard training. In Experiment 2, prior exposure to FT accelerated the development of temporal control on a FI schedule with a threefold increase in temporal value. Additionally, it was found that with prior FI 30s training, acquisition of temporal control on FI 90s was even faster than with prior FT 30s. Measures of head-entries into the feeder along the experiments indicated that temporal control was already developed during the periodic but not during the non-periodic histories and that this control transferred to lever press during FI testing phase.     

Mathematical principles of reinforcement and resistance to change

Although Killeen's mathematical principles of reinforcement (MPR) apply to the asymptotic rate of a free operant after extended exposure to a single schedule of reinforcement, they can be extended to resistance to change in multiple schedules via alterations in the parameter representing the activating effects of reinforcers. MPR's predictions of resistance to change in relation to reinforcer rate on variable-interval (VI) schedules are empirically correct and agree with behavioral momentum theory (BMT). However, both MPR and BMT encounter problems in accounting for the effects of delayed reinforcement on resistance to change, relative to immediate reinforcement at the same rate. Further problems are raised by differences in resistance to change between variable-ratio (VR) and variable-interval performances maintained by the same reinforcer rate. With both delayed versus immediate reinforcement and variable-ratio versus variable-interval reinforcement, differential resistance to change is negatively correlated with the log ratios of baseline response rates when reinforcer rates are equated. Cases where resistance to change varies despite equated reinforcer rates, and the correlations among behavioral measures, provide challenges and opportunities for both MPR and BMT.     

Effects of unpredictable changes in initial-link duration on choice and timing

Four pigeons responded in a concurrent-chains procedure in which terminal-link schedules were fixed-interval (FI) 10 s and FI 20 s. Across sessions, the location of the shorter terminal-link changed according to a pseudorandom binary sequence. Each session, the variable-interval initial-link schedule value was sampled from a uniform distribution that ranged from 0.01 to 30 s. On some terminal links, food was withheld to obtain measures of temporal control. Terminal-link delays determined choice (log initial-link response ratios) and timing (start and stop times on no-food trials) measures, which stabilized within the 1st half of each session. Preference for the shorter terminal-link delay was a monotonically decreasing function of initial-link duration. There was no evidence of control by initial-link durations from previous sessions.     

Body weight manipulation,reinforcement value and choice between sucrose and wheel running: A behavioral economic analysis

Twelve female Long-Evans rats were exposed to concurrent variable (VR) ratio schedules of sucrose and wheel-running reinforcement (Sucrose VR 10 Wheel VR 10; Sucrose VR 5 Wheel VR 20; Sucrose VR 20 Wheel VR 5) with predetermined budgets (number of responses). The allocation of lever pressing to the sucrose and wheel-running alternatives was assessed at high and low body weights. Results showed that wheel-running rate and lever-pressing rates for sucrose and wheel running increased, but the choice of wheel running decreased at the low body weight. A regression analysis of relative consumption as a function of relative price showed that consumption shifted toward sucrose and interacted with price differences in a manner consistent with increased substitutability. Demand curves showed that demand for sucrose became less elastic while demand for wheel running became more elastic at the low body weight. These findings reflect an increase in the difference in relative value of sucrose and wheel running as body weight decreased. Discussion focuses on the limitations of response rates as measures of reinforcement value. In addition, we address the commonalities between matching and demand curve equations for the analysis of changes in relative reinforcement value.     

Temporal generalization accounts for response resurgence in the peak procedure

The peak interval (PI) procedure is commonly used to evaluate animals' ability to produce timed intervals. It consists of presenting fixed interval (FI) schedules in which some of the trials are replaced by extended non-reinforced trials. Responding will often resume (resurge) at the end of the non-reinforced trials unless precautions are taken to prevent it. Response resurgence was replicated in rats and pigeons. Variation of the durations of the FI and the non-reinforced probe trials showed it to be dependent on the time when reinforcement is expected. Timing of both the normal time to reinforcement, and the subsequent time to reinforcement during the probe trials followed Weber's law. A quantitative model of resurgence is described, suggesting how animals respond to the signaling properties of reinforcement omission. Model results were simulated using a stochastic binary counter.     

Transformation of a waiting schedule into a temporal regulation schedule (DRRD) by addition of external stimuli in the dog

Waiting schedules do not impose temporal regulation but condition the animal to give the operant response during a given time. At the end of the required delay, a positive discriminative stimulus is presented. The suspension of the response while the discriminative stimulus is being given suspension of the response while the discriminative stimulus is being given is accompanied by reinforcement. The transformation of a waiting schedule into a temporal regulation schedule is generally achieved by suppressing the external facilitating factors or by physically modifying them. Our study reveals that a similar transformation can be achieved in the dog by the addition of a further stimulus. This stimulus, which is physically exactly the same as the excitatory stimulus and which punctuates the waiting period, is randomly introduced into the temporal delay. The absence of reinforcement in response to the added stimulus should force the animal to regulate its behavior in time and the additional negative discriminative stimulus favours the expression of the active nature of the inhibation. The results show that subjects can differentiate their response durations according to stimuli that only differ according to temporal location. Thus this pattern resembles a DRRD schedule. The peak of responses at the time of the inhibition stimulus reveals considerable behavioral conflict : either the response must be maintained or the inhibition suppressed. The positive or negative resolution of this conflict reveals noteworthy aspects of the behavioural inhibition process.     

Response-reinforcer relations and resistance to change

Behavioral momentum theory suggests that the relation between a response and a reinforcer (i.e., response-reinforcer relation) governs response rates and the relation between a stimulus and a reinforcer (i.e., stimulus-reinforcer relation) governs resistance to change. The present experiments compared the effects degrading response-reinforcer relations with response-independent or delayed reinforcers on resistance to change in conditions with equal stimulus-reinforcer relations. In Experiment 1, pigeons responded on equal variable-interval schedules of immediate reinforcement in three components of a multiple schedule. Additional response-independent reinforcers were available in one component and additional delayed reinforcers were available in another component. The results showed that resistance to disruption was greater in the components with added reinforcers than without them (i.e., better stimulus-reinforcer relations), but did not differ for the components with added response-independent and delayed reinforcement. In Experiment 2, a component presenting immediate reinforcement alternated with either a component that arranged equal rates of reinforcement with a proportion of those reinforcers being response independent or a component with a proportion of the reinforcers being delayed. Results showed that resistance to disruption tended to be either similar across components or slightly lower when response-reinforcer relations were degraded with either response-independent or delayed reinforcers. These findings suggest that degrading response-reinforcer relations can impact resistance to change, but that impact does not depend on the specific method and is small relative to the effects of the stimulus-reinforcer relation.     

Varying reinforcer duration produces behavioral interactions during multiple schedules

The experiments tested the idea that changes in habituation to the reinforcer contribute to behavioral interactions during multiple schedules. This idea predicts that changing an aspect of the reinforcer should disrupt habituation and produce an interaction. Pigeons and rats responded on multiple variable interval variable interval schedules. Introducing variability into the duration of reinforcers in one component increased response rates in both components when the schedules provided high, but not low, rates of reinforcement. The increases in constant-component response rates grew larger as the session progressed. Within-session decreases in responding were smaller when the other component provided variable-, rather than fixed-, duration reinforcers. These results are consistent with the idea that changes in habituation to the reinforcer contribute to behavioral interactions. They help to explain why interactions do not occur for some subjects under conditions that produce them for others. Finally, the results question the assumption that induction and behavioral contrast are always produced by different theoretical mechanisms.     

Tradeoffs among delay, rate, and amount of reinforcement

In Experiment 1, an adjusting-delay procedure was used to measure pigeons' choices between a single delayed reinforcer and a range of different variable-time schedules. Indifference points showed an inverse relation between rate of reinforcement and delay that was well described by a hyperbolic equation. An adjusting-amount procedure was used in Experiment 2, in which pigeons chose between an adjusting amount of food delivered after a 0.5-s delay and 3 s of food delivered after a range of different delays, and the effects of delay were similar to those found in Experiment 1. The results from both experiments indicated that, for pigeons, the strength of a reinforcer decreased rapidly with increasing delay. Estimates of a decay rate parameter in the hyperbolic equation were similar to those found in other studies with pigeons, but the rates of temporal discounting were three or four times faster than those found in studies with rats, suggesting a possible species difference.