Models for the pressing rate response of rats and pigeons on a certain class of variable interval schedules of behavior

Two mathematical models are proposed for a certain class of variable interval schedules. These schedules are derived from the fixed interval schedule by adding (with probabilityp on any interval) reinforcing stimuli at a fixed point within the interval. The first model is a deterministic one in which the excitation producing the response is treated as a superposition of an excitation generated by the added reinforcements upon that generated by the fixed interval reinforcements. A method for estimating some of the parameters is presented. The model is applied with slight variations to data from performances by pigeons and by normal and brain-operated rats. The second model is a probabilistic one in which the basic entity is the interval between successive responses. This quantity must be represented by a probability distribution which is changing throughout the intervals between reinforcements. A solution is presented for a class of interresponse time distributions. Results from a simulation of a variant of this model are discussed briefly.     

EMG Biofeedback: The Effects of CRF, FR, VR, FI, and VI Schedules of Reinforcement on the Acquisition and Extinction of Increases in Forearm Muscle Tension

Biofeedback was used to increase forearm-muscle tension. Feedback was delivered under continuous reinforcement (CRF), variable interval (VI), fixed interval (FI), variable ratio (VR), and fixed ratio (FR) schedules of reinforcement when college students increased their muscle tension (electromyograph, EMG) above a high threshold. There were three daily sessions of feedback, and Session 3 was immediately followed by a session without feedback (extinction). The CRF schedule resulted in the highest EMG, closely followed by the FR and VR schedules, and the lowest EMG scores were produced by the FI and VI schedules. Similarly, the CRF schedule resulted in the greatest amount of time-above-threshold and the VI and FI schedules produced the lowest time-above-threshold. The highest response rates were generated by the FR schedule, followed by the VR schedule. The CRF schedule produced relatively low response rates, comparable to the rates under the VI and FI schedules. Some of the data are consistent with the partial-reinforcement-extinction effect. The present data suggest that different schedules of feedback should be considered in muscle-strengthening contexts such as during the rehabilitation of muscles following brain damage or peripheral nervous-system injury.     

A further study of choice and percentage reinforcement

In Experiment I four pigeons were trained in a concurrent chains procedure with fixed-ratio schedules (FR1) in the initial components and fixed-time schedules in the terminal components. Pecking one of the keys when both keys were white initiated a fixed time schedule on that key. A peck to the left key produced three stripes on the key. At the termination of the fixed-time component food always occurred. Pecking the other key produced either a circle or a triangle. If a circle appeared, reinforcement occurred. If a triangle appeared a brief timeout was given. Initially the stripes appeared on the left key and the circle and triangle on the left. This was reversed during the course of the experiment. In addition, sessions were conducted in which both circle and triangle sometimes preceded reinforcement and sometimes timeout. For most birds under most conditions there was a preference for the key that produced the circle and triangle. When these were uncorrelated with reinforcement and time out three of the birds preferred the key producing 100% reinforcement.

In Experiment II three factors were varied and VI 20 sec schedules were used in the initial links instead of FR1. The results showed that pigeons preferred the 50% condition more 1) the greater the duration of the terminal links, 2) the smaller the value on the initial link VI schedules and 3) the less the probability of food in the terminal link with stripes on the key.     

Dishabituation with component transitions may contribute to the interactions observed during multiple schedules

Rates of responding by rats were usually higher during the variable interval (VI) 30-s component of a multiple VI 30-s fixed interval (FI) 30-s schedule than during the same component of a multiple VI 30-s VI 30-s schedule (Experiment 1). Response rates were also usually higher during the FI 30-s component of a multiple VI 30-s FI 30-s schedule than during the same component of a multiple FI 30-s FI 30-s schedule (Experiment 2). The differences in response rates were not observed when the components provided VI or FI 120-s schedules. These results were predicted by the idea that differences in habituation to the reinforcer between multiple schedules contribute to behavioral interactions, such as behavioral contrast. However, differences in habituation were not apparent in the within-session patterns of responding. Finding differences in response rates in both experiments violates widely-held assumptions about behavioral interactions, including that behavioral contrast does not occur for rats and that improving the conditions of reinforcement decreases, rather than increases, response rate in the alternative component.     

Does the terminal-link effect depend on duration or reinforcement rate?

Preference in concurrent chains for the richer terminal-link schedule becomes more extreme as the schedule values increase with their ratio held constant, a result known as the terminal-link effect. We report two experiments that attempt to determine whether this effect is related to terminal-link duration or the overall rate of reinforcement. These variables have been confounded in prior studies, but can be separated by comparing variable-duration schedules that end after a single reinforcer has been earned, with constant-duration schedules during which a variable number of reinforcers may be earned. In Experiment 1, the terminal-link effect was obtained with variable-duration schedules when duration and overall reinforcement rate were manipulated, but not with constant-duration schedules when overall reinforcement rate was changed with duration held constant. In Experiment 2, the terminal-link effect was obtained with constant-duration schedules when duration was manipulated with overall reinforcement rate held constant. Taken together, these results show that the terminal-link effect depends on changes in terminal-link duration, not overall reinforcement rate (or equivalently, average time to reinforcement). This accords with the account of the terminal-link effect provided by the contextual choice model [J. Exp. Anal. Behav. 61 (1994) 113] but not delay-reduction theory [J. Exp. Anal. Behav. 12 (1969) 723].     

Constraining response output on conjunctive fixed-ratio 1 fixed-time reinforcement schedules: Effects on the postreinforcement pause

Two experiments used response-restriction procedures in order to test the independence of the factors determining response rate and the factors determining the size of the postreinforcement pause on interval schedules. Responding was restricted by response-produced blackout or by retracting the lever. In Experiment 1 with a Conjunctive FR 1 FT schedule, the blackout procedure reduced the postreinforcement pause more than the lever-retraction procedure did, and both procedures produced shorter pauses than did the schedule without response restriction. In Experiment 2 the interreinforcement interval was also manipulated, and the size of the pause was an increasing function of the interreinforcement interval, but the rate of increase was lower than that produced by fixed interval schedules of comparable interval durations. The assumption of functional independence of the postreinforcement pause and terminal rate in fixed interval schedules is questioned since data suggest that pause reductions resulted from constraining variation in response number compared to equivalent periodic schedules in which response number was allowed to vary.     

Psychological distance to reward in monkeys

A concurrent-chains schedule was employed to examine crab-eating macaques' choice between segmented and unsegmented fixed-interval 30-s schedules. A stimulus change occurred during one of the fixed-interval schedules (segmented schedule), while no stimulus change occurred during the other (unsegmented schedule). The subjects were exposed to three consecutive stimulus-change conditions, in an attempt to explore whether correlation of the stimulus changes with a fixed interval of nonreinforcement affects the preference between the alternatives. When the stimulus change always occurred at a half point of the segmented schedule (the first and third conditions), all subjects preferred the fixed-interval schedule without stimulus changes. This replicated previous findings with pigeons and humans. On the other hand, when the intervals before and after the stimulus change varied on every entry to the segmented schedule with the sum of them being constant (the second condition), the subjects were indifferent to the alternatives. These results are consistent with the view that the preference for unsegmented schedules over segmented ones might be due to aversiveness which the initial component in the segmented schedule possesses as a result of its correlation with a fixed interval of nonreinforcement.     

Psychological distance to reward: equating the number of stimulus and response segments

Psychological distance to reward, or the segmentation effect, refers to the preference for a terminal link of a concurrent-chains schedule consisting of a simple reinforcement schedule (e.g. fixed interval [FI] 30s) relative to its chained-schedule counterpart (e.g. chained FI 15s FI 15s). This experiment was conducted to examine whether the segmentation effect is due to the number of terminal-link stimulus and response segments per se. Three pigeons pecked under a concurrent-chains schedule in which identical variable-interval (VI) schedules operated in the initial links. In each session, half the terminal-link entries followed one initial-link key and the other half followed the other initial-link key. The initial-link keys correlated with the different terminal links were manipulated across conditions. In the first three conditions, each terminal link contained a chained fixed-time (FT) FT schedule, and in the final three conditions, each terminal link contained a chained FI FI schedule. In each condition, in one terminal link (alternating), the order of two key colors correlated with the different schedule segments alternated across terminal-link entries, whereas in the other terminal link (constant), the order of two other key colors was identical for each entry. With the chained FT FT schedule terminal links, there was indifference between the alternating and constant terminal links within and across pigeons, as indexed by initial-link choice proportions. In addition, terminal-link response rates were relatively low. With the chained FI FI schedule terminal links, for each pigeon, there was relatively more preference for the alternating terminal link and terminal-link response rates increased relative to conditions with the chained FT FT schedule terminal links. These data suggest that the segmentation effect is not due simply to the number of terminal-link stimulus or response segments per se, but rather to a required period of responding during a stimulus segment that never is paired with reinforcement.     

Differences in corticosterone level due to inter-food interval length: implications for schedule-induced polydipsia

The purpose of this study was to determine the effects of different food-reinforcement schedules on plasma corticosterone (CORT), and its possible involvement in the acquisition and maintenance of schedule-induced polydipsia (SIP). In Experiment 1, three groups of rats were submitted to two different fixed-interval (FI) schedules with inter-food intervals of 30 and 120 s, and to a massed-feeding presentation for 40 days until SIP was well stabilized. In Experiment 2, six groups of rats were exposed to the same schedules, FI 30s and FI 120s, and to the massed-feeding condition, but no water bottles were presented. CORT levels were determined on Days 3 and 40. Results of Experiment 1 indicated that FI 30s schedule, but not FI 120s or the massed-feeding condition, induces excessive drinking from Day 3. Results in Experiment 2 indicated that CORT levels were similar for all the groups on Day 3. However, only animals on the FI 30s schedule did increase their CORT levels on Day 40, with no variation in the hormone in the other two conditions, FI 120s and massed-feeding presentations. The data are discussed in terms of the implications of these results for hypotheses of SIP as anxiolitic behavior.     

Instrumental performance following saccharin pre-feeding

The effect of pre-feeding rats with a saccharin solution on instrumental performance, established with the same reinforcer, was studied in two experiments. Experiment 1 demonstrated that such pre-feeding depresses subsequent instrumental performance in an extinction test after training on a variable ratio schedule. In contrast, training on a variable interval schedule in Experiment 2 rendered instrumental performance impervious to pre-feeding. The motivational effects of pre-feeding with a non-nutritive substance appear to act through an incentive rather than drive process.     

Energy budget versus temporal discounting as determinants of preference in risky choice

Four pigeons and three ringneck doves responded on an operant simulation of natural foraging. After satisfying a schedule of reinforcement associated with search time, subjects could "accept" or "reject" another schedule of reinforcement associated with handling time. Two schedules of reinforcement were available, a variable interval, and a fixed interval with the same mean value. Food available in the session (a variable related to the energy budget) was manipulated in the different conditions either by increases of the value of the search state schedule of reinforcement, or by increases in the mean value of the handling state schedules. The results indicate that the amount of food available in the session did not affect the preference for variable schedules of reinforcement, as would be predicted by an influential theory of risk sensitive foraging. Instead, the preference for variability depended on the relationship between the time spent in the search and the handling states, as is predicted by a family of models of choice that are based on the temporal proximity to the reinforcer.     

Free-choice preference under uncertainty

Preferences in pigeons for free choice over forced choice under uncertain contingencies were compared with the one under certain contingencies in multiple concurrent-chain schedules of reinforcement. The uncertain condition examined the preference for two alternatives over one alternative when reinforcement probability at the end of the terminal link equalled 0.5, and with all keys in each terminal link lit green (two mixed fixed-interval extinction keys versus one mixed fixed-interval extinction key). Key and schedule arrangement in the certain condition was the same except that a peck on any terminal-link key after the FI interval always produced food. When naive pigeons were first exposed to uncertain contingencies, preference for two lit keys over one lit key was observed, and the preference was confirmed by sequential reversals of the terminal-link contingencies. However, no consistent preference was observed when uncertain condition followed the certain condition. Under certain contingencies, unlike the earlier experiments, very small and inconsistent preferences for free choice were demonstrated. A possible reason for the different preferences in the uncertain conditions was that pigeons may lessen their sensitivity to the circumstances with uncertainty by any history or carry-over effect of the prior contingency.     

Within-session changes in responding during concurrent variable interval variable ratio schedules

Rats (Experiment 1) and pigeons (Experiment 2) responded on several concurrent variable interval (VI) variable ratio (VR) schedules. The rate of, but not the time spent, responding in each component usually changed within-sessions. The bias and sensitivity parameters of the generalized matching law (GML) did not change systematically within-sessions. The fit of the GML to the data did not change within-sessions for pigeons, but it was better in the middle than at the beginning or end of the session for some for rats. Both over- and under-matching occurred. These results imply that within-session changes in responding do not usually cause problems for assessing the validity of the GML when subjects respond on concurrent VI VR schedules. The results also suggest that under- and over-matching are not produced by different factors, but rather lie on a continuum.     

Choice behavior in spontaneously hypertensive rats: Variable vs. fixed schedules of reinforcement

Spontaneously hypertensive rats (SHR) and Wistar rats were evaluated in the successive-encounters procedure (an operant simulation of natural foraging) with the idea of assessing differences between them in their preference for variable schedules of reinforcement. In this procedure, after satisfying a schedule of reinforcement associated with search time, the subjects could “accept” or “reject” another schedule of reinforcement associated with handling time. Two schedules of reinforcement were available: a fixed interval (FI), and a variable interval (VI) with the same mean value. The results indicated preference for the variable schedule in both strains, as suggested by the observation that the VI was always accepted while the FI was often rejected. The difference in FI acceptability between strains was not statistically significant, a result which is relevant for the current debate of SHR as an adequate animal model of Attention Deficit Hyperactivity Disorder.     

Variability of force and interresponse time under random interval reinforcement schedules

Response variability was examined under five random interval schedules with minimum mean interreinforcement intervals of 15, 30, 60, 120, and 240 sec. Distributions of response force-time integrals showed no systematic variability changes with increases in reinforcement intermittency. Corresponding distributions of interresponse times revealed increases in variability accompanying increases in reinforcement intermittency. The relationship of variability to these two measures, and the potential effect of behavior outside the experimentally measured response class, on recorded variability within the class, are considered.     

Effect of unsignaled delays between stimuli in a chain schedule on responding and resistance to change

Behavioral momentum theory is an evolving theoretical account of the strength of behavior. One challenge for the theory is determining the role of signal stimuli in determining response strength. This study evaluated the effect of an unsignaled delay between the initial link and terminal link of a two-link chain schedule on resistance to change using a multiple schedule of reinforcement. Pigeons were presented two different signaled delay to reinforcement schedules. Both schedules employed a two-link chain schedule with a variable interval 120-s initial link followed by a 5-s fixed time terminal link schedule. One of the schedules included a 5-s unsignaled delay between the initial link and the terminal link. Resistance to change was assessed with two separate disruption procedures: extinction and adding a variable time 20-s schedule of reinforcement to the inter-component interval. Baseline responding was lower in the schedule with the unsignaled delay but resistance to change for the initial link was unaffected by the unsignaled delay. The results suggest that not all unsignaled delays are equal in their effect on resistance to change.     

An Assessment of Fixed Interval Timing in Free-Flying Honey Bees (Apis mellifera ligustica): An Analysis of Individual Performance

Interval timing is a key element of foraging theory, models of predator avoidance, and competitive interactions. Although interval timing is well documented in vertebrate species, it is virtually unstudied in invertebrates. In the present experiment, we used free-flying honey bees (Apis mellifera ligustica) as a model for timing behaviors. Subjects were trained to enter a hole in an automated artificial flower to receive a nectar reinforcer (i.e. reward). Responses were continuously reinforced prior to exposure to either a fixed interval (FI) 15-sec, FI 30-sec, FI 60-sec, or FI 120-sec reinforcement schedule. We measured response rate and post-reinforcement pause within each fixed interval trial between reinforcers. Honey bees responded at higher frequencies earlier in the fixed interval suggesting subject responding did not come under traditional forms of temporal control. Response rates were lower during FI conditions compared to performance on continuous reinforcement schedules, and responding was more resistant to extinction when previously reinforced on FI schedules. However, no “scalloped” or “break-and-run” patterns of group or individual responses reinforced on FI schedules were observed; no traditional evidence of temporal control was found. Finally, longer FI schedules eventually caused all subjects to cease returning to the operant chamber indicating subjects did not tolerate the longer FI schedules.     

Fixed interval and fixed time treadle pressing in the pigeon: A comparison with FI and FT keypecking

Experiment I used non-naive pigeons having previously performed on both keypecking and treadlepressing Fixed Interval schedules. In condition IT, treadlepressing was reinforced on successive Fixed Interval 60 seconds, Fixed Time 60 seconds and Fixed Interval 60 seconds schedules. Subsequently (condition IK), the same subjects pecked a key on an identical schedule sequence (FI60, FT60, FI60). In Experiment II, separate groups of naïve subjects were assigned either to treadlepressing (condition IIT) or keypecking (condition IIK) and to the same schedule sequence (FI60, FT60, FI60). Treadle pressing and keypecking decreased greatly in Fixed Time schedules. Curvature indices, pauses and running rates were less sensitive than response rates to the switching from one schedule to the other. Experiments I and II yielded similar results, experimental history accounting only for minor differences. The results were discussed in relation to interspecies differences in the temporal regulation of behavior and operant versus respondent control of the response and schedule-induced behaviour.     

Effect of signaling reinforcement on resistance to change in a multiple schedule

This study evaluated the effect of a signal on resistance to change using a multiple schedule of reinforcement. Experiment 1 presented pigeons with three schedules: a signaled delay to reinforcement schedule (a two-link chain schedule with a variable-interval 120-s initial link followed by a 5-s fixed-time schedule), an unsignaled delay schedule (a comparable two-link tandem schedule), and an immediate, zero-delay variable-interval 125-s schedule. Two separate disruption procedures assessed resistance to change: extinction and adding a variable-time 20-s schedule of reinforcement to the inter-component interval. Resistance to change tests were conducted twice, once with the signal stimulus (the terminal link of the chain schedule) present and once with it absent. Results from both disruption procedures showed that signal absence reduced resistance to change for the pre-signal stimulus. In probe choice tests subjects strongly preferred the signal stimulus over the unsignaled stimulus and exhibited no reliable preference when given a choice between the signal stimulus and immediate stimulus. Experiment 2 presented two equal signaled schedules where, during resistance to change tests, the signal remained for one schedule and was removed for the second. Resistance to change was consistently lower when the signal was absent.     

Schedules of reinforcement as regulators of dispersion patterns

An eight-rat eight-station operant conditioning arena was used to study the spatial structure and temporal stability of foraging dispersion patterns. Food was obtained by bar pressing as the population was exposed to an ascending series of the fixed and variable aspects of ratio and interval schedules of reinforcement. Dispersion patterns, defined by the number of rats simultaneously foraging at each of the eight stations, and the temporal changes in these patterns, were the dependent variables. Both variables exhibited a unique relationship to each schedule type and value. The absence of such relationships when either food supply or response costs were examined suggests that these factors were not the determinants of spatio-temporal structure. An account is provided of how schedules may interact with behavioral foraging chains to explain dispersion patterns.