The role of terminal-link stimuli in concurrent-chain schedules: revisited using a behavioral-history procedure

A behavioral-history procedure was used to study the function of terminal-link stimuli as conditioned reinforcers in multiple concurrent-chain schedules of reinforcement. First, three pigeons were exposed to multiple concurrent-chain schedules in which the two multiple-schedule components were correlated with a blue and a white stimulus, respectively. In each component the initial links were equal independent variable-interval (VI) 15 s schedules. A fixed-interval (FI) 10 s schedule operated on the red key in one terminal link while extinction operated on the green key in the alternative terminal link. When large preferences for the red stimulus had been established, two tests were conducted. In the terminal-link test, under new initial-link stimuli--purple and brown--an FI 10 s schedule operated for both the red and green terminal-link stimuli. In the subsequent initial-link test, the blue and white initial-link stimuli were reintroduced, and, as in the terminal-link test, FI 10s operated for both the red and the green terminal-link stimuli. In the terminal-link test, the three pigeons showed no preference for the terminal links with the red stimulus, but showed clear and consistent preferences for the red stimulus when blue and white stimuli were reintroduced as initial-link stimuli in the initial-link test. This suggests that there are multiple sources of control over initial-link response allocation in concurrent-chains, including control by both terminal- and initial-link stimuli.     

Early-session increases in responding during extinction

After training under a variable-interval 60-s schedule of reinforcement, four rats were exposed to 30-min extinction tests, which occurred either at the start or at the end of the session (each session being 50-min long). Response rate in extinction decreased when the extinction test occurred at the end of the session, but first increased and then decreased when the extinction test occurred at the start of the session. Consistent with other recent results, this finding suggests that some variable, other than reinforcement, contributes to early-session increases in responding.     

Behavioral contrast of a complex operant

Two experiments were conducted in which pigeons were trained to perform a complex operant consisting of a peck to one key followed by a peck to another key. In the first experment this performance was reinforced on a variable-interval schedule and the birds were then subjected to a multiple schedule in which the variable-interval was alternated with extinction. The first two pigeons trained showed some deterioration of the cohesiveness of the response sequence. After measures were taken to correct this the pigeon gave evidence of behavioral contrast in a condition where sessions with discrimination were alternated rapidly with sessions with no discrimination (variable-interval only). Another two birds were trained on variable-interval followed by discrimination. One of these birds showed evidence of contrast but the results of the other were not clear.Experiment II was a three phase experiment in which animals were first trained to make the complex response. In the second phase they were extinguished by providing noncontingent reinforcement in each of two components. In the third phase reinforcement was removed from one component, providing for a stimulus-reinforcer contingency. In the third phase the complex response reappeared temporarily.The results of these two studies indicate that a complex response may show behavioral contrast and that it may be enhanced by stimulus-reinforcer contingencies if it has already been trained.     

Resistance to extinction, generalization decrement, and conditioned reinforcement

This study investigated generalization decrement during an extinction resistance-to-change test for pigeon key pecking using a two-component multiple schedule with equal variable-interval 3-min schedules and different reinforcer amounts (one component presented 2-s access to reinforcement and the other 8s). After establishing baseline responding, subjects were assigned to one of the two extinction conditions: hopper stimuli (hopper and hopper light were activated but no food was available) or Control (inactive hopper and hopper light). Responding in the 8-s component was more resistant to extinction than responding in the 2-s component, the hopper stimuli group was more resistant to extinction compared to the Control group, and an interaction between amount of reinforcement, extinction condition, and session block was present. This finding supports generalization decrement as a factor that influences resistance to extinction. Hopper-time data (the amount of time subjects spent with their heads in the hopper) were compared to resistance-to-change data in an investigation of the role of conditioned reinforcement on resistance to change.     

Manipulating pre-feed, density of reinforcement, and extinction produces disruption in the Location variation of a temporal discrimination task in pigeons

When pharmacological and non-pharmacological agents are used to disrupt temporal discrimination, two major findings have emerged in the literature. One result reveals lateral shifts of the psychophysical curve for time due to disruptors, while the other is a decrease in accuracy for classifying short and long intervals and a flattening of the psychophysical curve. These results represent a discrepancy within the timing literature that requires clarification. The current study determined the effects of pre-feed, increased density of reinforcement during session, and extinction on the Location variation of a temporal discrimination procedure. The results showed that extinction and pre-feed (at higher levels), when presented in an acute fashion, led to right-ward shifts in the psychophysical curve. Our results, when compared to similar studies in the literature, suggest that lateral shifts are more likely to be found due to disruptors when the Location variation is being used and when procedures are less complicated.     

Effect of signaling reinforcement on resistance to change in a multiple schedule

This study evaluated the effect of a signal on resistance to change using a multiple schedule of reinforcement. Experiment 1 presented pigeons with three schedules: a signaled delay to reinforcement schedule (a two-link chain schedule with a variable-interval 120-s initial link followed by a 5-s fixed-time schedule), an unsignaled delay schedule (a comparable two-link tandem schedule), and an immediate, zero-delay variable-interval 125-s schedule. Two separate disruption procedures assessed resistance to change: extinction and adding a variable-time 20-s schedule of reinforcement to the inter-component interval. Resistance to change tests were conducted twice, once with the signal stimulus (the terminal link of the chain schedule) present and once with it absent. Results from both disruption procedures showed that signal absence reduced resistance to change for the pre-signal stimulus. In probe choice tests subjects strongly preferred the signal stimulus over the unsignaled stimulus and exhibited no reliable preference when given a choice between the signal stimulus and immediate stimulus. Experiment 2 presented two equal signaled schedules where, during resistance to change tests, the signal remained for one schedule and was removed for the second. Resistance to change was consistently lower when the signal was absent.     

Effects of unsignaled delays of reinforcement on fixed-interval schedule performance

Key pecking of pigeons was maintained by a fixed-interval (FI) 61-s schedule. The effects of resetting and nonresetting unsignaled delays of reinforcement then were examined. The resetting delay was programmed as a differential-reinforcement-of-other-behavior schedule, and the nonresetting delay as a fixed-time schedule. Three delay durations (0.5, 1 and 10 s) were examined. Overall response rates were decreased by one and 10-s delays and increased by 0.5-s delays. Response patterns changed from positively accelerated to more linear when resetting or nonresetting 10-s delays were imposed, but remained predominantly positively accelerated when resetting and nonresetting 0.5- and 1-s delays were in effect. In general, temporal control, as measured by quarter-life values, changed less than overall response rates when delays of reinforcement were in effect. The response patterns controlled by FI schedules are more resilient to the nominally disruptive effects of delays of reinforcement than are corresponding overall response rates.     

Superstitious responding and reinforcement rate under concurrent variable-interval extinction schedules

To examine superstitious responding, four pigeons key pecked under multiple concurrent variable-interval 45 s variable-interval 90 s concurrent variable-interval 90 s variable-interval 180 s schedules in the absence of a changeover delay. The two variable-interval 90 s schedules then were replaced by extinction, and key-peck responding during extinction was examined as a function of the prevailing reinforcement rate. During the first several sessions, extinction-key responding was maintained closer to baseline levels in the presence of the higher reinforcement rate, and this effect dissipated or even reversed with continued exposure to extinction. Although extinction-key responding generally decreased to near-zero levels after several sessions, in a few instances, it continued for 30 and 45 sessions. These results demonstrate how concurrent variable-interval extinction schedules can be used to investigate what often has been labeled superstitious responding.     

Reinforcement context and resistance to change

Eight pigeons responded in a multiple variable-interval (VI) schedule in which a constant component always delivered 40rft/h, and an alternated component was either rich (200rft/h) or lean (6.67rft/h) in different conditions. Four tests of resistance to change were conducted in each condition: prefeeding, full extinction, constant-component-only extinction, and response-independent food. Resistance to both prefeeding and full extinction in the constant component varied inversely with the reinforcement rate in the alternated component, but resistance to response-independent food did not. The extinction and response-independent food results were consistent with [J. Exp. Psychol.: Anim. Behav. Proc. 25 (1999) 256] behavioral momentum model. Maintaining reinforcement in the alternated component increased resistance to extinction in the constant component, as predicted by the behavioral momentum model but not accounts of multiple-schedule performance based on [J. Exp. Anal. Behav. 13 (1970) 243] equation. Overall, the momentum model gave a good account of the results with the exception of the prefeeding data. Possible ways to reconcile the prefeeding results with behavioral momentum theory are considered.     

Effects of prefeeding, intercomponent-interval food, and extinction on temporal discrimination and pacemaker rate

This experiment investigated the effects of nonpharmacological disruption on temporal discrimination. Pigeons responded on a multiple schedule composed of fixed interval, color-matching, and temporal-discrimination components. The effects of three different disruptors (prefeeding, intercomponent-interval food, and extinction) were assessed. All disruptors decreased response rates during the fixed interval. Prefeeding and intercomponent-interval food had unsystematic effects on response patterning during the fixed interval, whereas extinction increased the relative response rate in the initial portions of the fixed interval. Accuracy of color matching was decreased by prefeeding and was not systematically affected by intercomponent-interval food and extinction. In the temporal-discrimination component, all disruptors flattened the psychophysical functions relating proportion long responses to sample duration. This result indicates a general disruption of temporal discrimination. In addition, parameter estimates derived from the behavioral theory of timing indicated all disruptors decreased pacemaker rate, a result consistent with the predictions of the theory. These results highlight the similarities between disruption of temporal discrimination by pharmacological and nonpharmacological manipulations.     

Sensitivity to reinforcement in successive and delayed discriminations

Pigeons' responses were recorded in successive 15-s subintervals of 60-s components of several multiple variable-interval schedules of food reinforcement. In the standard multiple schedule or successive discrimination, discriminative stimuli were present throughout each component. In the delayed discrimination or memory procedure, red or green stimuli were present in the first 15 s of components and were followed by a white stimulus for the remainder of both components. Ratios of responses in the first 15 s of the two components, where discriminative stimuli were present, were sensitive to ratios of reinforcers obtained in the two components, to the same extent in both multiple and memory procedures. In both procedures, sensitivity to reinforcement decreased systematically over component subintervals, but to a greater extent in the memory procedure where discriminative stimuli were absent. The reduction in sensitivity with time since presentation of prior discriminative stimuli in the memory procedure was therefore influenced by two main factors: delayed stimulus control by the discriminative stimuli presented earlier in the component, and a decrease in sensitivity to reinforcement with increasing time since component alternation.     

Mathematical principles of reinforcement and resistance to change

Although Killeen's mathematical principles of reinforcement (MPR) apply to the asymptotic rate of a free operant after extended exposure to a single schedule of reinforcement, they can be extended to resistance to change in multiple schedules via alterations in the parameter representing the activating effects of reinforcers. MPR's predictions of resistance to change in relation to reinforcer rate on variable-interval (VI) schedules are empirically correct and agree with behavioral momentum theory (BMT). However, both MPR and BMT encounter problems in accounting for the effects of delayed reinforcement on resistance to change, relative to immediate reinforcement at the same rate. Further problems are raised by differences in resistance to change between variable-ratio (VR) and variable-interval performances maintained by the same reinforcer rate. With both delayed versus immediate reinforcement and variable-ratio versus variable-interval reinforcement, differential resistance to change is negatively correlated with the log ratios of baseline response rates when reinforcer rates are equated. Cases where resistance to change varies despite equated reinforcer rates, and the correlations among behavioral measures, provide challenges and opportunities for both MPR and BMT.     

Running response reinforcement by food-hoarding in the golden hamster

Partial reinforcement (PR) effects on animal locomotor behavior were studied in the golden hamster, using food-hoarding activity as a reinforcer. The first experiment demonstrated that hoarding reinforces a running response towards the goal section of a straight-alley runway, and that no such learning occurs when sated hamsters were not allowed to hoard food. However, a second experiment using various partial reinforcement schedules and a continuous reinforcement schedule did not give any evidence for the existence of a partial reinforcement acquisition effect (PRAE). The third experiment confirmed these results with an extended training procedure and showed a slight partial reinforcement extinction effect (PREE) mainly in the first sessions of the extinction phase.     

Response-reinforcer relations and resistance to change

Behavioral momentum theory suggests that the relation between a response and a reinforcer (i.e., response-reinforcer relation) governs response rates and the relation between a stimulus and a reinforcer (i.e., stimulus-reinforcer relation) governs resistance to change. The present experiments compared the effects degrading response-reinforcer relations with response-independent or delayed reinforcers on resistance to change in conditions with equal stimulus-reinforcer relations. In Experiment 1, pigeons responded on equal variable-interval schedules of immediate reinforcement in three components of a multiple schedule. Additional response-independent reinforcers were available in one component and additional delayed reinforcers were available in another component. The results showed that resistance to disruption was greater in the components with added reinforcers than without them (i.e., better stimulus-reinforcer relations), but did not differ for the components with added response-independent and delayed reinforcement. In Experiment 2, a component presenting immediate reinforcement alternated with either a component that arranged equal rates of reinforcement with a proportion of those reinforcers being response independent or a component with a proportion of the reinforcers being delayed. Results showed that resistance to disruption tended to be either similar across components or slightly lower when response-reinforcer relations were degraded with either response-independent or delayed reinforcers. These findings suggest that degrading response-reinforcer relations can impact resistance to change, but that impact does not depend on the specific method and is small relative to the effects of the stimulus-reinforcer relation.     

Shifts in the psychophysical function in rats

The primary goal was to compare results from a free-operant procedure with pigeons [Machado, A., Guilhardi, P., 2000. Shifts in the psychometric function and their implications for models of timing. J. Exp. Anal. Behav. 74, 25-54, Experiment 2] with new results obtained with rats. The secondary goal was to compare the results of both experiments with dependent variables that were not used in the original publication. As in the original study with pigeons, rats were trained on a two-alternative free-operant psychophysical procedure in which left lever press responses were reinforced during the first and second quarters of a 60-s trial, and right lever press responses were reinforced during the third and fourth quarters of the trial. The quarters were reinforced according to four independent variable interval (VI) schedules of reinforcement. The VI duration was manipulated in each quarter, and shifts in the psychophysical functions that relate response rate with time since trial onset were measured. The results obtained with rats were consistent with those previously obtained with pigeons. In addition, results not originally reported were also consistent between rats and pigeons, and provided insights into the perception, memory, and decision processes in Scalar Expectancy Theory and Learning-to-Time Theory.     

Signals, resistance to change, and conditioned reinforcement in a multiple schedule

The effect of signals on resistance to change was evaluated using pigeons responding on a three-component multiple schedule. Each component contained a variable-interval initial link followed by a fixed-time terminal link. One component was an unsignaled-delay schedule, and two were equivalent signaled-delay schedules. After baseline training, resistance to change was assessed through (a) extinction and (b) adding free food to the intercomponent interval. During these tests, the signal stimulus from one of the signaled-delay components (SIG-T) was replaced with the initial-link stimulus from that component, converting it to an unsignaled-delay schedule. That signal stimulus was added to the delay period of the unsignaled-delay component (UNS), converting it to a signaled-delay schedule. The remaining signaled component remained unchanged (SIG-C). Resistance-to-change tests showed removing the signal had a minimal effect on resistance to change in the SIG-T component compared to the unchanged SIG-C component except for one block during free-food testing. Adding the signal to the UNS component significantly increased response rates suggesting that component had low response strength. Interestingly, the direction of the effect was in the opposite direction from what is typically observed. Results are consistent with the conclusion that the signal functioned as a conditioned reinforcer and inconsistent with a generalization-decrement explanation.     

Rapid acquisition of preference in concurrent schedules: Effects of d-amphetamine on sensitivity to reinforcement amount

In the present study, effects of d-amphetamine on sensitivity to reinforcement amount under concurrent schedules were examined using a rapid-acquisition procedure. Four pigeons key pecked under single concurrent variable-interval 30-s schedules of grain presentation. Two different reinforcer-amount ratios (7:1 and 1:7) changed across sessions according to a 31-step pseudo-random binary sequence (PRBS). After at least four times through the PRBS, response ratios generally tracked the session-to-session changes in amount ratios; estimates of sensitivity ranged from 0.26 to 0.31 across the four pigeons. Effects of a range of doses of d-amphetamine (0.3-5.6 mg/kg) then were determined. For 3 of 4 pigeons, at least one dose, which did not dramatically alter overall response output or bias, decreased sensitivity to reinforcement amount. These results suggest that reducing sensitivity of responding to reinforcement amount may be one behavioral mechanism of stimulants, which may have implications for interpreting drug effects on self-control.     

Transitional choice behavior in concurrent-chain schedules

The choice responses of four pigeons were examined in 20 periods of transition in a concurrent-chain procedure with variable-interval schedules as initial links and fixed delays to reinforcement as terminal links. In some conditions, the delays to reinforcement were different for the two terminal links, and changes in preference were recorded after the delays for the two response keys were switched. In other conditions, the reinforcer delays were equal for the two keys, but which key delivered 80% of the reinforcers was periodically switched. Choice proportions changed more quickly after a switch in reinforcement percentages than after a switch in the delays, thereby contradicting the hypothesis that faster changes would occur when the switch in conditions was easier to discriminate. Analyses of response sequences showed that the effects of individual reinforcers were larger and lasted longer in conditions with changing reinforcement percentages than in conditions with changing terminal-link delays. Rates of change in choice behavior do not appear to be limited by the unpredictability of variable reinforcement schedules, because the changes in behavior were slow and gradual even when there was a large and sudden change in reinforcer delays.     

Effects of unpredictable changes in initial-link duration on choice and timing

Four pigeons responded in a concurrent-chains procedure in which terminal-link schedules were fixed-interval (FI) 10 s and FI 20 s. Across sessions, the location of the shorter terminal-link changed according to a pseudorandom binary sequence. Each session, the variable-interval initial-link schedule value was sampled from a uniform distribution that ranged from 0.01 to 30 s. On some terminal links, food was withheld to obtain measures of temporal control. Terminal-link delays determined choice (log initial-link response ratios) and timing (start and stop times on no-food trials) measures, which stabilized within the 1st half of each session. Preference for the shorter terminal-link delay was a monotonically decreasing function of initial-link duration. There was no evidence of control by initial-link durations from previous sessions.     

Extinction as discrimination: the molar view

The traditional molecular view of behavior explains extinction as the dissipation or inhibition of strength, formerly built up by contiguous reinforcement. In obstinate opposition to this explanation was the partial-reinforcement extinction effect: a partially reinforced response extinguishes more slowly than a continuously reinforced response. It suggests instead that extinction is discrimination. Four pigeons were exposed to daily sessions in which a variable period of food delivery, produced by pecking on a variable-interval schedule, was followed by extinction. The rate of food delivery was varied over a wide range across conditions. Varying the amount of food per delivery inversely with rate of delivery kept response rate from varying excessively. The results confirmed and extended the partial-reinforcement effect; persistence of pecking and time to extinction were inversely related to rate of obtaining food. The results support the molar view of extinction, not as loss of strength of a particular discrete response, but as a transition from one allocation of time among activities to another. Although molecular theories dismiss discrimination due to repeated training and extinction as an impurity or complication, repeated cycles of availability and privation are probably typical of the environment in which most vertebrate species evolved.