Response-reinforcer relations and resistance to change

Behavioral momentum theory suggests that the relation between a response and a reinforcer (i.e., response-reinforcer relation) governs response rates and the relation between a stimulus and a reinforcer (i.e., stimulus-reinforcer relation) governs resistance to change. The present experiments compared the effects degrading response-reinforcer relations with response-independent or delayed reinforcers on resistance to change in conditions with equal stimulus-reinforcer relations. In Experiment 1, pigeons responded on equal variable-interval schedules of immediate reinforcement in three components of a multiple schedule. Additional response-independent reinforcers were available in one component and additional delayed reinforcers were available in another component. The results showed that resistance to disruption was greater in the components with added reinforcers than without them (i.e., better stimulus-reinforcer relations), but did not differ for the components with added response-independent and delayed reinforcement. In Experiment 2, a component presenting immediate reinforcement alternated with either a component that arranged equal rates of reinforcement with a proportion of those reinforcers being response independent or a component with a proportion of the reinforcers being delayed. Results showed that resistance to disruption tended to be either similar across components or slightly lower when response-reinforcer relations were degraded with either response-independent or delayed reinforcers. These findings suggest that degrading response-reinforcer relations can impact resistance to change, but that impact does not depend on the specific method and is small relative to the effects of the stimulus-reinforcer relation.     

Segmentation and the pairing hypothesis

The effect of stimulus contiguity and response contingency on responding in chain schedules was examined in two experiments. In Experiment 1, four pigeons were trained on two simple three-link chain schedules that alternated within sessions. Initial links were correlated with a variable-interval 30s schedule, and middle and terminal links were correlated with interdependent variable-interval 30s variable-interval 30s schedules. The combined duration of the interdependent schedules summed to 60s. The two chains differed with respect to signaling of the schedule components: a two-stimulus chain had one stimulus paired with the initial link and one stimulus paired with both the middle and the terminal link, while a three-stimulus chain had a different stimulus paired with the each of the three links. The results showed that the two-stimulus chain maintained lower initial-link responding than the three-stimulus chain. In Experiment 2, four pigeons were exposed to three separate conditions, the two- and three-stimulus chains of Experiment 1 and a three-stimulus chain that had a 3s delay to terminal-link entry from the middle-link response that produced it. The two-stimulus chain maintained lower initial-link responding than the three-stimulus chain, as in Experiment 1, and a similar initial-link responding was maintained by the two-stimulus chain and the three-stimulus chain with the delay contingency. The results demonstrate that a stimulus noncontiguous with food can maintain responding that is sometimes greater than a stimulus contiguous with food, depending on the response contingency for terminal-link entry. The results are contrary to the pairing hypothesis of conditioned reinforcement.     

Resistance to change varies inversely with reinforcement context

We report two experiments which test whether resistance to prefeeding and satiation for a variable-interval (VI) schedule that delivers a constant rate of reinforcement varies inversely with the reinforcement rate for an alternative schedule. In Experiment 1, eight pigeons responded in a multiple schedule in which the red key was always associated with a VI 90-s schedule and the green key with either a richer (VI 18s) or leaner (VI 540s) schedule in different conditions. After baseline training in each condition, prefeeding test sessions were conducted in which 10g, 20g, 30g, 40g, and 50g food were provided one-hour prior to test. Additional baseline training was given between each test session. In Experiment 2, two groups of pigeons responded in a multiple schedule similar to Experiment 1. After baseline training, pigeons were exposed to a 5-h satiation test session in which the VI 90-s schedule was available continuously. Test sessions were conducted when pigeons were maintained at 85%, 95%, and 85% of their body weights in an ABA design. Results of both experiments showed that responding in the VI 90-s schedule that alternated with a leaner schedule during baseline was more resistant to prefeeding and satiation. These data rule out alternative explanations for results of previous studies, and confirm that resistance to change varies inversely with reinforcement context.     

Conditioning the pecking motions of pigeons

The spatio-temporal courses of head and neck motions of pigeons while pecking at small grains are described. Single and serial pecks are distinguished but the inter- and intraindividual variability of the peck kinetics is stressed. Pigeons were then trained with instrumental conditioning procedures to speed-up their pecking. A partial reinforcement schedule where pigeons had to peck repeatedly before receiving reward led to a mild shortening of inter-peck intervals at lower reinforcement rates but surprisingly, a lengthening at higher rates. A schedule where short inter-peck intervals were differentially rewarded yielded a pronounced abbreviation of the inter-peck intervals, but this was achieved by a reduction of the movement path rather than an increase in motion velocity. A schedule whereby increased approach velocities were differentially rewarded yielded marked movement accelerations. When pigeons were rewarded for diminished approach speeds they also showed significant movement decelerations. Finally, it is shown that pigeons could learn to reliably abort their peck approach movement when a visual stimulus signalling a penalty was occasionally presented during the approach movement. The proportion of successful peck interruptions decreased as these interruption signals occurred later during the approach phase. It is concluded that the pecking of pigeons is neither an innately fixed nor a visually ballistic movement. It is instead a multiply controlled and flexibly adaptable response pattern.     

Feature-short and feature-long discrimination learning in the pigeon: conditional control of a two-event temporal map

Eight pigeons were trained to peck an illuminated target key on discrete-trial fixed-interval schedules of reinforcement by food. Four birds were exposed to a feature-short (FS) task where a feature light signaled shortening of the forthcoming target-outcome interval from 30 to 15s, while the other four birds were exposed to a feature-long (FL) task where a feature light signaled extension of the forthcoming target-outcome interval from 15 to 30s. The discrimination performance measured by differential temporal distributions of pecks between featured and non-featured target trials suggested that the target-food temporal map was under conditional control of the feature light in both groups. The FS discrimination was more difficult to learn than the FL discrimination. This FS inferiority implies that our birds did not resort on the simple temporal discrimination by timing from the trial onset. The simple temporal discrimination account was also negated by the finding that increasing the feature-target gap did not have a predicted effect on the response distribution.     

Within-trial contrast: The effect of probability of reinforcement in training

There is evidence that pigeons prefer conditioned reinforcers that are preceded by greater effort over those that are preceded by less effort (an effect that has been attributed to within-trial contrast). In past research the probability of reinforcement for correct choice of the conditioned reinforcer has been 100%, however, the high level of reinforcement for both alternatives in training may result in a performance ceiling when choice between those alternatives is provided on test trials. In the present study we tested this hypothesis by including a group for which the probability of reinforcement in training was only 50%. Pigeons were trained on two simultaneous discriminations, one that was preceded by a 30 peck requirement the other by a single peck requirement. On test trials, we found a significant preference for the S+ that required the greater effort in training for pigeons trained with 100% and a small but nonsignificant effect for pigeons trained with 50% reinforcement. Although the hypothesis that the within-trial contrast effect was constrained by a performance ceiling was not confirmed, we did find a reliable within-trial contrast effect with 100% reinforcement.     

The effects of concurrent task and gap events on peak time in the peak procedure

The effect of a concurrent task on timing performance of pigeons was investigated with the peak interval procedure. Birds were trained to peck a side key on a discrete-trial schedule that included reinforced fixed-interval (FI) 30-s trials and nonreinforced extended probe trials. Then, in separate sessions, birds were trained to peck a 6-s center key for food. In a subsequent test phase, the FI procedure was in effect along with dual-task probe test trials. On those test trials, the 6-s center key (task cue) was presented at 3, 9, or 15s after probe trial onset. During another test phase, a 6-s gap (the FI keylight was extinguished) was presented at 3, 9, or 15s after probe trial onset. Peak time increased with center key time of onset, and was greater under task than gap conditions. Moreover, peak time under task conditions exceeded values predicted by stop and reset clock mechanisms. These results are at variance with current attentional accounts of timing behavior in dual-task conditions, and suggest a role of nontemporal factors in the control of timing behavior.     

Reinforcement context and resistance to change

Eight pigeons responded in a multiple variable-interval (VI) schedule in which a constant component always delivered 40rft/h, and an alternated component was either rich (200rft/h) or lean (6.67rft/h) in different conditions. Four tests of resistance to change were conducted in each condition: prefeeding, full extinction, constant-component-only extinction, and response-independent food. Resistance to both prefeeding and full extinction in the constant component varied inversely with the reinforcement rate in the alternated component, but resistance to response-independent food did not. The extinction and response-independent food results were consistent with [J. Exp. Psychol.: Anim. Behav. Proc. 25 (1999) 256] behavioral momentum model. Maintaining reinforcement in the alternated component increased resistance to extinction in the constant component, as predicted by the behavioral momentum model but not accounts of multiple-schedule performance based on [J. Exp. Anal. Behav. 13 (1970) 243] equation. Overall, the momentum model gave a good account of the results with the exception of the prefeeding data. Possible ways to reconcile the prefeeding results with behavioral momentum theory are considered.     

A further study of choice and percentage reinforcement

In Experiment I four pigeons were trained in a concurrent chains procedure with fixed-ratio schedules (FR1) in the initial components and fixed-time schedules in the terminal components. Pecking one of the keys when both keys were white initiated a fixed time schedule on that key. A peck to the left key produced three stripes on the key. At the termination of the fixed-time component food always occurred. Pecking the other key produced either a circle or a triangle. If a circle appeared, reinforcement occurred. If a triangle appeared a brief timeout was given. Initially the stripes appeared on the left key and the circle and triangle on the left. This was reversed during the course of the experiment. In addition, sessions were conducted in which both circle and triangle sometimes preceded reinforcement and sometimes timeout. For most birds under most conditions there was a preference for the key that produced the circle and triangle. When these were uncorrelated with reinforcement and time out three of the birds preferred the key producing 100% reinforcement.

In Experiment II three factors were varied and VI 20 sec schedules were used in the initial links instead of FR1. The results showed that pigeons preferred the 50% condition more 1) the greater the duration of the terminal links, 2) the smaller the value on the initial link VI schedules and 3) the less the probability of food in the terminal link with stripes on the key.     

Superstitious responding and reinforcement rate under concurrent variable-interval extinction schedules

To examine superstitious responding, four pigeons key pecked under multiple concurrent variable-interval 45 s variable-interval 90 s concurrent variable-interval 90 s variable-interval 180 s schedules in the absence of a changeover delay. The two variable-interval 90 s schedules then were replaced by extinction, and key-peck responding during extinction was examined as a function of the prevailing reinforcement rate. During the first several sessions, extinction-key responding was maintained closer to baseline levels in the presence of the higher reinforcement rate, and this effect dissipated or even reversed with continued exposure to extinction. Although extinction-key responding generally decreased to near-zero levels after several sessions, in a few instances, it continued for 30 and 45 sessions. These results demonstrate how concurrent variable-interval extinction schedules can be used to investigate what often has been labeled superstitious responding.     

Psychological distance to reward: equating the number of stimulus and response segments

Psychological distance to reward, or the segmentation effect, refers to the preference for a terminal link of a concurrent-chains schedule consisting of a simple reinforcement schedule (e.g. fixed interval [FI] 30s) relative to its chained-schedule counterpart (e.g. chained FI 15s FI 15s). This experiment was conducted to examine whether the segmentation effect is due to the number of terminal-link stimulus and response segments per se. Three pigeons pecked under a concurrent-chains schedule in which identical variable-interval (VI) schedules operated in the initial links. In each session, half the terminal-link entries followed one initial-link key and the other half followed the other initial-link key. The initial-link keys correlated with the different terminal links were manipulated across conditions. In the first three conditions, each terminal link contained a chained fixed-time (FT) FT schedule, and in the final three conditions, each terminal link contained a chained FI FI schedule. In each condition, in one terminal link (alternating), the order of two key colors correlated with the different schedule segments alternated across terminal-link entries, whereas in the other terminal link (constant), the order of two other key colors was identical for each entry. With the chained FT FT schedule terminal links, there was indifference between the alternating and constant terminal links within and across pigeons, as indexed by initial-link choice proportions. In addition, terminal-link response rates were relatively low. With the chained FI FI schedule terminal links, for each pigeon, there was relatively more preference for the alternating terminal link and terminal-link response rates increased relative to conditions with the chained FT FT schedule terminal links. These data suggest that the segmentation effect is not due simply to the number of terminal-link stimulus or response segments per se, but rather to a required period of responding during a stimulus segment that never is paired with reinforcement.     

Effects of free-food deliveries and delays on choice under concurrent-chains schedules

Pigeons responded on a concurrent-chains schedule with two equal variable-interval (VI) schedules as initial links and delays to food of 3 and 12s as the two terminal links. In even-numbered sessions, no other reinforcement schedule was present, and all pigeons showed a strong preference for the response key that had the shorter, 3-s terminal-link delay. In odd-numbered sessions, the initial links were interrupted at random times by one of three different types of events. When the interruptions were immediate food deliveries, the response percentages increased on the key that had the 3-s delay. When the interruptions were 30-s delays followed by food, the response percentages remained approximately unchanged. When the interruptions were 30-s delays with no food, the response percentages decreased. The results were used to compare the predictions of different mathematical models of concurrent-chains performance. The results favored models that assume that preference is determined by the relative amount of advantage that is gained when a terminal link is entered.     

Extinction as discrimination: the molar view

The traditional molecular view of behavior explains extinction as the dissipation or inhibition of strength, formerly built up by contiguous reinforcement. In obstinate opposition to this explanation was the partial-reinforcement extinction effect: a partially reinforced response extinguishes more slowly than a continuously reinforced response. It suggests instead that extinction is discrimination. Four pigeons were exposed to daily sessions in which a variable period of food delivery, produced by pecking on a variable-interval schedule, was followed by extinction. The rate of food delivery was varied over a wide range across conditions. Varying the amount of food per delivery inversely with rate of delivery kept response rate from varying excessively. The results confirmed and extended the partial-reinforcement effect; persistence of pecking and time to extinction were inversely related to rate of obtaining food. The results support the molar view of extinction, not as loss of strength of a particular discrete response, but as a transition from one allocation of time among activities to another. Although molecular theories dismiss discrimination due to repeated training and extinction as an impurity or complication, repeated cycles of availability and privation are probably typical of the environment in which most vertebrate species evolved.     

Rapid acquisition of preference in concurrent schedules: Effects of d-amphetamine on sensitivity to reinforcement amount

In the present study, effects of d-amphetamine on sensitivity to reinforcement amount under concurrent schedules were examined using a rapid-acquisition procedure. Four pigeons key pecked under single concurrent variable-interval 30-s schedules of grain presentation. Two different reinforcer-amount ratios (7:1 and 1:7) changed across sessions according to a 31-step pseudo-random binary sequence (PRBS). After at least four times through the PRBS, response ratios generally tracked the session-to-session changes in amount ratios; estimates of sensitivity ranged from 0.26 to 0.31 across the four pigeons. Effects of a range of doses of d-amphetamine (0.3-5.6 mg/kg) then were determined. For 3 of 4 pigeons, at least one dose, which did not dramatically alter overall response output or bias, decreased sensitivity to reinforcement amount. These results suggest that reducing sensitivity of responding to reinforcement amount may be one behavioral mechanism of stimulants, which may have implications for interpreting drug effects on self-control.     

Behavioral research in pigeons with ARENA: An Automated Remote Environmental Navigation Apparatus

Three experiments established the effectiveness of an Automated Remote Environmental Navigation Apparatus (ARENA) developed in our lab to study behavioral processes in pigeons. The technology utilizes one or more wireless modules, each capable of presenting colored lights as visual stimuli to signal reward and of detecting subject peck responses. In Experiment 1, subjects were instrumentally shaped to peck at a single ARENA module following an unsuccessful autoshaping procedure. In Experiment 2, pigeons were trained with a simultaneous discrimination procedure during which two modules were illuminated different colors; pecks to one color (S+) were reinforced while pecks to the other color (S−) were not. Pigeons learned to preferentially peck the module displaying the S+. In Experiment 3, two modules were lit the same color concurrently from a set of six colors in a conditional discrimination task. For three of the colors pecks to the module in one location (e.g., upper quadrant) were reinforced while for the remaining colors pecks at the other module (e.g., lower quadrant) were reinforced. After learning this discrimination, the color-reinforced location assignments were reversed. Pigeons successfully acquired the reversal. ARENA is an automated system for open-field studies and a more ecologically valid alternative to the touchscreen.     

Contextual determinants of temporal control: Behavioral contrast in a free-operant psychophysical procedure

The question of how temporal control of responding might be influenced by contingency changes in other contexts was investigated. Each of three pigeons first was exposed to a two-component multiple schedule in which a two-key free-operant psychophysical procedure operated in one component and a variable-interval schedule operated in the other component. The variable-interval schedule then was changed to extinction while the free-operant psychophysical procedure remained unchanged. Finally, the variable-interval schedule was reintroduced. Response rates on the left key and the estimated temporal threshold under the free-operant psychophysical procedure increased for each pigeon when the alternate component schedule was changed to extinction and then decreased again when the variable-interval schedule was reintroduced. The results suggest one way that temporal control is affected by its context, and may be interpreted through the direct effects of overall reinforcement rate on temporal control mechanisms or the disruptive effects of alternative sources of reinforcement on temporally controlled behavior.     

An application of the active time model to multiple concurrent variable-interval schedules

The current experiment investigates whether an active time model can account for anomalous results that have emerged from multiple schedule, concurrent variable-interval (VI) VI experiments. The model assumes that (1) during concurrent VI VI training pigeons learn a function that relates time since the most immediate response, i.e., active time, to changeover probabilities and (2) that molar preference is the result of an interaction between inter-response time frequencies and the learned active time changeover functions. Pigeons were trained under a concurrent VI 30-s VI 30-s schedule and a concurrent VI 60-s VI 60-s schedule. Probes were conducted in which VI 30-s and VI 60-s stimuli were paired. During these probes, birds allocated choices equally to the stimuli. The active time model accurately fit individual subject data. In contrast data were not fit by a variant of scalar expectancy theory proposed by Gibbon [Gibbon, J., 1995. Dynamics of time matching: arousal makes better seem worse. Psychon. Bull. Rev. 2, 208-215].     

Massed and spaced stimulus sequences in dimensional discrimination

In two experiments, pigeons' responding to a visual flicker-rate continuum was established by a maintained generalization procedure. For both experiments, variable-interval reinforcement was available for responses during stimuli from one half of the stimulus continuum while responses during other stimuli were extinguished. The first experiment compared gradients of dimensional stimulus control produced by randomly presented positive and negative stimuli with gradients produced by massed positive and negative stimuli. The second experiment alternated the order of massed stimulus sequences. In both cases, massing of stimulus sequences diminished positive dimensional contrast effects, without seriously disturbing discrimination between positive and negative stimuli. The results indicate that stimulus sequences can have an important role in the production of dimensional contrast effects.     

An instance of viewpoint consistency in pigeon object recognition

Pigeons were trained on a visual discrimination, a task using a TV monitor. Two different types of stimuli appeared as pictures on the TV screen, one was a feeder used in their home cages and the other a coffee mug. One group of the pigeons was trained to peck the screen when the feeder appeared on it, while the other group was trained to peck the screen when the mug appeared. The feeder was considered to be a `familiar object' but the mug an `unfamiliar object' for the subjects. After training, to peck the familiar object, the subjects showed generalization to unusual view pictures of the object, but they did not show such generalization after training to peck the unfamiliar object. These results suggest that view point consistency is limited to familiar objects in pigeons.