Effects of free-food deliveries and delays on choice under concurrent-chains schedules

Pigeons responded on a concurrent-chains schedule with two equal variable-interval (VI) schedules as initial links and delays to food of 3 and 12s as the two terminal links. In even-numbered sessions, no other reinforcement schedule was present, and all pigeons showed a strong preference for the response key that had the shorter, 3-s terminal-link delay. In odd-numbered sessions, the initial links were interrupted at random times by one of three different types of events. When the interruptions were immediate food deliveries, the response percentages increased on the key that had the 3-s delay. When the interruptions were 30-s delays followed by food, the response percentages remained approximately unchanged. When the interruptions were 30-s delays with no food, the response percentages decreased. The results were used to compare the predictions of different mathematical models of concurrent-chains performance. The results favored models that assume that preference is determined by the relative amount of advantage that is gained when a terminal link is entered.     

Segmentation and the pairing hypothesis

The effect of stimulus contiguity and response contingency on responding in chain schedules was examined in two experiments. In Experiment 1, four pigeons were trained on two simple three-link chain schedules that alternated within sessions. Initial links were correlated with a variable-interval 30s schedule, and middle and terminal links were correlated with interdependent variable-interval 30s variable-interval 30s schedules. The combined duration of the interdependent schedules summed to 60s. The two chains differed with respect to signaling of the schedule components: a two-stimulus chain had one stimulus paired with the initial link and one stimulus paired with both the middle and the terminal link, while a three-stimulus chain had a different stimulus paired with the each of the three links. The results showed that the two-stimulus chain maintained lower initial-link responding than the three-stimulus chain. In Experiment 2, four pigeons were exposed to three separate conditions, the two- and three-stimulus chains of Experiment 1 and a three-stimulus chain that had a 3s delay to terminal-link entry from the middle-link response that produced it. The two-stimulus chain maintained lower initial-link responding than the three-stimulus chain, as in Experiment 1, and a similar initial-link responding was maintained by the two-stimulus chain and the three-stimulus chain with the delay contingency. The results demonstrate that a stimulus noncontiguous with food can maintain responding that is sometimes greater than a stimulus contiguous with food, depending on the response contingency for terminal-link entry. The results are contrary to the pairing hypothesis of conditioned reinforcement.     

Effects of unpredictable changes in initial-link duration on choice and timing

Four pigeons responded in a concurrent-chains procedure in which terminal-link schedules were fixed-interval (FI) 10 s and FI 20 s. Across sessions, the location of the shorter terminal-link changed according to a pseudorandom binary sequence. Each session, the variable-interval initial-link schedule value was sampled from a uniform distribution that ranged from 0.01 to 30 s. On some terminal links, food was withheld to obtain measures of temporal control. Terminal-link delays determined choice (log initial-link response ratios) and timing (start and stop times on no-food trials) measures, which stabilized within the 1st half of each session. Preference for the shorter terminal-link delay was a monotonically decreasing function of initial-link duration. There was no evidence of control by initial-link durations from previous sessions.     

Effects of initial-link duration on preference and resistance to change in concurrent-chains schedules

Previous studies with concurrent-chains procedures have shown that preference for a terminal-link signaling a higher reinforcement rate decreases as initial-link durations increase. Using a concurrent-chains procedure, the present experiment examined the effects of manipulating initial-link duration on preference and resistance to disruption with rats nose poking for different rates of food reinforcement in the terminal links. Consistent with previous findings, preference for a terminal link with a higher reinforcement rate decreased with longer initial links. Conversely, relative resistance to disruption in the terminal link with a higher reinforcement rate increased with longer initial links. These findings are counter to the prediction of behavioral momentum theory that preference and resistance to change should be positively related.     

Transitional choice behavior in concurrent-chain schedules

The choice responses of four pigeons were examined in 20 periods of transition in a concurrent-chain procedure with variable-interval schedules as initial links and fixed delays to reinforcement as terminal links. In some conditions, the delays to reinforcement were different for the two terminal links, and changes in preference were recorded after the delays for the two response keys were switched. In other conditions, the reinforcer delays were equal for the two keys, but which key delivered 80% of the reinforcers was periodically switched. Choice proportions changed more quickly after a switch in reinforcement percentages than after a switch in the delays, thereby contradicting the hypothesis that faster changes would occur when the switch in conditions was easier to discriminate. Analyses of response sequences showed that the effects of individual reinforcers were larger and lasted longer in conditions with changing reinforcement percentages than in conditions with changing terminal-link delays. Rates of change in choice behavior do not appear to be limited by the unpredictability of variable reinforcement schedules, because the changes in behavior were slow and gradual even when there was a large and sudden change in reinforcer delays.     

Choice and timing in concurrent chains: effects of initial-link duration

Theories of timing have been applied to choice between delayed rewards by assuming that delays are represented in memory and that subjects sample from memory when choosing between alternatives. To search for covariation in single-trial measures of performance that might confirm this assumption, we used a procedure that allowed for convergent measurement of choice and timing behavior. Four pigeons responded in a concurrent chains/peak procedure in which the terminal links were fixed-interval (FI) 8s and FI 16s, across conditions the duration of the initial-link schedule was either short or long, and one quarter of the terminal links lasted for 48 s and ended without reinforcer delivery. Preference for the FI 8-s alternative was stronger with shorter initial links, replicating the 'initial-link effect'. Responding on no-food trials was unaffected by initial-link duration, and aggregated across trials, was typical of the peak procedure: response distributions were approximately Gaussian, with modes near the FI schedule durations, and variance was greater for the FI 16-s terminal link. Analysis of local measures of initial-link performance (e.g., pause to begin responding, time spent responding, number and duration of visits to each alternative, etc.) found that the initial-link effect was associated with an increase in the number and duration of visits per cycle to the nonpreferred alternative. Regression analyses showed that local initial-link measures contributed relatively little additional variance in predicting performance on individual no-food trials beyond that accounted for by FI schedule. Our results provide no clear evidence that initial- and terminal-link responding in concurrent chains are mediated by a common representation of terminal-link delays.     

The role of terminal-link stimuli in concurrent-chain schedules: revisited using a behavioral-history procedure

A behavioral-history procedure was used to study the function of terminal-link stimuli as conditioned reinforcers in multiple concurrent-chain schedules of reinforcement. First, three pigeons were exposed to multiple concurrent-chain schedules in which the two multiple-schedule components were correlated with a blue and a white stimulus, respectively. In each component the initial links were equal independent variable-interval (VI) 15 s schedules. A fixed-interval (FI) 10 s schedule operated on the red key in one terminal link while extinction operated on the green key in the alternative terminal link. When large preferences for the red stimulus had been established, two tests were conducted. In the terminal-link test, under new initial-link stimuli--purple and brown--an FI 10 s schedule operated for both the red and green terminal-link stimuli. In the subsequent initial-link test, the blue and white initial-link stimuli were reintroduced, and, as in the terminal-link test, FI 10s operated for both the red and the green terminal-link stimuli. In the terminal-link test, the three pigeons showed no preference for the terminal links with the red stimulus, but showed clear and consistent preferences for the red stimulus when blue and white stimuli were reintroduced as initial-link stimuli in the initial-link test. This suggests that there are multiple sources of control over initial-link response allocation in concurrent-chains, including control by both terminal- and initial-link stimuli.     

Psychological distance to reward: equating the number of stimulus and response segments

Psychological distance to reward, or the segmentation effect, refers to the preference for a terminal link of a concurrent-chains schedule consisting of a simple reinforcement schedule (e.g. fixed interval [FI] 30s) relative to its chained-schedule counterpart (e.g. chained FI 15s FI 15s). This experiment was conducted to examine whether the segmentation effect is due to the number of terminal-link stimulus and response segments per se. Three pigeons pecked under a concurrent-chains schedule in which identical variable-interval (VI) schedules operated in the initial links. In each session, half the terminal-link entries followed one initial-link key and the other half followed the other initial-link key. The initial-link keys correlated with the different terminal links were manipulated across conditions. In the first three conditions, each terminal link contained a chained fixed-time (FT) FT schedule, and in the final three conditions, each terminal link contained a chained FI FI schedule. In each condition, in one terminal link (alternating), the order of two key colors correlated with the different schedule segments alternated across terminal-link entries, whereas in the other terminal link (constant), the order of two other key colors was identical for each entry. With the chained FT FT schedule terminal links, there was indifference between the alternating and constant terminal links within and across pigeons, as indexed by initial-link choice proportions. In addition, terminal-link response rates were relatively low. With the chained FI FI schedule terminal links, for each pigeon, there was relatively more preference for the alternating terminal link and terminal-link response rates increased relative to conditions with the chained FT FT schedule terminal links. These data suggest that the segmentation effect is not due simply to the number of terminal-link stimulus or response segments per se, but rather to a required period of responding during a stimulus segment that never is paired with reinforcement.     

Does the terminal-link effect depend on duration or reinforcement rate?

Preference in concurrent chains for the richer terminal-link schedule becomes more extreme as the schedule values increase with their ratio held constant, a result known as the terminal-link effect. We report two experiments that attempt to determine whether this effect is related to terminal-link duration or the overall rate of reinforcement. These variables have been confounded in prior studies, but can be separated by comparing variable-duration schedules that end after a single reinforcer has been earned, with constant-duration schedules during which a variable number of reinforcers may be earned. In Experiment 1, the terminal-link effect was obtained with variable-duration schedules when duration and overall reinforcement rate were manipulated, but not with constant-duration schedules when overall reinforcement rate was changed with duration held constant. In Experiment 2, the terminal-link effect was obtained with constant-duration schedules when duration was manipulated with overall reinforcement rate held constant. Taken together, these results show that the terminal-link effect depends on changes in terminal-link duration, not overall reinforcement rate (or equivalently, average time to reinforcement). This accords with the account of the terminal-link effect provided by the contextual choice model [J. Exp. Anal. Behav. 61 (1994) 113] but not delay-reduction theory [J. Exp. Anal. Behav. 12 (1969) 723].     

A further study of choice and percentage reinforcement

In Experiment I four pigeons were trained in a concurrent chains procedure with fixed-ratio schedules (FR1) in the initial components and fixed-time schedules in the terminal components. Pecking one of the keys when both keys were white initiated a fixed time schedule on that key. A peck to the left key produced three stripes on the key. At the termination of the fixed-time component food always occurred. Pecking the other key produced either a circle or a triangle. If a circle appeared, reinforcement occurred. If a triangle appeared a brief timeout was given. Initially the stripes appeared on the left key and the circle and triangle on the left. This was reversed during the course of the experiment. In addition, sessions were conducted in which both circle and triangle sometimes preceded reinforcement and sometimes timeout. For most birds under most conditions there was a preference for the key that produced the circle and triangle. When these were uncorrelated with reinforcement and time out three of the birds preferred the key producing 100% reinforcement.

In Experiment II three factors were varied and VI 20 sec schedules were used in the initial links instead of FR1. The results showed that pigeons preferred the 50% condition more 1) the greater the duration of the terminal links, 2) the smaller the value on the initial link VI schedules and 3) the less the probability of food in the terminal link with stripes on the key.     

Rapid acquisition in concurrent chains: effects of initial-link duration

Pigeons were trained in a concurrent chains procedure in which the terminal-link schedules in each session were either fixed-interval (FI) 10s FI 20s or FI 20s FI 10s, as determined by a pseudorandom binary series. The initial-link was a variable-interval (VI) 10-s schedule. Training continued until initial-link response allocation stabilized about midway through each session and was sensitive to the terminal-link immediacy ratio in that session. The initial-link schedule was then varied across sessions between VI 0.01 s and VI 30s according to an ascending and descending sequence. Initial-link response allocation was a bitonic function over the full range of durations. Preference for the FI 10-s terminal-link at first increased as programmed initial-link duration varied from 0.01 to 7.5s, and then decreased as initial-link duration increased to 30s. The bitonic function poses a potential challenge for existing models for steady-state choice, such as delay-reduction theory (DRT) [Fantino, E., 1969. Choice and rate of reinforcement. J. Exp. Anal. Behav. 12, 723-730], which predict a monotonic function. However, an extension of Grace and McLean's [Grace, R.C., McLean, A.P., 2006. Rapid acquisition in concurrent chains: evidence for a decision model. J. Exp. Anal. Behav. 85, 181-202] decision model predicted the bitonic function, and may ultimately provide an integrated account of choice in concurrent chains under both steady-state and dynamic conditions.     

Free-choice preference under uncertainty

Preferences in pigeons for free choice over forced choice under uncertain contingencies were compared with the one under certain contingencies in multiple concurrent-chain schedules of reinforcement. The uncertain condition examined the preference for two alternatives over one alternative when reinforcement probability at the end of the terminal link equalled 0.5, and with all keys in each terminal link lit green (two mixed fixed-interval extinction keys versus one mixed fixed-interval extinction key). Key and schedule arrangement in the certain condition was the same except that a peck on any terminal-link key after the FI interval always produced food. When naive pigeons were first exposed to uncertain contingencies, preference for two lit keys over one lit key was observed, and the preference was confirmed by sequential reversals of the terminal-link contingencies. However, no consistent preference was observed when uncertain condition followed the certain condition. Under certain contingencies, unlike the earlier experiments, very small and inconsistent preferences for free choice were demonstrated. A possible reason for the different preferences in the uncertain conditions was that pigeons may lessen their sensitivity to the circumstances with uncertainty by any history or carry-over effect of the prior contingency.     

Effect of unsignaled delays between stimuli in a chain schedule on responding and resistance to change

Behavioral momentum theory is an evolving theoretical account of the strength of behavior. One challenge for the theory is determining the role of signal stimuli in determining response strength. This study evaluated the effect of an unsignaled delay between the initial link and terminal link of a two-link chain schedule on resistance to change using a multiple schedule of reinforcement. Pigeons were presented two different signaled delay to reinforcement schedules. Both schedules employed a two-link chain schedule with a variable interval 120-s initial link followed by a 5-s fixed time terminal link schedule. One of the schedules included a 5-s unsignaled delay between the initial link and the terminal link. Resistance to change was assessed with two separate disruption procedures: extinction and adding a variable time 20-s schedule of reinforcement to the inter-component interval. Baseline responding was lower in the schedule with the unsignaled delay but resistance to change for the initial link was unaffected by the unsignaled delay. The results suggest that not all unsignaled delays are equal in their effect on resistance to change.     

Constraining response output on conjunctive fixed-ratio 1 fixed-time reinforcement schedules: Effects on the postreinforcement pause

Two experiments used response-restriction procedures in order to test the independence of the factors determining response rate and the factors determining the size of the postreinforcement pause on interval schedules. Responding was restricted by response-produced blackout or by retracting the lever. In Experiment 1 with a Conjunctive FR 1 FT schedule, the blackout procedure reduced the postreinforcement pause more than the lever-retraction procedure did, and both procedures produced shorter pauses than did the schedule without response restriction. In Experiment 2 the interreinforcement interval was also manipulated, and the size of the pause was an increasing function of the interreinforcement interval, but the rate of increase was lower than that produced by fixed interval schedules of comparable interval durations. The assumption of functional independence of the postreinforcement pause and terminal rate in fixed interval schedules is questioned since data suggest that pause reductions resulted from constraining variation in response number compared to equivalent periodic schedules in which response number was allowed to vary.     

Quantitative analyses of methamphetamine's effects on self-control choices: implications for elucidating behavioral mechanisms of drug action

The purpose of the present research was to utilize quantitative methods to identify behavioral mechanisms involved in the effects of stimulant drugs on choice in a self-control procedure. A logarithmic equation based upon a combination of the matching law and hyperbolic discounting was used to separate drug-induced changes in sensitivity to reinforcement delay from drug-induced changes in sensitivity to reinforcement amount. Pigeons responded under a concurrent-chains schedule. In the initial link, two keys were illuminated simultaneously and access to the terminal link was controlled by a single random-interval (RI) schedule; pecks on one or the other key lead to its terminal link with a 0.5 probability. In the terminal links, one alternative provided 1-s access to food (the smaller reinforcer) and the other alternative provided 4-s access to food (the larger reinforcer). The signaled delay to the smaller reinforcer always was 2s, whereas the signaled delay to the larger reinforcer increased from 2 to 40s within each session, across 10-min blocks. In general, intermediate doses of methamphetamine increased preference for the larger more delayed reinforcer. Quantitative analyses indicated that, in most cases, methamphetamine decreased sensitivity to reinforcement delay. In a few instances, concomitant decreases in sensitivity to reinforcement amount also occurred. These results suggest that a reduced sensitivity to reinforcement delay may be important behavioral mechanism of the effects of stimulants on self-control choices, and that this effect sometimes can be accompanied by a decreased sensitivity to reinforcement amount.     

Transient dynamics and food-web complexity in the Lotka-Volterra cascade model

How does the long-term behaviour near equilibrium of model food webs correlate with their short-term transient dynamics? Here, simulations of the Lotka -Volterra cascade model of food webs provide the first evidence to answer this question. Transient behaviour is measured by resilience, reactivity, the maximum amplification of a perturbation and the time at which the maximum amplification occurs. Model food webs with a higher probability of local asymptotic stability may be less resilient and may have a larger transient growth of perturbations. Given a fixed connectance, the sizes and durations of transient responses to perturbations increase with the number of species. Given a fixed number of species, as connectance increases, the sizes and durations of transient responses to perturbations may increase or decrease depending on the type of link that is varied. Reactivity is more sensitive to changes in the number of donor-controlled links than to changes in the number of recipient-controlled links, while resilience is more sensitive to changes in the number of recipient-controlled links than to changes in the number of donor-controlled links. Transient behaviour is likely to be one of the important factors affecting the persistence of ecological communities.     

Preference for a segmented schedule using a brief S+ stimulus correlated with a great delay reduction in humans

Past studies using the concurrent-chain procedure showed that pigeons and humans generally prefer an unsegmented schedule to a segmented schedule. This finding is ostensibly inconsistent with theories of conditioned reinforcement such as delay-reduction theory. In the present study with humans, two changes in the basic segmented schedule were implemented to resolve this inconsistency. The first change was that in the segmented schedule the terminal-link stimulus (S+ stimulus) changed late in the terminal-link, close to reinforcement presentation. The second change was that the presentation of the segmenting stimulus, S+, was brief allowing a reinstatement of the early terminal-link stimulus, which is contiguous with reinforcement. Our data constitute the first demonstration of preference for the segmented schedule when a brief S+ is correlated with a greater reduction in delay to reinforcement.     

Need probability effects in animal short-term memory

Five pigeons performed in a delayed matching-to-sample (DMTS) procedure with five delay durations (0.5, 2.5, 5, 10 and 20 s) mixed within sessions. Contrary to the predictions of need probability theory, discriminability decreased when fewer short than long delays were included in each session. To test whether the decrease in discriminability was due to a decrease in obtained reinforcement at short delays, the number of trials at each delay was held constant and reinforcer probability was increased with increasing delay. This manipulation produced a similar decrease in discriminability as when the frequency of delays was manipulated. It was concluded that the effect of delay frequency on the forgetting function is mediated by the effect of the reinforcer distribution, which influences discriminability by weakening stimulus control.     

The role of temporal generalization in a temporal discrimination task

Two experiments trained rats to discriminate two or three stimulus durations using a temporal discrimination task. A standard peak shift effect was observed when training was administered with short versus long signals in Experiment 1. Both discrimination accuracy scores and the generalization gradients revealed that shorter intervals were discriminated more accurately, which may be due to the scalar property of timing. In Experiment 2, three signals (short, medium, and long) were associated with three different responses, or two of the intervals were associated with one response (short and long or short and medium) and the other interval with a different response. Here, the short/medium versus long discrimination was learned most readily of the three tasks. The results of both experiments indicated a strong contribution of learning of individual durations combined with scalar generalization gradients, but Experiment 2 indicated that categorical encoding of durations may have also been operating.