The stay/switch model describes choice among magnitudes of reinforcers

The stay/switch model is an alternative to the generalized matching law for describing choice in concurrent procedures. The purpose of the present experiment was to extend this model to choice among magnitudes of reinforcers. Rats were exposed to conditions in which the magnitude of reinforcers (number of food pellets) varied for staying at alternative 1, switching from alternative 1, staying at alternative 2 and switching from alternative 2. A changeover delay was not used. The results showed that the stay/switch model provided a good account of the data overall, and deviations from fits of the generalized matching law to response allocation data were in the direction predicted by the stay/switch model. In addition, comparisons among specific conditions suggested that varying the ratio of obtained reinforcers, as in the generalized matching law, was not necessary to change the response and time allocations. Other comparisons suggested that varying the ratio of obtained reinforcers was not sufficient to change response allocation. Taken together these results provide additional support for the stay/switch model of concurrent choice.     

Some effects of overall rate of earning reinforcers on run lengths and visit durations

In a concurrent schedule, responding at each alternative is controlled by a pair of schedules that arrange reinforcers for staying at that alternative and reinforcers for switching to the other alternative. Each pair of schedules operates only while at the associated alternative. When only one pair of stay and switch schedules is presented, the rates of earning reinforcers for staying divided by the rates of earning reinforcers for switching controls the mean number responses in a visit and the mean duration of visits. The purpose of the present experiment was to see whether the sum of the rates of earning stay and switch reinforcers changed the way that run length and visit duration were affected by the ratio of the rates of stay to switch reinforcers. Rats were exposed to pairs of stay and switch schedules that varied both the ratio of the rates of earning stay and switch reinforcers and the sum of the rates of earning stay and switch reinforcers. Run lengths and visit durations were joint functions of the ratio of the rates of earning stay and switch reinforcers and the sum of the rates of earning stay and switch reinforcers. These results shows that the effect of the ratio of the sum of the rates of earning stay and switch reinforcers results from processes operating at the alternative, rather than from processes operating at both alternatives.     

Contingency discriminability and the generalized matching law describe choice on concurrent ratio schedules of wheel-running reinforcement

Belke (2010) showed that on concurrent ratio schedules, the difference in ratio requirements required to produce near exclusive preference for the lower ratio alternative was substantively greater when the reinforcer was wheel running than when it was sucrose. The current study replicated this finding and showed that this choice behavior can be described by the matching law and the contingency discriminability model. Eight female Long Evans rats were exposed to concurrent VR schedules of wheel-running reinforcement (30s) and the schedule value of the initially preferred alternative was systematically increased. Two rats rapidly developed exclusive preference for the lower ratio alternative, but the majority did not - even when ratios differed by 20:1. Analysis showed that estimates of slopes from the matching law and the proportion of reinforcers misattributed from the contingency discriminability model were related to the ratios at which near exclusive preference developed. The fit of these models would be consistent with misattribution of reinforcers or poor discrimination between alternatives due to the long duration of wheel running.     

Momentary and global maximizing

Four adult females responded at a computer console, on three constant probability concurrent variable-interval reinforcement schedules. The subjects were instructed to try to obtain as many reinforcers as possible, but were not given any instructions on how to accomplish this task. Three of the four subjects typically allocated responses to the schedule offering the higher reinforcement probability. These results show that some subjects trying to maximize overall reinforcement may respond in accordance with a momentary maximizing strategy.     

Quantitative analyses of observing and attending

We review recent experiments examining whether simple models of the allocation and persistence of operant behavior are applicable to attending. In one series of experiments, observing responses of pigeons were used as an analog of attending. Maintenance of observing is often attributed to the conditioned reinforcing effects of a food-correlated stimulus (i.e., S+), so these experiments also may inform our understanding of conditioned reinforcement. Rates and allocations of observing were governed by rates of food or S+ delivery in a manner consistent with the matching law. Resistance to change of observing was well described by behavioral momentum theory only when rates of primary reinforcement in the context were considered. Rate and value of S+ deliveries did not affect resistance to change. Thus, persistence of attending to stimuli appears to be governed by primary reinforcement rates in the training context rather than conditioned reinforcing effects of the stimuli. An additional implication of these findings is that conditioned "reinforcers" may affect response rates through some mechanism other than response-strengthening. In a second series of experiments, we examined the applicability of the matching law to the allocation of attending to the elements of compound stimuli in a divided-attention task. The generalized matching law described performance well, and sensitivity to relative reinforcement varied with sample duration. The bias and sensitivity terms of the generalized matching law may provide measures of stimulus-driven and goal-driven control of divided attention. Further application of theories of operant behavior to performance on attention tasks may provide insights into what is referred to variously as endogenous, top-down, or goal-directed control of attention.     

Modeling the dynamics of choice

A simple linear-operator model both describes and predicts the dynamics of choice that may underlie the matching relation. We measured inter-food choice within components of a schedule that presented seven different pairs of concurrent variable-interval schedules for 12 food deliveries each with no signals indicating which pair was in force. This measure of local choice was accurately described and predicted as obtained reinforcer sequences shifted it to favor one alternative or the other. The effect of a changeover delay was reflected in one parameter, the asymptote, whereas the effect of a difference in overall rate of food delivery was reflected in the other parameter, rate of approach to the asymptote. The model takes choice as a primary dependent variable, not derived by comparison between alternatives—an approach that agrees with the molar view of behaviour.     

Small and large fixed ratios with the same unit price

Key pressing of rats was maintained under multiple and discrete-trial choice schedules with reinforcer units of 45 mg food pellets or 3.5 s dips of sucrose solution. Both smaller and larger fixed ratio (FR) schedules were associated with the same unit price in a manner, for example, that each of eight iterations of FR120 was associated with delivery of a single reinforcer unit and one instance of FR960 was associated with eight reinforcer units. FR requirement varied between 20 and 1560 per aggregate reinforcer and unit price varied between 20 and 240 per reinforcer unit. During multiple schedules with food reinforcers, rates and patterns of responding were comparable over nearly a 50-fold range of FR requirements (20-1380) when unit price was 20; over nearly a six-fold range of FR requirements (120-720) when unit price was 120; and was only marginally maintained when unit price was 240. Demand for food pellets was comparatively inelastic at FRs between 20 and 120, during which subjects did not receive supplemental feeding outside experimental sessions, but was elastic at FRs greater than 240, when subjects sometimes did receive supplemental feeding. In a discrete-trial choice procedure with a constant unit price of 120 for sucrose solution, subjects were indifferent between smaller FRs and alternative FRs as large as 480, but began switching away from larger FRs that were 600 or greater. Because responding had been comparably maintained under both FR120 and FRs as large as 960 in the multiple schedule, results from the choice procedure indicated that choice performance was influenced by variables other than FR requirement and unit price. Because aggregate reinforcers were the same for smaller and larger FRs, the most likely reason for preferring smaller FRs was the nearness in time to some reinforcer.     

Undermatching is an emergent property of selection by consequences

A computational theory of selection by consequences [McDowell, J.J, 2004. A computational model of selection by consequences. J. Exp. Anal. Behav. 81, 297-317] was tested by studying the responding of virtual organisms that were animated by the theory on random interval schedules of reinforcement. The theory generated responding by applying principles of selection, reproduction, and mutation to a population of potential behaviors that evolved in response to the selection pressure exerted by reinforcement. The organisms' equilibrium response rates were well described by the modern version of the Herrnstein hyperbola, which includes an exponent on reinforcement rate. Under strong selection pressure this exponent decreased with increasing mutation rate from a value near 1.0 at 1% mutation to an asymptotic value of 0.83 at mutation rates of 10% and greater. This asymptotic value is consistent with values obtained by fitting the equation to data from live organisms responding on single schedules, and with the value of about 0.80 that is expected on the basis of extensive research with live organisms responding on concurrent schedules. These results show that the computational theory is consistent with the modern theory of matching [McDowell, J.J, 2005. On the classic and modern theories of matching. J. Exp. Anal. Behav. 84, 111-127], and that it is a viable candidate for a mathematical dynamics of behavior.     

Transitional choice behavior in concurrent-chain schedules

The choice responses of four pigeons were examined in 20 periods of transition in a concurrent-chain procedure with variable-interval schedules as initial links and fixed delays to reinforcement as terminal links. In some conditions, the delays to reinforcement were different for the two terminal links, and changes in preference were recorded after the delays for the two response keys were switched. In other conditions, the reinforcer delays were equal for the two keys, but which key delivered 80% of the reinforcers was periodically switched. Choice proportions changed more quickly after a switch in reinforcement percentages than after a switch in the delays, thereby contradicting the hypothesis that faster changes would occur when the switch in conditions was easier to discriminate. Analyses of response sequences showed that the effects of individual reinforcers were larger and lasted longer in conditions with changing reinforcement percentages than in conditions with changing terminal-link delays. Rates of change in choice behavior do not appear to be limited by the unpredictability of variable reinforcement schedules, because the changes in behavior were slow and gradual even when there was a large and sudden change in reinforcer delays.     

Context: a central concept

Seminal research in several areas has underscored the central role played by context in the control of behavior. Landmark studies in classical conditioning (with both conditioned suppression and autoshaping procedures) and in conditioned reinforcement (using the observing paradigm) are reviewed. The role of context also proved central in the study of choice (including the matching law and delay-reduction theory). This latter work contributed to the development of experimental analogs to foraging behavior. Research on foraging has also highlighted the importance of context and has led to some counterintuitive predictions that are mediated by context.     

Within-session changes in responding during concurrent variable interval variable ratio schedules

Rats (Experiment 1) and pigeons (Experiment 2) responded on several concurrent variable interval (VI) variable ratio (VR) schedules. The rate of, but not the time spent, responding in each component usually changed within-sessions. The bias and sensitivity parameters of the generalized matching law (GML) did not change systematically within-sessions. The fit of the GML to the data did not change within-sessions for pigeons, but it was better in the middle than at the beginning or end of the session for some for rats. Both over- and under-matching occurred. These results imply that within-session changes in responding do not usually cause problems for assessing the validity of the GML when subjects respond on concurrent VI VR schedules. The results also suggest that under- and over-matching are not produced by different factors, but rather lie on a continuum.     

Coyotes (Canis latrans) and the matching law

Environmental change is accelerating due to anthropogenic influence. Species that have greater behavioral flexibility may be better adapted to exploit new or constantly changing habitats. There are few mammals and even fewer carnivores that better illustrate widespread adaptability and behavioral flexibility in the wake of human disturbance than coyotes (Canis latrans). Yet how such predators successfully track resources, enabling them to survive and extend their range in stochastic environments remains unknown. We tested eight wild-born, captive coyotes individually on an operant two-choice test using concurrent variable interval (VI) schedules. We held the overall rate of reinforcement constant but manipulated the ratio of reinforcement available from the two choices. We analyzed sensitivity of coyotes’ tracking of resource change by fitting the generalized matching equation to the data. Results showed all coyotes efficiently tracked changes in reinforcement ratios within the first few sessions of each new condition and matched their relative rate of foraging time to relative rate of resources. We suggest the matching paradigm provides a methodology to explore coyote foraging strategies, and a potential framework to compare behavioral flexibility across species, by measuring the ability to track resource change under variable resource conditions.     

Demarcated response sequences and generalised matching

The generalised matching law was applied for the first time to sequences of responses. Pigeons responded on dependently arranged two-operandum concurrent schedules in which all four possible two-response sequences on these operanda were demarcated and reinforced. Reinforcer rates for the sequences were varied over a wide range, including extinction. Sequences of responses, rather than individual responses, followed the generalised matching law. We compared single-sensitivity and multiple-sensitivity versions of the generalised matching law. The multiple sensitivities were not significantly different from each other, so the single-sensitivity model provided the more efficient fit. As in previous research, responding was biased against the emission of sequences that required changeovers.     

Reinforcement omission in concurrent fixed-interval and random-interval schedules

The present experiment examined overall and local effects of omission of reinforcers in a choice situation. Pigeons' key-pecking responses were reinforced under concurrent fixed-interval and random-interval schedules of food presentation. After some weeks of baseline sessions in which the probability of reinforcement was 1.00, approximately 25% of food presentations from the fixed-interval schedule were omitted and replaced by timeout periods. In such omission sessions, the overall relative rates of responding to the fixed-interval schedule became lower than those in the baseline sessions. On the other hand, when relative rates of responding to the fixed-interval schedule in the omission sessions were calculated separately for fixed-interval cycles preceded by timeout periods and those preceded by food presentations, the relative rates in the former type of fixed-interval cycles were higher than those in the latter type for three out of four pigeons. These results mean that relative rates of responding cannot always be regarded as reflecting a relative value of an alternative, and that the overall effect of the omission of fixed-interval reinforcers is not reducible to the local effect of omission.     

Optimal Foraging and Predator–Prey Dynamics

A system consisting of a population of predators and two types of prey is considered. The dynamics of the system is described by differential equations with controls. The controls model how predators forage on each of the two types of prey. The choice of these controls is based on the standard assumption in the theory of optimal foraging which requires that each predator maximizes the net rate of energy intake during foraging. Since this choice depends on the densities of populations involved, this allows us to link the optimal behavior of an individual with the dynamics of the whole system. Simple qualitative analysis and some simulations show the qualitative behavior of such a system. The effect of the optimal diet choice on the stability of the system is discussed.     

Matching by fixing and sampling: a local model based on internality

Undermatching and overmatching in concurrent schedules of reinforcement have been traditionally described as changes in the slope of the Generalized Matching Law function. More recently, it has been suggested that deviations from strict matching may be better described as following a policy of mostly fixing on the preferred schedule, and occasionally sampling the alternative schedule. So far, no model of local performance predicts the global outcome of this policy. We describe one such model; it assumes immediate and long-term effects of reinforcement on local performance. The model assumes long-term effects as changes in the internal state of the organism. Formally, the model is analogous to the Axiom of Repeated Choice [Lefebvre, V.A., 2004. Bipolarity, choice, and entro-field. In: Proceedings of the Eighth World Multi-Conference on Systemics, Cybernetics and Informatics, vol. IV, pp. 95-99].     

Rate maximizing and state variable theories of diet selection

In behavioral ecology, diet selection is often predicted by using a theory in which it is assumed that behaviors have evolved to maximize the rate of intake of energy. In this theory, it is assumed that fitness is a monotonic function of energy intake. An alternative is to deal directly with fitness, measured in terms of expected reproduction, and thus connect short term behavior with reproduction. Following the usual assumptions of the theory of diet choice, fitness satisfies a partial differential-difference equation. Conditions under which rate maximizing and state variable theories of diet selection yield identical predictions are identified. When predation, as well as starvation, is a source of mortality, the identification is not as complete.     

Human choice behaviour in a frequently changing environment

The present experiment provided a replication in humans of an experimental procedure that has been used frequently with nonhumans to investigate choice behaviour in a changing environment. Six volunteers played a computer game, which required tracking of a moving balloon on two simultaneously available response panels for monetary reinforcers. Each of the 15 sessions randomly arranged the following concurrent variable-interval reinforcement schedules, which were in effect until six reinforcers had been obtained: 27:1, 9:1, 3:1, 1:1, 1:3, 1:9, and 1:27. Although many aspects of human performance appeared to be qualitatively similar to that of nonhumans on this procedure, such as the rapid preference shifts towards the within-session reinforcer ratios and the presence of local effects of reinforcers, values of sensitivity to reinforcement were very variable in the present study, as commonly reported in human choice studies. Future variations and refinements of the experimental methods are needed to explore how this variability may be reduced.     

Temporal discounting predicts individual competitive success in a human analogue of group foraging

Twenty groups of college undergraduates (N=9-12) participated in a discrete-trials analogue of group foraging in which points, exchangeable for course credit, were the resource to be acquired. Group matching of foragers to patch resource availability was well described by the generalized Ideal Free Distribution, with undermatching (imperfect sensitivity to resource differentials) the norm. Individuals differed on several measures of competitive success, and a measure of temporal discounting (TD) accounted for a significant proportion of the variance in these measures. Tendencies in switch patterns and visit durations differed for the most-impulsive and least-impulsive individuals in a fashion that was consistent with a TD interpretation. When competitive weights, defined in terms of TD scores, were substituted for counts of individuals in the generalized Ideal Free Distribution, group-matching slopes more closely approximated perfect sensitivity than in standard analyses. This was true for groups incorporating all impulsive individuals, all non-impulsive individuals, or a broad range of TD scores. The results suggest that well-understood individual psychological processes are associated with competitive ability in group choice.