Choice behavior in spontaneously hypertensive rats: Variable vs. fixed schedules of reinforcement

Spontaneously hypertensive rats (SHR) and Wistar rats were evaluated in the successive-encounters procedure (an operant simulation of natural foraging) with the idea of assessing differences between them in their preference for variable schedules of reinforcement. In this procedure, after satisfying a schedule of reinforcement associated with search time, the subjects could “accept” or “reject” another schedule of reinforcement associated with handling time. Two schedules of reinforcement were available: a fixed interval (FI), and a variable interval (VI) with the same mean value. The results indicated preference for the variable schedule in both strains, as suggested by the observation that the VI was always accepted while the FI was often rejected. The difference in FI acceptability between strains was not statistically significant, a result which is relevant for the current debate of SHR as an adequate animal model of Attention Deficit Hyperactivity Disorder.     

EMG Biofeedback: The Effects of CRF, FR, VR, FI, and VI Schedules of Reinforcement on the Acquisition and Extinction of Increases in Forearm Muscle Tension

Biofeedback was used to increase forearm-muscle tension. Feedback was delivered under continuous reinforcement (CRF), variable interval (VI), fixed interval (FI), variable ratio (VR), and fixed ratio (FR) schedules of reinforcement when college students increased their muscle tension (electromyograph, EMG) above a high threshold. There were three daily sessions of feedback, and Session 3 was immediately followed by a session without feedback (extinction). The CRF schedule resulted in the highest EMG, closely followed by the FR and VR schedules, and the lowest EMG scores were produced by the FI and VI schedules. Similarly, the CRF schedule resulted in the greatest amount of time-above-threshold and the VI and FI schedules produced the lowest time-above-threshold. The highest response rates were generated by the FR schedule, followed by the VR schedule. The CRF schedule produced relatively low response rates, comparable to the rates under the VI and FI schedules. Some of the data are consistent with the partial-reinforcement-extinction effect. The present data suggest that different schedules of feedback should be considered in muscle-strengthening contexts such as during the rehabilitation of muscles following brain damage or peripheral nervous-system injury.     

Schedules of reinforcement as regulators of dispersion patterns

An eight-rat eight-station operant conditioning arena was used to study the spatial structure and temporal stability of foraging dispersion patterns. Food was obtained by bar pressing as the population was exposed to an ascending series of the fixed and variable aspects of ratio and interval schedules of reinforcement. Dispersion patterns, defined by the number of rats simultaneously foraging at each of the eight stations, and the temporal changes in these patterns, were the dependent variables. Both variables exhibited a unique relationship to each schedule type and value. The absence of such relationships when either food supply or response costs were examined suggests that these factors were not the determinants of spatio-temporal structure. An account is provided of how schedules may interact with behavioral foraging chains to explain dispersion patterns.     

Simulation of patch use by monkeys using operant conditioning

Food patch use by Japanese monkeys was examined using an operant conditioning procedure. Modified progressive ratio schedules, in which the probability of reinforcement decreases exponentially with the number of bar presses, were presented to 2 Japanese monkeys. Two types of schedule were used in each experimental session. One represented high quality food patches, where the probability of reinforcement was twice as high as in the other, which represented low quality food patches. The number of bar presses in each food patch was counted. Monkeys responded more frequently in high quality patches. The probability of reinforcement for the last response in each patch was the same in both types of schedule. The number of responses increased with a decrease in the occurrence of high quality patches, and with an increase in the inter-patch time interval. These results are in agreement with the predictions of Charnov’s marginal value theorem (Charnov, 1976). The pattern of patch use by monkeys observed in this study is discussed in terms of optimal foraging strategy.     

Effect of unsignaled delays between stimuli in a chain schedule on responding and resistance to change

Behavioral momentum theory is an evolving theoretical account of the strength of behavior. One challenge for the theory is determining the role of signal stimuli in determining response strength. This study evaluated the effect of an unsignaled delay between the initial link and terminal link of a two-link chain schedule on resistance to change using a multiple schedule of reinforcement. Pigeons were presented two different signaled delay to reinforcement schedules. Both schedules employed a two-link chain schedule with a variable interval 120-s initial link followed by a 5-s fixed time terminal link schedule. One of the schedules included a 5-s unsignaled delay between the initial link and the terminal link. Resistance to change was assessed with two separate disruption procedures: extinction and adding a variable time 20-s schedule of reinforcement to the inter-component interval. Baseline responding was lower in the schedule with the unsignaled delay but resistance to change for the initial link was unaffected by the unsignaled delay. The results suggest that not all unsignaled delays are equal in their effect on resistance to change.     

Schedule discrimination in a mixed schedule: implications for models of the variable-ratio, variable-interval rate difference

In Experiment 1, each of three humans knowledgeable about operant schedules used mouse clicks to respond to a "work key" presented on a monitor. On a random half of the presentations, work-key responses that completed a variable ratio (VR) 12 produced a tone. After five tones, the work key was replaced by two report keys. Pressing the right or left report key, respectively, added or subtracted yen50 from a counter and produced the work key. On the other half of the presentations, a variable interval (VI) associated with the work key was defined so its interreinforcer intervals approximated the time it took to complete the variable ratio. After five tone-producing completions of this schedule, the report keys were presented. Left or right report-key presses, respectively, added or subtracted yen50 from the counter. Subjects achieved high yen totals. In Experiment 2, the procedure was changed by requiring an interresponse time after completion of the variable interval that approximated the duration of the reinforced interresponse time on the variable ratio. Prior to beginning, subjects were shown how a sequence of response bouts and pauses could be used to predict schedule type. Subjects again achieved high levels of accuracy. These results show humans can discriminate ratio from interval schedules even when those schedules provide the same rate of reinforcement and reinforced interresponse times.     

Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement

Across two experiments, a peak procedure was used to assess the timing of the onset and offset of an opportunity to run as a reinforcer. The first experiment investigated the effect of reinforcer duration on temporal discrimination of the onset of the reinforcement interval. Three male Wistar rats were exposed to fixed-interval (FI) 30-s schedules of wheel-running reinforcement and the duration of the opportunity to run was varied across values of 15, 30, and 60s. Each session consisted of 50 reinforcers and 10 probe trials. Results showed that as reinforcer duration increased, the percentage of postreinforcement pauses longer than the 30-s schedule interval increased. On probe trials, peak response rates occurred near the time of reinforcer delivery and peak times varied with reinforcer duration. In a second experiment, seven female Long-Evans rats were exposed to FI 30-s schedules leading to 30-s opportunities to run. Timing of the onset and offset of the reinforcement period was assessed by probe trials during the schedule interval and during the reinforcement interval in separate conditions. The results provided evidence of timing of the onset, but not the offset of the wheel-running reinforcement period. Further research is required to assess if timing occurs during a wheel-running reinforcement period.     

Dishabituation with component transitions may contribute to the interactions observed during multiple schedules

Rates of responding by rats were usually higher during the variable interval (VI) 30-s component of a multiple VI 30-s fixed interval (FI) 30-s schedule than during the same component of a multiple VI 30-s VI 30-s schedule (Experiment 1). Response rates were also usually higher during the FI 30-s component of a multiple VI 30-s FI 30-s schedule than during the same component of a multiple FI 30-s FI 30-s schedule (Experiment 2). The differences in response rates were not observed when the components provided VI or FI 120-s schedules. These results were predicted by the idea that differences in habituation to the reinforcer between multiple schedules contribute to behavioral interactions, such as behavioral contrast. However, differences in habituation were not apparent in the within-session patterns of responding. Finding differences in response rates in both experiments violates widely-held assumptions about behavioral interactions, including that behavioral contrast does not occur for rats and that improving the conditions of reinforcement decreases, rather than increases, response rate in the alternative component.     

Coyotes (Canis latrans) and the matching law

Environmental change is accelerating due to anthropogenic influence. Species that have greater behavioral flexibility may be better adapted to exploit new or constantly changing habitats. There are few mammals and even fewer carnivores that better illustrate widespread adaptability and behavioral flexibility in the wake of human disturbance than coyotes (Canis latrans). Yet how such predators successfully track resources, enabling them to survive and extend their range in stochastic environments remains unknown. We tested eight wild-born, captive coyotes individually on an operant two-choice test using concurrent variable interval (VI) schedules. We held the overall rate of reinforcement constant but manipulated the ratio of reinforcement available from the two choices. We analyzed sensitivity of coyotes’ tracking of resource change by fitting the generalized matching equation to the data. Results showed all coyotes efficiently tracked changes in reinforcement ratios within the first few sessions of each new condition and matched their relative rate of foraging time to relative rate of resources. We suggest the matching paradigm provides a methodology to explore coyote foraging strategies, and a potential framework to compare behavioral flexibility across species, by measuring the ability to track resource change under variable resource conditions.     

Properties of behavior under different random ratio and random interval schedules: A parametric study

Four pigeons were trained to peck a key under different values of a temporally defined independent variable (T) and different probabilities of reinforcement (p). Parameter T is a fixed repeating time cycle and p the probability of reinforcement for the first response of each cycle T. Two dependent variables were used: mean response rate and mean postreinforcement pause. For all values of p a critical value for the independent variable T was found (T=1 sec) in which marked changes took place in response rate and postreinforcement pauses. Behavior typical of random ratio schedules was obtained at T

1 sec and behavior typical of random interval schedules at T

1 sec.     

Effect of signaling reinforcement on resistance to change in a multiple schedule

This study evaluated the effect of a signal on resistance to change using a multiple schedule of reinforcement. Experiment 1 presented pigeons with three schedules: a signaled delay to reinforcement schedule (a two-link chain schedule with a variable-interval 120-s initial link followed by a 5-s fixed-time schedule), an unsignaled delay schedule (a comparable two-link tandem schedule), and an immediate, zero-delay variable-interval 125-s schedule. Two separate disruption procedures assessed resistance to change: extinction and adding a variable-time 20-s schedule of reinforcement to the inter-component interval. Resistance to change tests were conducted twice, once with the signal stimulus (the terminal link of the chain schedule) present and once with it absent. Results from both disruption procedures showed that signal absence reduced resistance to change for the pre-signal stimulus. In probe choice tests subjects strongly preferred the signal stimulus over the unsignaled stimulus and exhibited no reliable preference when given a choice between the signal stimulus and immediate stimulus. Experiment 2 presented two equal signaled schedules where, during resistance to change tests, the signal remained for one schedule and was removed for the second. Resistance to change was consistently lower when the signal was absent.     

Can flower constancy in nectaring butterflies be explained by Darwin’s interference hypothesis?

 When foraging for nectar many insects exhibit flower constancy (a preference for flower species which they have previously visited) and frequently ignore rewarding flowers of other species. Darwin proposed the favoured explanation for this behaviour, hypothesizing that learning of handling skills for one flower species interferes with the ability to recall handling skills for previously learned species. A crucial element of this hypothesis is that savings in handling time resulting from constancy must exceed increases in travelling time necessitated by ignoring other suitable species. A convincing quantification of this trade-off has not been achieved and tests to date on bumblebees indicate that savings in handling time are too small to offset an increase in travelling time. To assess further the validity of Darwin’s hypothesis, handling and flight times of the butterfly, Thymelicus flavus, were measured under natural conditions, and the abundance and reward provided by the available flower species quantified to enable estimation of foraging efficiency. Butterflies exhibited a mean increase in handling time of 0.85 s per flower associated with switching between flower species, although the magnitude of this difference varied greatly among flower species. Switching was not associated with a decrease in travelling time, contrary to expectation. Switching was more frequent following a lower than average reward from the last flower visited. In butterflies, flights serve functions other than movement between nectar sources, such as mate location (unlike worker bees). Hence constancy may be a viable strategy to reduce time spent in handling flowers and increase time available for other activities. Although savings in handling time may be small, Darwin’s interference hypothesis remains a valid explanation for flower constancy in foraging butterflies. Received: 27 January 1997 / Accepted: 5 June 1997     

Oscillations following periodic reinforcement

Three experiments examined behavior in extinction following periodic reinforcement. During the first phase of Experiment 1, four groups of pigeons were exposed to fixed interval (FI 16 s or FI 48 s) or variable interval (VI 16 s or VI 48 s) reinforcement schedules. Next, during the second phase, each session started with reinforcement trials and ended with an extinction segment. Experiment 2 was similar except that the extinction segment was considerably longer. Experiment 3 replaced the FI schedules with a peak procedure, with FI trials interspersed with non-food peak interval (PI) trials that were four times longer. One group of pigeons was exposed to FI 20 s PI 80 s trials, and another to FI 40 s PI 160 s trials. Results showed that, during the extinction segment, most pigeons trained with FI schedules, but not with VI schedules, displayed pause-peck oscillations with a period close to, but slightly greater than the FI parameter. These oscillations did not start immediately after the onset of extinction. Comparing the oscillations from Experiments 1 and 2 suggested that the alternation of reconditioning and re-extinction increases the reliability and earlier onset of the oscillations. In Experiment 3 the pigeons exhibited well-defined pause-peck cycles since the onset of extinction. These cycles had periods close to twice the value of the FI and lasted for long intervals of time. We discuss some hypotheses concerning the processes underlying behavioral oscillations following periodic reinforcement.     

Risk-sensitive foraging in rats: the effects of response-effort and reward-amount manipulations on choice behavior

The literature on risk-sensitive foraging theory provides several accounts of species that fluctuate between risk-averse and risk-prone strategies. The daily energy budget rule suggests that shifts in foraging strategy are precipitated by changes in the forager's energy budget. Researchers have attempted to alter the organism's energy budget using a variety of techniques such as food deprivation, manipulation of ambient temperatures, and delays to food reward; however, response-effort manipulations have been relatively neglected. A choice preparation using a wheel-running response and rats examined risk-sensitive preferences when both response effort and reward amounts were manipulated. Concurrently available reinforcement schedules (FI/60 and VI/60) yielded equivalent food amounts per unit time in all treatments. Two levels of response effort (20 or 120 g tangential resistance) and two levels of reward amount (three or nine pellets) were combined to form four distinct response-effort/reward-amount pairings. Increasing reward amounts significantly shifted choice toward the FI schedule in both response-effort conditions. The incidence of choice preference and the magnitude of shifts in choice were greater for the high response-effort conditions than for the low response-effort conditions. Implications of the significant interaction between response effort and reward amount are discussed in terms of a general energy-budget model.     

Effects of experience on food-searching behavior in the antFormica schaufussi (Hymenoptera: Formicidae)

Foragers of the ant Formica schaufussitend to return to and search at a site of a previous food find. The search tactic employed by a forager on its return trip is related to the type of food previously encountered: search is more persistent in response to carbohydrate than to protein food. Using different reinforcement schedules with carbohydrate and protein food rewards, we show that, on a short-term as well as on a long-term basis, the basic pattern of search observed in naive foragers is only slightly modified through foraging experience. Foragers do not increase their search effort or adjust their search pattern when either type of food is systematically renewed on a fixed reward schedule and thus do not seem to be able to learn to assess the food predictability. Collective responses that could compensate for this lack of individual flexibility and increase foraging efficiency at the colony level are discussed.     

Increased effort requirements and risk sensitivity: a comparison of delay and magnitude manipulations

Reward magnitude and delay to reward were independently manipulated in two separate experiments examining risk-sensitive choice in rats. A dual-running wheel apparatus was used and the tangential force resistance required to displace both wheels was low (50g) for half of the subjects, and high (120g) for the remaining subjects. Concurrent FI30-s and FI60-s schedules delivered equivalent amounts of food reward per unit time (i.e. 5 and 10 pellets of food, respectively), and these conditions served as the baseline treatment for all subjects. Variability, either in reward magnitude or delay, was introduced on the long-delay (60s) schedule during the second phase. All subjects were returned to the baseline condition in the third phase, and variability was introduced on the short-delay (30s) interval schedule during phase four. The subjects were again returned to the baseline condition in the fifth and final phase, ultimately yielding a five-phase ABACA design. Original baseline performance was characterized by a slight short-delay interval preference, and this pattern of performance was recovered with each subsequent presentation of the baseline condition. Overall, the data obtained from the reward magnitude and delay-to-reward manipulations were indistinguishable; subjects experiencing low-response effort requirement behaved in a risk-indifferent manner and subjects experiencing high-response effort requirement preferred the variable schedule. Implications for the daily energy budget rule on risk-sensitive foraging are discussed in light of these findings.     

Greater effort boosts the affective taste properties of food

Actions can create preferences, increasing the value ascribed to commodities acquired at greater cost. This behavioural finding has been observed in a variety of species; however, the causal factors underlying the phenomenon are relatively unknown. We sought to develop a behavioural platform to examine the relationship between effort and reinforcer value in mice trained under demanding or lenient schedules of reinforcement to obtain food. In the initial experiment, expenditure of effort enhanced the value of the associated food via relatively lasting changes in its hedonic attributes, promoting an acquired preference for these reinforcers when tested outside of the training environment. Moreover, otherwise neutral cues associated with those reinforcers during training similarly acquired greater reinforcing value, as assessed under conditioned reinforcement. In a separate experiment, expenditure of effort was also capable of enhancing the value of less-preferred low-caloric reinforcers. Analysis of licking microstructure revealed the basis for this increased valuation was, in part, due to increased palatability of the associated reinforcer. This change in the hedonic taste properties of the food can not only serve as a basis for preference, but also guide decision-making and foraging behaviour by coordinating a potentially adaptive repertoire of incentive motivation, goal-directed action and consumption.     

Economic context and pigeons' risk-taking: an integrative approach

The present work tested pigeons' (Columba livia) risk sensitivity to changes in resource availability in the concurrent-chains choice procedure. Subjects were provided choices, generally between variable-ratio (VR) and fixed-ratio (FR) outcome schedules with equal or different mean values. Predictions from ecology's risk sensitivity, behavioral economic's rate maximization, and psychology's delay-reduction theory were contrasted under settings where budget was adversely affected by one of three manipulations in seven experiments: (1) session length was dramatically reduced; (2) choice-phase duration was substantially increased by increasing the response requirements in the choice phase, or (3) outcome duration was significantly increased by increasing the response requirements in the outcome phase while holding session length constant. Although preference measures were sensitive to every budget manipulation, the nature of the changes observed were dependent upon the manner in which resource availability was manipulated and the choice alternatives offered. For example, choice was affected in an opposite manner when budget was adversely altered by lengthening choice duration ("search time") as opposed to lengthening outcome duration ("handling time"). Findings revealed partial support for pigeons' sensitivity both to changes in budget and to changes in conditioned reinforcer value. The overall pattern of results thus favors an integrative approach to describing the effect of economic context on general risk sensitivity.     

Psychological distance to reward: equating the number of stimulus and response segments

Psychological distance to reward, or the segmentation effect, refers to the preference for a terminal link of a concurrent-chains schedule consisting of a simple reinforcement schedule (e.g. fixed interval [FI] 30s) relative to its chained-schedule counterpart (e.g. chained FI 15s FI 15s). This experiment was conducted to examine whether the segmentation effect is due to the number of terminal-link stimulus and response segments per se. Three pigeons pecked under a concurrent-chains schedule in which identical variable-interval (VI) schedules operated in the initial links. In each session, half the terminal-link entries followed one initial-link key and the other half followed the other initial-link key. The initial-link keys correlated with the different terminal links were manipulated across conditions. In the first three conditions, each terminal link contained a chained fixed-time (FT) FT schedule, and in the final three conditions, each terminal link contained a chained FI FI schedule. In each condition, in one terminal link (alternating), the order of two key colors correlated with the different schedule segments alternated across terminal-link entries, whereas in the other terminal link (constant), the order of two other key colors was identical for each entry. With the chained FT FT schedule terminal links, there was indifference between the alternating and constant terminal links within and across pigeons, as indexed by initial-link choice proportions. In addition, terminal-link response rates were relatively low. With the chained FI FI schedule terminal links, for each pigeon, there was relatively more preference for the alternating terminal link and terminal-link response rates increased relative to conditions with the chained FT FT schedule terminal links. These data suggest that the segmentation effect is not due simply to the number of terminal-link stimulus or response segments per se, but rather to a required period of responding during a stimulus segment that never is paired with reinforcement.     

Self-control and impulsiveness with asynchronous presentation of reinforcement schedules

IN DISCRETE TRIALS, PIGEONS WERE PRESENTED WITH TWO ALTERNATIVES: to wait for a larger reinforcer, or to respond and obtain a smaller reinforcer immediately. The choice of the former was defined as self-control, and the choice of the latter as impulsiveness. The stimulus that set the opportunity for an impulsive choice was presented after a set interval from the onset of the stimulus that signaled the waiting period. That interval increased or decreased from session to session so that the opportunity for an impulsive choice became available either more removed from or closer in time to the presentation of the larger reinforcer. In three separate conditions, the larger reinforcer was delivered according to either a fixed interval (FI) schedule, a fixed time (FT) schedule, or a differential reinforcement of other behavior (DRO) schedule. The results showed that impulsive choices increased as the opportunity for such a choice was more distant in time from presentation of the larger reinforcer. Although the schedule of the larger reinforcer affected the rate of response in the waiting period, the responses themselves had no effect on choice unless the responses postponed presentation of the larger reinforcer.