Dynamics of temporal control in rats: the effects of a brief transition in interval duration

Five rats lever-pressed for liquid reinforcers delivered according to a fixed-interval (FI) reinforcement schedule, where the interval requirement changed at an unpredictable point within a session. In a short square wave (SSW) condition, eight 30-s intervals were intercalated in a series of 120-s intervals so that the intervals changed from 120 to 30 s then back to 120 s. In a long square wave (LSW) condition the intervals changed from 120 to 480 s then back to 120 s. We observed rapid temporal control of post-reinforcement wait time duration by the IFI duration in the SSW condition only: Wait times decreased significantly during a transition to shorter (30 s) intervals; whereas wait times did not reliably increase during a transition to longer (480 s) intervals. Furthermore, in the SSW condition, wait time in post-transition intervals was shorter than that observed during pre-transition intervals. The results show that rats' wait times are sensitive to moment-to-moment changes in interval duration and that the dynamics depend on the direction in which the intervals change.     

Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement

Across two experiments, a peak procedure was used to assess the timing of the onset and offset of an opportunity to run as a reinforcer. The first experiment investigated the effect of reinforcer duration on temporal discrimination of the onset of the reinforcement interval. Three male Wistar rats were exposed to fixed-interval (FI) 30-s schedules of wheel-running reinforcement and the duration of the opportunity to run was varied across values of 15, 30, and 60s. Each session consisted of 50 reinforcers and 10 probe trials. Results showed that as reinforcer duration increased, the percentage of postreinforcement pauses longer than the 30-s schedule interval increased. On probe trials, peak response rates occurred near the time of reinforcer delivery and peak times varied with reinforcer duration. In a second experiment, seven female Long-Evans rats were exposed to FI 30-s schedules leading to 30-s opportunities to run. Timing of the onset and offset of the reinforcement period was assessed by probe trials during the schedule interval and during the reinforcement interval in separate conditions. The results provided evidence of timing of the onset, but not the offset of the wheel-running reinforcement period. Further research is required to assess if timing occurs during a wheel-running reinforcement period.     

Choice and timing in concurrent chains: effects of initial-link duration

Theories of timing have been applied to choice between delayed rewards by assuming that delays are represented in memory and that subjects sample from memory when choosing between alternatives. To search for covariation in single-trial measures of performance that might confirm this assumption, we used a procedure that allowed for convergent measurement of choice and timing behavior. Four pigeons responded in a concurrent chains/peak procedure in which the terminal links were fixed-interval (FI) 8s and FI 16s, across conditions the duration of the initial-link schedule was either short or long, and one quarter of the terminal links lasted for 48 s and ended without reinforcer delivery. Preference for the FI 8-s alternative was stronger with shorter initial links, replicating the 'initial-link effect'. Responding on no-food trials was unaffected by initial-link duration, and aggregated across trials, was typical of the peak procedure: response distributions were approximately Gaussian, with modes near the FI schedule durations, and variance was greater for the FI 16-s terminal link. Analysis of local measures of initial-link performance (e.g., pause to begin responding, time spent responding, number and duration of visits to each alternative, etc.) found that the initial-link effect was associated with an increase in the number and duration of visits per cycle to the nonpreferred alternative. Regression analyses showed that local initial-link measures contributed relatively little additional variance in predicting performance on individual no-food trials beyond that accounted for by FI schedule. Our results provide no clear evidence that initial- and terminal-link responding in concurrent chains are mediated by a common representation of terminal-link delays.     

The effects of concurrent task and gap events on peak time in the peak procedure

The effect of a concurrent task on timing performance of pigeons was investigated with the peak interval procedure. Birds were trained to peck a side key on a discrete-trial schedule that included reinforced fixed-interval (FI) 30-s trials and nonreinforced extended probe trials. Then, in separate sessions, birds were trained to peck a 6-s center key for food. In a subsequent test phase, the FI procedure was in effect along with dual-task probe test trials. On those test trials, the 6-s center key (task cue) was presented at 3, 9, or 15s after probe trial onset. During another test phase, a 6-s gap (the FI keylight was extinguished) was presented at 3, 9, or 15s after probe trial onset. Peak time increased with center key time of onset, and was greater under task than gap conditions. Moreover, peak time under task conditions exceeded values predicted by stop and reset clock mechanisms. These results are at variance with current attentional accounts of timing behavior in dual-task conditions, and suggest a role of nontemporal factors in the control of timing behavior.     

An Assessment of Fixed Interval Timing in Free-Flying Honey Bees (Apis mellifera ligustica): An Analysis of Individual Performance

Interval timing is a key element of foraging theory, models of predator avoidance, and competitive interactions. Although interval timing is well documented in vertebrate species, it is virtually unstudied in invertebrates. In the present experiment, we used free-flying honey bees (Apis mellifera ligustica) as a model for timing behaviors. Subjects were trained to enter a hole in an automated artificial flower to receive a nectar reinforcer (i.e. reward). Responses were continuously reinforced prior to exposure to either a fixed interval (FI) 15-sec, FI 30-sec, FI 60-sec, or FI 120-sec reinforcement schedule. We measured response rate and post-reinforcement pause within each fixed interval trial between reinforcers. Honey bees responded at higher frequencies earlier in the fixed interval suggesting subject responding did not come under traditional forms of temporal control. Response rates were lower during FI conditions compared to performance on continuous reinforcement schedules, and responding was more resistant to extinction when previously reinforced on FI schedules. However, no “scalloped” or “break-and-run” patterns of group or individual responses reinforced on FI schedules were observed; no traditional evidence of temporal control was found. Finally, longer FI schedules eventually caused all subjects to cease returning to the operant chamber indicating subjects did not tolerate the longer FI schedules.     

Interval timing in Siamese fighting fish (Betta splendens)

The present study evaluated the temporal performance of Siamese fighting fish (Betta splendens) given short-term exposure to four fixed interval (FI) schedules of reinforcement, FI 30, 60, 120, and 240 s, during which a reinforcer (mirror image) was given for the first response (swimming through a hoop) after the interval requirement had elapsed. Response levels were generally low early in an interval and increased as the interval elapsed; wait times and break points in an interval increased with increases in the FI requirement. The results were similar to that obtained with other species and different types of responses and reinforcers, and demonstrate that the procedure is a feasible method for studying interval timing in fish.     

Rapid timing of transitions in inter-reinforcement interval duration in infants

Infants trained on FI schedules are able to adjust their post-reinforcement pause, but many sessions at a given parameter value are necessary before the FI cumulative record appears stable. We analysed transitions from one FI value to another from our earlier experiments. Infants were exposed to FI schedules from 10 to 80 s. Under these schedules, they showed long post-reinforcement pauses with most intervals involving the emission of a single, reinforced response. The passage from one FI value to another was unpredictable for infants. We observed rapid timing effects during transitions (passage from one FI value to another, from FI 10 s to FI 20 s, etc.). Wait times increased during the first session of the new FI value, and accurate adjustments of wait times quickly appeared. Dynamic effects observed were similar to those obtained from animals in experiments on rapid-timing effects.     

Dishabituation with component transitions may contribute to the interactions observed during multiple schedules

Rates of responding by rats were usually higher during the variable interval (VI) 30-s component of a multiple VI 30-s fixed interval (FI) 30-s schedule than during the same component of a multiple VI 30-s VI 30-s schedule (Experiment 1). Response rates were also usually higher during the FI 30-s component of a multiple VI 30-s FI 30-s schedule than during the same component of a multiple FI 30-s FI 30-s schedule (Experiment 2). The differences in response rates were not observed when the components provided VI or FI 120-s schedules. These results were predicted by the idea that differences in habituation to the reinforcer between multiple schedules contribute to behavioral interactions, such as behavioral contrast. However, differences in habituation were not apparent in the within-session patterns of responding. Finding differences in response rates in both experiments violates widely-held assumptions about behavioral interactions, including that behavioral contrast does not occur for rats and that improving the conditions of reinforcement decreases, rather than increases, response rate in the alternative component.     

Effects of breaks in the interval cycle on temporal tracking in pigeons

Two experiments examined the effects of inserting a break in a cyclic interval schedule on the temporal control of keypecking responses in pigeons. In Experiment 1, pigeons were exposed to intervals that changed from 45 to 15s and returned to 45 s. A break was inserted between the last 15-s and following 45-s interval and was in effect for either 0, 60, or 120 s. Either a blackout of lights in the test chamber or turning off the response key alone signaled breaks. In Experiment 2, we examined the effects of a wider range of breaks-0, 120, and 360 s. Post-reinforcement pause (PRP) tracked changes in the interval requirement across all conditions. However, breaks in the schedule, even one lasting 360 s, did not disrupt the overall time course of responding. The only effect that a break had on temporal performance was an elevation in the rate of responding and a shorter PRP in the interval following a break. The results suggest that breaks did not affect the birds' memory for short intervals, and that the momentary increase in responding may be related to the reinforcement omission effect.     

Differences in corticosterone level due to inter-food interval length: implications for schedule-induced polydipsia

The purpose of this study was to determine the effects of different food-reinforcement schedules on plasma corticosterone (CORT), and its possible involvement in the acquisition and maintenance of schedule-induced polydipsia (SIP). In Experiment 1, three groups of rats were submitted to two different fixed-interval (FI) schedules with inter-food intervals of 30 and 120 s, and to a massed-feeding presentation for 40 days until SIP was well stabilized. In Experiment 2, six groups of rats were exposed to the same schedules, FI 30s and FI 120s, and to the massed-feeding condition, but no water bottles were presented. CORT levels were determined on Days 3 and 40. Results of Experiment 1 indicated that FI 30s schedule, but not FI 120s or the massed-feeding condition, induces excessive drinking from Day 3. Results in Experiment 2 indicated that CORT levels were similar for all the groups on Day 3. However, only animals on the FI 30s schedule did increase their CORT levels on Day 40, with no variation in the hormone in the other two conditions, FI 120s and massed-feeding presentations. The data are discussed in terms of the implications of these results for hypotheses of SIP as anxiolitic behavior.     

Psychological distance to reward: equating the number of stimulus and response segments

Psychological distance to reward, or the segmentation effect, refers to the preference for a terminal link of a concurrent-chains schedule consisting of a simple reinforcement schedule (e.g. fixed interval [FI] 30s) relative to its chained-schedule counterpart (e.g. chained FI 15s FI 15s). This experiment was conducted to examine whether the segmentation effect is due to the number of terminal-link stimulus and response segments per se. Three pigeons pecked under a concurrent-chains schedule in which identical variable-interval (VI) schedules operated in the initial links. In each session, half the terminal-link entries followed one initial-link key and the other half followed the other initial-link key. The initial-link keys correlated with the different terminal links were manipulated across conditions. In the first three conditions, each terminal link contained a chained fixed-time (FT) FT schedule, and in the final three conditions, each terminal link contained a chained FI FI schedule. In each condition, in one terminal link (alternating), the order of two key colors correlated with the different schedule segments alternated across terminal-link entries, whereas in the other terminal link (constant), the order of two other key colors was identical for each entry. With the chained FT FT schedule terminal links, there was indifference between the alternating and constant terminal links within and across pigeons, as indexed by initial-link choice proportions. In addition, terminal-link response rates were relatively low. With the chained FI FI schedule terminal links, for each pigeon, there was relatively more preference for the alternating terminal link and terminal-link response rates increased relative to conditions with the chained FT FT schedule terminal links. These data suggest that the segmentation effect is not due simply to the number of terminal-link stimulus or response segments per se, but rather to a required period of responding during a stimulus segment that never is paired with reinforcement.     

Increased effort requirements and risk sensitivity: a comparison of delay and magnitude manipulations

Reward magnitude and delay to reward were independently manipulated in two separate experiments examining risk-sensitive choice in rats. A dual-running wheel apparatus was used and the tangential force resistance required to displace both wheels was low (50g) for half of the subjects, and high (120g) for the remaining subjects. Concurrent FI30-s and FI60-s schedules delivered equivalent amounts of food reward per unit time (i.e. 5 and 10 pellets of food, respectively), and these conditions served as the baseline treatment for all subjects. Variability, either in reward magnitude or delay, was introduced on the long-delay (60s) schedule during the second phase. All subjects were returned to the baseline condition in the third phase, and variability was introduced on the short-delay (30s) interval schedule during phase four. The subjects were again returned to the baseline condition in the fifth and final phase, ultimately yielding a five-phase ABACA design. Original baseline performance was characterized by a slight short-delay interval preference, and this pattern of performance was recovered with each subsequent presentation of the baseline condition. Overall, the data obtained from the reward magnitude and delay-to-reward manipulations were indistinguishable; subjects experiencing low-response effort requirement behaved in a risk-indifferent manner and subjects experiencing high-response effort requirement preferred the variable schedule. Implications for the daily energy budget rule on risk-sensitive foraging are discussed in light of these findings.     

Rapid acquisition in concurrent chains: effects of initial-link duration

Pigeons were trained in a concurrent chains procedure in which the terminal-link schedules in each session were either fixed-interval (FI) 10s FI 20s or FI 20s FI 10s, as determined by a pseudorandom binary series. The initial-link was a variable-interval (VI) 10-s schedule. Training continued until initial-link response allocation stabilized about midway through each session and was sensitive to the terminal-link immediacy ratio in that session. The initial-link schedule was then varied across sessions between VI 0.01 s and VI 30s according to an ascending and descending sequence. Initial-link response allocation was a bitonic function over the full range of durations. Preference for the FI 10-s terminal-link at first increased as programmed initial-link duration varied from 0.01 to 7.5s, and then decreased as initial-link duration increased to 30s. The bitonic function poses a potential challenge for existing models for steady-state choice, such as delay-reduction theory (DRT) [Fantino, E., 1969. Choice and rate of reinforcement. J. Exp. Anal. Behav. 12, 723-730], which predict a monotonic function. However, an extension of Grace and McLean's [Grace, R.C., McLean, A.P., 2006. Rapid acquisition in concurrent chains: evidence for a decision model. J. Exp. Anal. Behav. 85, 181-202] decision model predicted the bitonic function, and may ultimately provide an integrated account of choice in concurrent chains under both steady-state and dynamic conditions.     

Effects of unpredictable changes in initial-link duration on choice and timing

Four pigeons responded in a concurrent-chains procedure in which terminal-link schedules were fixed-interval (FI) 10 s and FI 20 s. Across sessions, the location of the shorter terminal-link changed according to a pseudorandom binary sequence. Each session, the variable-interval initial-link schedule value was sampled from a uniform distribution that ranged from 0.01 to 30 s. On some terminal links, food was withheld to obtain measures of temporal control. Terminal-link delays determined choice (log initial-link response ratios) and timing (start and stop times on no-food trials) measures, which stabilized within the 1st half of each session. Preference for the shorter terminal-link delay was a monotonically decreasing function of initial-link duration. There was no evidence of control by initial-link durations from previous sessions.     

Interval timing by an invertebrate, the bumble bee Bombus impatiens

Sensitivity to temporal information and the ability to adjust behavior to the temporal structure of the environment should be phylogenetically widespread. Some timing abilities, such as sensitivity to circadian cycles, appear in a wide range of invertebrate and vertebrate taxa [1,2]. Interval timing--sensitivity to the duration of time intervals--has, however, only been shown to occur in vertebrates [3,4]. Insect pollinators make a variety of decisions that would appear to require the ability to estimate elapsed durations. We exposed bumble bees to conditions in which proboscis extension was reinforced after a fixed duration had elapsed or after either of two fixed durations had elapsed. Two groups of bees were trained with a short duration (either 6 s or 12 s) and a long duration (36 s) in separate experimental phases (independent timing groups), whereas two other groups were trained with a short duration (either 6 s or 12 s) and long duration (36 s) always intermixed unpredictably (multiple timing groups). On long intervals, independent timing groups waited longer than mixed timing groups to generate the first response and responded maximally near the end of the interval. Multiple timing groups waited the same amount of time on average before generating the first response on both long and short intervals. On individual trials, multiple timing groups appeared to time either the long duration only or both the short and long durations: most trials were characterized by a single burst of responding that began between the short and long duration values or by two bursts of responding with the first burst bracketing the short value and the second burst beginning in anticipation of the long value. These results show that bumble bees learn to time interval durations and can flexibly time multiple durations simultaneously. The results indicate no phylogenetic divide between vertebrates and invertebrates in interval timing ability.     

Self-control and impulsiveness with asynchronous presentation of reinforcement schedules

IN DISCRETE TRIALS, PIGEONS WERE PRESENTED WITH TWO ALTERNATIVES: to wait for a larger reinforcer, or to respond and obtain a smaller reinforcer immediately. The choice of the former was defined as self-control, and the choice of the latter as impulsiveness. The stimulus that set the opportunity for an impulsive choice was presented after a set interval from the onset of the stimulus that signaled the waiting period. That interval increased or decreased from session to session so that the opportunity for an impulsive choice became available either more removed from or closer in time to the presentation of the larger reinforcer. In three separate conditions, the larger reinforcer was delivered according to either a fixed interval (FI) schedule, a fixed time (FT) schedule, or a differential reinforcement of other behavior (DRO) schedule. The results showed that impulsive choices increased as the opportunity for such a choice was more distant in time from presentation of the larger reinforcer. Although the schedule of the larger reinforcer affected the rate of response in the waiting period, the responses themselves had no effect on choice unless the responses postponed presentation of the larger reinforcer.     

Choice behavior in spontaneously hypertensive rats: Variable vs. fixed schedules of reinforcement

Spontaneously hypertensive rats (SHR) and Wistar rats were evaluated in the successive-encounters procedure (an operant simulation of natural foraging) with the idea of assessing differences between them in their preference for variable schedules of reinforcement. In this procedure, after satisfying a schedule of reinforcement associated with search time, the subjects could “accept” or “reject” another schedule of reinforcement associated with handling time. Two schedules of reinforcement were available: a fixed interval (FI), and a variable interval (VI) with the same mean value. The results indicated preference for the variable schedule in both strains, as suggested by the observation that the VI was always accepted while the FI was often rejected. The difference in FI acceptability between strains was not statistically significant, a result which is relevant for the current debate of SHR as an adequate animal model of Attention Deficit Hyperactivity Disorder.     

When to respond? And how much? : Temporal control and response output on mixed-fixed-interval schedules with unequally probable components

Rats were trained on mixed-fixed-interval (FI) schedules, with component FIs of 30 and 60s. The probability of reinforcement according to FI 30s varied between conditions, across values of 0.1, 0.3, 0.5, 0.7 and 0.9. When response rate in the 60s intervals was measured, separate response peaks, one close to 30s, the other at 60s, could be identified when the probability of reinforcement at 30s was 0.3 or greater. Nonlinear regression found that the location of the earlier peak was always close to 30s, that the coefficient of variation of the response functions at 30 and 60s were unaffected by reinforcement probability, but that the 30s component appeared to be timed slightly more precisely than the 60s one. Response rate at around 30s increased with increasing probability of reinforcement according to FI 30s, but responding at 60s was unaffected by reinforcement probability. The data are discussed with respect to a number of contemporary models of animal timing (scalar expectancy theory, the Behavioural Theory of Timing and the Learning to Time model), and a recent account of response output on FI-like schedules.     

The perception of empty and filled time intervals by rats

In experiment 1, rats were trained in a within-subjects design to discriminate durations of a filled interval, and durations of an empty interval (an unfilled interval marked at the beginning and end by a 500 ms tone). Training and psychophysical testing was conducted with three sets of anchor durations. Rats made more long responses for filled than for empty intervals at signal durations greater than the geometric mean. In experiment 2, group same was trained similarly to the rats in experiment 1 with the ambient conditions (houselight illumination) remaining the same during the inter-trial interval and the empty intervals. Group different was trained with the houselight turned off during empty and filled intervals. The similarity of ambient conditions during the inter-trial interval and the empty intervals did not significantly affect timing. Filled intervals were timed more precisely and they were perceived as longer than empty intervals of the same duration. The psychophysical functions superimposed across anchor duration sets. These results are the first clear evidence of a filled interval illusion in rats, and they suggest that this difference may reflect a clock rate effect (greater for filled intervals) rather than a switch latency effect (slower for empty intervals).     

A neurocomputational model for optimal temporal processing

Humans can estimate the duration of intervals of time, and psychophysical experiments show that these estimations are subject to timing errors. According to standard theories of timing, these errors increase linearly with the interval to be estimated (Weber's law), and both at longer and shorter intervals, deviations from linearity are reported. This is not easily reconciled with the accumulation of neuronal noise, which would only lead to an increase with the square root of the interval. Here, we offer a neuronal model which explains the form of the error function as a result of a constrained optimization process. The model consists of a number of synfire chains with different transmission times, which project onto a set of readout neurons. We show that an increase in the transmission time corresponds to a superlinear increase of the timing errors. Under the assumption of a fixed chain length, the experimentally observed error function emerges from optimal selection of chains for each given interval. Furthermore, we show how this optimal selection could be implemented by competitive spike-timing dependent plasticity in the connections from the chains to the readout network, and discuss implications of our model on selective temporal learning and possible neural architectures of interval timing.