Contextual determinants of temporal control: Behavioral contrast in a free-operant psychophysical procedure

The question of how temporal control of responding might be influenced by contingency changes in other contexts was investigated. Each of three pigeons first was exposed to a two-component multiple schedule in which a two-key free-operant psychophysical procedure operated in one component and a variable-interval schedule operated in the other component. The variable-interval schedule then was changed to extinction while the free-operant psychophysical procedure remained unchanged. Finally, the variable-interval schedule was reintroduced. Response rates on the left key and the estimated temporal threshold under the free-operant psychophysical procedure increased for each pigeon when the alternate component schedule was changed to extinction and then decreased again when the variable-interval schedule was reintroduced. The results suggest one way that temporal control is affected by its context, and may be interpreted through the direct effects of overall reinforcement rate on temporal control mechanisms or the disruptive effects of alternative sources of reinforcement on temporally controlled behavior.     

Species differences between rats and pigeons in choices with probabilistic and delayed reinforcers

An adjusting-delay procedure was used to study rats' choices with probabilistic and delayed reinforcers, and to compare them with previous results from pigeons. A left lever press led to a 5-s delay signaled by a light and a tone, followed by a food pellet on 50% of the trials. A right lever press led to an adjusting delay signaled by a light followed by a food pellet on 100% of the trials. In some conditions, the light and tone for the probabilistic reinforcer were present only on trials that delivered food. In other conditions, the light and tone were present on all trials that the left lever was chosen. Similar studies with pigeons [Mazur, J.E., 1989. Theories of probabilistic reinforcement. J. Exp. Anal. Behav. 51, 87-99; Mazur, J.E., 1991. Conditioned reinforcement and choice with delayed and uncertain primary reinforcers. J. Exp. Anal. Behav. 63, 139-150] found that choice of the probabilistic reinforcer increased dramatically when the delay-interval stimuli were omitted on no-food trials, but this study found no such effect with the rats. In other conditions, the probability of food was varied, and comparisons to previous studies with pigeons indicated that rats showed greater sensitivity to decreasing reinforcer probabilities. The results support the hypothesis that rats' choices in these situations depend on the total time between a choice response and a reinforcer, whereas pigeons' choices are strongly influenced by the presence of delay-interval stimuli.     

Spatial associative memory: a possible species difference in rats and pigeons

Previous research has shown that pigeons can remember which of four spatially distinct responses was last reinforced, for at least 72 h. The present study sought to replicate this finding using rats. Rats were tested in an operant chamber containing four spatially distinct levers. In each session one lever was randomly selected to provide reinforcement for 15 min. This reinforced period was preceded by a non-reinforced period that was 30 s long, on average. During the non-reinforced period the amount the rat pressed on the previously reinforced lever was compared to responding on the other three levers, and was taken as a measure of memory. Sessions were separated either by 17 min, 24 or 72 h. Unlike pigeons, rats responded at chance levels following each of these retention intervals. This finding adds to previous research suggesting differences in cognitive processes in rats and pigeons.     

Progesterone attenuates impulsive action in a Go/No-Go task for sucrose pellets in female and male rats

Impulsivity, or a tendency to act without anticipation of future consequences, is associated with drug abuse. Impulsivity is typically separated into two main measures, impulsive action and impulsive choice. Given the association of impulsivity and drug abuse, treatments that reduce impulsivity have been proposed as an effective method for countering drug addiction. Progesterone has emerged as a promising treatment, as it is associated with decreased addiction-related behaviors and impulsive action. The goal of the present study was to determine the effects of progesterone (PRO) on impulsive action for food: a Go/No-Go task. Female and male rats responded for sucrose pellets during a Go component when lever pressing was reinforced on a variable-interval 30-s schedule. During the alternate No-Go component, withholding a lever press was reinforced on a differential reinforcement of other (DRO) behavior 30-s schedule, where a lever press reset the DRO timer. Impulsive action was operationally defined as the inability to withhold a response during the No-Go component (i.e. the number of DRO resets). Once Go/No-Go behavior was stable, responding between rats treated with PRO (0.5 mg/kg) or vehicle was examined. Progesterone significantly decreased the total number of DRO resets in both males and females, but it did not affect VI responding for sucrose pellets. This suggests that PRO decreases motor impulsivity for sucrose pellets without affecting motivation for food. Thus, PRO may reduce motor impulsivity, a behavior underlying drug addiction.     

Comparison of Scalar Expectancy Theory (SET) and the Learning-to-Time (LeT) model in a successive temporal bisection task

The present research tested the generality of the "context effect" previously reported in experiments using temporal double bisection tasks [e.g., Arantes, J., Machado, A. Context effects in a temporal discrimination task: Further tests of the Scalar Expectancy Theory and Learning-to-Time models. J. Exp. Anal. Behav., in press]. Pigeons learned two temporal discriminations in which all the stimuli appear successively: 1s (red) vs. 4s (green) and 4s (blue) vs. 16s (yellow). Then, two tests were conducted to compare predictions of two timing models, Scalar Expectancy Theory (SET) and the Learning-to-Time (LeT) model. In one test, two psychometric functions were obtained by presenting pigeons with intermediate signal durations (1-4s and 4-16s). Results were mixed. In the critical test, pigeons were exposed to signals ranging from 1 to 16s and followed by the green or the blue key. Whereas SET predicted that the relative response rate to each of these keys should be independent of the signal duration, LeT predicted that the relative response rate to the green key (compared with the blue key) should increase with the signal duration. Results were consistent with LeT's predictions, showing that the context effect is obtained even when subjects do not need to make a choice between two keys presented simultaneously.     

Disruptive effects of prefeeding and haloperidol administration on multiple measures of food-maintained behavior in rats

Four rats responded under a choice reaction-time procedure. At the beginning of each trial, the rats were required to hold down a center lever for a variable duration, release it following a high- or low-pitched tone, and press either a left or right lever, conditionally on the tone. Correct choices were reinforced with a probability of .95 or .05 under blinking or static houselights, respectively. After performance stabilized, disruptive effects of free access to food pellets prior to sessions (prefeeding) and intraperitoneal injection of haloperidol were examined on multiple behavioral measures (i.e., the number of trials completed, percent of correct responses, and reaction time). Resistance to prefeeding depended on the probability of food delivery for the number of trials completed and reaction time. Resistance to haloperidol, on the other hand, was not systematically affected by the probability of food delivery for all dependent measures.     

60 Hz electric fields and incandescent light as aversive stimuli controlling the behavior of rats responding under concurrent schedules of reinforcement

Several reports have shown that animals will sometimes engage in behaviors that reduce their exposure to a 60 Hz electric field (E-field). The field, therefore, can function as an aversive stimulus. In other studies, the E-field at equivalent strengths failed to function as an aversive stimulus. The present experiment, using rats, demonstrates how factors other than field strength can influence whether a subject engages in behavior that reduces field exposure. The general design consisted of giving the rat a choice between two alternatives, one of which sometimes included an added stimulus. Each subject was trained to press each of two levers to obtain food. Pressing one lever was reinforced intermittently under a variable interval 2 min schedule (VI 2); pressing the other lever was reinforced by a second VI 2 schedule operating independently of the first. Under this concurrent schedule the rat spent 50% of the daily 50 min session responding to each of the levers, indicating that they were equally “valued.” Next, while the schedules remained in effect, the first response to one of the levers turned on a 100 kV/m E-field which remained on until the rat pressed the other lever. The time spent responding under the schedule associated with the field was reduced by about 5–10%. When the procedure was changed so that no lever presses produced food, i.e., extinction, but the added stimulus contingency remained, the rats spent even less time in the presence of the field. Similar outcomes were observed during both the concurrent food or extinction schedules when incandescent light was used. Thus, both an E-field and incandescent light functioned as aversive stimuli, but the magnitude of the aversiveness was small. Aversiveness depended not only on stimulus intensity, but also on behavioral factors. Bioelectromagnetics 19:210–221, 1998. © 1998 Wiley-Liss, Inc.     

Further tests of the Scalar Expectancy Theory (SET) and the Learning-to-Time (LeT) model in a temporal bisection task

To contrast two models of timing, Scalar Expectancy Theory (SET) and Learning to Time (LeT), pigeons were exposed to a double temporal bisection procedure. On half of the trials, they learned to choose a red key after a 1s signal and a green key after a 4s signal; on the other half of the trials, they learned to choose a blue key after a 4-s signal and a yellow key after a 16-s signal. This was Phase A of an ABA design. On Phase B, the pigeons were divided into two groups and exposed to a new bisection task in which the signals ranged from 1 to 16s and the choice keys were blue and green. One group was reinforced for choosing blue after 1-s signals and green after 16-s signals and the other group was reinforced for the opposite mapping (green after 1-s signals and blue after 16-s signals). Whereas SET predicted no differences between the groups, LeT predicted that the former group would learn the new discrimination faster than the latter group. The results were consistent with LeT. Finally, the pigeons returned to Phase A. Only LeT made specific predictions regarding the reacquisition of the four temporal discriminations. These predictions were only partly consistent with the results.     

Differential resurgence and response elimination

Resurgence refers to the transient recovery of previously reinforced, but presently not reinforced, responding when more recently reinforced responding is extinguished. The primary purpose of our research was to determine how differential resurgence results from the procedures used to eliminate that responding. There were three conditions in each of five experiments. In Condition 1, key pecking by pigeons was maintained under a two-component multiple variable-interval (VI) 30-s VI 30-s schedule. In Condition 2, this pecking was eliminated in different ways across components. In Condition 3, extinction was in effect for all responses, and resurgence of key pecking was compared across components. These three conditions were repeated for most pigeons, and the procedures used to eliminate responding in Condition 2 varied across experiments. In Experiment 1, there was greater resurgence, and an earlier onset of it, after a differential-reinforcement-of-other-behavior (DRO) schedule than after a VI schedule was correlated with pecking an alternative key. Experiments 2 and 3 showed that the differential resurgence in Experiment 1 probably was not due to conditional stimulus control or the periodicity of food delivery, respectively. In Experiment 4, there was no systematic difference in resurgence after either a DRO schedule or a VI schedule correlated with treadle pressing. In Experiment 5, there was greater resurgence, and/or an earlier onset of it, after a VI schedule correlated with treadle pressing than after a VI schedule correlated with pecking an alternative key. Taken together, the results showed that the reinforcement of an alternative key-peck response was the most effective means of reducing subsequent key-peck resurgence. The relation of these results to an understanding of resurgence is discussed.     

Discriminative stimulus properties of L-phenylisopropyl adenosine: blockade by caffeine and generalization to 2-chloroadenosine

Recent neurochemical data on the effects of activation and blockade of adenosine A1 receptors has suggested a direct role of adenosine in neurotransmission. The present research used a drug discrimination procedure to test the hypotheses that A1 adenosine receptor activation could serve as a discriminative stimulus and that caffeine, a drug believed to be an A1 receptor antagonist, could block the adenosine discrimination. Food-deprived rats were trained to press one of two levers on an FR 10 schedule of food-pellet delivery. Responses on one lever were reinforced following i.p. injection of N6 - (L-phenylisopropyl) adenosine (L-PIA); responses on the other lever were reinforced following i.p. injection of saline. L-PIA training dose was increased from 0.064 to 0.08 mg/kg L-PIA in the course of the study. Subjects required an average of 91 sessions to acquire this discrimination. Stimulus control by L-PIA was dose-dependent, with the ED-50 being approximately 0.03 mg/kg. 2-Chloroadenosine (2CA) generalized to L-PIA with a tenth the potency. Caffeine blocked L-PIA-induced lever selection. These results indicate that 1) rats can be trained to discriminate L-PIA from saline in a two-lever food-reinforced task and 2) the discriminative stimuli produced by L-PIA are based on its agonistic action at the adenosine A1 receptor.     

Comparison of the instrumental reinforcer devaluation effect in two strains of rats (Wistar and Lister)

In this experiment, the effect of the reinforcer devaluation upon instrumental performance was analysed in two strains of rats (Wistar and Lister): Food deprived rats were trained to press a lever for sucrose pellets in a single session. Immediately after the fulfilment of this session, half of the Wistar and Lister rats received an injection of lithium chloride (LiCl), while the remaining animals were injected with a sodium chloride (NaCl) solution. A subsequent extinction test showed that the subjects who had received immediate LiCl did not press the lever as often as those injected with NaCl. No differences in response suppression were found between the two strains of rats. These results also show that a single devaluation experience is sufficient for an impact on instrumental performance.     

Context effect in a temporal bisection task with the choice keys available during the sample

In a symbolic matching to sample task, six pigeons learned to associate different sample durations with different comparison stimuli. On “Short” trials, choice of Red and Green keylight comparisons were reinforced following 3-s and 9-s samples, respectively; on “Long” trials, Blue and Yellow keylight comparisons were reinforced following 9-s and 27-s samples, respectively. In contrast with previous studies, the comparison keys were available during the samples. After the temporal discriminations were learned, new pairs of comparison keys were presented and the preference for each was assessed during 27-s samples. One pair in particular, Green and Blue, was critical because it tested the predictions of two timing models, Scalar Expectancy Theory (SET) and the Learning-to-Time (LeT) model. The results showed that preference for Green increased during the sample, a result consistent with LeT but not with SET. Other test results, however, were predicted by neither model.     

Appetitive latent inhibition in rats: preexposure performance does not predict conditioned performance

Nonreinforced preexposure to a conditioned stimulus impairs subsequent conditioning with that stimulus. The goal of these studies was to assess the extent to which acquisition performance could be predicted from preexposure performance using a correlational approach. For both preexposure and autoshaping, four measures of performance were computed, including overall average lever pressing, lever pressing in the initial session, percentage change in lever pressing, and slopes. These measures were correlated in a large sample of rats trained in an autoshaping situation. None of the three measures of autoshaping performance was consistently predicted by any of the three measures of preexposure performance. These results are consistent with the view that latent inhibition is not reducible to long-term habituation.     

Recovery of Pavlovian sign-tracking (autoshaping) following the discontinuation of inter-trial interval food in rats

In pigeons, Pavlovian autoshaped keypecking produced by keylight-food pairings has been eliminated by introducing food during periods between CS presentations (i.e., during the inter-trial intervals). Keypecking eliminated in this manner reappears when the inter-trial USs are discontinued even though the CS is no longer paired with US. The present experiment investigated whether this recovery of responding produced by discontinuing unpaired inter-trial US presentations could be extended to another species, rats, within a Pavlovian sign-tracking paradigm. Rats were initially trained on a procedure where insertion of one retractable lever (CS(+)) was followed, response independently, with food, while insertion of another lever (CS(-)) was not paired with food. Rats quickly came to contact the CS(+) lever at high rates, but contacted the CS(-) lever infrequently. In the next phase, CS(+) was no longer followed by food. Explicitly unpaired food was presented only during the inter-trial intervals when both levers were absent. This treatment essentially eliminated the sign-tracking response. In the final phase, the unpaired inter-trial food presentations were discontinued while both CSs continued to be presented without food. This produced a significant recovery of the sign-tracking elicited by the CS(+) lever, extending the species generality of the Pavlovian resurgence phenomenon that has previously only been reported in pigeons, to rats.     

Associative symmetry and stimulus-class formation by pigeons: The role of non-reinforced baseline relations

Two experiments tested the assumption of Urcuioli's (2008) theory of pigeons’ equivalence-class formation that consistent non-reinforcement of certain stimulus combinations in successive matching juxtaposed with consistent reinforcement of other combinations generates stimulus classes containing the elements of the reinforced combinations. In Experiment 1, pigeons were concurrently trained on symbolic (AB) and two identity (AA and BB) successive tasks in which half of all identity trials ended in non-reinforcement but all AB trials were reinforced, contingent upon either responding or not responding to the comparisons. Subsequent symmetry (BA) probe trials showed evidence of symmetry in one of four pigeons. In Experiment 2, pigeons learned three pair-comparison tasks in which left versus right spatial choices were reinforced after the various sample-comparison combinations comprising AB, AA, and BB conditional discriminations. Non-differentially reinforced BA probe trials following acquisition showed some indication of symmetrical choice responding. The overall results contradict the theoretical predictions derived from Urcuioli (2008) and those from Experiment 2 challenge other stimulus-class analyses as well.     

Rats' memory for event duration: differential effects of delaying the discriminative choice cue as opposed to the opportunity to execute the choice response

Rats were trained to discriminate short or long durations of houselight illumination using a choice procedure. During the test phase of each trial, the left and right levers were presented with an auditory cue above one of them on (cued lever) while the other was off (uncued lever). The auditory cue was presented immediately after sample offset and the levers were inserted after the auditory cue had been presented for 2 s. For half of the rats, the correct response following the short sample was to press the cued lever, while following the long sample, it was to press the uncued lever. This was reversed for the remaining rats. Following acquisition of the discrimination, two different types of delay tests were administered. In the first set, the delay between offset of the sample and onset of the auditory cue was manipulated (Cue Delay Test). In the second set, the delay between onset of the auditory cue and entry of the levers into the chamber was manipulated (Response Delay Test). Cue Delay testing resulted in a choose-long bias at the longer delays. Response Delay testing did not result in a systematic response bias and there was little forgetting over the delay interval. These data suggest that the rats did not stop the internal clock when the nominal sample was offset, but allowed it to keep running until the auditory cue was presented. The data from the Response Delay Test indicate that either a response decision was made based on the clock reading as soon as the auditory cue was presented, or the clock reading itself was retained over the delay with no subjective shortening and little forgetting.     

Passive immunization against nicotine attenuates nicotine discrimination

Ten rats were trained in a two lever operant chamber to press different levers after a nicotine injection (0.14 mg/kg s.c.) or a saline injection on an FR10 schedule. The rats were then injected i.p. with either 150 mg nicotine-specific IgG or the same amount of control IgG from non-immunized rabbits. On successive days, they were retested with both levers active after a saline injection, a full training dose of nicotine and a half dose of nicotine (0.07 mg/kg s.c.). After saline injection, both groups pressed the saline lever almost exclusively. After each of the nicotine doses, the immunized rats performed a significantly lower percentage of their lever presses on the nicotine lever than did non-immunized rats. The results suggest that passive immunization can interfere with the stimulus properties of nicotine.     

Ethanol seeking by long evans rats is not always a goal-directed behavior

Background

Two parallel and interacting processes are said to underlie animal behavior, whereby learning and performance of a behavior is at first via conscious and deliberate (goal-directed) processes, but after initial acquisition, the behavior can become automatic and stimulus-elicited (habitual). With respect to instrumental behaviors, animal learning studies suggest that the duration of training and the action-outcome contingency are two factors involved in the emergence of habitual seeking of “natural” reinforcers (e.g., sweet solutions, food or sucrose pellets). To rigorously test whether behaviors reinforced by abused substances such as ethanol, in particular, similarly become habitual was the primary aim of this study.

Methodology/Principal Findings

Male Long Evans rats underwent extended or limited operant lever press training with 10% sucrose/10% ethanol (10S10E) reinforcement (variable interval (VI) or (VR) ratio schedule of reinforcement), or with 10% sucrose (10S) reinforcement (VI schedule only). Once training and pretesting were complete, the impact of outcome devaluation on operant behavior was evaluated after lithium chloride injections were paired with the reinforcer, or unpaired 24 hours later. After limited, but not extended instrumental training, lever pressing by groups trained under VR with 10S10E and under VI with 10S was sensitive to outcome devaluation. In contrast, responding by both the extended and limited training 10S10E VI groups was not sensitive to ethanol devaluation during the test for habitual behavior.

Conclusions/Significance

Operant behavior by rats trained to self-administer an ethanol-sucrose solution showed variable sensitivity to a change in the value of ethanol, with relative insensitivity developing sooner in animals that received time-variable ethanol reinforcement during training sessions. One important implication, with respect to substance abuse in humans, is that initial learning about the relationship between instrumental actions and the opportunity to consume ethanol-containing drinks can influence the time course for the development or expression of habitual ethanol seeking behavior.     

The role of contextual conditioning in the effect of reinforcer devaluation on instrumental performance by rats

Different groups of rats received different amounts of training to lever press for a food reinforcer before an aversion was conditioned to the food. This devaluation of the reinforcer reduced responding in both subsequent extinction and reinforced tests of responding to a degree that was independent of the amount of instrumental training. Moreover, interpolating context extinction between aversion conditioning and the extinction test reduced the magnitude of the devaluation effect, thereby indicating that Pavlovian contextual conditioning may play a role in the instrumental devaluation effect.     

Delayed matching-to-position performance in C57BL/6N mice

Delayed matching-to-sample is one of the most frequently employed behavioral tasks for assessing spatial working memory in animals. Although the advantages of the task have been widely acknowledged and it is used in the study of a variety of species, its application to mice has been rare. In the present study, we reported the efficacy of a delayed matching-to-position task in C57BL mice lever-pressing in an operant-conditioning chamber. Each trial started with the press of a back lever, followed by the presentation of either a left or right front lever. When the ratio requirement for presses to the front lever (sample) was met, a delay interval started. Delay interval continued until the mice made the first response after the elapse of the programmed delay interval. This was followed by the presentation of a choice of left or right front levers. The choice of the same front lever as the sample was reinforced, whereas the other was not. The proportion of correct choices showed a delay-dependent decrement. A higher ratio of response requirement to the sample resulted in increased accuracy, but the duration of the intertrial interval had no effect. Preceding trials also influenced response accuracy, indicating proactive interference. Overall, the results replicated the effects of parametric manipulations reported in other species, and thus, our findings validate the efficacy of the task for assessing spatial working memory in laboratory mice.