A comparison of the effects of discriminative and Pavlovian inhibitors and excitors on instrumental responding

In two experiments, the effects of Pavlovian or discriminative conditioned inhibitors on operant responding were investigated in rats. Experiment 1 found that a Pavlovian conditioned inhibitor for food suppressed food-reinforced lever pressing more than a non-differentially trained control stimulus did. Experiment 2 demonstrated that an operant discriminative inhibitor produced greater suppression of lever pressing than a Pavlovian conditioned inhibitor. Experiment 2 also found that compounding an operant discriminative stimulus (SD) for food-reinforced responding with another SD for food-reinforced responding resulted in more additive summation than when an SD was compounded with a Pavlovian conditioned excitor for food. The results of these experiments support two-factor theories that postulate that incentive and response discriminative processes summate algebraically when the processes are inhibitory or excitatory.     

Neurochemical changes associated with schedule-controlled behavior.

Studies of monoamine metabolites within the cerebrospinal fluid compartment have indicated that this approach may be useful in examining central metabolic changes in vivo. By combining the technologies of radioisotope chemistry, operant behavior control and modification, and brain perfusion with push-pull cannulas, we have been able to examine minute to minute changes in the disposition of radiolabeled monoamine transmitter candidates and their metabolites. These substances appear to co-vary with changes in complex behavior maintained by operant schedules of reinforcement and affected by changes in schedules or administration of psychotropic drugs. In agreement with other perfusion studies, we have observed changes in fractional distribution of radiolabeled urea, a so-called extracellular marker, along with shifts in monoamines; but the former appear more transient. These observations nevertheless support the concept of dynamic changes within the extracellular environment of the CNS that may be part of a hormone-like communicating system with functional significance. Furthermore, the presence of peaks and/or troughs, in perfusates of [14C]urea of similar substances should not be taken as a priori evidence for nonspecificity of the technic, since selective release or inhibition of release of monoamines can be shown with appropriate drugs that are thought to act through these aminergic systems. Destruction of catecholamine nerve terminals with 6-hydroxydopamine likewise attenuates the signal-locked release of radiolabeled norepinephrine by a conditioned stimulus after conditioning occurs. No such release is seen on presentation of the to-be-conditioned neutral stimulus in control or 6-hydroxydopamine treated rats. These initial studies indicate the availability of a powerful tool for the study of drug-neurochemical-behavioral interactions using subjects as their own controls for extended periods of time so that phenomena of plasticity, tolerance and dependence may likewise be examined.     

The Effects of Environmental Control on Cognition in Rats (Rattus norvegicus)

The aim of this study was to allow nonhuman animals to control their environment using operant conditioning procedures and to assess the effect of control on cognitive tasks. The study tested 4 predictions: (a) rats (Rattus norvegicus) will control a light stimulus; (b) animals will exhibit preferences for particular stimulus strengths; (c) animals who exert control over environmental stimuli will show improved performance on cognitive tasks compared with animals who lack control; and (d) at the end of the operant phase, experimental subjects will have lower corticosterone levels than animals who lack control. Experimental subjects did show control over a light stimulus and performed significantly better over time in a discrimination task compared with subjects who could not control their environment. There was no difference in corticosterone levels between control and experimental subjects. The results will both contribute to our understanding of how control of environmental stimuli affects the welfare of animals in captive environments and aid in designing experimental conditions that will increase validity and reliability in research.     

Phencyclidine-like behavioral effects in pigeons induced by systemic administration of the excitatory amino acid antagonist, 2-amino-5-phosphonovalerate

A selective N-methyl-D-aspartate antagonist, DL-2-amino-5-phosphonovalerate, was found to produce PCP-like catalepsy, discriminative stimulus effects, and stereotyped operant responding in pigeons when administered intramuscularly. These results support the hypothesis that the behavioral effects of PCP-like drugs result at least in part from reduced neurotransmission at excitatory amino acid synapses utilizing N-methyl-D-aspartate preferring receptors.     

Whisking as a “voluntary” response: operant control of whisking parameters and effects of whisker denervation

The rat’s ability to vary its whisking “strategies” to meet the functional demands of a discriminative task suggests that whisking may be characterized as a “voluntary” behavior—an operant—and like other operants, should be modifiable by appropriate manipulations of response–reinforcer contingencies. To test this hypothesis we have used high-resolution, optoelectronic “real-time” recording procedures to monitor the movements of individual whiskers and reinforce specific movement parameters (amplitude, frequency). In one operant paradigm (N = 9) whisks with protractions above a specified amplitude were reinforced (Variable Interval 30?s) in the presence of a tone, but extinguished (EXT) in its absence. In a second paradigm (N = 3), rats were reinforced on two different VI schedules (VI-20s/VI-120s) signaled, respectively, by the presence or absence of the tone. Selective reinforcement of whisking movements maintained the behavior over many weeks of testing and brought it under stimulus and schedule control. Subjects in the first paradigm learned to increase responding in the presence of the tone and inhibit responding in its absence. In the second paradigm, subjects whisked at significantly different rates in the two stimulus conditions. Bilateral deafferentation of the whisker pad did not impair conditioned whisking or disrupt discrimination behavior. Our results confirm the hypothesis that rodent whisking has many of the properties of an operant response. The ability to bring whisking movement parameters under operant control should facilitate electrophysiological and lesion/behavioral studies of this widely used “model” sensorimotor system.     

The effects of environmental control on cognition in rats (Rattus norvegicus)

The aim of this study was to allow nonhuman animals to control their environment using operant conditioning procedures and to assess the effect of control on cognitive tasks. The study tested 4 predictions: (a) rats (Rattus norvegicus) will control a light stimulus; (b) animals will exhibit preferences for particular stimulus strengths; (c) animals who exert control over environmental stimuli will show improved performance on cognitive tasks compared with animals who lack control; and (d) at the end of the operant phase, experimental subjects will have lower corticosterone levels than animals who lack control. Experimental subjects did show control over a light stimulus and performed significantly better over time in a discrimination task compared with subjects who could not control their environment. There was no difference in corticosterone levels between control and experimental subjects. The results will both contribute to our understanding of how control of environmental stimuli affects the welfare of animals in captive environments and aid in designing experimental conditions that will increase validity and reliability in research.     

Whisking as a "voluntary" response: operant control of whisking parameters and effects of whisker denervation

The rat's ability to vary its whisking "strategies" to meet the functional demands of a discriminative task suggests that whisking may be characterized as a "voluntary" behavior--an operant--and like other operants, should be modifiable by appropriate manipulations of response-reinforcer contingencies. To test this hypothesis we have used high-resolution, optoelectronic "real-time" recording procedures to monitor the movements of individual whiskers and reinforce specific movement parameters (amplitude, frequency). In one operant paradigm (N = 9) whisks with protractions above a specified amplitude were reinforced (Variable Interval 30 s) in the presence of a tone, but extinguished (EXT) in its absence. In a second paradigm (N = 3), rats were reinforced on two different VI schedules (VI-20s/VI-120s) signaled, respectively, by the presence or absence of the tone. Selective reinforcement of whisking movements maintained the behavior over many weeks of testing and brought it under stimulus and schedule control. Subjects in the first paradigm learned to increase responding in the presence of the tone and inhibit responding in its absence. In the second paradigm, subjects whisked at significantly different rates in the two stimulus conditions. Bilateral deafferentation of the whisker pad did not impair conditioned whisking or disrupt discrimination behavior. Our results confirm the hypothesis that rodent whisking has many of the properties of an operant response. The ability to bring whisking movement parameters under operant control should facilitate electrophysiological and lesion/behavioral studies of this widely used "model" sensorimotor system.     

Locomotor activity in a novel environment predicts both responding for a visual stimulus and self-administration of a low dose of methamphetamine in rats

There is evidence that visual stimuli used to signal drug delivery in self-administration procedures have primary reinforcing properties, and that drugs of abuse enhance the reinforcing properties of such stimuli. Here, we explored the relationships between locomotor activity, responding for a visual stimulus, and self-administration of methamphetamine (METH). Rats were classified as high or low responders based on activity levels in a novel locomotor chamber and were subsequently tested for responding to produce a visual stimulus followed by self-administration of a low dose of METH (0.025 mg/kg/infusion) paired with the visual stimulus. High responder rats responded more for the visual stimulus than low responder rats indicating that the visual stimulus was reinforcing and that operant responding for a visual stimulus has commonalities with locomotor activity in a novel environment. Similarly, high responder rats responded more for METH paired with a visual stimulus than low responder rats. Because of the reinforcing properties of the visual stimulus, it was not possible to determine if the rats were responding to produce the visual stimulus, METH or the combination. We speculate that responding to produce sensory reinforcers may be a measure of sensation seeking. These results indicate that visual stimuli have unconditioned reinforcing effects which may have a significant role in acquisition of drug self-administration, a role that is not yet well understood.     

Disentangling the nature of the nicotine stimulus

Learning involving interoceptive stimuli likely plays an important role in many diseases and psychopathologies. Within this area, there has been extensive research investigating the interoceptive stimulus effects of abused drugs. In this pursuit, behavioral pharmacologists have taken advantage of what is known about learning processes and adapted the techniques to investigate the behavioral and receptor mechanisms of drug stimuli. Of particular interest is the nicotine stimulus and the use of the two-lever operant drug discrimination task and the Pavlovian drug discriminated goal-tracking task. There is strong concordance between the two methods when using "standard" testing protocols that minimize learning on test days. For example, ABT-418, nornicotine, and varenicline all fully evoked nicotine-appropriate responding. Notably, research from our laboratory with the discriminated goal-tracking task has used an alternative testing protocol. This protocol assesses stimulus substitution based on how well extinction learning using a non-nicotine ligand transfers back to the nicotine stimulus. These findings challenge conclusions based on more "standard" testing procedures (e.g., ABT-418 is not nicotine-like). As a starting point, we propose Thurstone scaling as a quantitative method for more precisely comparing transfer of extinction across doses, experiments, and investigators. We close with a discussion of future research directions and potential implications of the research for understanding interoceptive stimuli.     

Performance of mice in an automated olfactometer: odor detection, discrimination and odor memory

Mice were trained on a variety of odor detection and discrimination tasks in 100- or 200-trial sessions using a go, no-go discrete trials operant conditioning procedure. Odors, presented for 1 s on each trial, were generated by an air dilution olfactometer (for threshold tests) and an easily constructed eight-channel liquid dilution unit (for two- and multiple-odor discrimination tasks). Mice rapidly acquired the operant task and demonstrated excellent stimulus control by odor vapors. Their absolute detection threshold for ethyl acetate was similar to that obtained with rats using similar methods. They readily acquired four separate two-odor discrimination tasks and continued to perform well when all eight odors were presented in random order in the same session and when reinforcement probability for correct responding was decreased from 1 to 0.5. Memory for these eight odors, assessed under extinction after a 32 day rest period, was essentially perfect. Time spent sampling the odor on S+ and S- trials was highly correlated with response accuracy. When accuracy was at chance levels (e.g. initial trials on a novel task), stimulus sampling time on both S+ and S- trials was approximately 0.5-0.7 s. As response accuracy increased, sampling time on S+ trials tended to increase and remain higher than sampling time on S- trials.     

Varying temporal placement during CS of an added stimulus correlated with non-delivery of UCS.

This experiment extends Pavlov's method of contrasts for training a stimulus discrimination to the case of the cardiac conditional response in the rhesus monkey. It explores the parameter of temporal placement of an additional stimulus ("CS2") within a 10-sec CS (or "CS1"), with the appearance of the former stimulus on any trial signalling the absence of UCS (electric shock) on that trial. This experimental paradigm is a parallel to that of the "intruded stimulus" studies in operant conditioning. In both cases, several ways of describing the function of the added stimulus are possible, but all seem reducible to the same operational terms. Data were taken in the present study with respect to the form and latency of the cardiac rate changes produced by intrusion of CS2 (light), across a range of placements varying from simultaneity with CS1 (a different light) onset to two sec before UCS would have been delivered. The control of CS2 over the cardiac rate CR was occasionally exhibited with a latency as short as three beats after stimulus onset. The order of CS2 temporal placements to which a subject was exposed was a factor in determining the form of the conditioned cardiac rate response to CS1.     

Quantifying Social Motivation in Mice Using Operant Conditioning

In this protocol, social motivation is measured in mice through a pair of operant conditioning paradigms. To conduct the experiments, two-chambered shuttle boxes were equipped with two operant levers (left and right) and a food receptacle in one chamber, which was then divided from the second chamber by an automated guillotine door covered by a wire grid. Different stimulus mice, rotated across testing days, served as a social stimulus behind the wire grid, and were only visible following the opening of the guillotine door. Test mice were trained to lever press in order to open the door and gain access to the stimulus partner for 15 sec. The number of lever presses required to obtain the social reward progressively increased on a fixed schedule of 3. Testing sessions ended after test mice stopped lever pressing for 5 consecutive minutes. The last reinforced ratio or breakpoint can be used as a quantitative measure of social motivation. For the second paradigm, test mice were trained to discriminate between left and right lever presses in order to obtain either a food reward or the social reward. Mice were rewarded for every 3 presses of each respective lever. The number of food and social rewards can be compared as a measurement of the value placed upon each reward. The ratio of each reward type can also be compared between mouse strains and the change in this ratio can be monitored within testing sessions to measure satiation with a given reward type. Both of these operant conditioning paradigms are highly useful for the quantification of social motivation in mouse models of autism and other disorders of social behavior.     

The Relationship Between Foraging,Learning Abilities and Neophobia in Two Species of Darwin’s Finches

The ability to unlearn a previously established association is an important component of behavioural flexibility and may vary according to species ecology. Previously, two closely related sympatric Darwin’s finches were found to differ in their learning abilities. Small tree finches (Camarhynchus parvulus) outperformed woodpecker finches (Cactospiza pallida) in reversal learning but performed worse in an operant task. We attributed this difference to the habit of woodpecker finches to engage in long bouts of energetic pecking during extractive foraging. Persistently repeating one action without reward could favour performance in operant tasks but also limit behavioural flexibility. Here, we tested whether perseverance is the reason for woodpecker finches’ depressed reversal learning performance. Two new reversal conditions allowed the disentanglement of two sources of error in reversal learning: perseverant choice of the previously rewarded stimulus and failure to respond to the previously non‐rewarded stimulus. For the within‐species comparison, we predicted that woodpecker finches should find it more difficult to learn to avoid the previously rewarded stimulus than learning to choose the previously non‐rewarded stimulus. For the species comparison, we predicted the woodpecker finches should make more errors of perseverance than small tree finches. As performance could also be influenced by reaction to novelty, we compared neophobic responses between species and related them to reversal learning proficiency. We found no significant difference in reversal learning in the predicted direction, but found a negative correlation between neophobia and reversal learning at the inter‐ and the intraspecific level, which points towards a general relationship between reaction to novelty and flexibility.     

Extinction, relapse, and behavioral momentum

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a discriminative stimulus depends upon the baseline rate or magnitude of reinforcement associated with that stimulus (i.e., the Pavlovian stimulus-reinforcer relation). Recently, we have shown that relapse of operant behavior in reinstatement, resurgence, and context renewal preparations also is a function of baseline stimulus-reinforcer relations. In this paper we present new data examining the role of baseline stimulus-reinforcer relations on resistance to extinction and relapse using a variety of baseline training conditions and relapse operations. Furthermore, we evaluate the adequacy of a behavioral momentum based model in accounting for the results. The model suggests that relapse occurs as a result of a decrease in the disruptive impact of extinction precipitated by a change in circumstances associated with extinction, and that the degree of relapse is a function of the pre-extinction baseline Pavlovian stimulus-reinforcer relation. Across experiments, relative resistance to extinction and relapse were greater in the presence of stimuli associated with more favorable conditions of reinforcement and were positively related to one another. In addition, the model did a good job in accounting for these effects. Thus, behavioral momentum theory may provide a useful quantitative approach for characterizing how differential reinforcement conditions contribute to relapse of operant behavior.     

Stimulus change dis-habituates operant responding supported by water reinforcers

The present experiment examined whether habituation contributes to within-session decreases in operant responding for water reinforcers. The experiment asked if this responding can be dishabituated, a fundamental property of habituated behavior. During baseline, rats’ lever pressing was reinforced by water on a variable interval 15-s schedule. During experimental conditions, rats responded on the same schedule and a new stimulus was introduced for 5 min at 15, 30 or 45 min into the 60-min session. The new stimulus was extinction, continuous reinforcement or flashing lights in different conditions. Rate of responding primarily decreased within the session during baseline. Introducing a new stimulus sometimes suppressed (extinction, continuous reinforcement) and sometimes increased (flashing lights) responding while it was in effect. The new stimulus increased responding after it ended and before it was presented in the session. The results are incompatible with the idea that non-habituation satiety factors (e.g., cellular hydration and blood volume) contributed to within-session changes in responding. These satiety factors should increase with increases in consumption, decrease with decreases in consumption and remain constant with constant consumption of water. Nevertheless, all stimulus changes increased operant responding for water. These results support the idea that habituation contributes to within-session decreases in responding for water reinforcers.     

The effects of cue competition on timing in pigeons

We studied the effects of cue competition on timing in both overshadowing and blocking operant procedures with pigeons. A white center key delivered reward when pecked 30 s after a red or green sidekey was presented and 10 s after presentation of the alternate color on the other sidekey. In Experiment 1, key presentations were concurrent during training trials for overshadow-condition pigeons, while side key presentations were separated across training trials for control birds. In Experiment 2, half of the birds (Blocking group) were given pre-exposure trials to either the 10-s or 30-s sidekey condition. Both blocking-condition and control birds were then given trials of concurrent side key presentations. Peak time curves were compared between experimental and control conditions. The results showed blocking of timing accuracy of a long (30-s) stimulus by a short (10-s) stimulus, but no evidence for overshadowing of timing accuracy.     

Operant conditioning of cortical steady potential responses in rats

This study was conducted to determine if amplitudes of rat corticol steady potential (SP) response to an auditory stimulus could be altered by operant conditioning procedures using food reinforcement. The negative SP responses to the 5-sec tone alone diminished with repeated presentation of the stimulus. When food reinforcement was given immediately following the tone, SP response amplitudes increased and stabilized after 4–5 sessions. Thereafter, the animals were required to increase or decrease the amplitudes of response in order to obtain reinforcement. Two of three rats required to increase amplitudes were successful and three of four rats required to decrease amplitudes were successful. It is concluded that changes in cortical SP responses can be operantly conditioned.     

Extinguished operant responding shows stimulus specificity

The experiment tested for stimulus specificity in extinguished operant responding. Eight pigeons pecked keys for food reinforcers delivered by a variable interval (VI) 60-s schedule. The key was illuminated with red light during some sessions and white light during others. Then, responding was placed on extinction. During some sessions of extinction, the color of the key light remained constant throughout the session (red or white). During other sessions the color changed at 30 min into the session (red to white or white to red). Response rate increased after the change of key color in extinction. If it is assumed that key color is part of the stimulus to which subjects habituate, then these results are consistent with McSweeney and Swindell's [J. Gen. Psychol. 129 (2002) 364] suggestion that responding declines in extinction partly because subjects habituate to the stimuli that support conditioned responding. Habituation is relatively specific to the exact nature of the stimulus presented. Therefore, changes in the stimulus violate stimulus specificity and restore habituated responding. The results are also consistent with other theories that attribute extinction to a reduction of stimulus control [e.g., Psychol. Bull. 114 (1993) 80; J. Exp. Psychol.: Anim. Behav. Process. 16 (1990) 235], but considerations such as parsimony and testability favor the habituation hypothesis over these theories.     

Stimulus enhancement in greylag geese: socially mediated learning of an operant task

We recently observed the spreading of a novel tradition in a flock of semiferal greylag geese, Anser anser: an increasing number of individuals began to bite and chew the stems of butterbur, Petasites hybridus. Because this behaviour spread particularly fast within families, social learning seemed to be involved. We therefore designed an experiment with hand-reared goslings, which were socially imprinted on humans, to investigate whether and how the observation of an experienced tutor affects the acquisition of a novel skill. Goslings had to open the gliding lid of a box to get at a food reward. To each of seven hand-reared observers a human tutor demonstrated where and how to open the lid, whereas seven controls remained untutored. All observers learned to perform the task but only one of the controls succeeded. The observers explored more often at the position shown by the tutor than elsewhere and seemingly learned by trial and error. In contrast, control birds explored primarily at positions that did not allow them to open the box. These results indicate that in greylag goslings the observation of an experienced model facilitates the learning of an operant task. We conclude that stimulus enhancement followed by operant conditioning were the mechanisms involved, which may have accounted for the fast spread of the stem-chewing tradition between family members. Copyright 2000 The Association for the Study of Animal Behaviour.     

Inhaled toluene produces pentobarbital-like discriminative stimulus effects in mice

The abuse of volatile solvents may be due to their ability to produce an intoxication similar to that produced by classical central nervous system depressants such as the barbiturates and ethanol. To evaluate this hypothesis, mice were trained to discriminate pentobarbital from saline injections in a two-lever operant task. Stimulus generalization was examined following 20-min inhalation exposures to toluene (300-5400 ppm). In 8 of 10 subjects, pentobarbital-lever responding occurred following toluene exposure indicating an overlap in the discriminative stimulus properties of toluene and pentobarbital.