Passage of time leaves response-outcome associations intact

The effect of the passage of time on the association between an instrumental response and its outcome was examined in three experiments with rats using outcome devaluation. In Experiments 1 and 2, rats were initially trained to make one pair of instrumental responses for different outcomes (e.g., lever press-->pellets and chain pull-->sucrose). Then, after a delay of several weeks, a second pair of responses for those different outcomes (e.g., nose poke-->pellets and handle pull-->sucrose) was trained in Experiment 1 but not in Experiment 2. One of the outcomes was then devalued and the responses were tested in extinction. There was no evidence that the passage of time eliminated the sensitivity of a response to a change in the value of its outcome. Experiment 3 used the same design as Experiment 1 to examine the effect of the passage of time on extinguished instrumental responses. First, one pair of responses was trained with different outcomes and then extinguished prior to the training and subsequent extinction of a second pair of responses. After retraining both pairs of responses with a polycose outcome, one of the original outcomes was devalued. Each pair of responses was then tested in extinction. The results of the extinction tests revealed no evidence that the passage of time removed the sensitivity of an extinguished response to devaluation of its original outcome. These findings are consistent with other reports that the response-outcome association survives various response elimination techniques.     

Instrumental outcome devaluation with representation-mediated conditioning

In three experiments, rats were trained to perform two instrumental behaviours (R1 and R2) in the presence of discriminative stimuli (Sd1 and Sd2, respectively) to obtain a common food outcome (O1). Acquisition of the two discriminations was followed by switching the outcome accompanying R2 performance from O1 to a new one (O2). Experiment 1 showed paired presentations of O2 with a lithium chloride (LiCl) injection resulted in a reduction in the R2 performance. In the subsequent two experiments, each Sd was paired with LiCl injection and its effects on outcome consumption and instrumental performance were investigated. A reduction in the O2 consumption subsequent to the Sd devaluation was found in Experiments 2 and 3. Experiment 3 revealed a reduced R2 performance in an extinction test, following the animals’ consummatory access to the outcomes in training context. These results demonstrate representation-mediated outcome devaluation in the course of the Sd devaluation.     

The Effect of Ratio and Interval Training on Pavlovian-Instrumental Transfer in Mice

Conditional stimuli (CS) that are paired with reward can be used to motivate instrumental responses. This process is called Pavlovian-instrumental transfer (PIT). A recent study in rats suggested that habitual responses are particularly sensitive to the motivational effects of reward cues. The current experiments examined this idea using ratio and interval training in mice. Two groups of animals were trained to lever press for food pellets that were delivered on random ratio or random interval schedules. Devaluation tests revealed that interval training led to habitual responding while ratio training produced goal-directed actions. The presentation of CSs paired with reward led to positive transfer in both groups, however, the size of this effect was much larger in mice that were trained on interval schedules. This result suggests that habitual responses are more sensitive to the motivational influence of reward cues than goal-directed actions. The implications for neurobiological models of motivation and drug seeking behaviors are discussed.     

Reduction of instrumental discrimination performance by post-conditioning devaluation of discriminative stimulus: the effects of novelty in reinforcing outcome and extended training

In three experiments the effects of post-conditioning pairings of a discriminative stimulus (Sd) with an illness-inducing agent (lithium chloride, LiCl) on subsequent discrimination performance in extinction and consumption of reinforcing outcome were investigated. Rats were trained to choose a correct lever to obtain food pellets, with a light presented on a bulb just above the correct lever serving for the Sd on each trial. After achievement of a criterion of the discrimination, animals received paired or unpaired presentations of the Sds and LiCl injection. In Experiment 1, in which a familiar outcome was given throughout the discrimination training, Sd-LiCl pairings did not reduce either lever-press performance during presentation of the Sds or amount of consumption of outcomes. On the other hand, in Experiment 2 where a novel outcome was introduced in the final two sessions of the discrimination training, subsequent Sd devaluation reduced lever-press performance during presentations of the Sds. Similar findings were obtained in Experiment 3, in which animals were given extended discrimination training with introduction of novel outcomes in the final two sessions. These findings suggest that a representation of the outcome, evoked by presentation of the Sd, and illness were associated in the course of Sd-LiCl pairings but only when a novel outcome was used.     

Matching unrelated stimuli with same discriminative functions: training order effects

Previous research has shown that after training simple discriminations (A1+/A2−, B1+/B2−), bringing these tasks under conditional control (J1–A1, J2–A2) leads to transfer of discriminative control (J1+/J2−) and to generalized matching on the basis of same discriminative functions (e.g. J1–B1, J2–B2). The same occurs when conditional discriminations are trained (D1–E1, D2–E2; F1–G1, F2–G2). When the subjects are then trained to demonstrate correct relations (D1–E1, D2–E2) when given X1 and to demonstrate incorrect relations when given X2 (XD–E), transfer of discriminative control (X1+/X2−) and generalized matching on the basis of same discriminative functions emerges (e.g. X1F1–G1, X2F1–G2). The present study investigated if these performances are dependent on the training and/or testing order. In Experiment 1, the lower-order contingency tasks were trained before the higher-order contingency tasks (A1+/A2−, B1+/B2− before J–A, and D–E, F–G before XD–E). Half the subjects received the J–B test before the more complex XF–G test (Condition A), while for the other subjects, this testing order was reversed (Condition B). Finally, all subjects received additional tests in which they were given the opportunity to demonstrate the discriminative properties of the J and X stimuli (J1+/J2−, X1+/X2−), and to match the A, J, and X stimuli with newly introduced stimuli of same discriminative properties (e.g. J1-POLITE, J2-RUDE). Experiment 2 was the same except that the training order was reversed (J–A before A1+/A2−, B1+/B2−, and XD–E before D–E, F–G). The results were affected by the training order but not by the testing order. Transfer of discriminative functions and generalized matching on the basis of same functions only occurred reliably when the lower-order contingency tasks were trained first. A stimulus-control account of the data is offered.     

Ethanol seeking by long evans rats is not always a goal-directed behavior

Background

Two parallel and interacting processes are said to underlie animal behavior, whereby learning and performance of a behavior is at first via conscious and deliberate (goal-directed) processes, but after initial acquisition, the behavior can become automatic and stimulus-elicited (habitual). With respect to instrumental behaviors, animal learning studies suggest that the duration of training and the action-outcome contingency are two factors involved in the emergence of habitual seeking of “natural” reinforcers (e.g., sweet solutions, food or sucrose pellets). To rigorously test whether behaviors reinforced by abused substances such as ethanol, in particular, similarly become habitual was the primary aim of this study.

Methodology/Principal Findings

Male Long Evans rats underwent extended or limited operant lever press training with 10% sucrose/10% ethanol (10S10E) reinforcement (variable interval (VI) or (VR) ratio schedule of reinforcement), or with 10% sucrose (10S) reinforcement (VI schedule only). Once training and pretesting were complete, the impact of outcome devaluation on operant behavior was evaluated after lithium chloride injections were paired with the reinforcer, or unpaired 24 hours later. After limited, but not extended instrumental training, lever pressing by groups trained under VR with 10S10E and under VI with 10S was sensitive to outcome devaluation. In contrast, responding by both the extended and limited training 10S10E VI groups was not sensitive to ethanol devaluation during the test for habitual behavior.

Conclusions/Significance

Operant behavior by rats trained to self-administer an ethanol-sucrose solution showed variable sensitivity to a change in the value of ethanol, with relative insensitivity developing sooner in animals that received time-variable ethanol reinforcement during training sessions. One important implication, with respect to substance abuse in humans, is that initial learning about the relationship between instrumental actions and the opportunity to consume ethanol-containing drinks can influence the time course for the development or expression of habitual ethanol seeking behavior.     

A comparison of the effects of discriminative and Pavlovian inhibitors and excitors on instrumental responding

In two experiments, the effects of Pavlovian or discriminative conditioned inhibitors on operant responding were investigated in rats. Experiment 1 found that a Pavlovian conditioned inhibitor for food suppressed food-reinforced lever pressing more than a non-differentially trained control stimulus did. Experiment 2 demonstrated that an operant discriminative inhibitor produced greater suppression of lever pressing than a Pavlovian conditioned inhibitor. Experiment 2 also found that compounding an operant discriminative stimulus (SD) for food-reinforced responding with another SD for food-reinforced responding resulted in more additive summation than when an SD was compounded with a Pavlovian conditioned excitor for food. The results of these experiments support two-factor theories that postulate that incentive and response discriminative processes summate algebraically when the processes are inhibitory or excitatory.     

A discriminative two-lever test of dizocilpine's ability to reinstate ethanol-seeking behavior

Because ethanol has N-methyl-D-aspartate (NMDA) antagonist effects, we tested whether dizocilpine, an NMDA antagonist, reinstates ethanol-seeking behavior. Rats were trained to lever-press for a 10% ethanol/2% sucrose (EtOH) or a 3% sucrose (Suc) solution using a two-lever (one lever active) procedure (FR2). After extinction, rats were injected with ethanol (0.5 g/kg). The EtOH group emitted more active than inactive lever presses and the Suc group showed minimal responding. Thus, ethanol reinstated ethanol-seeking behavior in a specific manner. In contrast, dizocilpine (0.175 mg/kg) increased responding on both levers in both groups suggesting a loss of discriminative control. Dizocilpine fails to reinstate ethanol-seeking behavior. These data also demonstrate the necessity of using a discriminative, two-lever test for drug reinstatement.     

Object-picture equivalence in the pigeon: An analysis with natural concept and pseudoconcept discriminations

Pigeons were trained on a natural concept (food vs. non-food) and on a pseudoconcept (arbitrary classification of edible and not edible stimuli). The birds were trained with real objects and then tested with colour photographs in the object-to-picture transfer group, and they were trained and tested in the reverse order in a picture-to-object transfer group. The subjects showed good transfer of discrimination in both directions when the task involved a natural concept, but they did not show transfer of a pseudoconcept discrimination. Because all birds saw the same stimuli during the discriminative training, the difference in transfer was due to the type of classification of the stimuli. These results suggest object-picture equivalence based on functional classification.     

Effect of the passage of time on the contribution of initial response-outcome associations to instrumental performance

The effect of the passage of time on the contribution of initial response-outcome associations to subsequent instrumental performance was explored in three experiments with rats using outcome devaluation. Experiments 1 and 2 showed that a response that had been trained first with one outcome and then given identical training with a second outcome was more sensitive to devaluation of the second outcome than the first if the two training episodes were separated in time. Experiment 3 showed that inserting a delay between training with the second outcome and testing after outcome devaluation appeared to mitigate this effect of temporally separating first and second outcome training. Inserting this delay also made a response slightly more sensitive to devaluation of the first outcome than the second when there was no delay between the two training episodes. These results suggest that the passage of time can shift the balance between the contributions of first and second trained outcomes to instrumental performance.     

The sweet-bitter taste of alcohol: aversion generalization to various sweet-quinine mixtures in the rat

Rats were trained to avoid a 5% alcohol solution and then testedwith either sweet + quinine hydrochloride solutions (Experiment1) or sweet + hydrochloric acid solutions (Experiment 2). Thesweet stimuli used were sucrose, glucose, fructose and saccharin.Significant aversion generalization was found only in Experiment1 where trained rats generalized to all four test stimuli, thussuggesting that alcohol has a sweet taste (in combination withbitter) not specific to one sweetener. No significant aversiongeneralization was noted in Experiment 2 when sweet + hydrochloricacid solutions were tested. In Experiment 3, rats were trainedto avoid 6% alcohol and tested with sucrose + quinine hydrochloridemixtures with varying concentrations of each component. In general,rats showed generalization of the alcohol aversions across thevarious concentrations of sucrose and quinine hydrochloridetested.     

Duration, response, and location: the influence of upcoming 32% sucrose on rats' licking or lever pressing for 1% liquid sucrose

The present study investigated whether rats' rates of licking or pressing a lever for 1% liquid sucrose delivered by a continuous reinforcement schedule would decrease (contrast) or increase (induction) when the upcoming period would allow access to 32% sucrose and whether such changes would be influenced by how long each substance was available. In Experiment 1, different groups of rats licked a spout or pressed a lever for 1% sucrose in the first half of the session and, in different conditions, for 1% or 32% sucrose in the second half. Across conditions, halves of the session were 3, 6, 12, or 24 min long. Upcoming 32% sucrose significantly decreased rates of licking at each duration whereas it increased rates of lever pressing except when access duration was 3 min. Experiment 2 replicated Experiment 1 with the exception that rats that licked did so from the same spout in both halves of the session and rats that pressed a lever collected the sucrose reinforcers in the different halves at different locations. In these procedures, upcoming 32% sucrose significantly increased rates of licking. Significant, but small, increases in rates of lever pressing were still observed. The present results suggest that continuous reinforcement or duration of access to sucrose are not primary determinants of whether contrast or induction is observed. Rather, they suggest that the type of behavior (licking versus pressing a lever) and the location at which the substances are collected and consumed play a large role in which effect occurs.     

Discriminative stimulus properties of L-phenylisopropyl adenosine: blockade by caffeine and generalization to 2-chloroadenosine

Recent neurochemical data on the effects of activation and blockade of adenosine A1 receptors has suggested a direct role of adenosine in neurotransmission. The present research used a drug discrimination procedure to test the hypotheses that A1 adenosine receptor activation could serve as a discriminative stimulus and that caffeine, a drug believed to be an A1 receptor antagonist, could block the adenosine discrimination. Food-deprived rats were trained to press one of two levers on an FR 10 schedule of food-pellet delivery. Responses on one lever were reinforced following i.p. injection of N6 - (L-phenylisopropyl) adenosine (L-PIA); responses on the other lever were reinforced following i.p. injection of saline. L-PIA training dose was increased from 0.064 to 0.08 mg/kg L-PIA in the course of the study. Subjects required an average of 91 sessions to acquire this discrimination. Stimulus control by L-PIA was dose-dependent, with the ED-50 being approximately 0.03 mg/kg. 2-Chloroadenosine (2CA) generalized to L-PIA with a tenth the potency. Caffeine blocked L-PIA-induced lever selection. These results indicate that 1) rats can be trained to discriminate L-PIA from saline in a two-lever food-reinforced task and 2) the discriminative stimuli produced by L-PIA are based on its agonistic action at the adenosine A1 receptor.     

Three tests of 'anticipatory responding' as an account for induction produced by upcoming food-pellet reinforcement

Previous research has demonstrated that rats' rates of lever pressing for low-concentration liquid-sucrose reinforcers are increased when food-pellet, rather than sucrose, reinforcement will be upcoming in the same session (i.e. induction). The present experiments were designed to determine whether this induction was the product of 'anticipatory responses' for the upcoming food pellets being added to the responses being made for the currently available sucrose reinforcement. Experiment 1 tested this idea by summing sucrose-reinforced responding and 'anticipatory responding' from different conditions and comparing the sum to responding from a third condition in which subjects responded for sucrose when food-pellet reinforcement was upcoming. The comparison yielded similar response rates. Experiment 2 employed a blackout, of different durations in different conditions, to delay the upcoming food-pellet reinforcement. Consistent with the anticipatory-responding account, the delay decreased the size of the induction. However, results from the blackouts were not entirely consistent with the anticipatory-responding explanation. Experiment 3 provided, in some conditions, sucrose and food-pellet reinforcement in the first and second halves of the session, respectively, for responding on separate levers. These conditions separated 'anticipatory responses' for the food pellets from responses for the sucrose reinforcers. However, induction in responding for sucrose was still present. Together, these experiments demonstrate that, although anticipatory responses may contribute to induction in some instances, they are not solely responsible for the effect.     

Excitatory backward conditioning in an appetitive conditioned reinforcement preparation with rats

Four experiments were conducted to examine appetitive backward conditioning in a conditioned reinforcement preparation. In all experiments, off-line classical conditioning was conducted following lever-press training on two levers. Presentations of a sucrose solution by a liquid dipper served as an unconditioned stimulus (US) and two auditory stimuli served as conditioned stimuli (CSs); one was paired with the US in either a forward (Experiment 1a) or a backward (Experiments 1b, 2, and 3) relationship, and the other served as a control CS, which was not paired with the US. In testing, each lever-press response produced a presentation of one of the CSs instead of appetitive reinforcers. The response to a lever was facilitated, compared to the response to another lever, when the response produced the backward CS presentation as well as when it produced the forward CS presentation; that is, the backward CS served as an excitatory conditioned reinforcer.     

Interoceptive "satiety" signals produced by leptin and CCK

The present studies assessed the extent to which the adiposity signal leptin and the brain-gut hormone cholecystokinin (CCK), administered alone or in combination, give rise to interoceptive sensory cues like those that are produced by a low (1h) level of food deprivation. Rats were trained with cues arising from 1 to 24-h food deprivation as discriminative stimuli. For one group, 24-h food deprivation predicted the delivery of sucrose pellets, whereas 1-h food deprivation did not. Another group received the reversed deprivation level-sucrose contingency. After asymptotic performance was achieved, the effects of leptin and CCK on food intake and on discrimination performance were tested under 24-h food deprivation. In Experiment 1a, leptin administered into the third cerebroventricle (i3vt) at 3.5 or 7.0 microg doses had little effect, compared to saline on food intake or discriminative responding. In Experiment 1b, leptin (7.0 microg, i3vt) combined with CCK-8 (2 microg/kg, i.p.) reduced food intake significantly, but the findings indicated that CCK-8 alone produces interoceptive discriminative cues more like those produced by 1- than 24-h food deprivation. Experiment 2a tested rats with i.p. leptin (0.3 and 0.5mg/kg). Although neither dose suppressed intake, the 0.3mg/kg dose produced interoceptive cues like 1-h food deprivation. Experiment 2b tested two doses of CCK-8 (2 and 4 mg/kg, i.p.) and found significant intake suppression and generalization of discrimination with both doses of CCK-8. These findings suggest a role for both leptin and CCK in the production of sensory consequences that correspond to "satiety".     

Comparison of the instrumental reinforcer devaluation effect in two strains of rats (Wistar and Lister)

In this experiment, the effect of the reinforcer devaluation upon instrumental performance was analysed in two strains of rats (Wistar and Lister): Food deprived rats were trained to press a lever for sucrose pellets in a single session. Immediately after the fulfilment of this session, half of the Wistar and Lister rats received an injection of lithium chloride (LiCl), while the remaining animals were injected with a sodium chloride (NaCl) solution. A subsequent extinction test showed that the subjects who had received immediate LiCl did not press the lever as often as those injected with NaCl. No differences in response suppression were found between the two strains of rats. These results also show that a single devaluation experience is sufficient for an impact on instrumental performance.     

Strategy of auditory discrimination of scale in Java sparrows: they use both "imagery" and specific cues

We examined whether Java sparrows use imagery of auditory stimuli (imagery is a subject's mental representation of a stimulus by which the subject's behaviour may be governed under stimulus control even in the absence of the physical stimulus). Three types of ascending tone sequences were used. In the intact scale, sequence tones were played in ascending order. In the intact-masked scale, part of the sequence was masked by noise but the remaining scale was identical with the intact scale, whereas in the violated scale, the sequence could be heard as if tones were played slowly (Experiment 1) or quickly (Experiment 2). Subjects were divided into two groups: one group was trained to respond to the intact and intact-masked scales and to suppress response to the violation scale (imagery-positive group). The contingency was reversed for the other (violation-positive) group. In Experiment 1, all the birds acquired discrimination, but successful transfer to novel stimuli was observed only in the imagery-positive group, suggesting that the imagery of the tone sequence was used as a discriminative cue. Experiment 2 confirmed that the stimulus duration was a discriminative cue for both groups, suggesting that the birds also acquired discrimination using only specific cues.     

Discriminative effects of morphine administered intracerabrally in the rat

Rats were trained in a two-choice discrete trial avoidance paradigm to discriminate between saline and 3.0 mg/kg of morphine administered S.C. The microinjection of 0.3–3.0 μg of morphine into the lateral ventricle produced discriminative effects equivalent to those of the systemic training dose as measured by responding on the morphine-appropriate choice lever. Discriminative effects equivalent to those of the morphine training dose were not consistently produced by administration of morphine into the periaqueductal gray, lateral septum or dorsomedial thalamus in doses as high as 10 μg. However, the discriminative effects of systematically administered morphine were blocked by 10–30 μg of naloxone administered intracerebrally at all of the brain sites tested. Thus, the primary site at which morphine acts to produce discriminative effects in the rat is central, although the specific brain areas mediating these effects remain unidentified. The actions of naloxone could be the result of diffusion of the drug into the ventricular system or into the systemic circulation.     

Categorical perception of orientation in monkeys

Structural and functional substrates of orientation processing in monkeys have been clarified. However, orientation perception in monkeys has not been fully studied. In this study, the cognitive mechanism that controls monkeys' perception of orientation was evaluated. After the monkeys were trained to discriminate between a cardinal and an oblique orientation (e.g., 0 degrees and 30 degrees), their perceptual mechanisms underlying orientation discrimination were tested by using six orientations, ranging from 0 degrees to 150 degrees, including ones used in the discrimination training. Generalization tests showed that the monkeys who were trained with cardinal orientations (e.g., 0 degrees) as positive stimuli generalized their responses to the other cardinal orientation (e.g., 90 degrees). Similarly, the monkeys who were trained with oblique orientations (e.g., 30 degrees) as positive stimuli generalized their responses to all other oblique orientations (e.g., 60 degrees, 120 degrees, and 150 degrees). These findings indicated that the monkeys abstracted the quality of the cardinal/oblique category from the physical features of orientation stimuli although they were not trained to do so. Such an abstraction also suggested a discrepancy between a continuously and orderly arranged cortical map and a discontinuously categorized perception of orientation. The present findings provide insight into the learning-correlated plasticity of cortical orientation preference.