Concomitant Pavlovian conditioning of heart rate and leg flexion responses in the rat

Pavlovian conditioning was studied in male Fischer 344 rats using tones as the conditioned stimulus (CS) and footshock as the unconditioned stimulus (UCS). Different groups of animals received (a) contiguous CS-UCS pairings with a 0.5 sec CS, (b) contiguous CS-UCS pairings with a 4.0 sec CS, or (c) a random sequence of noncontiguous tones and shocks using either a 0.5 sec or a 4.0 sec CS. Heart rate (HR) and leg flexion (LF) responses were recorded. Leg flexion conditioning occurred only in the 0.5 sec contiguous group. Decelerative HR CRs occurred only in the 4.0 sec contiguous group. Accelerative HR changes occurred in the other two groups but were significantly greater in the 0.5 sec contiguous group. These results are similar to but not identical to those obtained during eyeblink or nictitating membrane conditioning in rabbits, and suggest that the topography of the Pavlovian HR CR is dependent on the simultaneous occurrence of other classically conditioned responses.     

Quantifying individual variation in the propensity to attribute incentive salience to reward cues

If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience") to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878) subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience), to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers") vs. others that approached the location of reward delivery ("goal-trackers"). This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.     

Pavlovian Conditioned Approach Training in Rats

Cues that are contingently paired with unconditioned, rewarding stimuli can acquire rewarding properties themselves through a process known as the attribution of incentive salience, or the transformation of neutral stimuli into attractive, "wanted'' stimuli capable of motivating behavior. Pavlovian conditioned approach (PCA) develops after the response-independent presentation of a conditioned stimulus (CS; e.g., a lever) that predicts the delivery of an unconditioned stimulus (US; e.g., a food pellet) and can be used to measure incentive salience. During training, three patterns of conditioned responses (CRs) can develop: sign-tracking behavior (CS-directed CR), goal-tracking behavior (US-directed CR), and an intermediate response (both CRs). Sign-trackers attribute incentive salience to reward-related cues and are more vulnerable to cue-induced reinstatement of drug-seeking as well as other addiction-related behaviors, making PCA a potentially valuable procedure for studying addiction vulnerability. Here, we describe materials and methods used to elicit PCA behavior from rats as well as analyze and interpret PCA behavior in individual experiments.     

Variation in the Form of Pavlovian Conditioned Approach Behavior among Outbred Male Sprague-Dawley Rats from Different Vendors and Colonies: Sign-Tracking vs. Goal-Tracking

Even when trained under exactly the same conditions outbred male Sprague-Dawley (SD) rats vary in the form of the Pavlovian conditioned approach response (CR) they acquire. The form of the CR (i.e. sign-tracking vs. goal-tracking) predicts to what degree individuals attribute incentive salience to cues associated with food or drugs. However, we have noticed variation in the incidence of these two phenotypes in rats obtained from different vendors. In this study, we quantified sign- and goal-tracking behavior in a reasonably large sample of SD rats obtained from two vendors (Harlan or Charles River), as well as from individual colonies operated by both vendors. Our sample of rats acquired from Harlan had, on average, more sign-trackers than goal-trackers, and vice versa for our sample of rats acquired from Charles River. Furthermore, there were significant differences among colonies of the same vendor. Although it is impossible to rule out environmental variables, SD rats at different vendors and barriers may have reduced phenotypic heterogeneity as a result of genetic variables, such as random genetic drift or population bottlenecks. Consistent with this hypothesis, we identified marked population structure among colonies from Harlan. Therefore, despite sharing the same name, investigators should be aware that important genetic and phenotypic differences exist among SD rats from different vendors or even from different colonies of the same vendor. If used judiciously this can be an asset to experimental design, but it can also be a pitfall for those unaware of the issue.     

Characterizing psychophysical measures of discrimination thresholds and the effects of concentration on discrimination learning in the moth Manduca sexta

What is the spatial and temporal nature of odor representations within primary olfactory networks at the threshold of an animal's ability to discriminate? Although this question is of central importance to olfactory neuroscience, it can only be answered in model systems where neural representations can be measured and discrimination thresholds between odors can be characterized. Here, we establish these thresholds for a panel of odors using a Pavlovian paradigm in the moth Manduca sexta. Moths were differentially conditioned to respond to one odor (CS+) but not another (CS-) using undiluted odorants to minimize salience-dependent learning effects. At 24 and 48 h postconditioning, moths were tested for the presence of a conditioned response (CR) with a blank, then the CS+ and CS- (pseudorandomly) across a 5-log step series of increasing concentration. Results identified discrimination thresholds and established that differential CRs to the CS+ and CS- increased with stimulus concentration. Next, 3 separate groups of moths were differentially conditioned at either one-log step below, at, or one log step above the identified discrimination threshold. At 24 and 48 h postconditioning, moths were tested sequentially with a blank, the concentration used for conditioning, and then undiluted odor. Conditioning at one log step below the discrimination threshold established a CR, indicating both stimulus detection and learning, but was insufficient to establish evidence of discrimination. Moths conditioned at the discrimination threshold were able to discriminate but only when stimulated with undiluted odors, indicating learning, but discrimination measures were hampered. When conditioned above the discrimination threshold, moths had no difficulty in discriminating. These results establish methods for psychophysical characterization of discrimination and indicate that differential conditioning at lowered concentrations biases threshold measures.     

Reinstatement of short-latency responses after asymptotic Pavlovian conditioning training by the presentation of an extraneous stimulus

The purpose of this study was to examine whether the progressive disappearance of short-latency conditioned responses, or inhibition of delay, observed in Pavlovian conditioning with long inter-stimulus intervals, could be reverted by the presentation of a novel stimulus. In one experiment, two groups of rabbits received extensive training with a short (250 ms) or a long (1500 ms) tone that overlapped and terminated with a periorbital shock unconditioned stimulus. After training, the presentation of an extraneous stimulus prior to tone onset produced a reinstatement of short latency CRs in the group trained with the long CS, but did not affect CR latency in the group trained with the short CS. This finding is consistent with Pavlov's (1927) view that conditioning with long conditioned stimuli involves the acquisition of response tendencies in the early portion of the stimulus that are subsequently suppressed by the development of an inhibitory process.     

Accounting for Negative Automaintenance in Pigeons: A Dual Learning Systems Approach and Factored Representations

Animals, including Humans, are prone to develop persistent maladaptive and suboptimal behaviours. Some of these behaviours have been suggested to arise from interactions between brain systems of Pavlovian conditioning, the acquisition of responses to initially neutral stimuli previously paired with rewards, and instrumental conditioning, the acquisition of active behaviours leading to rewards. However the mechanics of these systems and their interactions are still unclear. While extensively studied independently, few models have been developed to account for these interactions. On some experiment, pigeons have been observed to display a maladaptive behaviour that some suggest to involve conflicts between Pavlovian and instrumental conditioning. In a procedure referred as negative automaintenance, a key light is paired with the subsequent delivery of food, however any peck towards the key light results in the omission of the reward. Studies showed that in such procedure some pigeons persisted in pecking to a substantial level despite its negative consequence, while others learned to refrain from pecking and maximized their cumulative rewards. Furthermore, the pigeons that were unable to refrain from pecking could nevertheless shift their pecks towards a harmless alternative key light. We confronted a computational model that combines dual-learning systems and factored representations, recently developed to account for sign-tracking and goal-tracking behaviours in rats, to these negative automaintenance experimental data. We show that it can explain the variability of the observed behaviours and the capacity of alternative key lights to distract pigeons from their detrimental behaviours. These results confirm the proposed model as an interesting tool to reproduce experiments that could involve interactions between Pavlovian and instrumental conditioning. The model allows us to draw predictions that may be experimentally verified, which could help further investigate the neural mechanisms underlying theses interactions.     

Altering the synchrony of stimulus trace processes: tests of a neural-network model

A previously described neural-network model (Desmond 1991; Desmond and Moore 1988; Moore et al. 1989) predicts that both CS-onset-evoked and CS-offset-evoked stimulus trace processes acquire associative strength during classical conditioning, and that CR waveforms can be altered by manipulating the time at which the processes are activated. In a trace conditioning paradigm, where CS offset precedes US onset, the model predicts that onset and offset traces act in synchrony to generate unimodal CR waveforms. However, if the CS duration is subsequently lengthened on CS-alone probe trials, the model predicts that onset and offset traces will asynchronously contribute to CR output and bimodal CRs will be generated. In a delay conditioning paradigm, in which US onset occurs prior to CS offset, the model predicts that only the onset process will gain associative strength, and hence, only unimodal CRs will occur. Using the rabbit conditioned nictitating membrane response preparation, we found experimental support for these predictions.This research was supported by National Science Foundation grant BNS 88-10624 and Air Force Office of Scientific Research grant 89-0391.     

Autoshaping in the rat: conditioned licking response to a stimulus that signals sucrose reinforcement

The present experiments were designed to determine if repeated presentations of an empty sipper tube (the conditioned stimulus or CS) with the response-independent delivery of a sucrose solution (the unconditioned stimulus or US) from a second spout results in the development of Pavlovian conditioned responding. In Experiment 1, rats in the experimental condition received paired CS-US presentations whereas subjects in the control condition were exposed to random presentations of CS and US. In Experiment 2, an additional control condition (CS alone) was included and, to encourage generalized responding between the US and CS, the CS tube was filled with water for all groups. The results of both experiments indicate that the CS-directed responding in the paired CS-US condition was Pavlovian in nature. Thus, the present procedure serves as an autoshaping task in which conditioned licking is generated.     

External inhibition in a goldfish (Carassius auratus) classical conditioning situation

Goldfish were classically conditioned with a light as the CS and shock as the US. The UR was a decrease in respiration. After 15 or 60 conditioning trials the fish were tested with novel stimuli (clicks) during the CS-US interval. High and moderate intensity novel stimuli produced a significant decrease in CRs (external inhibition) for fish with 60 conditioning trials (5.5 or 10.5 sec CS-US interval), but not fish with 15 conditioning trials. Low intensity novel stimuli produced no evidence for disinhibition (an increase in CRs). Control groups (e.g., groups with random presentations of the CS and US) showed that the external inhibition for fish with 60 conditioning trials was inhibition of a true CR.     

Temporal difference models and reward-related learning in the human brain

Temporal difference learning has been proposed as a model for Pavlovian conditioning, in which an animal learns to predict delivery of reward following presentation of a conditioned stimulus (CS). A key component of this model is a prediction error signal, which, before learning, responds at the time of presentation of reward but, after learning, shifts its response to the time of onset of the CS. In order to test for regions manifesting this signal profile, subjects were scanned using event-related fMRI while undergoing appetitive conditioning with a pleasant taste reward. Regression analyses revealed that responses in ventral striatum and orbitofrontal cortex were significantly correlated with this error signal, suggesting that, during appetitive conditioning, computations described by temporal difference learning are expressed in the human brain.     

Expression of freezing and fear‐potentiated startle during sustained fear in mice

Fear‐potentiated acoustic startle paradigms have been used to investigate phasic and sustained components of conditioned fear in rats and humans. This study describes a novel training protocol to assess phasic and sustained fear in freely behaving C57BL/6J mice, using freezing and/or fear‐potentiated startle as measures of fear, thereby, if needed, allowing in vivo application of various techniques, such as optogenetics, electrophysiology and pharmacological intervention, in freely behaving animals. An auditory Pavlovian fear conditioning paradigm, with pseudo‐randomized conditioned–unconditioned stimulus presentations at various durations, is combined with repetitive brief auditory white noise burst presentations during fear memory retrieval 24 h after fear conditioning. Major findings are that (1) a motion sensitive platform built on mechano‐electrical transducers enables measurement of startle responses in freely behaving mice, (2) absence or presence of startle stimuli during retrieval as well as unpredictability of a given threat determine phasic and sustained fear response profiles and (3) both freezing and startle responses indicate phasic and sustained components of behavioral fear, with sustained freezing reflecting unpredictability of conditioned stimulus (CS)/unconditioned stimulus (US) pairings. This paradigm and available genetically modified mouse lines will pave the way for investigation of the molecular and neural mechanisms relating to the transition from phasic to sustained fear.     

A novel role for acid‐sensing ion channels in Pavlovian reward conditioning

Pavlovian fear conditioning has been shown to depend on acid‐sensing ion channel‐1A (ASIC1A); however, it is unknown whether conditioning to rewarding stimuli also depends on ASIC1A. Here, we tested the hypothesis that ASIC1A contributes to Pavlovian conditioning to a non‐drug reward. We found effects of ASIC1A disruption depended on the relationship between the conditional stimulus (CS) and the unconditional stimulus (US), which was varied between five experiments. In experiment 1, when the CS preceded the US signaling an upcoming reward, Asic1a^?/? mice exhibited a deficit in conditioning compared to Asic1a^+/+ mice. Alternatively, in experiment 2, when the CS coinitiated with the US and signaled immediate reward availability, the Asic1a^?/? mice exhibited an increase in conditioned responses compared to Asic1a^+/+ mice, which contrasted with the deficits in the first experiment. Furthermore, in experiments 3 and 4, when the CS partially overlapped in time with the US, or the CS was shortened and coinitiated with the US, the Asic1a^?/? mice did not differ from control mice. The contrasting outcomes were likely because of differences in conditioning because in experiment 5 neither the Asic1a^?/? nor Asic1a^+/+ mice acquired conditioned responses when the CS and US were explicitly unpaired. Taken together, these results suggest that the effects of ASIC1A disruption on reward conditioning depend on the temporal relationship between the CS and US. Furthermore, these results suggest that ASIC1A plays a critical, yet nuanced role in Pavlovian conditioning. More research will be needed to deconstruct the roles of ASIC1A in these fundamental forms of learning and memory.     

Nicotine-evoked conditioned responding is dependent on concentration of sucrose unconditioned stimulus

Previous studies have shown that the interoceptive nicotine conditional stimulus (CS) functions similarly to exteroceptive CSs such as lights or environments. For instance, the appetitive conditioned response (CR) evoked when nicotine is repeatedly paired with sucrose presentations (the unconditioned stimulus; US) is sensitive to changes in training dose (CS salience) and the contiguity between the CS effects and sucrose. The current study was conducted to extend this research by examining the possible role of US intensity in CR acquisition and maintenance. Rats were trained using one of four sucrose concentrations: 0, 4, 16, or 32% (w/v). On nicotine sessions (0.4 mg base/kg), rats received 36 deliveries (4 s each) of their assigned concentration intermittently throughout the session; sucrose was withheld on saline sessions. In all groups, an appetitive goal-tracking CR was acquired at a similar rate. However, the asymptotic CR level varied with sucrose concentration. The magnitude of the CR was increased in rats trained with higher sucrose US concentrations. These findings are consistent with previous Pavlovian conditioning research, and extend the conditions under which the nicotine state functions as an interoceptive conditional stimulus.     

Can Honey Bees Learn the Removal of a Stimulus as a Conditioning Cue?

Experiments investigated a Pavlovian conditioning situation where the presence and absence of the stimulus are reversed temporally with respect to the presentation of a reward. Instead of a conditioned stimulus (e.g. odor) signaling the presence of a reward, the stimulus (e.g. odor) is present in the environment except just prior to the presence of the reward. Thus, the absence of the stimulus, or offset of the stimulus (e.g. absence of odor), serves as a conditioned stimulus and is the reward cue. Honey bees (Apis mellifera) were used as a model invertebrate system, and the proboscis‐conditioning paradigm was used as the test procedure. Using both simple Pavlovian conditioning and discrimination‐learning protocols, animals learned to associate the onset of an odor as conditioned stimuli when paired with a sucrose reward. They could also learn to associate the onset of a puff of air with a sucrose reward. However, bees could not associate the offset of an order stimulus with the presentation of a sucrose reward in either a simple conditioning or a discrimination‐learning situation. These results support the model that a very different cognitive architecture is used by invertebrates to deal with certain environmental situations, including signaled avoidance.     

A comparison of the effects of vicariously instigated classical conditioning and direct classical conditioning procedures.

Two groups of Ss received either two or 16 paired classical conditioning trails beyond the peak CR. A third group received the same stimuli as the 16 postpeak condition but in an unpaired and random order. The stimuli in all three groups were delivered directly to S. Subsequently, all three groups, including a fourth which was not given any prior direct classical conditioning, were exposed to vicariously instigated classical conditioning. This consisted of having S observe someone (model) employed by E who received the same CS as was delivered during direct conditioning. The CS was paired with the feigned arm movement of the model, simulating a reaction to shock. This vicarious classical conditioning procedure when compared to direct classical conditioning resulted in smaller GSR magnitudes for both the CRs and UCRs. Previous experience with direct classical conditional seems to have had an attenuating effect on GSR magnitude during the vicarious situation. A postexperimental questionnaire tended to support the results, and the relationship between the present study and current classical conditioning theory is discussed.     

Towards a unified model of pavlovian conditioning: short review of trace conditioning models

There are three basic paradigms of classical conditioning: delay, trace and context conditioning where presentation of a conditioned stimulus (CS) or a context typically predicts an unconditioned stimulus (US). In delay conditioning CS and US normally coterminate, whereas in trace conditioning an interval of time exists between CS termination and US onset. The modeling of trace conditioning is a rather difficult computational problem and is a challenge to the behavior and connectionist approaches mainly due to a time gap between CS and US. To account for trace conditioning, Pavlov (Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex, Oxford University Press, London, 1927) postulated the existence of a stimulus “trace” in the nervous system. Meanwhile, there exist many other options for solving this association problem. There are several excellent reviews of computational models of classical conditioning but none has thus far been devoted to trace conditioning. Eight representative models of trace conditioning aimed at building a prospective model are being reviewed below in a brief form. As a result, one of them, comprising the most important features of its predecessors, can be suggested as a real candidate for a unified model of trace conditioning.     

Nucleus accumbens neurons dynamically respond to appetitive and aversive associative learning

To survive, individuals must learn to associate cues in the environment with emotionally relevant outcomes. This association is partially mediated by the nucleus accumbens (NAc), a key brain region of the reward circuit that is mainly composed by GABAergic medium spiny neurons (MSNs), that express either dopamine receptor D1 or D2. Recent studies showed that both populations can drive reward and aversion, however, the activity of these neurons during appetitive and aversive Pavlovian conditioning remains to be determined. Here, we investigated the relevance of D1- and D2-neurons in associative learning, by measuring calcium transients with fiber photometry during appetitive and aversive Pavlovian tasks in mice. Sucrose was used as a positive valence unconditioned stimulus (US) and foot shock was used as a negative valence US. We show that during appetitive Pavlovian conditioning, D1- and D2-neurons exhibit a general increase in activity in response to the conditioned stimuli (CS). Interestingly, D1- and D2-neurons present distinct changes in activity after sucrose consumption that dynamically evolve throughout learning. During the aversive Pavlovian conditioning, D1- and D2-neurons present an increase in the activity in response to the CS and to the US (shock). Our data support a model in which D1- and D2-neurons are concurrently activated during appetitive and aversive conditioning.

     

Conditioning of stress in Nile tilapia

A Pavlovian conditioning paradigm was used to induce a connection between a conditioned stimulus, light (CS), associated with an unconditioned stimulus, confinement (US) in Nile tilapia Oreochromis niloticus , which resulted in a conditioned endocrine response (CR) to the CS alone manifested as an increase in plasma cortisol. Individual isolated Nile tilapia were submitted for 10 days to the conditioning treatment consisting of turning on a light (CS) for 1 min with subsequent 30 min confinement (US). On the 10th day of the experiment, plasma cortisol was not increased when fish were subjected to no handling at all, or only light, or even a daily stressor for the 9 days. On the other hand, at the 10th day cortisol was significantly increased only when light was presented either with or without pairing with the stressor. These results confirmed that the cue, light (CS), was not stressful in itself, but when given as the CS in the absence of the US post conditioning the hypothalamus–pituitary–interrenal axis was activated. Therefore, it was concluded that memory of a previous experience with a stressor can be recalled by a conditioned stimulus and induce stress, which is the first demonstration of a memory‐induced stress in fishes.     

Ghrelin regulates phasic dopamine and nucleus accumbens signaling evoked by food‐predictive stimuli

Environmental stimuli that signal food availability hold powerful sway over motivated behavior and promote feeding, in part, by activating the mesolimbic system. These food‐predictive cues evoke brief (phasic) changes in nucleus accumbens (NAc) dopamine concentration and in the activity of individual NAc neurons. Phasic fluctuations in mesolimbic signaling have been directly linked to goal‐directed behaviors, including behaviors elicited by food‐predictive cues. Food‐seeking behavior is also strongly influenced by physiological state (i.e., hunger vs. satiety). Ghrelin, a stomach hormone that crosses the blood‐brain barrier, is linked to the perception of hunger and drives food intake, including intake potentiated by environmental cues. Notwithstanding, whether ghrelin regulates phasic mesolimbic signaling evoked by food‐predictive stimuli is unknown. Here, rats underwent Pavlovian conditioning in which one cue predicted the delivery of rewarding food (CS+) and a second cue predicted nothing (CS?). After training, we measured the effect of ghrelin infused into the lateral ventricle (LV) on sub‐second fluctuations in NAc dopamine using fast‐scan cyclic voltammetry and individual NAc neuron activity using in vivo electrophysiology in separate groups of rats. LV ghrelin augmented both phasic dopamine and phasic increases in the activity of NAc neurons evoked by the CS+. Importantly, ghrelin did not affect the dopamine nor NAc neuron response to the CS?, suggesting that ghrelin selectively modulated mesolimbic signaling evoked by motivationally significant stimuli. These data demonstrate that ghrelin, a hunger signal linked to physiological state, can regulate cue‐evoked mesolimbic signals that underlie food‐directed behaviors.