Exposure to cold: aversive Pavlovian conditioning in individual Drosophila melanogaster

Abstract. Temperature changes can be especially threatening for ectotherms, such as Drosophila melanogaster (Diptera: Drosophilidea Meigen, 1830 ), and in this study we tested whether flies can associate olfactory stimuli with a sudden drop in temperature. Such Pavlovian conditioning would allow them to make appropriate behavioural and/or physiological responses in the future. We found that exposing individual flies to one of two odours in the presence of a sudden drop in temperature resulted in Pavlovian conditioning with flies subsequently avoiding the odour paired with cold. The characteristics of Pavlovian conditioning in flies were comparable to those observed for mammalian species. Specifically, the strength of conditioning increased with increasing intensity of the cold and decreased as the time interval between the olfactory stimulus (CS) and cold (US) was lengthened. Finally, the order in which CS and US were presented affected the strength of conditioning. Learning was observed when the CS preceded US and when the US immediately preceded the CS, but not when the CS preceded the US by 30 s or more. These results provide further evidence for learning in individual flies, and confirm that Pavlovian conditioning is a general mechanism used by organisms to obtain information about their environment.     

Quantifying individual variation in the propensity to attribute incentive salience to reward cues

If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience") to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878) subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience), to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers") vs. others that approached the location of reward delivery ("goal-trackers"). This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.     

Accounting for Negative Automaintenance in Pigeons: A Dual Learning Systems Approach and Factored Representations

Animals, including Humans, are prone to develop persistent maladaptive and suboptimal behaviours. Some of these behaviours have been suggested to arise from interactions between brain systems of Pavlovian conditioning, the acquisition of responses to initially neutral stimuli previously paired with rewards, and instrumental conditioning, the acquisition of active behaviours leading to rewards. However the mechanics of these systems and their interactions are still unclear. While extensively studied independently, few models have been developed to account for these interactions. On some experiment, pigeons have been observed to display a maladaptive behaviour that some suggest to involve conflicts between Pavlovian and instrumental conditioning. In a procedure referred as negative automaintenance, a key light is paired with the subsequent delivery of food, however any peck towards the key light results in the omission of the reward. Studies showed that in such procedure some pigeons persisted in pecking to a substantial level despite its negative consequence, while others learned to refrain from pecking and maximized their cumulative rewards. Furthermore, the pigeons that were unable to refrain from pecking could nevertheless shift their pecks towards a harmless alternative key light. We confronted a computational model that combines dual-learning systems and factored representations, recently developed to account for sign-tracking and goal-tracking behaviours in rats, to these negative automaintenance experimental data. We show that it can explain the variability of the observed behaviours and the capacity of alternative key lights to distract pigeons from their detrimental behaviours. These results confirm the proposed model as an interesting tool to reproduce experiments that could involve interactions between Pavlovian and instrumental conditioning. The model allows us to draw predictions that may be experimentally verified, which could help further investigate the neural mechanisms underlying theses interactions.     

Spontaneous decisions and operant conditioning in fruit flies

Already in the 1930s Skinner, Konorskiand colleagues debated the commonalities, differences and interactions among the processes underlying what was then known as “conditioned reflexes type I and II”, but which is today more well-known as classical (Pavlovian) and operant (instrumental) conditioning. Subsequent decades of research have confirmed that the interactions between the various learning systems engaged during operant conditioning are complex and difficult to disentangle. Today, modern neurobiological tools allow us to dissect the biological processes underlying operant conditioning and study their interactions. These processes include initiating spontaneous behavioral variability, world-learning and self-learning. The data suggest that behavioral variability is generated actively by the brain, rather than as a by-product of a complex, noisy input-output system. The function of this variability, in part, is to detect how the environment responds to such actions. World-learning denotes the biological process by which value is assigned to environmental stimuli. Self-learning is the biological process which assigns value to a specific action or movement. In an operant learning situation using visual stimuli for flies, world-learning inhibits self-learning via a prominent neuropil region, the mushroom-bodies. Only extended training can overcome this inhibition and lead to habit formation by engaging the self-learning mechanism. Self-learning transforms spontaneous, flexible actions into stereotyped, habitual responses.     

Optimism as a Prior Belief about the Probability of Future Reward

Optimists hold positive a priori beliefs about the future. In Bayesian statistical theory, a priori beliefs can be overcome by experience. However, optimistic beliefs can at times appear surprisingly resistant to evidence, suggesting that optimism might also influence how new information is selected and learned. Here, we use a novel Pavlovian conditioning task, embedded in a normative framework, to directly assess how trait optimism, as classically measured using self-report questionnaires, influences choices between visual targets, by learning about their association with reward progresses. We find that trait optimism relates to an a priori belief about the likelihood of rewards, but not losses, in our task. Critically, this positive belief behaves like a probabilistic prior, i.e. its influence reduces with increasing experience. Contrary to findings in the literature related to unrealistic optimism and self-beliefs, it does not appear to influence the iterative learning process directly.     

Comparison of differential Pavlovian conditioning in whole animals and physiological preparations of Pleurobranchaea: implications of motor pattern variability

The present study compares differential Pavlovian conditioning in whole animals with the behavior of the same animals during electrophysiological recording. Untrained specimens of the sea slug Pleurobranchaea did not discriminate between two appetitive stimuli, one derived from an extract of beer (Sbr) and the other from a homogenate of squid muscle (Ssq). When animals received Sbr as the CS+ and Ssq as the CS- in a single day of five-trial, differential Pavlovian conditioning they learned to avoid selectively the Sbr but continued to exhibit appetitive responses to Ssq. Quantitative measures show that there was over a 1000-fold increase in the thresholds of the proboscis extension and bite-strike responses, many animals ceased all feeding behavior, and exhibited withdrawal responses to Sbr. We examined the behavior of the same trained animals immediately before preparing them for physiological recording and during the recording session. There was a close one-to-one relationship between these behavioral observations, showing that the qualitative and quantitative features of whole-animal Pavlovian conditioning persist into the physiological preparations. Unexpectedly, motor patterns from untrained preparations showed considerable variability both within the same preparation at different times and between preparations; conditioning appeared to increase such variability. Thus, it was not possible to state unequivocally the behavior of the animal by examining the electromyogram recording alone. Many of the trained preparations not only exhibited suppressed feeding behavior and withdrawal responses to Sbr, but, as a consequence of the multifunctional nature of the Pleurobranchaea buccal-oral system, also regurgitated previously ingested Ssq or squid meat when they were stimulated with Sbr. We discuss the findings with respect to self-organizing mechanisms that may establish motor patterns in multifunctional systems, and suggest that such mechanisms may lead to the generation of behaviors that are not specifically encoded by the conditioned cellular changes.     

The influence of serotonin on fear learning

Learning of associations between aversive stimuli and predictive cues is the basis of Pavlovian fear conditioning and is driven by a mismatch between expectation and outcome. To investigate whether serotonin modulates the formation of such aversive cue-outcome associations, we used functional magnetic resonance imaging (fMRI) and dietary tryptophan depletion to reduce brain serotonin (5-HT) levels in healthy human subjects. In a Pavlovian fear conditioning paradigm, 5-HT depleted subjects compared to a non-depleted control group exhibited attenuated autonomic responses to cues indicating the upcoming of an aversive event. These results were closely paralleled by reduced aversive learning signals in the amygdala and the orbitofrontal cortex, two prominent structures of the neural fear circuit. In agreement with current theories of serotonin as a motivational opponent system to dopamine in fear learning, our data provide first empirical evidence for a role of serotonin in representing formally derived learning signals for aversive events.     

Pavlovian conditional stimuli cannot always be defined solely in physical terms

Twenty-four university students received differential Pavlovian conditioning with two colored stimuli separately accompanied by shock, and two other colored stimuli separately presented without shock. The reinforced and nonreinforced pairs of stimuli both contained complementary elements. After differentiation between the reinforced and nonreinforced elements was established, the complementary pairs were each additively mixed, (i.e., presented at the same time and in the same locus), producing two identical white compounds (established by pilot study). The subjects' skin-conductance responses to the two compounds showed that their different conditioning histories did not result in different responses. Rather, a simple declining function was obtained, resembling habituation or extinction. It was concluded that the definition of the conditional stimulus as a physical event is inappropriate in studies in which physically different stimuli may result in identical internal processes (or phenomenologic experiences)--for example, in additive color mixture.     

Environmental variables that ameliorate extinction learning deficits in the 129S1/SvlmJ mouse strain

Fear conditioning is an associative learning process by which organisms learn to avoid environmental stimuli that are predictive of aversive outcomes. Fear extinction learning is a process by which avoidance of fear‐conditioned stimuli is attenuated when the environmental stimuli is no longer predictive of the aversive outcome. Aberrant fear conditioning and extinction learning are key elements in the development of several anxiety disorders. The 129S1 inbred strain of mice is used as an animal model for maladaptive fear learning because this strain has been shown to generalize fear to other nonaversive stimuli and is less capable of extinguishing fear responses relative to other mouse strains, such as the C57BL/6. Here we report new environmental manipulations that enhance fear and extinction learning, including the ability to discriminate between an aversively paired tone and a neutral tone, in both the 129S1 and C57BL/6 strains of mice. Specifically, we show that discontinuous (“pipped”) tone stimuli significantly enhance within‐session extinction learning and the discrimination between neutral and aversively paired stimuli in both strains. Furthermore, we find that extinction training in novel contexts significantly enhances the consolidation and recall of extinction learning for both strains. Cumulatively, these results underscore how environmental changes can be leveraged to ameliorate maladaptive learning in animal models and may advance cognitive and behavioral therapeutic strategies.     

Japanese quail as a model system for studying the neuroendocrine control of reproductive and social behaviors

Japanese quail (Coturnix japonica; referred to simply as quail in this article) readily exhibit sexual behavior and related social behaviors in captive conditions and have therefore proven valuable for studies of how early social experience can shape adult mate preference and sexual behavior. Quail have also been used in sexual conditioning studies illustrating that natural stimuli predict successful reproduction via Pavlovian processes. In addition, they have proven to be a good model to study how variation in photoperiod regulates reproduction and how variation in gonadal steroid hormones controls sexual behavior. For example, studies have shown that testosterone activates male-typical behaviors after being metabolized into estrogenic and androgenic metabolites. A critical site of action for these metabolites is the medial preoptic nucleus (POM), which is larger in males than in females. The enzyme aromatase converts testosterone to estradiol and is enriched in the POM in a male-biased fashion. Quail studies were the first to show that this enzyme is regulated both relatively slowly via genomic actions of steroids and more quickly via phosphorylation. With this base of knowledge and the recent cloning of the entire genome of the closely related chicken, quail will be valuable for future studies connecting gene expression to sexual and social behaviors.     

A novel role for acid‐sensing ion channels in Pavlovian reward conditioning

Pavlovian fear conditioning has been shown to depend on acid‐sensing ion channel‐1A (ASIC1A); however, it is unknown whether conditioning to rewarding stimuli also depends on ASIC1A. Here, we tested the hypothesis that ASIC1A contributes to Pavlovian conditioning to a non‐drug reward. We found effects of ASIC1A disruption depended on the relationship between the conditional stimulus (CS) and the unconditional stimulus (US), which was varied between five experiments. In experiment 1, when the CS preceded the US signaling an upcoming reward, Asic1a^?/? mice exhibited a deficit in conditioning compared to Asic1a^+/+ mice. Alternatively, in experiment 2, when the CS coinitiated with the US and signaled immediate reward availability, the Asic1a^?/? mice exhibited an increase in conditioned responses compared to Asic1a^+/+ mice, which contrasted with the deficits in the first experiment. Furthermore, in experiments 3 and 4, when the CS partially overlapped in time with the US, or the CS was shortened and coinitiated with the US, the Asic1a^?/? mice did not differ from control mice. The contrasting outcomes were likely because of differences in conditioning because in experiment 5 neither the Asic1a^?/? nor Asic1a^+/+ mice acquired conditioned responses when the CS and US were explicitly unpaired. Taken together, these results suggest that the effects of ASIC1A disruption on reward conditioning depend on the temporal relationship between the CS and US. Furthermore, these results suggest that ASIC1A plays a critical, yet nuanced role in Pavlovian conditioning. More research will be needed to deconstruct the roles of ASIC1A in these fundamental forms of learning and memory.     

Odor Detection in Manduca sexta Is Optimized when Odor Stimuli Are Pulsed at a Frequency Matching the Wing Beat during Flight

Sensory systems sample the external world actively, within the context of self-motion induced disturbances. Mammals sample olfactory cues within the context of respiratory cycles and have adapted to process olfactory information within the time frame of a single sniff cycle. In plume tracking insects, it remains unknown whether olfactory processing is adapted to wing beating, which causes similar physical effects as sniffing. To explore this we first characterized the physical properties of our odor delivery system using hotwire anemometry and photo ionization detection, which confirmed that odor stimuli were temporally structured. Electroantennograms confirmed that pulse trains were tracked physiologically. Next, we quantified odor detection in moths in a series of psychophysical experiments to determine whether pulsing odor affected acuity. Moths were first conditioned to respond to a target odorant using Pavlovian olfactory conditioning. At 24 and 48 h after conditioning, moths were tested with a dilution series of the conditioned odor. On separate days odor was presented either continuously or as 20 Hz pulse trains to simulate wing beating effects. We varied pulse train duty cycle, olfactometer outflow velocity, pulsing method, and odor. Results of these studies, established that detection was enhanced when odors were pulsed. Higher velocity and briefer pulses also enhanced detection. Post hoc analysis indicated enhanced detection was the result of a significantly lower behavioral response to blank stimuli when presented as pulse trains. Since blank responses are a measure of false positive responses, this suggests that the olfactory system makes fewer errors (i.e. is more reliable) when odors are experienced as pulse trains. We therefore postulate that the olfactory system of Manduca sexta may have evolved mechanisms to enhance odor detection during flight, where the effects of wing beating represent the norm. This system may even exploit temporal structure in a manner similar to sniffing.     

Associative learning: Pavlovian conditioning without awareness

Can Pavlovian conditioning occur outside of awareness? Yes, according to a new study showing that, under a particular set of circumstances, visual stimuli can become associated with aversive outcomes without participants ever seeing the stimuli.     

Nucleus accumbens neurons dynamically respond to appetitive and aversive associative learning

To survive, individuals must learn to associate cues in the environment with emotionally relevant outcomes. This association is partially mediated by the nucleus accumbens (NAc), a key brain region of the reward circuit that is mainly composed by GABAergic medium spiny neurons (MSNs), that express either dopamine receptor D1 or D2. Recent studies showed that both populations can drive reward and aversion, however, the activity of these neurons during appetitive and aversive Pavlovian conditioning remains to be determined. Here, we investigated the relevance of D1- and D2-neurons in associative learning, by measuring calcium transients with fiber photometry during appetitive and aversive Pavlovian tasks in mice. Sucrose was used as a positive valence unconditioned stimulus (US) and foot shock was used as a negative valence US. We show that during appetitive Pavlovian conditioning, D1- and D2-neurons exhibit a general increase in activity in response to the conditioned stimuli (CS). Interestingly, D1- and D2-neurons present distinct changes in activity after sucrose consumption that dynamically evolve throughout learning. During the aversive Pavlovian conditioning, D1- and D2-neurons present an increase in the activity in response to the CS and to the US (shock). Our data support a model in which D1- and D2-neurons are concurrently activated during appetitive and aversive conditioning.

     

Modelling Individual Differences in the Form of Pavlovian Conditioned Approach Responses: A Dual Learning Systems Approach with Factored Representations

Reinforcement Learning has greatly influenced models of conditioning, providing powerful explanations of acquired behaviour and underlying physiological observations. However, in recent autoshaping experiments in rats, variation in the form of Pavlovian conditioned responses (CRs) and associated dopamine activity, have questioned the classical hypothesis that phasic dopamine activity corresponds to a reward prediction error-like signal arising from a classical Model-Free system, necessary for Pavlovian conditioning. Over the course of Pavlovian conditioning using food as the unconditioned stimulus (US), some rats (sign-trackers) come to approach and engage the conditioned stimulus (CS) itself – a lever – more and more avidly, whereas other rats (goal-trackers) learn to approach the location of food delivery upon CS presentation. Importantly, although both sign-trackers and goal-trackers learn the CS-US association equally well, only in sign-trackers does phasic dopamine activity show classical reward prediction error-like bursts. Furthermore, neither the acquisition nor the expression of a goal-tracking CR is dopamine-dependent. Here we present a computational model that can account for such individual variations. We show that a combination of a Model-Based system and a revised Model-Free system can account for the development of distinct CRs in rats. Moreover, we show that revising a classical Model-Free system to individually process stimuli by using factored representations can explain why classical dopaminergic patterns may be observed for some rats and not for others depending on the CR they develop. In addition, the model can account for other behavioural and pharmacological results obtained using the same, or similar, autoshaping procedures. Finally, the model makes it possible to draw a set of experimental predictions that may be verified in a modified experimental protocol. We suggest that further investigation of factored representations in computational neuroscience studies may be useful.     

Amygdala: Neuroanatomy and neurophysiology of fear

This review is devoted to neuroanatomical and neurophysiological mechanisms of Pavlovian fear conditioning with a focus on the amydgalae, two subcortical nuclear groups, as primary structures responsible for controlling conditioned fear responses, and synaptic plasticity at their afferent and efferent projections as a cellular mechanism to mediate the formation and retention of fear memory. We survey current data on anatomical organization of the amygdaloid complex, as well as on its afferent and efferent projections and their functional significance. A special consideration is given to auditory inputs to the amygdala to analyze the mechanisms of aversive conditioning to sensory (acoustic) stimuli.     

Differential Pavlovian conditioning in the mollusc Pleurobranchaea

The present differential Pavlovian conditioning experiments on the sea slug Pleurobranchaea californica extend conditioning described in a preceding paper and provide the conditioning foundation for studies reported in another accompanying paper comparing learned behavior in whole animals with the behavior and motor patterns of electrophysiological preparations. All animals received two appetitive-conditioned stimuli (CSs), one derived from beer (Sbr) and the other derived from squid muscle (Ssq), in different temporal relationships to an electric shock unconditioned stimulus (UCS). Two groups of animals were run concurrently. One group (n = 19) received Sbr as the CS+ in close temporal pairing with the UCS, and Ssq as the CS- explicitly unpaired with the UCS (Sbr +/Ssq-). The second group (n = 20) received the opposite contingencies (Sbr-/Ssq+). All animals received only one day of conditioning involving 5 trials with an intertrial interval of 2 h. There were two replicate experiments, each involving about half of the total n, and each yielding similar results as the sum we report here. Before conditioning, animals exhibited feeding behavior (extension of the proboscis and bite-strike responses) to both stimuli at similar low thresholds. Conditioning produced long-term behavioral changes in all animals throughout the 4.5-day postconditioning observation period. However, only the Sbr+/Ssq- animals consistently exhibited the appropriate differentially conditioned food-aversion behavior which consisted of strong withdrawal and high-threshold feeding responses to Sbr, and low-threshold feeding responses to Ssq. We discuss the possibility that such differences between Sbr+/Ssq- and Sbr-/Ssq+ conditioning may arise either from inherent differences in the responses of the animals to Sbr and Ssq, or, as seems more likely to us, from training and testing effects produced by differences in the compositions of the two stimuli.     

Aversive reinforcement improves visual discrimination learning in free-flying honeybees

Background

Learning and perception of visual stimuli by free-flying honeybees has been shown to vary dramatically depending on the way insects are trained. Fine color discrimination is achieved when both a target and a distractor are present during training (differential conditioning), whilst if the same target is learnt in isolation (absolute conditioning), discrimination is coarse and limited to perceptually dissimilar alternatives. Another way to potentially enhance discrimination is to increase the penalty associated with the distractor. Here we studied whether coupling the distractor with a highly concentrated quinine solution improves color discrimination of both similar and dissimilar colors by free-flying honeybees. As we assumed that quinine acts as an aversive stimulus, we analyzed whether aversion, if any, is based on an aversive sensory input at the gustatory level or on a post-ingestional malaise following quinine feeding.

Methodology/Principal Findings

We show that the presence of a highly concentrated quinine solution (60 mM) acts as an aversive reinforcer promoting rejection of the target associated with it, and improving discrimination of perceptually similar stimuli but not of dissimilar stimuli. Free-flying bees did not use remote cues to detect the presence of quinine solution; the aversive effect exerted by this substance was mediated via a gustatory input, i.e. via a distasteful sensory experience, rather than via a post-ingestional malaise.

Conclusion

The present study supports the hypothesis that aversion conditioning is important for understanding how and what animals perceive and learn. By using this form of conditioning coupled with appetitive conditioning in the framework of a differential conditioning procedure, it is possible to uncover discrimination capabilities that may remain otherwise unsuspected. We show, therefore, that visual discrimination is not an absolute phenomenon but can be modulated by experience.     

Conditioning and sexual behavior: a review

Sexual behavior is directed by a sophisticated interplay between steroid hormone actions in the brain that give rise to sexual arousability and experience with sexual reward that gives rise to expectations of competent sexual activity, sexual desire, arousal, and performance. Sexual experience allows animals to form instrumental associations between internal or external stimuli and behaviors that lead to different sexual rewards. Furthermore, Pavlovian associations between internal and external stimuli allow animals to predict sexual outcomes. These two types of learning build upon instinctual mechanisms to create distinctive, and seemingly "automated," patterns of sexual response. This article reviews the literature on conditioning and sexual behavior with a particular emphasis on incentive sequences of sexual behavior that move animals from distal to proximal with regard to sexual stimuli during appetitive phases of behavior and ultimately result in copulatory interaction and mating during consummatory phases of behavior. Accordingly, the role of learning in sexual excitement, in behaviors that bring about the opportunity to mate, in courtship and solicitation displays, in sexual arousal and copulatory behaviors, in sexual partner preferences, and the short- and long-term influence of copulatory experience on sexual and reproductive function is examined. Although hormone actions set the stage for sexual activity by generating the ability of animals to become sexually excited and aroused, it is each animal's unique experience with sexual behavior and sexual reward that molds the strength of responses made toward sexual incentives.