Light-induced activation of distinct modulatory neurons triggers appetitive or aversive learning in Drosophila larvae

During classical conditioning, a positive or negative value is assigned to a previously neutral stimulus, thereby changing its significance for behavior. If an odor is associated with a negative stimulus, it can become repulsive. Conversely, an odor associated with a reward can become attractive. By using Drosophila larvae as a model system with minimal brain complexity, we address the question of which neurons attribute these values to odor stimuli. In insects, dopaminergic neurons are required for aversive learning, whereas octopaminergic neurons are necessary and sufficient for appetitive learning. However, it remains unclear whether two independent neuronal populations are sufficient to mediate such antagonistic values. We report the use of transgenically expressed channelrhodopsin-2, a light-activated cation channel, as a tool for optophysiological stimulation of genetically defined neuronal populations in Drosophila larvae. We demonstrate that distinct neuronal populations can be activated simply by illuminating the animals with blue light. Light-induced activation of dopaminergic neurons paired with an odor stimulus induces aversive memory formation, whereas activation of octopaminergic/tyraminergic neurons induces appetitive memory formation. These findings demonstrate that antagonistic modulatory subsystems are sufficient to substitute for aversive and appetitive reinforcement during classical conditioning.     

Contrasting Role of Octopamine in Appetitive and Aversive Learning in the Crab Chasmagnathus

Background

Biogenic amines are implicated in reinforcing associative learning. Octopamine (OA) is considered the invertebrate counterpart of noradrenaline and several studies in insects converge on the idea that OA mediates the reward in appetitive conditioning. However, it is possible to assume that OA could have a different role in an aversive conditioning.

Methodology/Principal Findings

Here we pharmacologically studied the participation of OA in two learning processes in the crab Chasmagnathus granulatus, one appetitive and one aversive. It is shown that the aversive memory is impaired by an OA injection applied immediately or 30 minutes after the last training trial. By contrast, the appetitive memory is blocked by OA antagonists epinastine and mianserine, but enhanced by OA when injected together with the supply of a minimum amount of reinforcement. Finally, double-learning experiments in which crabs are given the aversive and the appetitive learning either successively or simultaneously allow us to study the interaction between both types of learning and analyze the presumed action of OA. We found that the appetitive training offered immediately, but not one hour, after an aversive training has an amnesic effect on the aversive memory, mimicking the effect and the kinetic of an OA injection.

Conclusions/Significance

Our results demonstrate that the role of OA is divergent in two memory processes of opposite signs: on the one hand it would mediate the reinforcement in appetitive learning, and on the other hand it has a deleterious effect over aversive memory consolidation.     

Aversive learning in honeybees revealed by the olfactory conditioning of the sting extension reflex

Invertebrates have contributed greatly to our understanding of associative learning because they allow learning protocols to be combined with experimental access to the nervous system. The honeybee Apis mellifera constitutes a standard model for the study of appetitive learning and memory since it was shown, almost a century ago, that bees learn to associate different sensory cues with a reward of sugar solution. However, up to now, no study has explored aversive learning in bees in such a way that simultaneous access to its neural bases is granted. Using odorants paired with electric shocks, we conditioned the sting extension reflex, which is exhibited by harnessed bees when subjected to a noxious stimulation. We show that this response can be conditioned so that bees learn to extend their sting in response to the odorant previously punished. Bees also learn to extend the proboscis to one odorant paired with sugar solution and the sting to a different odorant paired with electric shock, thus showing that they can master both appetitive and aversive associations simultaneously. Responding to the appropriate odorant with the appropriate response is possible because two different biogenic amines, octopamine and dopamine subserve appetitive and aversive reinforcement, respectively. While octopamine has been previously shown to substitute for appetitive reinforcement, we demonstrate that blocking of dopaminergic, but not octopaminergic, receptors suppresses aversive learning. Therefore, aversive learning in honeybees can now be accessed both at the behavioral and neural levels, thus opening new research avenues for understanding basic mechanisms of learning and memory.     

A Predictive Reinforcement Model of Dopamine Neurons for Learning Approach Behavior

A neural network model of how dopamine and prefrontal cortex activity guides short- and long-term information processing within the cortico-striatal circuits during reward-related learning of approach behavior is proposed. The model predicts two types of reward-related neuronal responses generated during learning: (1) cell activity signaling errors in the prediction of the expected time of reward delivery and (2) neural activations coding for errors in the prediction of the amount and type of reward or stimulus expectancies. The former type of signal is consistent with the responses of dopaminergic neurons, while the latter signal is consistent with reward expectancy responses reported in the prefrontal cortex. It is shown that a neural network architecture that satisfies the design principles of the adaptive resonance theory of Carpenter and Grossberg (1987) can account for the dopamine responses to novelty, generalization, and discrimination of appetitive and aversive stimuli. These hypotheses are scrutinized via simulations of the model in relation to the delivery of free food outside a task, the timed contingent delivery of appetitive and aversive stimuli, and an asymmetric, instructed delay response task.     

The Role of Dopamine in Drosophila Larval Classical Olfactory Conditioning

Learning and memory is not an attribute of higher animals. Even Drosophila larvae are able to form and recall an association of a given odor with an aversive or appetitive gustatory reinforcer. As the Drosophila larva has turned into a particularly simple model for studying odor processing, a detailed neuronal and functional map of the olfactory pathway is available up to the third order neurons in the mushroom bodies. At this point, a convergence of olfactory processing and gustatory reinforcement is suggested to underlie associative memory formation. The dopaminergic system was shown to be involved in mammalian and insect olfactory conditioning. To analyze the anatomy and function of the larval dopaminergic system, we first characterize dopaminergic neurons immunohistochemically up to the single cell level and subsequent test for the effects of distortions in the dopamine system upon aversive (odor-salt) as well as appetitive (odor-sugar) associative learning. Single cell analysis suggests that dopaminergic neurons do not directly connect gustatory input in the larval suboesophageal ganglion to olfactory information in the mushroom bodies. However, a number of dopaminergic neurons innervate different regions of the brain, including protocerebra, mushroom bodies and suboesophageal ganglion. We found that dopamine receptors are highly enriched in the mushroom bodies and that aversive and appetitive olfactory learning is strongly impaired in dopamine receptor mutants. Genetically interfering with dopaminergic signaling supports this finding, although our data do not exclude on naïve odor and sugar preferences of the larvae. Our data suggest that dopaminergic neurons provide input to different brain regions including protocerebra, suboesophageal ganglion and mushroom bodies by more than one route. We therefore propose that different types of dopaminergic neurons might be involved in different types of signaling necessary for aversive and appetitive olfactory memory formation respectively, or for the retrieval of these memory traces. Future studies of the dopaminergic system need to take into account such cellular dissociations in function in order to be meaningful.     

Memory retention capacity using two different training methods,appetitive and aversive learning,in juvenile red sea bream Chrysophrys major

Memory retention based on appetitive and aversive learning was studied in juvenile red sea bream Chrysophrys major. The fish were individually trained via appetitive and aversive learning. In general, they retained appetitive memories for 30 days, but not for 60 days. Conversely, aversive memory endured for 1 day, but not for 3 days or longer. Analyses at the individual level revealed that some fish retained appetitive memories for 60 days, whereas others lost it within 3 days; this suggests considerable variability in memory retention capacity among individual fish. The memory duration for aversive learning was remarkably short, which should be considered when releasing trained fish into the wild for stock enhancement. Furthermore, the high inter-individual variability suggests that evaluating memory retention capacity through group experiments might lead to overestimation of fishes’ ability.     

Beetles, boxes and brain cells: neural mechanisms underlying valuation and learning

Sensory cues in the environment can predict the availability of reward. Through experience, humans and animals learn these predictions and use them to guide their actions. For example, we can learn to discriminate chanterelles from ordinary champignons through experience. Assuming the development of a taste for the complex and lingering flavors of chanterelles, we therefore learn to value the same action--picking mushrooms--differentially depending upon the appearance of a mushroom. One major goal of cognitive neuroscience is to understand the neural mechanisms that underlie this sort of learning. Because the acquisition of rewards motivates much behavior, recent efforts have focused on describing the neural signals related to learning the value of stimuli and actions. Neurons in the basal ganglia, in midbrain dopamine areas, in frontal and parietal cortices and in other brain areas, all modulate their activity in relation to aspects of learning. By training monkeys on various behavioral tasks, recent studies have begun to characterize how neural signals represent distinct processes, such as the timing of events, motivation, absolute (objective) and relative (subjective) valuation, and the formation of associative links between stimuli and potential actions. In addition, a number of studies have either further characterized dopamine signals or sought to determine how such signaling might interact with target structures, such as the striatum and rhinal cortex, to underlie learning.     

Analysis of associative learning in the terrestrial mollusc Limax maximus. II. Appetitive learning

Summary The odor and taste processing systems of the terrestrial mollusc Limax maximus have been shown capable of a number of complex computations. Most of the complex higher-order features of Limax learning have been demonstrated using differential aversive conditioning. The present experiments probe the appetitive learning ability of Limax. In the first experiment a differential appetitive classical conditioning procedure was used. An aversive CS⁺ odor was paired with an attractive taste while a CS^– odor was explicitly unpaired with the attractive taste. This appetitive conditioning procedure dramatically increased the preference for the CS⁺ odor. Further experiments determined the time course of acquisition, the effect of an extinction procedure and long-term retention of the appetitive conditioning. Now that Limax has been shown capable of appetitive conditioning, the neural network simulation of Limax learning, called LIMAX, can be examined for its ability to display appetitive conditioning.Abbreviations AA amyl acetate - CS conditioned stimulus - ISI interstimulus interval - ITI intertrial interval - MCH methylcyclohexanol - US unconditioned stimulus     

Genotypic Influence on Aversive Conditioning in Honeybees,Using a Novel Thermal Reinforcement Procedure

In Pavlovian conditioning, animals learn to associate initially neutral stimuli with positive or negative outcomes, leading to appetitive and aversive learning respectively. The honeybee (Apis mellifera) is a prominent invertebrate model for studying both versions of olfactory learning and for unraveling the influence of genotype. As a queen bee mates with about 15 males, her worker offspring belong to as many, genetically-different patrilines. While the genetic dependency of appetitive learning is well established in bees, it is not the case for aversive learning, as a robust protocol was only developed recently. In the original conditioning of the sting extension response (SER), bees learn to associate an odor (conditioned stimulus - CS) with an electric shock (unconditioned stimulus - US). This US is however not a natural stimulus for bees, which may represent a potential caveat for dissecting the genetics underlying aversive learning. We thus first tested heat as a potential new US for SER conditioning. We show that thermal stimulation of several sensory structures on the bee’s body triggers the SER, in a temperature-dependent manner. Moreover, heat applied to the antennae, mouthparts or legs is an efficient US for SER conditioning. Then, using microsatellite analysis, we analyzed heat sensitivity and aversive learning performances in ten worker patrilines issued from a naturally inseminated queen. We demonstrate a strong influence of genotype on aversive learning, possibly indicating the existence of a genetic determinism of this capacity. Such determinism could be instrumental for efficient task partitioning within the hive.     

Salt Appetite Is Reduced by a Single Experience of Drinking Hypertonic Saline in the Adult Rat

Salt appetite, the primordial instinct to favorably ingest salty substances, represents a vital evolutionary important drive to successfully maintain body fluid and electrolyte homeostasis. This innate instinct was shown here in Sprague-Dawley rats by increased ingestion of isotonic saline (IS) over water in fluid intake tests. However, this appetitive stimulus was fundamentally transformed into a powerfully aversive one by increasing the salt content of drinking fluid from IS to hypertonic saline (2% w/v NaCl, HS) in intake tests. Rats ingested HS similar to IS when given no choice in one-bottle tests and previous studies have indicated that this may modify salt appetite. We thus investigated if a single 24 h experience of ingesting IS or HS, dehydration (DH) or 4% high salt food (HSD) altered salt preference. Here we show that 24 h of ingesting IS and HS solutions, but not DH or HSD, robustly transformed salt appetite in rats when tested 7 days and 35 days later. Using two-bottle tests rats previously exposed to IS preferred neither IS or water, whereas rats exposed to HS showed aversion to IS. Responses to sweet solutions (1% sucrose) were not different in two-bottle tests with water, suggesting that salt was the primary aversive taste pathway recruited in this model. Inducing thirst by subcutaneous administration of angiotensin II did not overcome this salt aversion. We hypothesised that this behavior results from altered gene expression in brain structures important in thirst and salt appetite. Thus we also report here lasting changes in mRNAs for markers of neuronal activity, peptide hormones and neuronal plasticity in supraoptic and paraventricular nuclei of the hypothalamus following rehydration after both DH and HS. These results indicate that a single experience of drinking HS is a memorable one, with long-term changes in gene expression accompanying this aversion to salty solutions.     

Coordinated activity of ventral tegmental neurons adapts to appetitive and aversive learning

Our understanding of how value-related information is encoded in the ventral tegmental area (VTA) is based mainly on the responses of individual putative dopamine neurons. In contrast to cortical areas, the nature of coordinated interactions between groups of VTA neurons during motivated behavior is largely unknown. These interactions can strongly affect information processing, highlighting the importance of investigating network level activity. We recorded the activity of multiple single units and local field potentials (LFP) in the VTA during a task in which rats learned to associate novel stimuli with different outcomes. We found that coordinated activity of VTA units with either putative dopamine or GABA waveforms was influenced differently by rewarding versus aversive outcomes. Specifically, after learning, stimuli paired with a rewarding outcome increased the correlation in activity levels between unit pairs whereas stimuli paired with an aversive outcome decreased the correlation. Paired single unit responses also became more redundant after learning. These response patterns flexibly tracked the reversal of contingencies, suggesting that learning is associated with changing correlations and enhanced functional connectivity between VTA neurons. Analysis of LFP recorded simultaneously with unit activity showed an increase in the power of theta oscillations when stimuli predicted reward but not an aversive outcome. With learning, a higher proportion of putative GABA units were phase locked to the theta oscillations than putative dopamine units. These patterns also adapted when task contingencies were changed. Taken together, these data demonstrate that VTA neurons organize flexibly as functional networks to support appetitive and aversive learning.     

Driving opposing behaviors with ensembles of piriform neurons

Anatomic and physiologic studies have suggested a model in which neurons of the piriform cortex receive convergent input from random collections of glomeruli. In this model, odor representations can only be afforded behavioral significance upon experience. We have devised an experimental strategy that permits us to ask whether the activation of an arbitrarily chosen subpopulation of neurons in piriform cortex can elicit different behavioral responses dependent upon learning. Activation of a small subpopulation of piriform neurons expressing channelrhodopsin at multiple loci in the piriform cortex, when paired with reward or shock, elicits either appetitive or aversive behavior. Moreover, we demonstrate that different subpopulations of piriform neurons expressing ChR2 can be discriminated and independently entrained to elicit distinct behaviors. These observations demonstrate that the piriform cortex is sufficient to elicit learned behavioral outputs in the absence of sensory input. These data imply that the piriform does not use spatial order to map odorant identity or behavioral output.     

Nucleus accumbens neurons dynamically respond to appetitive and aversive associative learning

To survive, individuals must learn to associate cues in the environment with emotionally relevant outcomes. This association is partially mediated by the nucleus accumbens (NAc), a key brain region of the reward circuit that is mainly composed by GABAergic medium spiny neurons (MSNs), that express either dopamine receptor D1 or D2. Recent studies showed that both populations can drive reward and aversion, however, the activity of these neurons during appetitive and aversive Pavlovian conditioning remains to be determined. Here, we investigated the relevance of D1- and D2-neurons in associative learning, by measuring calcium transients with fiber photometry during appetitive and aversive Pavlovian tasks in mice. Sucrose was used as a positive valence unconditioned stimulus (US) and foot shock was used as a negative valence US. We show that during appetitive Pavlovian conditioning, D1- and D2-neurons exhibit a general increase in activity in response to the conditioned stimuli (CS). Interestingly, D1- and D2-neurons present distinct changes in activity after sucrose consumption that dynamically evolve throughout learning. During the aversive Pavlovian conditioning, D1- and D2-neurons present an increase in the activity in response to the CS and to the US (shock). Our data support a model in which D1- and D2-neurons are concurrently activated during appetitive and aversive conditioning.

     

Distinct traces for appetitive versus aversive olfactory memories in DPM neurons of Drosophila

The global logic used by the brain for differentially encoding positive and negative experiences remains unknown along with how such experiences are represented by collections of memory traces at the cellular level. Here we contrast the cellular memory traces that form in the dorsal paired medial (DPM) neurons of Drosophila after conditioning flies with odors associated with aversive or appetitive unconditioned stimuli (US). Our results show that the appetitive DPM neuron trace is distinguished from the aversive in three fundamental ways: (1) The DPM neurons do not respond to an appetitive US of sucrose by itself, in contrast to their robust response to an aversive US. (2) The appetitive trace persists for twice as long as the aversive trace. (3) The appetitive trace is expressed in both neurite branches of the neuron, rather than being confined to a single branch like the aversive trace. In addition, we demonstrate that training flies with nonnutritive sugars that elicit a behavioral memory that decays within 24 hr generates, like aversive conditioning, a short-lived and branch-restricted memory trace. These results indicate that the persistence and breadth of the DPM neuron memory trace influences the duration of behavioral memory.     

The role of the striatum in aversive learning and aversive prediction errors

Neuroeconomic studies of decision making have emphasized reward learning as critical in the representation of value-driven choice behaviour. However, it is readily apparent that punishment and aversive learning are also significant factors in motivating decisions and actions. In this paper, we review the role of the striatum and amygdala in affective learning and the coding of aversive prediction errors (PEs). We present neuroimaging results showing aversive PE-related signals in the striatum in fear conditioning paradigms with both primary (shock) and secondary (monetary loss) reinforcers. These results and others point to the general role for the striatum in coding PEs across a broad range of learning paradigms and reinforcer types.     

桔小实蝇雄成虫联系性嗅觉学习

【目的】为了探究桔小实蝇 Bactrocera dorsalis (Hendel)雄成虫的嗅觉学习能力。【方法】本研究采用经典性嗅觉条件反射训练法（classical olfactory conditioning）在室内对固定的羽化后14-17日龄的桔小实蝇雄成虫进行气味与食物的联合学习训练, 即薄荷精油和10%蔗糖溶液联合的奖赏性训练（appetitive conditioning）以及甲基丁香酚(methyl eugenol, ME)和饱和盐溶液联合的惩罚性训练（aversive conditioning）,并以伸喙反射行为（proboscis extension reflex, PER）作为学习与否的判定标准。【结果】经过奖赏性训练后,桔小实蝇雄成虫对薄荷精油的伸喙反射率可从0%增加至68%;而经过惩罚性训练后,桔小实蝇对甲基丁香酚的伸喙反射率可从100%降低至36.54%,且这种伸喙反射率的变化是通过气味条件刺激（conditioned stimulus）和食物非条件刺激（unconditioned stimulus）的对称性联合而产生的。【结论】结果表明,桔小实蝇雄性成虫具有较强的联系性嗅觉学习能力,并且两种刺激的联合是形成学习记忆的必要条件。     

Dynamic Excitatory and Inhibitory Gain Modulation Can Produce Flexible,Robust and Optimal Decision-making

Behavioural and neurophysiological studies in primates have increasingly shown the involvement of urgency signals during the temporal integration of sensory evidence in perceptual decision-making. Neuronal correlates of such signals have been found in the parietal cortex, and in separate studies, demonstrated attention-induced gain modulation of both excitatory and inhibitory neurons. Although previous computational models of decision-making have incorporated gain modulation, their abstract forms do not permit an understanding of the contribution of inhibitory gain modulation. Thus, the effects of co-modulating both excitatory and inhibitory neuronal gains on decision-making dynamics and behavioural performance remain unclear. In this work, we incorporate time-dependent co-modulation of the gains of both excitatory and inhibitory neurons into our previous biologically based decision circuit model. We base our computational study in the context of two classic motion-discrimination tasks performed in animals. Our model shows that by simultaneously increasing the gains of both excitatory and inhibitory neurons, a variety of the observed dynamic neuronal firing activities can be replicated. In particular, the model can exhibit winner-take-all decision-making behaviour with higher firing rates and within a significantly more robust model parameter range. It also exhibits short-tailed reaction time distributions even when operating near a dynamical bifurcation point. The model further shows that neuronal gain modulation can compensate for weaker recurrent excitation in a decision neural circuit, and support decision formation and storage. Higher neuronal gain is also suggested in the more cognitively demanding reaction time than in the fixed delay version of the task. Using the exact temporal delays from the animal experiments, fast recruitment of gain co-modulation is shown to maximize reward rate, with a timescale that is surprisingly near the experimentally fitted value. Our work provides insights into the simultaneous and rapid modulation of excitatory and inhibitory neuronal gains, which enables flexible, robust, and optimal decision-making.     

Electrical stimulation of the primate lateral habenula suppresses saccadic eye movement through a learning mechanism

The lateral habenula (LHb) is a brain structure which represents negative motivational value. Neurons in the LHb are excited by unpleasant events such as reward omission and aversive stimuli, and transmit these signals to midbrain dopamine neurons which are involved in learning and motivation. However, it remains unclear whether these phasic changes in LHb neuronal activity actually influence animal behavior. To answer this question, we artificially activated the LHb by electrical stimulation while monkeys were performing a visually guided saccade task. In one block of trials, saccades to one fixed direction (e.g., right direction) were followed by electrical stimulation of the LHb while saccades to the other direction (e.g., left direction) were not. The direction-stimulation contingency was reversed in the next block. We found that the post-saccadic stimulation of the LHb increased the latencies of saccades in subsequent trials. Notably, the increase of the latency occurred gradually as the saccade was repeatedly followed by the stimulation, suggesting that the effect of the post-saccadic stimulation was accumulated across trials. LHb stimulation starting before saccades, on the other hand, had no effect on saccade latency. Together with previous studies showing LHb activation by reward omission and aversive stimuli, the present stimulation experiment suggests that LHb activity contributes to learning to suppress actions which lead to unpleasant events.     

The political left rolls with the good and the political right confronts the bad: connecting physiology and cognition to preferences

We report evidence that individual-level variation in people's physiological and attentional responses to aversive and appetitive stimuli are correlated with broad political orientations. Specifically, we find that greater orientation to aversive stimuli tends to be associated with right-of-centre and greater orientation to appetitive (pleasing) stimuli with left-of-centre political inclinations. These findings are consistent with recent evidence that political views are connected to physiological predispositions but are unique in incorporating findings on variation in directed attention that make it possible to understand additional aspects of the link between the physiological and the political.     

Flexibility of memory for future-oriented cognition

Memories of daily experiences contain incidental details unique to each experience as well as common latent patterns shared with others. Neural representations focusing on the latter aspect can be reinstated by similar new experiences even though their perceptual features do not match the original experiences perfectly. Such flexible memory use allows for faster learning and better decision-making in novel situations. Here, I review evidence from rodent and primate electrophysiological studies to discuss how memory flexibility is implemented in the spiking activity of neuronal ensembles. These findings uncovered innate and learned coding properties and their potential refinement during sleep that support flexible integration and application of memories for better future adaptation.