Aversive learning in honeybees revealed by the olfactory conditioning of the sting extension reflex

Invertebrates have contributed greatly to our understanding of associative learning because they allow learning protocols to be combined with experimental access to the nervous system. The honeybee Apis mellifera constitutes a standard model for the study of appetitive learning and memory since it was shown, almost a century ago, that bees learn to associate different sensory cues with a reward of sugar solution. However, up to now, no study has explored aversive learning in bees in such a way that simultaneous access to its neural bases is granted. Using odorants paired with electric shocks, we conditioned the sting extension reflex, which is exhibited by harnessed bees when subjected to a noxious stimulation. We show that this response can be conditioned so that bees learn to extend their sting in response to the odorant previously punished. Bees also learn to extend the proboscis to one odorant paired with sugar solution and the sting to a different odorant paired with electric shock, thus showing that they can master both appetitive and aversive associations simultaneously. Responding to the appropriate odorant with the appropriate response is possible because two different biogenic amines, octopamine and dopamine subserve appetitive and aversive reinforcement, respectively. While octopamine has been previously shown to substitute for appetitive reinforcement, we demonstrate that blocking of dopaminergic, but not octopaminergic, receptors suppresses aversive learning. Therefore, aversive learning in honeybees can now be accessed both at the behavioral and neural levels, thus opening new research avenues for understanding basic mechanisms of learning and memory.     

桔小实蝇雄成虫联系性嗅觉学习

【目的】为了探究桔小实蝇 Bactrocera dorsalis (Hendel)雄成虫的嗅觉学习能力。【方法】本研究采用经典性嗅觉条件反射训练法（classical olfactory conditioning）在室内对固定的羽化后14-17日龄的桔小实蝇雄成虫进行气味与食物的联合学习训练, 即薄荷精油和10%蔗糖溶液联合的奖赏性训练（appetitive conditioning）以及甲基丁香酚(methyl eugenol, ME)和饱和盐溶液联合的惩罚性训练（aversive conditioning）,并以伸喙反射行为（proboscis extension reflex, PER）作为学习与否的判定标准。【结果】经过奖赏性训练后,桔小实蝇雄成虫对薄荷精油的伸喙反射率可从0%增加至68%;而经过惩罚性训练后,桔小实蝇对甲基丁香酚的伸喙反射率可从100%降低至36.54%,且这种伸喙反射率的变化是通过气味条件刺激（conditioned stimulus）和食物非条件刺激（unconditioned stimulus）的对称性联合而产生的。【结论】结果表明,桔小实蝇雄性成虫具有较强的联系性嗅觉学习能力,并且两种刺激的联合是形成学习记忆的必要条件。     

Appetitive and aversive learning in Spodoptera littoralis larvae

Adult Lepidoptera are capable of associative learning. This helps them to forage flowers or to find suitable oviposition sites. Larval learning has never been seriously considered because they have limited foraging capabilities and usually depend on adults as concerns their food choices. We tested if Spodoptera littoralis larvae can learn to associate an odor with a tastant using a new classical conditioning paradigm. Groups of larvae were exposed to an unconditioned stimulus (US: fructose or quinine mixed with agar) paired with a conditioned stimulus (CS: hexanol, geraniol or pentyl acetate) in a petri dish. Their reaction to CS was subsequently tested in a petri dish at different time intervals after conditioning. Trained larvae showed a significant preference or avoidance to CS when paired with US depending on the reinforcer used. The training was more efficient when larvae were given a choice between an area where CS-US was paired and an area with no CS (or another odor). In these conditions, the memory formed could be recalled at least 24 h after pairing with an aversive stimulus and only 5 min after pairing with an appetitive stimulus. This learning was specific to CS because trained larvae were able to discriminate CS from another odor that was present during the training but unrewarded. These results suggest that Lepidoptera larvae exhibit more behavioral plasticity than previously appreciated.     

Social Modulation of Stress Reactivity and Learning in Young Worker Honey Bees

Alarm pheromone and its major component isopentylacetate induce stress-like responses in forager honey bees, impairing their ability to associate odors with a food reward. We investigated whether isopentylacetate exposure decreases appetitive learning also in young worker bees. While isopentylacetate-induced learning deficits were observed in guards and foragers collected from a queen-right colony, learning impairments resulting from exposure to this pheromone could not be detected in bees cleaning cells. As cell cleaners are generally among the youngest workers in the colony, effects of isopentylacetate on learning behavior were examined further using bees of known age. Adult workers were maintained under laboratory conditions from the time of adult emergence. Fifty percent of the bees were exposed to queen mandibular pheromone during this period, whereas control bees were not exposed to this pheromone. Isopentylacetate-induced learning impairments were apparent in young (less than one week old) controls, but not in bees of the same age exposed to queen mandibular pheromone. This study reveals young worker bees can exhibit a stress-like response to alarm pheromone, but isopentylacetate-induced learning impairments in young bees are suppressed by queen mandibular pheromone. While isopentylacetate exposure reduced responses during associative learning (acquisition), it did not affect one-hour memory retrieval.     

Nucleus accumbens neurons dynamically respond to appetitive and aversive associative learning

To survive, individuals must learn to associate cues in the environment with emotionally relevant outcomes. This association is partially mediated by the nucleus accumbens (NAc), a key brain region of the reward circuit that is mainly composed by GABAergic medium spiny neurons (MSNs), that express either dopamine receptor D1 or D2. Recent studies showed that both populations can drive reward and aversion, however, the activity of these neurons during appetitive and aversive Pavlovian conditioning remains to be determined. Here, we investigated the relevance of D1- and D2-neurons in associative learning, by measuring calcium transients with fiber photometry during appetitive and aversive Pavlovian tasks in mice. Sucrose was used as a positive valence unconditioned stimulus (US) and foot shock was used as a negative valence US. We show that during appetitive Pavlovian conditioning, D1- and D2-neurons exhibit a general increase in activity in response to the conditioned stimuli (CS). Interestingly, D1- and D2-neurons present distinct changes in activity after sucrose consumption that dynamically evolve throughout learning. During the aversive Pavlovian conditioning, D1- and D2-neurons present an increase in the activity in response to the CS and to the US (shock). Our data support a model in which D1- and D2-neurons are concurrently activated during appetitive and aversive conditioning.

     

Dopamine and octopamine influence avoidance learning of honey bees in a place preference assay

Biogenic amines are widely characterized in pathways evaluating reward and punishment, resulting in appropriate aversive or appetitive responses of vertebrates and invertebrates. We utilized the honey bee model and a newly developed spatial avoidance conditioning assay to probe effects of biogenic amines octopamine (OA) and dopamine (DA) on avoidance learning. In this new protocol non-harnessed bees associate a spatial color cue with mild electric shock punishment. After a number of experiences with color and shock the bees no longer enter the compartment associated with punishment. Intrinsic aspects of avoidance conditioning are associated with natural behavior of bees such as punishment (lack of food, explosive pollination mechanisms, danger of predation, heat, etc.) and their association to floral traits or other spatial cues during foraging. The results show that DA reduces the punishment received whereas octopamine OA increases the punishment received. These effects are dose-dependent and specific to the acquisition phase of training. The effects during acquisition are specific as shown in experiments using the antagonists Pimozide and Mianserin for DA and OA receptors, respectively. This study demonstrates the integrative role of biogenic amines in aversive learning in the honey bee as modeled in a novel non-appetitive avoidance learning assay.     

Associative learning: Hebbian flies

Fruit flies can learn to associate an odor with an aversive stimulus, such as a shock. New findings indicate that disrupting the expression of N-methyl-D-aspartate (NMDA) receptors in flies impairs olfactory conditioning. The findings provide support for a critical role for NMDA receptors in associative learning.     

Analysis of associative learning in the terrestrial mollusc Limax maximus. II. Appetitive learning

Summary The odor and taste processing systems of the terrestrial mollusc Limax maximus have been shown capable of a number of complex computations. Most of the complex higher-order features of Limax learning have been demonstrated using differential aversive conditioning. The present experiments probe the appetitive learning ability of Limax. In the first experiment a differential appetitive classical conditioning procedure was used. An aversive CS⁺ odor was paired with an attractive taste while a CS^– odor was explicitly unpaired with the attractive taste. This appetitive conditioning procedure dramatically increased the preference for the CS⁺ odor. Further experiments determined the time course of acquisition, the effect of an extinction procedure and long-term retention of the appetitive conditioning. Now that Limax has been shown capable of appetitive conditioning, the neural network simulation of Limax learning, called LIMAX, can be examined for its ability to display appetitive conditioning.Abbreviations AA amyl acetate - CS conditioned stimulus - ISI interstimulus interval - ITI intertrial interval - MCH methylcyclohexanol - US unconditioned stimulus     

Can Honey Bees Learn the Removal of a Stimulus as a Conditioning Cue?

Experiments investigated a Pavlovian conditioning situation where the presence and absence of the stimulus are reversed temporally with respect to the presentation of a reward. Instead of a conditioned stimulus (e.g. odor) signaling the presence of a reward, the stimulus (e.g. odor) is present in the environment except just prior to the presence of the reward. Thus, the absence of the stimulus, or offset of the stimulus (e.g. absence of odor), serves as a conditioned stimulus and is the reward cue. Honey bees (Apis mellifera) were used as a model invertebrate system, and the proboscis‐conditioning paradigm was used as the test procedure. Using both simple Pavlovian conditioning and discrimination‐learning protocols, animals learned to associate the onset of an odor as conditioned stimuli when paired with a sucrose reward. They could also learn to associate the onset of a puff of air with a sucrose reward. However, bees could not associate the offset of an order stimulus with the presentation of a sucrose reward in either a simple conditioning or a discrimination‐learning situation. These results support the model that a very different cognitive architecture is used by invertebrates to deal with certain environmental situations, including signaled avoidance.     

Memory retention of appetitive and aversive conditioning in the damselfish Stegastes fuscus

We compared the memory of damselfish Stegastes fuscus in an aversive and appetitive conditioning task. Fish were trained to associate the sides of the tank that corresponded to the presence of a positive (conspecific presence) or negative (electroshock) stimulus. After two conditioning sessions, they were tested for learning. The fish conditioned to the stimulus were then re-tested for memory retention after 5, 10 or 15 days. Both the positive and negative rewards were associated with a specific side of the tank, indicating learning ability. Additionally, in both contexts, S. fuscus stored the information learned and showed similar behavioural patterns after 5, 10 and 15 days, suggesting long-lasting memory. For the ecological context, long lasting memories of social encounters outcomes and negative experiences of threatening situations may confer advantages that ultimately affect fishes’ fitness.     

Cognitive components of color vision in honey bees: how conditioning variables modulate color learning and discrimination

Since the demonstration of color vision in honey bees 100 years ago by Karl von Frisch, appetitive conditioning to color targets has been used as the principal way to access behavioral aspects of bee color vision. Yet, analyses on how conditioning parameters affect color perception remained scarce. Conclusions on bee color vision have often been made without referring them to the experimental context in which they were obtained, and thus presented as absolute facts instead of realizing that subtle variations in conditioning procedures might yield different results. Here, we review evidence showing that color learning and discrimination in bees are not governed by immutable properties of their visual system, but depend on how the insects are trained and thus learn a task. The use of absolute or differential conditioning protocols, the presence of aversive reinforcement in differential conditioning and the degrees of freedom of motor components determine dramatic variations in color discrimination. We, thus, suggest top-down attentional modulation of color vision to explain the changes in color learning and discrimination reviewed here. We discuss the possible neural mechanisms of this modulation and conclude that color vision experiments require a careful consideration of how training parameters shape behavioral responses.     

Light-induced activation of distinct modulatory neurons triggers appetitive or aversive learning in Drosophila larvae

During classical conditioning, a positive or negative value is assigned to a previously neutral stimulus, thereby changing its significance for behavior. If an odor is associated with a negative stimulus, it can become repulsive. Conversely, an odor associated with a reward can become attractive. By using Drosophila larvae as a model system with minimal brain complexity, we address the question of which neurons attribute these values to odor stimuli. In insects, dopaminergic neurons are required for aversive learning, whereas octopaminergic neurons are necessary and sufficient for appetitive learning. However, it remains unclear whether two independent neuronal populations are sufficient to mediate such antagonistic values. We report the use of transgenically expressed channelrhodopsin-2, a light-activated cation channel, as a tool for optophysiological stimulation of genetically defined neuronal populations in Drosophila larvae. We demonstrate that distinct neuronal populations can be activated simply by illuminating the animals with blue light. Light-induced activation of dopaminergic neurons paired with an odor stimulus induces aversive memory formation, whereas activation of octopaminergic/tyraminergic neurons induces appetitive memory formation. These findings demonstrate that antagonistic modulatory subsystems are sufficient to substitute for aversive and appetitive reinforcement during classical conditioning.     

Reversal Learning and Risk‐Averse Foraging Behavior in the Monarch Butterfly,Danaus plexippus (Lepidoptera: Nymphalidae)

Learning ability allows insects to respond to a variable environment, and to adjust their behaviors in response to positive or negative experiences. Pollinating insects readily learn to associate floral characteristics, such as color, shape, or pattern, with appetitive stimuli, such as the presence of a nectar reward. However, in nature pollinators may also encounter flowers that contain distasteful or toxic nectar, or offer highly variable nectar volumes, providing opportunities for aversive learning or risk‐averse foraging behavior. Whereas some bees learn to avoid flowers with unpalatable or unreliable nectar rewards, little is known about how Lepidoptera respond to such stimuli. We used a reversal learning paradigm to establish that monarch butterflies learn to discriminate against colored artificial flowers that contain salt solution, decreasing both number of probes and probing time on flowers of a preferred color and altogether avoiding artificial flowers of a non‐preferred color. In addition, when we offered butterflies artificial flowers of two different colors, both of which contained the same mean nectar volume but which differed in variance, the monarchs exhibited risk‐averse foraging: they probed the constant flowers significantly more than the variable ones, regardless of flower color or butterfly sex. Our results add to our understanding of butterfly foraging behavior, as they demonstrate that monarchs can respond to aversive as well as appetitive stimuli, and can also adjust their foraging behavior to avoid floral resources with high variance rewards.     

Effect of Flumethrin on Survival and Olfactory Learning in Honeybees

Flumethrin has been widely used as an acaricide for the control of Varroa mites in commercial honeybee keeping throughout the world for many years. Here we test the mortality of the Asian honeybee Apis cerana cerana after treatment with flumethrin. We also ask (1) how bees react to the odor of flumethrin, (2) whether its odor induces an innate avoidance response, (3) whether its taste transmits an aversive reinforcing component in olfactory learning, and (4) whether its odor or taste can be associated with reward in classical conditioning. Our results show that flumethrin has a negative effect on Apis ceranàs lifespan, induces an innate avoidance response, acts as a punishing reinforcer in olfactory learning, and interferes with the association of an appetitive conditioned stimulus. Furthermore flumethrin uptake within the colony reduces olfactory learning over an extended period of time.     

Apis mellifera bees acquire long-term olfactory memories within the colony

Early studies indicate that Apis mellifera bees learn nectar odours within their colonies. This form of olfactory learning, however, has not been analysed by measuring well-quantifiable learning performances and the question remains whether it constitutes a 'robust' form of learning. Hence, we asked whether bees acquire long-term olfactory memories within the colony. To this end, we used the bee proboscis extension response. We found that within-the-nest bees do indeed associate the odour (as the conditioned stimulus) with the sugar (as the unconditioned stimulus) present in the incoming nectar, and that the distribution of scented nectar within the colony allows them to establish long-term olfactory memories. This finding is discussed in the context of efficient foraging.     

Conditioning honeybees to discriminate between heritable odors from full and half sisters

Summary Differential conditioning of the proboscis extension reflex in honeybees is used to assess whether worker honeybees can be trained to discriminate between volatile odors emanating from different kin groups consisting of 2 or 20 workers. These odor source group workers are all reared and maintained under identical environmental conditions. They are the progeny of a queen that has been instrumentally inseminated so that eclosing adult workers can be sorted into colormorph full sister patrilines (workers are half sisters across patrilines). We demonstrate that workers are able to discriminate between the odors from groups of 20 individuals only if the groups represent individuals from different patrilines. However, discrimination occurs between groups of 2 individuals even if groups do not represent different patrilines. A number of environmental control experiments are also conducted. From our results we infer that there is heritable variation in the production of volatile odors by worker honeybees at a level that can be detected by the workers.     

Distinct traces for appetitive versus aversive olfactory memories in DPM neurons of Drosophila

The global logic used by the brain for differentially encoding positive and negative experiences remains unknown along with how such experiences are represented by collections of memory traces at the cellular level. Here we contrast the cellular memory traces that form in the dorsal paired medial (DPM) neurons of Drosophila after conditioning flies with odors associated with aversive or appetitive unconditioned stimuli (US). Our results show that the appetitive DPM neuron trace is distinguished from the aversive in three fundamental ways: (1) The DPM neurons do not respond to an appetitive US of sucrose by itself, in contrast to their robust response to an aversive US. (2) The appetitive trace persists for twice as long as the aversive trace. (3) The appetitive trace is expressed in both neurite branches of the neuron, rather than being confined to a single branch like the aversive trace. In addition, we demonstrate that training flies with nonnutritive sugars that elicit a behavioral memory that decays within 24 hr generates, like aversive conditioning, a short-lived and branch-restricted memory trace. These results indicate that the persistence and breadth of the DPM neuron memory trace influences the duration of behavioral memory.     

Aversive reinforcement improves visual discrimination learning in free-flying honeybees

Background

Learning and perception of visual stimuli by free-flying honeybees has been shown to vary dramatically depending on the way insects are trained. Fine color discrimination is achieved when both a target and a distractor are present during training (differential conditioning), whilst if the same target is learnt in isolation (absolute conditioning), discrimination is coarse and limited to perceptually dissimilar alternatives. Another way to potentially enhance discrimination is to increase the penalty associated with the distractor. Here we studied whether coupling the distractor with a highly concentrated quinine solution improves color discrimination of both similar and dissimilar colors by free-flying honeybees. As we assumed that quinine acts as an aversive stimulus, we analyzed whether aversion, if any, is based on an aversive sensory input at the gustatory level or on a post-ingestional malaise following quinine feeding.

Methodology/Principal Findings

We show that the presence of a highly concentrated quinine solution (60 mM) acts as an aversive reinforcer promoting rejection of the target associated with it, and improving discrimination of perceptually similar stimuli but not of dissimilar stimuli. Free-flying bees did not use remote cues to detect the presence of quinine solution; the aversive effect exerted by this substance was mediated via a gustatory input, i.e. via a distasteful sensory experience, rather than via a post-ingestional malaise.

Conclusion

The present study supports the hypothesis that aversion conditioning is important for understanding how and what animals perceive and learn. By using this form of conditioning coupled with appetitive conditioning in the framework of a differential conditioning procedure, it is possible to uncover discrimination capabilities that may remain otherwise unsuspected. We show, therefore, that visual discrimination is not an absolute phenomenon but can be modulated by experience.     

The Sound and the Fury—Bees Hiss when Expecting Danger

Honey bees are important model systems for the investigation of learning and memory and for a better understanding of the neuronal basics of brain function. Honey bees also possess a rich repertoire of tones and sounds, from queen piping and quacking to worker hissing and buzzing. In this study, we tested whether the worker bees’ sounds can be used as a measure of learning. We therefore conditioned honey bees aversively to odours in a walking arena and recorded both their sound production and their movement. Bees were presented with two odours, one of which was paired with an electric shock. Initially, the bees did not produce any sound upon odour presentation, but responded to the electric shock with a strong hissing response. After learning, many bees hissed at the presentation of the learned odour, while fewer bees hissed upon presentation of another odour. We also found that hissing and movement away from the conditioned odour are independent behaviours that can co-occur but do not necessarily do so. Our data suggest that hissing can be used as a readout for learning after olfactory conditioning, but that there are large individual differences between bees concerning their hissing reaction. The basis for this variability and the possible ecological relevance of the bees’ hissing remain to be investigated.