Mushroom Bodies Regulate Habit Formation in Drosophila

To make good decisions, we evaluate past choices to guide later decisions. In most situations, we have the opportunity to simultaneously learn about both the consequences of our choice (i.e., operantly) and the stimuli associated with correct or incorrect choices (i.e., classically) [1]. Interestingly, in many species, including humans, these learning processes occasionally lead to irrational decisions [2]. An extreme case is the habitual drug user consistently administering the drug despite the negative consequences, but we all have experience with our own, less severe habits. The standard animal model employs a combination of operant and classical learning components to bring about habit formation in rodents [3] and [4]. After extended training, these animals will press a lever even if the outcome associated with lever-pressing is no longer desired [5]. In this study, experiments with wild-type and transgenic flies revealed that a prominent insect neuropil, the mushroom bodies (MBs), regulates habit formation in flies by inhibiting the operant learning system when a predictive stimulus is present. This inhibition enables generalization of the classical memory and prevents premature habit formation. Extended training in wild-type flies produced a phenocopy of MB-impaired flies, such that generalization was abolished and goal-directed actions were transformed into habitual responses.     

Associative learning and memory in Drosophila: beyond olfactory conditioning

The associative learning abilities of the fruit fly, Drosophila melanogaster, have been demonstrated in both classical and operant conditioning paradigms. Efforts to identify the neural pathways and cellular mechanisms of learning have focused largely on olfactory classical conditioning. Results derived from various genetic and molecular manipulations provide considerable evidence that this form of associative learning depends critically on neural activity and cAMP signaling in brain neuropil structures called mushroom bodies. Three other behavioral learning paradigms in Drosophila serve as the main subject of this review. These are (1) visual and motor learning of flies tethered in a flight simulator, (2) a form of spatial learning that is independent of visual and olfactory cues, and (3) experience-dependent changes in male courtship behavior. The present evidence suggests that at least some of these modes of learning are independent of mushroom bodies. Applying targeted genetic manipulations to these behavioral paradigms should allow for a more comprehensive understanding of neural mechanisms responsible for diverse forms of associative learning and memory.     

Conditioning and learning in relation to disease

Of the two generally recognized processes through which learning occurs--imprinting and conditioning--only the latter with its two paradigms, classical and operant, has both practical and heuristic implications for disease. From the classical conditioning experiments of Pavlov's laboratory over 100 years ago to the later work in operant conditioning by Skinner and others in the past four decades has evolved much of the basis of modern learning theory and its applications to disease in the form of behavior therapy. Variants of behavior therapy have been employed in the treatment of wide variety of medical and psychiatric illnesses. Recent developments in the study of brain function and biochemistry have led to renewed interest in the conditioning paradigm and its value as tool in these areas of research.     

Evaluation of the role of the adaptive result in the theory of the functional systems

The fundamental of the theory of the functional systems, i.e., the concept of the useful adaptive result as a universal system-forming factor is considered. It is suggested that the adaptive result is not system-forming in behaviors actualized exclusively due to activity of systems developed earlier. It is argued that positive mutations may serve as the system-forming factor for hereditary determined behavioral forms. In all other cases of goal-directed behavior (except conditioning) the aim of performance as a model of the future result plays the decisive role. Only in conditioning the classical concept of the system-forming role of the adaptive result seems to be undeniable. The refined ideas about the mechanisms of formation of the functional systems may be useful in analysis of a number of animal and human functions (learning, emotional stress, neuroses, etc.).     

Individual vocal recognition and the effect of partial lesions to HVc on discrimination, learning, and categorization of conspecific song in adult songbirds

Among songbirds, the capacity to associate particular songs with particular singers (i.e., vocal recognition) forms the cognitive basis for more complex communication behaviors such as female choice and territoriality. In the present study, we combine operant conditioning techniques and excitotoxic lesions to the forebrain nucleus HVc to examine the role of this region in the discrimination, associative learning, and categorization of conspecific song. We trained adult male and female European starlings, Sturnus vulgaris, to recognize simultaneously the songs of several conspecific males. Then, using a series of transfer procedures, we demonstrate that correct recognition does not generalize to song bouts containing novel motifs from familiar singers. This suggests that starlings do not make use of individually invariant source or filter characteristics for vocal recognition. We then lesioned a portion of HVc bilaterally with ibotenic acid, and exposed the birds to a series of manipulations testing the discrimination, associative learning, and categorization of conspecific song. The lesions attenuated song production among males, but retention of the basic recognition task (i.e., maintenance of the discrimination) was unaffected. However, when the response contingencies were reversed-as a test of associative learning independent of discrimination-the initial performance and subsequent learning rate were negatively correlated with the size of the HVc lesions. This suggests that HVc plays a role in the formation of associations between a song and some referent. The results of this study are discussed in light of earlier claims regarding the role of HVc in the perceptual processing of conspecific song.     

Parallel processing of appetitive short- and long-term memories in Drosophila

It is broadly accepted that long-term memory (LTM) is formed sequentially after learning and short-term memory (STM) formation, but the nature of the relationship between early and late memory traces remains heavily debated [1-5]. To shed light on this issue, we used an olfactory appetitive conditioning in Drosophila, wherein starved flies learned to associate an odor with the presence of sugar [6]. We took advantage of the fact that both STM and LTM are generated after a unique conditioning cycle [7, 8] to demonstrate that appetitive LTM is able to form independently of STM. More specifically, we show that (1) STM retrieval involves output from γ neurons of the mushroom body (MB), i.e., the olfactory memory center [9, 10], whereas LTM retrieval involves output from αβ MB neurons; (2) STM information is not transferred from γ neurons to αβ neurons for LTM formation; and (3) the adenylyl cyclase RUT, which is thought to operate as a coincidence detector between the olfactory stimulus and the sugar stimulus [11-14], is required independently in γ neurons to form appetitive STM and in αβ neurons to form LTM. Taken together, these results demonstrate that appetitive short- and long-term memories are formed and processed in parallel.     

Molecular mechanisms underlying a cellular analog of operant reward learning

Operant conditioning is a ubiquitous but mechanistically poorly understood form of associative learning in which an animal learns the consequences of its behavior. Using a single-cell analog of operant conditioning in neuron B51 of Aplysia, we examined second-messenger pathways engaged by activity and reward and how they may provide a biochemical association underlying operant learning. Conditioning was blocked by Rp-cAMP, a peptide inhibitor of PKA, a PKC inhibitor, and by expressing a dominant-negative isoform of Ca2+-dependent PKC (apl-I). Thus, both PKA and PKC were necessary for operant conditioning. Injection of cAMP into B51 mimicked the effects of operant conditioning. Activation of PKC also mimicked conditioning but was dependent on both cAMP and PKA, suggesting that PKC acted at some point upstream of PKA activation. Our results demonstrate how these molecules can interact to mediate operant conditioning in an individual neuron important for the expression of the conditioned behavior.     

The Genetic Control of the Formation and Propagation of the [PSI+] Prion of Yeast

It is over 40 years since it was first reported that the yeast Saccahromyces cerevisiae contains two unusual cytoplasmic ‘genetic’ elements: [PSI+] and [URE3]. Remarkably the underlying determinants are protein-based rather than nucleic acid-based, i.e., that they are prions, and we have already learned much about their inheritance and phenotypic effects from the application of ‘classical’ genetic studies alongside the more modern molecular, cellular and biochemical approaches. Of particular value has been the exploitation of chemical mutagens and ‘antagonistic’ mutants which directly affect the replication and/or transmission of yeast prions. In this chapter we describe what has emerged from the application of classical and molecular genetic studies, to the most intensively studied of the three native yeast prions, the [PSI+] prion.     

High and low temperatures have unequal reinforcing properties in Drosophila spatial learning

Small insects regulate their body temperature solely through behavior. Thus, sensing environmental temperature and implementing an appropriate behavioral strategy can be critical for survival. The fly Drosophila melanogaster prefers 24°C, avoiding higher and lower temperatures when tested on a temperature gradient. Furthermore, temperatures above 24°C have negative reinforcing properties. In contrast, we found that flies have a preference in operant learning experiments for a low-temperature-associated position rather than the 24°C alternative in the heat-box. Two additional differences between high- and low-temperature reinforcement, i.e., temperatures above and below 24°C, were found. Temperatures equally above and below 24°C did not reinforce equally and only high temperatures supported increased memory performance with reversal conditioning. Finally, low- and high-temperature reinforced memories are similarly sensitive to two genetic mutations. Together these results indicate the qualitative meaning of temperatures below 24°C depends on the dynamics of the temperatures encountered and that the reinforcing effects of these temperatures depend on at least some common genetic components. Conceptualizing these results using the Wolf–Heisenberg model of operant conditioning, we propose the maximum difference in experienced temperatures determines the magnitude of the reinforcement input to a conditioning circuit.     

Avoidance conditioning in bamboo sharks (Chiloscyllium griseum and C. punctatum): behavioral and neuroanatomical aspects

Animals face different threats; to survive, they have to anticipate how to react or how to avoid these. It has already been shown in teleosts that selected regions in the telencephalon, i.e., the medial pallium, are involved in avoidance learning strategies. No such study exists for any chondrichthyan. In nature, an avoidance reaction may vary, ranging from a ‘freeze’ reaction to a startling response and quick escape. This study investigated whether elasmobranchs (Chiloscyllium griseum and C. punctatum) can be conditioned in an aversive classical conditioning paradigm. Upon successful conditioning, the dorsal, medial and lateral pallium were removed (group 1) and performance tested again. In a second group, the same operation was performed prior to training. While conditioning was successful in individuals of both groups, no escape responses were observed. Post-operative performance was assessed and compared between individual and groups to reveal if the neural substrates governing avoidance behavior or tasks learned in a classical conditioning paradigm are located within the telencephalon, as has been shown for teleosts such as goldfish.     

Coexistence of Reward and Unsupervised Learning During the Operant Conditioning of Neural Firing Rates

A fundamental goal of neuroscience is to understand how cognitive processes, such as operant conditioning, are performed by the brain. Typical and well studied examples of operant conditioning, in which the firing rates of individual cortical neurons in monkeys are increased using rewards, provide an opportunity for insight into this. Studies of reward-modulated spike-timing-dependent plasticity (RSTDP), and of other models such as R-max, have reproduced this learning behavior, but they have assumed that no unsupervised learning is present (i.e., no learning occurs without, or independent of, rewards). We show that these models cannot elicit firing rate reinforcement while exhibiting both reward learning and ongoing, stable unsupervised learning. To fix this issue, we propose a new RSTDP model of synaptic plasticity based upon the observed effects that dopamine has on long-term potentiation and depression (LTP and LTD). We show, both analytically and through simulations, that our new model can exhibit unsupervised learning and lead to firing rate reinforcement. This requires that the strengthening of LTP by the reward signal is greater than the strengthening of LTD and that the reinforced neuron exhibits irregular firing. We show the robustness of our findings to spike-timing correlations, to the synaptic weight dependence that is assumed, and to changes in the mean reward. We also consider our model in the differential reinforcement of two nearby neurons. Our model aligns more strongly with experimental studies than previous models and makes testable predictions for future experiments.     

Spontaneous decisions and operant conditioning in fruit flies

Already in the 1930s Skinner, Konorskiand colleagues debated the commonalities, differences and interactions among the processes underlying what was then known as “conditioned reflexes type I and II”, but which is today more well-known as classical (Pavlovian) and operant (instrumental) conditioning. Subsequent decades of research have confirmed that the interactions between the various learning systems engaged during operant conditioning are complex and difficult to disentangle. Today, modern neurobiological tools allow us to dissect the biological processes underlying operant conditioning and study their interactions. These processes include initiating spontaneous behavioral variability, world-learning and self-learning. The data suggest that behavioral variability is generated actively by the brain, rather than as a by-product of a complex, noisy input-output system. The function of this variability, in part, is to detect how the environment responds to such actions. World-learning denotes the biological process by which value is assigned to environmental stimuli. Self-learning is the biological process which assigns value to a specific action or movement. In an operant learning situation using visual stimuli for flies, world-learning inhibits self-learning via a prominent neuropil region, the mushroom-bodies. Only extended training can overcome this inhibition and lead to habit formation by engaging the self-learning mechanism. Self-learning transforms spontaneous, flexible actions into stereotyped, habitual responses.     

Psychopharmacologic facilitation of psychosocial therapy of violence and hyperkinesis

Experiments on naturally occurring hyperkinetic and violent dogs and cats demonstrated the usefulness of low dosages of amphetamine (0,2-1,0 mg/kg per os) in inhibiting these nonadaptive forms of behavior, permitting the development of discriminated Pavlovian and operant conditional responses. When amphetamine therapy was combined systematically with conditioning experiments and psychosocial therapy, for long enough periods of time (many weeks), the beneficial effects of this drug persisted in the nodrug state, i.e. the learning was not state-dependent. Amphetamine also ameliorated significantly conditional emotional visceral responses in dogs with low adaptation to psychologically stressful situations. The same low dosage of amphetamine which improved the behavior and learning of hyperkinetic and violent dogs disrupted the behavior and produced disorientation in normal dogs with previously stable conditional responses.     

Persistent sodium current is a nonsynaptic substrate for long-term associative memory

Although synaptic plasticity is widely regarded as the primary mechanism of memory [1], forms of nonsynaptic plasticity, such as increased somal or dendritic excitability or membrane potential depolarization, also have been implicated in learning in both vertebrate and invertebrate experimental systems [2], [3], [4], [5], [6] and [7]. Compared to synaptic plasticity, however, there is much less information available on the mechanisms of specific types of nonsynaptic plasticity involved in well-defined examples of behavioral memory. Recently, we have shown that learning-induced somal depolarization of an identified modulatory cell type (the cerebral giant cells, CGCs) of the snail Lymnaea stagnalis encodes information that enables the expression of long-term associative memory [8]. The Lymnaea CGCs therefore provide a highly suitable experimental system for investigating the ionic mechanisms of nonsynaptic plasticity that can be linked to behavioral learning. Based on a combined behavioral, electrophysiological, immunohistochemical, and computer simulation approach, here we show that an increase of a persistent sodium current of this neuron underlies its delayed and persistent depolarization after behavioral single-trial classical conditioning. Our findings provide new insights into how learning-induced membrane level changes are translated into a form of long-lasting neuronal plasticity already known to contribute to maintained adaptive modifications at the network and behavioral level [8].     

Comment on Gallistel: behavior theory and information theory: some parallels

In this article, Gallistel proposes information theory as an approach to some enduring problems in the study of operant and classical conditioning.     

Comment on Gallistel: Behavior theory and information theory: Some parallels

In this article, Gallistel proposes information theory as an approach to some enduring problems in the study of operant and classical conditioning.     

Dissociation of visual associative and motor learning in Drosophila at the flight simulator

Ever since operant conditioning was studied experimentally, the relationship between associative learning and possible motor learning has become controversial. Although motor learning and its underlying neural substrates have been extensively studied in mammals, it is still poorly understood in invertebrates. The visual discriminative avoidance paradigm of Drosophila at the flight simulator has been widely used to study the flies' visual associative learning and related functions, but it has not been used to study the motor learning process. In this study, newly-designed data analysis was employed to examine the flies' solitary behavioural variable that was recorded at the flight simulator-yaw torque. Analysis was conducted to explore torque distributions of both wild-type and mutant flies in conditioning, with the following results: (1) Wild-type Canton-S flies had motor learning performance in conditioning, which was proved by modifications of the animal's behavioural mode in conditioning. (2) Repetition of training improved the motor learning performance of wild-type Canton-S flies. (3) Although mutant dunce(1) flies were defective in visual associative learning, they showed essentially normal motor learning performance in terms of yaw torque distribution in conditioning. Finally, we tentatively proposed that both visual associative learning and motor learning were involved in the visual operant conditioning of Drosophila at the flight simulator, that the two learning forms could be dissociated and they might have different neural bases.     

Effect of Circadian Phase on Memory Acquisition and Recall: Operant Conditioning vs. Classical Conditioning

There have been several studies on the role of circadian clocks in the regulation of associative learning and memory processes in both vertebrate and invertebrate species. The results have been quite variable and at present it is unclear to what extent the variability observed reflects species differences or differences in methodology. Previous results have shown that following differential classical conditioning in the cockroach, Rhyparobia maderae, in an olfactory discrimination task, formation of the short-term and long-term memory is under strict circadian control. In contrast, there appeared to be no circadian regulation of the ability to recall established memories. In the present study, we show that following operant conditioning of the same species in a very similar olfactory discrimination task, there is no impact of the circadian system on either short-term or long-term memory formation. On the other hand, ability to recall established memories is strongly tied to the circadian phase of training. On the basis of these data and those previously reported for phylogenetically diverse species, it is suggested that there may be fundamental differences in the way the circadian system regulates learning and memory in classical and operant conditioning.     

Behavioral modification in choice process of Drosophila

Because of its clear genetic and developmental background, diversity of behavioral paradigms and neuroanatomy of the brain, Drosophila has become an important animal model for studying genetic, molecular and cellular bases of learning and memory[1]. Extensive research has explored the visual operant conditioning of Drosophila and related molecular bases[2—8]; recently, researchers began to address cognition-like functions and involved neural substrates[9—11]. In these studies, behavioral ana…