Double dissociation of PKC and AC manipulations on operant and classical learning in Drosophila

Learning about relationships between stimuli (i.e., classical conditioning [1]) and learning about consequences of one's own behavior (i.e., operant conditioning [2]) constitute the major part of our predictive understanding of the world. Since these forms of learning were recognized as two separate types 80 years ago [3], a recurrent concern has been the issue of whether one biological process can account for both of them [4, 5, 6, 7, 8, 9]. Today, we know the anatomical structures required for successful learning in several different paradigms, e.g., operant and classical processes can be localized to different brain regions in rodents [9] and an identified neuron in Aplysia shows opposite biophysical changes after operant and classical training, respectively [5]. We also know to some detail the molecular mechanisms underlying some forms of learning and memory consolidation. However, it is not known whether operant and classical learning can be distinguished at the molecular level. Therefore, we investigated whether genetic manipulations could differentiate between operant and classical learning in Drosophila. We found a double dissociation of protein kinase C and adenylyl cyclase on operant and classical learning. Moreover, the two learning systems interacted hierarchically such that classical predictors were learned preferentially over operant predictors.     

Spontaneous decisions and operant conditioning in fruit flies

Already in the 1930s Skinner, Konorskiand colleagues debated the commonalities, differences and interactions among the processes underlying what was then known as “conditioned reflexes type I and II”, but which is today more well-known as classical (Pavlovian) and operant (instrumental) conditioning. Subsequent decades of research have confirmed that the interactions between the various learning systems engaged during operant conditioning are complex and difficult to disentangle. Today, modern neurobiological tools allow us to dissect the biological processes underlying operant conditioning and study their interactions. These processes include initiating spontaneous behavioral variability, world-learning and self-learning. The data suggest that behavioral variability is generated actively by the brain, rather than as a by-product of a complex, noisy input-output system. The function of this variability, in part, is to detect how the environment responds to such actions. World-learning denotes the biological process by which value is assigned to environmental stimuli. Self-learning is the biological process which assigns value to a specific action or movement. In an operant learning situation using visual stimuli for flies, world-learning inhibits self-learning via a prominent neuropil region, the mushroom-bodies. Only extended training can overcome this inhibition and lead to habit formation by engaging the self-learning mechanism. Self-learning transforms spontaneous, flexible actions into stereotyped, habitual responses.     

Dissociation of visual associative and motor learning in Drosophila at the flight simulator

Ever since operant conditioning was studied experimentally, the relationship between associative learning and possible motor learning has become controversial. Although motor learning and its underlying neural substrates have been extensively studied in mammals, it is still poorly understood in invertebrates. The visual discriminative avoidance paradigm of Drosophila at the flight simulator has been widely used to study the flies' visual associative learning and related functions, but it has not been used to study the motor learning process. In this study, newly-designed data analysis was employed to examine the flies' solitary behavioural variable that was recorded at the flight simulator-yaw torque. Analysis was conducted to explore torque distributions of both wild-type and mutant flies in conditioning, with the following results: (1) Wild-type Canton-S flies had motor learning performance in conditioning, which was proved by modifications of the animal's behavioural mode in conditioning. (2) Repetition of training improved the motor learning performance of wild-type Canton-S flies. (3) Although mutant dunce(1) flies were defective in visual associative learning, they showed essentially normal motor learning performance in terms of yaw torque distribution in conditioning. Finally, we tentatively proposed that both visual associative learning and motor learning were involved in the visual operant conditioning of Drosophila at the flight simulator, that the two learning forms could be dissociated and they might have different neural bases.     

Actions,Action Sequences and Habits: Evidence That Goal-Directed and Habitual Action Control Are Hierarchically Organized

Behavioral evidence suggests that instrumental conditioning is governed by two forms of action control: a goal-directed and a habit learning process. Model-based reinforcement learning (RL) has been argued to underlie the goal-directed process; however, the way in which it interacts with habits and the structure of the habitual process has remained unclear. According to a flat architecture, the habitual process corresponds to model-free RL, and its interaction with the goal-directed process is coordinated by an external arbitration mechanism. Alternatively, the interaction between these systems has recently been argued to be hierarchical, such that the formation of action sequences underlies habit learning and a goal-directed process selects between goal-directed actions and habitual sequences of actions to reach the goal. Here we used a two-stage decision-making task to test predictions from these accounts. The hierarchical account predicts that, because they are tied to each other as an action sequence, selecting a habitual action in the first stage will be followed by a habitual action in the second stage, whereas the flat account predicts that the statuses of the first and second stage actions are independent of each other. We found, based on subjects'' choices and reaction times, that human subjects combined single actions to build action sequences and that the formation of such action sequences was sufficient to explain habitual actions. Furthermore, based on Bayesian model comparison, a family of hierarchical RL models, assuming a hierarchical interaction between habit and goal-directed processes, provided a better fit of the subjects'' behavior than a family of flat models. Although these findings do not rule out all possible model-free accounts of instrumental conditioning, they do show such accounts are not necessary to explain habitual actions and provide a new basis for understanding how goal-directed and habitual action control interact.     

Operant conditioning in Drosophila melanogaster wild-type and learning mutants with defects in the cyclic AMP metabolism

Wild-type Drosophila melanogaster and the learning mutants dunce, amnesiac and rutabaga, were tested using a new operant conditioning paradigm for single flies. All strains are able to learn to different extents, but no evidence of memory was found in the mutants amnesiac and rutabaga, while dunce has a reduced but extended memory. The relationship between this characteristic and cAMP levels are discussed. The three mutants have previously been shown, using classical conditioning paradigms to be deficient in olfactory learning and/or memory, and show reduced visual learning. The variability of the response of the mutants in the different paradigms is discussed in relation to the generality of the Aplysia model of the cellular mechanism underlying learning. In the operant conditioning paradigm described here, 93% of the wild-type flies learned to criterion. The performance of individual flies was consistent.     

Behavioral modification in choice process of Drosophila

Because of its clear genetic and developmental background, diversity of behavioral paradigms and neuroanatomy of the brain, Drosophila has become an important animal model for studying genetic, molecular and cellular bases of learning and memory[1]. Extensive research has explored the visual operant conditioning of Drosophila and related molecular bases[2—8]; recently, researchers began to address cognition-like functions and involved neural substrates[9—11]. In these studies, behavioral ana…     

Fuzzy ARTMAP prediction of biological activities for potential HIV-1 protease inhibitors using a small molecular data set

Obtaining satisfactory results with neural networks depends on the availability of large data samples. The use of small training sets generally reduces performance. Most classical Quantitative Structure-Activity Relationship (QSAR) studies for a specific enzyme system have been performed on small data sets. We focus on the neuro-fuzzy prediction of biological activities of HIV-1 protease inhibitory compounds when inferring from small training sets. We propose two computational intelligence prediction techniques which are suitable for small training sets, at the expense of some computational overhead. Both techniques are based on the FAMR model. The FAMR is a Fuzzy ARTMAP (FAM) incremental learning system used for classification and probability estimation. During the learning phase, each sample pair is assigned a relevance factor proportional to the importance of that pair. The two proposed algorithms in this paper are: 1) The GA-FAMR algorithm, which is new, consists of two stages: a) During the first stage, we use a genetic algorithm (GA) to optimize the relevances assigned to the training data. This improves the generalization capability of the FAMR. b) In the second stage, we use the optimized relevances to train the FAMR. 2) The Ordered FAMR is derived from a known algorithm. Instead of optimizing relevances, it optimizes the order of data presentation using the algorithm of Dagher et al. In our experiments, we compare these two algorithms with an algorithm not based on the FAM, the FS-GA-FNN introduced in [4], [5]. We conclude that when inferring from small training sets, both techniques are efficient, in terms of generalization capability and execution time. The computational overhead introduced is compensated by better accuracy. Finally, the proposed techniques are used to predict the biological activities of newly designed potential HIV-1 protease inhibitors.     

Classical reward conditioning in Drosophila melanogaster

Negatively reinforced olfactory conditioning has been widely employed to identify learning and memory genes, signal transduction pathways and neural circuitry in Drosophila. To delineate the molecular and cellular processes underlying reward-mediated learning and memory, we developed a novel assay system for positively reinforced olfactory conditioning. In this assay, flies were involuntarily exposed to the appetitive unconditioned stimulus sucrose along with a conditioned stimulus odour during training and their preference for the odour previously associated with sucrose was measured to assess learning and memory capacities. After one training session, wild-type Canton S flies displayed reliable performance, which was enhanced after two training cycles with 1-min or 15-min inter-training intervals. Higher performance scores were also obtained with increasing sucrose concentration. Memory in Canton S flies decayed slowly when measured at 30 min, 1 h and 3 h after training; whereas, it had declined significantly at 6 h and 12 h post-training. When learning mutant t beta h flies, which are deficient in octopamine, were challenged, they exhibited poor performance, validating the utility of this assay. As the Drosophila model offers vast genetic and transgenic resources, the new appetitive conditioning described here provides a useful tool with which to elucidate the molecular and cellular underpinnings of reward learning and memory. Similar to negatively reinforced conditioning, this reward conditioning represents classical olfactory conditioning. Thus, comparative analyses of learning and memory mutants in two assays may help identify the molecular and cellular components that are specific to the unconditioned stimulus information used in conditioning.     

Parallel processing of appetitive short- and long-term memories in Drosophila

It is broadly accepted that long-term memory (LTM) is formed sequentially after learning and short-term memory (STM) formation, but the nature of the relationship between early and late memory traces remains heavily debated [1-5]. To shed light on this issue, we used an olfactory appetitive conditioning in Drosophila, wherein starved flies learned to associate an odor with the presence of sugar [6]. We took advantage of the fact that both STM and LTM are generated after a unique conditioning cycle [7, 8] to demonstrate that appetitive LTM is able to form independently of STM. More specifically, we show that (1) STM retrieval involves output from γ neurons of the mushroom body (MB), i.e., the olfactory memory center [9, 10], whereas LTM retrieval involves output from αβ MB neurons; (2) STM information is not transferred from γ neurons to αβ neurons for LTM formation; and (3) the adenylyl cyclase RUT, which is thought to operate as a coincidence detector between the olfactory stimulus and the sugar stimulus [11-14], is required independently in γ neurons to form appetitive STM and in αβ neurons to form LTM. Taken together, these results demonstrate that appetitive short- and long-term memories are formed and processed in parallel.     

Ethanol seeking by long evans rats is not always a goal-directed behavior

Background

Two parallel and interacting processes are said to underlie animal behavior, whereby learning and performance of a behavior is at first via conscious and deliberate (goal-directed) processes, but after initial acquisition, the behavior can become automatic and stimulus-elicited (habitual). With respect to instrumental behaviors, animal learning studies suggest that the duration of training and the action-outcome contingency are two factors involved in the emergence of habitual seeking of “natural” reinforcers (e.g., sweet solutions, food or sucrose pellets). To rigorously test whether behaviors reinforced by abused substances such as ethanol, in particular, similarly become habitual was the primary aim of this study.

Methodology/Principal Findings

Male Long Evans rats underwent extended or limited operant lever press training with 10% sucrose/10% ethanol (10S10E) reinforcement (variable interval (VI) or (VR) ratio schedule of reinforcement), or with 10% sucrose (10S) reinforcement (VI schedule only). Once training and pretesting were complete, the impact of outcome devaluation on operant behavior was evaluated after lithium chloride injections were paired with the reinforcer, or unpaired 24 hours later. After limited, but not extended instrumental training, lever pressing by groups trained under VR with 10S10E and under VI with 10S was sensitive to outcome devaluation. In contrast, responding by both the extended and limited training 10S10E VI groups was not sensitive to ethanol devaluation during the test for habitual behavior.

Conclusions/Significance

Operant behavior by rats trained to self-administer an ethanol-sucrose solution showed variable sensitivity to a change in the value of ethanol, with relative insensitivity developing sooner in animals that received time-variable ethanol reinforcement during training sessions. One important implication, with respect to substance abuse in humans, is that initial learning about the relationship between instrumental actions and the opportunity to consume ethanol-containing drinks can influence the time course for the development or expression of habitual ethanol seeking behavior.     

Voting Intention and Choices: Are Voters Always Rational and Deliberative?

Human rationality–the ability to behave in order to maximize the achievement of their presumed goals (i.e., their optimal choices)–is the foundation for democracy. Research evidence has suggested that voters may not make decisions after exhaustively processing relevant information; instead, our decision-making capacity may be restricted by our own biases and the environment. In this paper, we investigate the extent to which humans in a democratic society can be rational when making decisions in a serious, complex situation–voting in a local political election. We believe examining human rationality in a political election is important, because a well-functioning democracy rests largely upon the rational choices of individual voters. Previous research has shown that explicit political attitudes predict voting intention and choices (i.e., actual votes) in democratic societies, indicating that people are able to reason comprehensively when making voting decisions. Other work, though, has demonstrated that the attitudes of which we may not be aware, such as our implicit (e.g., subconscious) preferences, can predict voting choices, which may question the well-functioning democracy. In this study, we systematically examined predictors on voting intention and choices in the 2014 mayoral election in Taipei, Taiwan. Results indicate that explicit political party preferences had the largest impact on voting intention and choices. Moreover, implicit political party preferences interacted with explicit political party preferences in accounting for voting intention, and in turn predicted voting choices. Ethnic identity and perceived voting intention of significant others were found to predict voting choices, but not voting intention. In sum, to the comfort of democracy, voters appeared to engage mainly explicit, controlled processes in making their decisions; but findings on ethnic identity and perceived voting intention of significant others may suggest otherwise.     

Ectopic G-protein expression in dopamine and serotonin neurons blocks cocaine sensitization in Drosophila melanogaster

Sensitization to repeated doses of psychostimulants is thought to be an important component underlying the addictive process in humans [1] [2] [3] [4]. In all vertebrate animal models, including humans [5], and even in fruit flies, sensitization is observed after repeated exposure to volatilized crack cocaine [6]. In vertebrates, sensitization is thought to be initiated by processes occurring in brain regions that contain dopamine cell bodies [2] [7]. Here, we show that modulated cell signaling in the Drosophila dopamine and serotonin neurons plays an essential role in cocaine sensitization. Targeted expression of either a stimulatory (Galpha(s)) or inhibitory (Galpha(i)) Galpha subunit, or tetanus toxin light chain (TNT) in dopamine and serotonin neurons of living flies blocked behavioral sensitization to repeated cocaine exposures. These flies showed alterations in their initial cocaine responsiveness that correlated with compensatory adaptations of postsynaptic receptor sensitivity. Finally, repeated drug stimulation of a nerve cord preparation that is postsynaptic to the brain amine cells failed to induce sensitization, further showing the importance of presynaptic modulation in sensitization.     

Classical conditioning and retention in normal and mutantDrosophila melanogaster

Summary By changing the conditioned discrimination paradigm of Quinn et al. (1974) from an instrumental procedure to a classical (Pavlovian) one, we have demonstrated strong learning in type flies. About 150 flies were sequestered in a closed chamber and trained by explosing them sequentially to two odors in air currents. Flies received twelve electric shock pulses in the presence of the first odor (CS+) but not in the presence of the second odor (CS–). To test for conditioned avoidance responses, flies were transported to a Tmaze choice point, between converging currents of the two odors. Typically, 95% of trained flies avoided the shock-associated odor (CS+).Acquisition of learning was a function of the number of shock pulses received during CS+ presentation and was asymptotic within one training cycle. Conditioned avoidance increased with increasing shock intensity or odor concentration and was very resistant to extinction. Learning was best when CS+ presentations overlap shock (delay conditioning) and then decreased with increasing CS-US interstimulus intervals. Shocking flies immediately before CS+ presentation (backward conditioning) produced no learning. Nonassociative control procedures (CS Alone, US Alone and Explicitly Unpaired) produced slight decreases in avoidance responses, but these affected both odors equally and did not alter our associative learning index (A).Memory in wild-type flies decayed gradually over the first seven hours after training and still was present 24 h later. The mutantsamnesiac, rutabaga anddunce showed appreciable learning acquisition, but their memories decayed very rapidly during the first 30 min. After this, the rates of decay slowed sharply; conditioned avoidance still was measurable at least three hours after training.Abbreviations OCT 3-octanol - MCH 4-methylcyclohexanol - C-S Canton-Special - CS conditioned stimulus - US unconditioned stimulus     

Effect of Circadian Phase on Memory Acquisition and Recall: Operant Conditioning vs. Classical Conditioning

There have been several studies on the role of circadian clocks in the regulation of associative learning and memory processes in both vertebrate and invertebrate species. The results have been quite variable and at present it is unclear to what extent the variability observed reflects species differences or differences in methodology. Previous results have shown that following differential classical conditioning in the cockroach, Rhyparobia maderae, in an olfactory discrimination task, formation of the short-term and long-term memory is under strict circadian control. In contrast, there appeared to be no circadian regulation of the ability to recall established memories. In the present study, we show that following operant conditioning of the same species in a very similar olfactory discrimination task, there is no impact of the circadian system on either short-term or long-term memory formation. On the other hand, ability to recall established memories is strongly tied to the circadian phase of training. On the basis of these data and those previously reported for phylogenetically diverse species, it is suggested that there may be fundamental differences in the way the circadian system regulates learning and memory in classical and operant conditioning.     

Cortical and Hippocampal Correlates of Deliberation during Model-Based Decisions for Rewards in Humans

How do we use our memories of the past to guide decisions we''ve never had to make before? Although extensive work describes how the brain learns to repeat rewarded actions, decisions can also be influenced by associations between stimuli or events not directly involving reward — such as when planning routes using a cognitive map or chess moves using predicted countermoves — and these sorts of associations are critical when deciding among novel options. This process is known as model-based decision making. While the learning of environmental relations that might support model-based decisions is well studied, and separately this sort of information has been inferred to impact decisions, there is little evidence concerning the full cycle by which such associations are acquired and drive choices. Of particular interest is whether decisions are directly supported by the same mnemonic systems characterized for relational learning more generally, or instead rely on other, specialized representations. Here, building on our previous work, which isolated dual representations underlying sequential predictive learning, we directly demonstrate that one such representation, encoded by the hippocampal memory system and adjacent cortical structures, supports goal-directed decisions. Using interleaved learning and decision tasks, we monitor predictive learning directly and also trace its influence on decisions for reward. We quantitatively compare the learning processes underlying multiple behavioral and fMRI observables using computational model fits. Across both tasks, a quantitatively consistent learning process explains reaction times, choices, and both expectation- and surprise-related neural activity. The same hippocampal and ventral stream regions engaged in anticipating stimuli during learning are also engaged in proportion to the difficulty of decisions. These results support a role for predictive associations learned by the hippocampal memory system to be recalled during choice formation.     

How stress alters memory in 'smart' snails

Cognitive ability varies within species, but whether this variation alters the manner in which memory formation is affected by environmental stress is unclear. The great pond snail, Lymnaea stagnalis, is commonly used as model species in studies of learning and memory. The majority of those studies used a single laboratory strain (i.e. the Dutch strain) originating from a wild population in the Netherlands. However, our recent work has identified natural populations that demonstrate significantly enhanced long-term memory (LTM) formation relative to the Dutch strain following operant conditioning of aerial respiratory behaviour. Here we assess how two populations with enhanced memory formation (i.e. 'smart' snails), one from Canada (Trans Canada 1: TC1) and one from the U.K. (Chilton Moor: CM) respond to ecologically relevant stressors. In control conditions the Dutch strain forms memory lasting 1-3 h following a single 0.5 h training session in our standard calcium pond water (80 mg/l [Ca(2+)]), whereas the TC1 and CM populations formed LTM lasting 5+ days following this training regime. Exposure to low environmental calcium pond water (20 mg/l [Ca(2+)]), which blocks LTM in the Dutch strain, reduced LTM retention to 24 h in the TC1 and CM populations. Crowding (20 snails in 100 ml) immediately prior to training blocks LTM in the Dutch strain, and also did so in TC1 and CM populations. Therefore, snails with enhanced cognitive ability respond to these ecologically relevant stressors in a similar manner to the Dutch strain, but are more robust at forming LTM in a low calcium environment. Despite the two populations (CM and TC1) originating from different continents, LTM formation was indistinguishable in both control and stressed conditions. This indicates that the underlying mechanisms controlling cognitive differences among populations may be highly conserved in L. stagnalis.     

The evolutionary origins of human patience: temporal preferences in chimpanzees, bonobos, and human adults

To make adaptive choices, individuals must sometimes exhibit patience, forgoing immediate benefits to acquire more valuable future rewards [1-3]. Although humans account for future consequences when making temporal decisions [4], many animal species wait only a few seconds for delayed benefits [5-10]. Current research thus suggests a phylogenetic gap between patient humans and impulsive, present-oriented animals [9, 11], a distinction with implications for our understanding of economic decision making [12] and the origins of human cooperation [13]. On the basis of a series of experimental results, we reject this conclusion. First, bonobos (Pan paniscus) and chimpanzees (Pan troglodytes) exhibit a degree of patience not seen in other animals tested thus far. Second, humans are less willing to wait for food rewards than are chimpanzees. Third, humans are more willing to wait for monetary rewards than for food, and show the highest degree of patience only in response to decisions about money involving low opportunity costs. These findings suggest that core components of the capacity for future-oriented decisions evolved before the human lineage diverged from apes. Moreover, the different levels of patience that humans exhibit might be driven by fundamental differences in the mechanisms representing biological versus abstract rewards.     

What can we teach Lymnaea and what can Lymnaea teach us?

This review describes the advantages of adopting a molluscan complementary model, the freshwater snail Lymnaea stagnalis, to study the neural basis of learning and memory in appetitive and avoidance classical conditioning; as well as operant conditioning of its aerial respiratory and escape behaviour. We firstly explored ‘what we can teach Lymnaea’ by discussing a variety of sensitive, solid, easily reproducible and simple behavioural tests that have been used to uncover the memory abilities of this model system. Answering this question will allow us to open new frontiers in neuroscience and behavioural research to enhance our understanding of how the nervous system mediates learning and memory. In fact, from a translational perspective, Lymnaea and its nervous system can help to understand the neural transformation pathways from behavioural output to sensory coding in more complex systems like the mammalian brain. Moving on to the second question: ‘what can Lymnaea teach us?’, it is now known that Lymnaea shares important associative learning characteristics with vertebrates, including stimulus generalization, generalization of extinction and discriminative learning, opening the possibility to use snails as animal models for neuroscience translational research.     

The learning characteristics of rats under free-choice conditions]

The ability to organize a four-link operant food-procuring habit in a multiple alternative maze using the free-choice method was studied in albino rats. Three types of animals were observed which were different in the character of learning. The learning curve of 20% of rats had of exponential character (type I). Some animals (37%) acquired the skill through "insight" and the process of learning in these cases could be described by a logistic regression function (type II). The remaining rats (43%) refused from solving the intricate task and were able to acquire only the simplest form of a response, i.e., running to feeders. It is suggested that learning differences between the I and II types of animals may be associated with different strategies of problem solving: "procedural" (algorithmic) and "conceptual" (semantic).     

Speed/accuracy trade-off between the habitual and the goal-directed processes

Instrumental responses are hypothesized to be of two kinds: habitual and goal-directed, mediated by the sensorimotor and the associative cortico-basal ganglia circuits, respectively. The existence of the two heterogeneous associative learning mechanisms can be hypothesized to arise from the comparative advantages that they have at different stages of learning. In this paper, we assume that the goal-directed system is behaviourally flexible, but slow in choice selection. The habitual system, in contrast, is fast in responding, but inflexible in adapting its behavioural strategy to new conditions. Based on these assumptions and using the computational theory of reinforcement learning, we propose a normative model for arbitration between the two processes that makes an approximately optimal balance between search-time and accuracy in decision making. Behaviourally, the model can explain experimental evidence on behavioural sensitivity to outcome at the early stages of learning, but insensitivity at the later stages. It also explains that when two choices with equal incentive values are available concurrently, the behaviour remains outcome-sensitive, even after extensive training. Moreover, the model can explain choice reaction time variations during the course of learning, as well as the experimental observation that as the number of choices increases, the reaction time also increases. Neurobiologically, by assuming that phasic and tonic activities of midbrain dopamine neurons carry the reward prediction error and the average reward signals used by the model, respectively, the model predicts that whereas phasic dopamine indirectly affects behaviour through reinforcing stimulus-response associations, tonic dopamine can directly affect behaviour through manipulating the competition between the habitual and the goal-directed systems and thus, affect reaction time.