Leg position learning by an insect: II. Motor strategies underlying learned leg extension

The patterns of myographic activity in the flexor and extensor tibiae muscles of the locust which accompany learned tibial extension were examined. Three distinct motor strategies were identified: (1) repeated flexion-extension movements, each of which resulted in a momentary excursion beyond the required, pre-set joint angle (demand angle) and in sum met the criterion for learning; (2) changes in basic muscle tonus, which resulted in maintained shifts in tibial position without discernible myographic activity; (3) tonic activity in the single slow excitatory motoneuron of the extensor tibiae ( SETi ) which produced maintained tibial extension. These strategies were selectively employed depending on the particular range of joint angle required. These strategies were compared and their effectiveness evaluated using a variety of behavioral criteria. Neuronal mechanisms which might underlie each of these strategies are discussed.     

Coordinated activity of ventral tegmental neurons adapts to appetitive and aversive learning

Our understanding of how value-related information is encoded in the ventral tegmental area (VTA) is based mainly on the responses of individual putative dopamine neurons. In contrast to cortical areas, the nature of coordinated interactions between groups of VTA neurons during motivated behavior is largely unknown. These interactions can strongly affect information processing, highlighting the importance of investigating network level activity. We recorded the activity of multiple single units and local field potentials (LFP) in the VTA during a task in which rats learned to associate novel stimuli with different outcomes. We found that coordinated activity of VTA units with either putative dopamine or GABA waveforms was influenced differently by rewarding versus aversive outcomes. Specifically, after learning, stimuli paired with a rewarding outcome increased the correlation in activity levels between unit pairs whereas stimuli paired with an aversive outcome decreased the correlation. Paired single unit responses also became more redundant after learning. These response patterns flexibly tracked the reversal of contingencies, suggesting that learning is associated with changing correlations and enhanced functional connectivity between VTA neurons. Analysis of LFP recorded simultaneously with unit activity showed an increase in the power of theta oscillations when stimuli predicted reward but not an aversive outcome. With learning, a higher proportion of putative GABA units were phase locked to the theta oscillations than putative dopamine units. These patterns also adapted when task contingencies were changed. Taken together, these data demonstrate that VTA neurons organize flexibly as functional networks to support appetitive and aversive learning.     

Food aversion learning in the polyphagous grasshopper Schistocerca americana

ABSTRACT. The ability of the acridid Schistocerca americana (Drury) (Orthoptera: Acrididae) to learn to avoid a food when it is associated with an artificial aversive stimulus was examined. The aversive stimulus used was an injection of nicotine hydrogen tartrate at a concentration which usually did not cause knockdown. Insects learned to avoid a food which already had limited sustained acceptability. The learning was not found when the aversive stimulus followed a meal on a fully acceptable food.     

Neural encoding in ventral striatum during olfactory discrimination learning

A growing body of evidence implicates the ventral striatum in using information acquired through associative learning. The present study examined the activity of ventral striatal neurons in awake, behaving rats during go/no-go odor discrimination learning and reversal. Many neurons fired selectively to odor cues predictive of either appetitive (sucrose) or aversive (quinine) outcomes. Few neurons were selective when first exposed to the odors, but many acquired this differential activity as rats learned the significance of the cues. A substantial proportion of these neurons encoded the cues' learned motivational significance, and these neurons tended to reverse their firing selectivity after reversal of odor-outcome contingencies. Other neurons that became selectively activated during learning did not reverse, but instead appeared to encode specific combinations of cues and associated motor responses. The results support a role for ventral striatum in using the learned significance, both appetitive and aversive, of predictive cues to guide behavior.     

Learning,using natural reinforcements,in insect preparations that permit cellular neuronal analysis

A general paradigm is described that permits testing the ability of an arthropod to learn (by operant conditioning) to alter the position of a single leg segment in order to relate to behaviorally appropriate reinforcement. The paradigm was designed so that intracellular recording from identified neurons involved would be possible during the training of a locust or grasshopper, for which extensive neuron maps are available. As a prelude to such studies, electromyograms were made from the antagonistic muscles that move the conditioned limb, which in the present experiments was the tibia of the metathoracic leg. Negative (aversive) reinforcement was provided by a loud sound/vibration and positive (reward) reinforcement by food in the form of sugar-water or fresh-growing grass. In the aversive reinforcement experiments the sound, which reflexly caused flexion, was on continually except when the tibia of one hind leg was voluntarily placed in an electronically set position “window” displaced, in extension, away from the preferred position. In feeding experiments, food was brought automatically to the mouth by a motor-driven arm when the tibia was held within a position window set away from the preferred position in either extension or flexion. Whole or headless insects learned to turn off the sound permanently, except for sporadic brief interruptions, by tonic shifting of tibial position. Insects learned to bring food to the mouth by modifying the plateau phase of a position displacement lasting for a few minutes, that was found to occur from time to time also in controls. In aversive learning, minimum times to turn off the sound were 22 sec for the easiest position and 4 min for the most difficult. The longest time in the easiest position was 1 min 40 sec and in the most difficult 39 min; excluding measurement for individuals that did not learn. In reward learning, the minimum time in the easiest position was just under 1 min, and 12 min in the most difficult position. The longest times were about 8 hr regardless of difficulty.     

Social learning and evolution: the cultural intelligence hypothesis

If social learning is more efficient than independent individual exploration, animals should learn vital cultural skills exclusively, and routine skills faster, through social learning, provided they actually use social learning preferentially. Animals with opportunities for social learning indeed do so. Moreover, more frequent opportunities for social learning should boost an individual's repertoire of learned skills. This prediction is confirmed by comparisons among wild great ape populations and by social deprivation and enculturation experiments. These findings shaped the cultural intelligence hypothesis, which complements the traditional benefit hypotheses for the evolution of intelligence by specifying the conditions in which these benefits can be reaped. The evolutionary version of the hypothesis argues that species with frequent opportunities for social learning should more readily respond to selection for a greater number of learned skills. Because improved social learning also improves asocial learning, the hypothesis predicts a positive interspecific correlation between social-learning performance and individual learning ability. Variation among primates supports this prediction. The hypothesis also predicts that more heavily cultural species should be more intelligent. Preliminary tests involving birds and mammals support this prediction too. The cultural intelligence hypothesis can also account for the unusual cognitive abilities of humans, as well as our unique mechanisms of skill transfer.     

Worrying affects associative fear learning: a startle fear conditioning study

A valuable experimental model for the pathogenesis of anxiety disorders is that they originate from a learned association between an intrinsically non-aversive event (Conditioned Stimulus, CS) and an anticipated disaster (Unconditioned Stimulus, UCS). Most anxiety disorders, however, do not evolve from a traumatic experience. Insights from neuroscience show that memory can be modified post-learning, which may elucidate how pathological fear can develop after relatively mild aversive events. Worrying--a process frequently observed in anxiety disorders--is a potential candidate to strengthen the formation of fear memory after learning. Here we tested in a discriminative fear conditioning procedure whether worry strengthens associative fear memory. Participants were randomly assigned to either a Worry (n = 23) or Control condition (n = 25). After fear acquisition, the participants in the Worry condition processed six worrisome questions regarding the personal aversive consequences of an electric stimulus (UCS), whereas the Control condition received difficult but neutral questions. Subsequently, extinction, reinstatement and re-extinction of fear were tested. Conditioned responding was measured by fear-potentiated startle (FPS), skin conductance (SCR) and UCS expectancy ratings. Our main results demonstrate that worrying resulted in increased fear responses (FPS) to both the feared stimulus (CS(+)) and the originally safe stimulus (CS(-)), whereas FPS remained unchanged in the Control condition. In addition, worrying impaired both extinction and re-extinction learning of UCS expectancy. The implication of our findings is that they show how worry may contribute to the development of anxiety disorders by affecting associative fear learning.     

Learning performances in young horses using two different learning tests

To achieve optimal performance in equine sports as well as in leisure not only the physical abilities of the horse should be considered, but also the horse’s personality. Besides temperamental aspects, like emotionality, or the horse’s reactivity towards humans in handling situations, the learning ability of the horse is another relevant personality trait. To study whether differences in learning performance are consistent over time and whether individual learning performance differs between learning tests or is affected by emotionality, 39 young horses (Dutch Warmblood) were tested repeatedly in two learning tests. An aversive stimulus (AS) was used in one learning test (the avoidance learning test) and a reward was used in the other learning test (the reward learning test). During both learning tests behaviour as well as heart rate were measured. Each test was executed four times, twice when horses were 1 year of age, and twice when they were 2 years of age. Half of the horses received additional physical training from 6 months onwards. In both tests horses could be classified as either performers, i.e. completing the daily session, or as non-performers, i.e. returning to the home environment without having completed the daily session. There were some indications that emotionality might have caused non-performing behaviour, but these indications are not convincing enough to exclude other causes. Furthermore, there seem to be no simple relationships between measures of heart rate, behavioural responses putatively related to emotionality and learning performance. Horses revealed consistent individual learning performances within years in both tests, and in the avoidance learning test also between years. There was no significant correlation between learning performances in the avoidance learning test and the learning performances in the reward learning test. It is concluded that individual learning abilities are consistent over a short time interval for an avoidance learning test and a reward learning test and over a longer time for the avoidance learning test. Furthermore, results indicate that some horses perform better when they have to learn to avoid an aversive stimulus while others perform better when they are rewarded after a correct response. It is suggested that these differences may be relevant to design optimal individual training programmes and methods.     

Evolution of learning capacities and learning levels

Humans strongly depend on individual and social learning, both of which are highly effective and accurate. I study the effects of environmental change on the evolution of the effectiveness and accuracy of individual and social learning (individual and social learning levels) and the number of pieces of information learned individually and socially (individual and social learning capacities) by analyzing a mathematical model. I show that individual learning capacity decreases and social learning capacity increases when the environment becomes more stable; both decrease when the environment becomes milder. I also show that individual learning capacity increases when individual learning level increases or social learning level decreases, while social learning capacity increases when individual or social learning level increases. The evolution of high learning levels can be triggered when the environment becomes severe, but a high social learning level can evolve only when a high individual learning level can simultaneously evolve with it.     

Memory retention capacity using two different training methods,appetitive and aversive learning,in juvenile red sea bream Chrysophrys major

Memory retention based on appetitive and aversive learning was studied in juvenile red sea bream Chrysophrys major. The fish were individually trained via appetitive and aversive learning. In general, they retained appetitive memories for 30 days, but not for 60 days. Conversely, aversive memory endured for 1 day, but not for 3 days or longer. Analyses at the individual level revealed that some fish retained appetitive memories for 60 days, whereas others lost it within 3 days; this suggests considerable variability in memory retention capacity among individual fish. The memory duration for aversive learning was remarkably short, which should be considered when releasing trained fish into the wild for stock enhancement. Furthermore, the high inter-individual variability suggests that evaluating memory retention capacity through group experiments might lead to overestimation of fishes’ ability.     

Rapid evolution of social learning

Culture is widely thought to be beneficial when social learning is less costly than individual learning and thus may explain the enormous ecological success of humans. Rogers (1988. Does biology constrain culture. Am. Anthropol.  90 : 819–831) contradicted this common view by showing that the evolution of social learning does not necessarily increase the net benefits of learned behaviours in a variable environment. Using simulation experiments, we re‐analysed extensions of Rogers’ model after relaxing the assumption that genetic evolution is much slower than cultural evolution. Our results show that this assumption is crucial for Rogers’ finding. For many parameter settings, genetic and cultural evolution occur on the same time scale, and feedback effects between genetic and cultural dynamics increase the net benefits. Thus, by avoiding the costs of individual learning, social learning can increase ecological success. Furthermore, we found that rapid evolution can limit the evolution of complex social learning strategies, which have been proposed to be widespread in animals.     

Two antagonistic functions of neural groups of the femoral chordotonal organ underlie thanatosis in the cricket Gryllus bimaculatus DeGeer

The cricket Gryllus bimaculatus displayed freezing (thanatosis) after struggling while the femoro-tibial joints of the walking legs were forcibly restrained. Myographic recording indicated that strong contraction of the flexor tibia muscle “leg flexion response” occurred under this restrained condition. During thanatosis, when the femoro-tibial joint was passively displaced and held for several seconds, it maintained its new position (catalepsy). Only discharge of the slow flexor units was mechanically indispensable for maintaining thanatosis and catalepsy. Differing roles of identified neuron subgroups of the femoral chordotonal organ were elucidated using this behavioral substrate. Ablation of the dorsal group neurons in the ventral scoloparium strengthened the leg flexion response and the normal resistance reflex, while ablation of the ventral group weakened both motor outputs. Ablation of the dorsal scoloparium neurons, or other main sensory nerves caused no detectable deficiency in femoro-tibial joint control. These results imply that both modes of flexor muscle activation promoted by the ventral group neurons are normally held under inhibitory control by the dorsal group. It is hypothesized that this antagonistic function causes immobilization of the femoro-tibial joint in a wide range of angles in thanatosis and catalepsy. Accepted: 12 November 1998     

Evolution of learning in fluctuating environments: when selection favors both social and exploratory individual learning

Cumulative cultural change requires organisms that are capable of both exploratory individual learning and faithful social learning. In our model, an organism's phenotype is initially determined innately (by its genotypic value) or by social learning (copying a phenotype from the parental generation), and then may or may not be modified by individual learning (exploration around the initial phenotype). The environment alternates periodically between two states, each defined as a certain range of phenotypes that can survive. These states may overlap, in which case the same phenotype can survive in both states, or they may not. We find that a joint social and exploratory individual learning strategy-the strategy that supports cumulative culture-is likely to spread when the environmental states do not overlap. In particular, when the environmental states are contiguous and mutation is allowed among the genotypic values, this strategy will spread in either moderately or highly stable environments, depending on the exact nature of the individual learning applied. On the other hand, natural selection often favors a social learning strategy without exploration when the environmental states overlap. We find only partial support for the "consensus" view, which holds that individual learning, social learning, and innate determination of behavior will evolve at short, intermediate, and long environmental periodicities, respectively.     

Light-induced activation of distinct modulatory neurons triggers appetitive or aversive learning in Drosophila larvae

During classical conditioning, a positive or negative value is assigned to a previously neutral stimulus, thereby changing its significance for behavior. If an odor is associated with a negative stimulus, it can become repulsive. Conversely, an odor associated with a reward can become attractive. By using Drosophila larvae as a model system with minimal brain complexity, we address the question of which neurons attribute these values to odor stimuli. In insects, dopaminergic neurons are required for aversive learning, whereas octopaminergic neurons are necessary and sufficient for appetitive learning. However, it remains unclear whether two independent neuronal populations are sufficient to mediate such antagonistic values. We report the use of transgenically expressed channelrhodopsin-2, a light-activated cation channel, as a tool for optophysiological stimulation of genetically defined neuronal populations in Drosophila larvae. We demonstrate that distinct neuronal populations can be activated simply by illuminating the animals with blue light. Light-induced activation of dopaminergic neurons paired with an odor stimulus induces aversive memory formation, whereas activation of octopaminergic/tyraminergic neurons induces appetitive memory formation. These findings demonstrate that antagonistic modulatory subsystems are sufficient to substitute for aversive and appetitive reinforcement during classical conditioning.     

Teaching and learning in a probabilistic prisoner's dilemma

The prisoner's dilemma is much studied in social psychology and decision-making because it models many real-world conflicts. In everyday terms, the choice to 'cooperate' (maximize reward for the group) or 'defect' (maximize reward for the individual) is often attributed to altruistic or selfish motives. Alternatively, behavior during a dilemma may be understood as a function of reinforcement and punishment. Human participants played a prisoner's-dilemma-type game (for points exchangeable for money) with a computer that employed either a teaching strategy (a probabilistic version of tit-for-tat), in which the computer reinforced or punished participants' cooperation or defection, or a learning strategy (a probabilistic version of Pavlov), in which the computer's responses were reinforced and punished by participants' cooperation and defection. Participants learned to cooperate against both computer strategies. However, in a second experiment which varied the context of the game, they learned to cooperate only against one or other strategy; participants did not learn to cooperate against tit-for-tat when they believed that they were playing against another person; participants did not learn to cooperate against Pavlov when the computer's cooperation probability was signaled by a spinner. The results are consistent with the notion that people are biased not only to cooperate or defect on individual social choices, but also to employ one or other strategy of interaction in a pattern across social choices.     

Specialized learning in antlions (Neuroptera: Myrmeleontidae), pit-digging predators, shortens vulnerable larval stage

Unique in the insect world for their extremely sedentary predatory behavior, pit-dwelling larval antlions dig pits, and then sit at the bottom and wait, sometimes for months, for prey to fall inside. This sedentary predation strategy, combined with their seemingly innate ability to detect approaching prey, make antlions unlikely candidates for learning. That is, although scientists have demonstrated that many species of insects possess the capacity to learn, each of these species, which together represent multiple families from every major insect order, utilizes this ability as a means of navigating the environment, using learned cues to guide an active search for food and hosts, or to avoid noxious events. Nonetheless, we demonstrate not only that sedentary antlions can learn, but also, more importantly, that learning provides an important fitness benefit, namely decreasing the time to pupate, a benefit not yet demonstrated in any other species. Compared to a control group in which an environmental cue was presented randomly vis-à-vis daily prey arrival, antlions given the opportunity to associate the cue with prey were able to make more efficient use of prey and pupate significantly sooner, thus shortening their long, highly vulnerable larval stage. Whereas "median survival time," the point at which half of the animals in each group had pupated, was 46 days for antlions receiving the Learning treatment, that point never was reached in antlions receiving the Random treatment, even by the end of the experiment on Day 70. In addition, we demonstrate a novel manifestation of antlions' learned response to cues predicting prey arrival, behavior that does not match the typical "learning curve" but which is well-adapted to their sedentary predation strategy. Finally, we suggest that what has long appeared to be instinctive predatory behavior is likely to be highly modified and shaped by learning.     

Evolutionary ecology of learning: insights from fruit flies

Ecologically and evolutionarily oriented research on learning has traditionally been carried out on vertebrates and bees. While less sophisticated than those animals, fruit flies (Drosophila) are capable of several forms of learning, and have the advantage of a short generation time, which makes them an ideal system for experimental evolution studies. This review summarizes the insights into evolutionary questions about learning gained in the last decade from evolutionary experiments on Drosophila. These experiments demonstrate that Drosophila has the genetic potential to evolve a substantially improved learning performance in ecologically relevant learning tasks. In at least one set of selected populations, the improved learning generalized to a task other than that used to impose selection, involving a different behavior, different stimuli, and a different sensory channel for the aversive reinforcement. This improvement in learning ability was associated with reductions in other fitness-related traits, such as larval competitive ability and lifespan, pointing to evolutionary trade-offs for improved learning. These trade-offs were confirmed by other evolutionary experiments where a reduction in learning performance was observed as a correlated response to selection for tolerance to larval nutritional stress or for delayed aging. Such trade-offs could be one reason why fruit flies have not fully used up their evolutionary potential for learning. Finally, another evolutionary experiment with Drosophila provided the first direct evidence for the long-standing idea that learning can under some circumstances accelerate and in others slow down genetically based evolutionary change. These results demonstrate the usefulness of fruit flies as a model system to address evolutionary questions about learning.     

Dopamine and octopamine influence avoidance learning of honey bees in a place preference assay

Biogenic amines are widely characterized in pathways evaluating reward and punishment, resulting in appropriate aversive or appetitive responses of vertebrates and invertebrates. We utilized the honey bee model and a newly developed spatial avoidance conditioning assay to probe effects of biogenic amines octopamine (OA) and dopamine (DA) on avoidance learning. In this new protocol non-harnessed bees associate a spatial color cue with mild electric shock punishment. After a number of experiences with color and shock the bees no longer enter the compartment associated with punishment. Intrinsic aspects of avoidance conditioning are associated with natural behavior of bees such as punishment (lack of food, explosive pollination mechanisms, danger of predation, heat, etc.) and their association to floral traits or other spatial cues during foraging. The results show that DA reduces the punishment received whereas octopamine OA increases the punishment received. These effects are dose-dependent and specific to the acquisition phase of training. The effects during acquisition are specific as shown in experiments using the antagonists Pimozide and Mianserin for DA and OA receptors, respectively. This study demonstrates the integrative role of biogenic amines in aversive learning in the honey bee as modeled in a novel non-appetitive avoidance learning assay.     

Aversion learning techniques to evaluate dairy cattle handling practices

Fear of humans is a source of stress for Holstein dairy cattle and can result when animals are handled aversively. We used aversion learning techniques to determine which handling practices cattle find most aversive. In an aversion race, the cows are repeatedly walked down a race and treatments applied when they reached the end; the time and force required for cows to walk down the race are measured. The animal learns to associate walking down the race with the treatment received; if the treatment is aversive, the animal will take more time and require more force to reach the end of the race than if the treatment is positive. In experiment 1, 54 cows were assigned to four treatments (hit/shout, brushing, control, and food). Treatments of 2-min duration were applied three times a day for 4 days. Cows on the hit/shout treatment took more time and required more force to walk through the race than cows on other treatments, while brushed cows took longer to move through the race than cows given food. In experiment 2, 60 cows were assigned to five treatments (electric prod, shouting, hitting, tail twist, and control). Treatments of 1-min duration were applied three times a day for 3 days. Cows on the shout and electric prod treatments took more time and required more force to walk down the race than cows on the control treatment. In experiment 3, thirty-six 1-1.5-year-old heifers were assigned to three treatments (hand feeding, gentling, or control) applied as in experiment 2. Treatments did not affect the time or force required to walk down the race. The aversion race successfully discriminated between handling treatments that differ greatly in aversiveness but lacked sensitivity to distinguish between treatments that were similar. Although many procedural factors must be considered, aversion learning techniques are an effective method to determine which handling practices cattle find aversive or rewarding.