Win-stay, lose-shift strategies for repeated games-memory length, aspiration levels and noise.

Win-stay, lose-shift, the principle to retain a successful action is a simple and general learning rule that can be applied to all types of repeated decision problems. In this paper I consider win-stay, lose-shift strategies with diverse memory sizes and strategies that adapt their aspiration levels, i.e. the payoff level considered as "success". I study their evolution for the Prisoner's Dilemma, as well as in a rapidly changing environment, where a randomly selected game is assigned to the players. For win-stay, lose-shift strategies with memory one the average payoffs are computed and their evolutionary stability is discussed. Using computer simulations I show that the win-stay, lose-shift strategies with longer memory are very successful both for the Prisoner's Dilemma, where cooperation dominates even for high noise levels, and the randomly assigned games, where the players achieve nearly the expected Pareto optimal payoffs. I discuss the impact of noise and show that the memory length of the players increases with the noise level. These results indicate that the win-stay, lose-shift principle is a very successful strategy in repeated games with noise.     

Rhesus monkey (Macaca mulatta) complex learning skills reassessed

Results from three experiments on basic learning and transfer in rhesus monkeys (Macaca mulatta) are reported in which fully automated testing paradigms, afforded by the Language Research Center's Computerized Test System (LRC-CTS), were employed. Performance levels for discrimination learning set, transfer index, and mediational-learning testing were uniformly higher than was predicted from the literature, in contrast to previous reports of compromised learning under similar conditions (automated apparatus, planimetric stimuli, spatial discontiguity between stimuli and response loci). Analyses reveal relatively advanced learning set performance, transfer-index ratios, and positive transfer of learning even at stringent criterion levels. Moreover, the data suggest that rhesus monkeys tested in these experiments exhibit mediational instead of associative learning strategies, as do great apes and in contrast to previous reports of rhesus learning. We argue that the LRC-CTS enhances learning by nonhuman primate subjects, obviating those factors, reported in the literature from experiments in which manual or other automated systems were employed, that compromise learning.     

Investigations on training, recall and reversal learning of a Y-maze by dwarf goats (Capra hircus): the impact of lateralisation

We investigated maze learning in dwarf goats (Capra hircus) and the impact of lateralisation on learning. Lateralisation refers to the collection of phenomena in which external stimuli are perceived and processed differentially on the two sides of the brain and/or certain behaviours are preferentially performed by one side of the body. We trained 29 dwarf goats in a Y-maze, directing them to the opposite alley from that chosen in a free pre-run. In total, 13 goats were trained to the left alley (L-goats) and 16 goats to the right alley (R-goats). Recall of the trained alley was tested three months later. We then analysed reversal learning across 10 reversals. During training, the direction of the alley had an impact on learning. The number of runs required to reach the learning criterion was significantly lower in the L- than the R-goats. The goats recalled the trained alley three months later, with no difference between the L- and the R-goats. During the reversal learning, the reversal only tended to impact learning performance, whereas the directions of the new and the initially trained alley did not. Goats did not adopt a general rule with which to master the maze (e.g., win-stay/lose-shift) across the 10 reversals. Our results indicate a right hemisphere bias in the processing of visuospatial cues in the maze during initial training; however, no such impact was detected during reversal learning.     

Associative learning and memory in Drosophila: beyond olfactory conditioning

The associative learning abilities of the fruit fly, Drosophila melanogaster, have been demonstrated in both classical and operant conditioning paradigms. Efforts to identify the neural pathways and cellular mechanisms of learning have focused largely on olfactory classical conditioning. Results derived from various genetic and molecular manipulations provide considerable evidence that this form of associative learning depends critically on neural activity and cAMP signaling in brain neuropil structures called mushroom bodies. Three other behavioral learning paradigms in Drosophila serve as the main subject of this review. These are (1) visual and motor learning of flies tethered in a flight simulator, (2) a form of spatial learning that is independent of visual and olfactory cues, and (3) experience-dependent changes in male courtship behavior. The present evidence suggests that at least some of these modes of learning are independent of mushroom bodies. Applying targeted genetic manipulations to these behavioral paradigms should allow for a more comprehensive understanding of neural mechanisms responsible for diverse forms of associative learning and memory.     

Testing the social dog hypothesis: Are dogs also more skilled than chimpanzees in non-communicative social tasks?

Relative to non-human primates, domestic dogs possess a number of social skills that seem exceptional—particularly in solving problems involving cooperation and communication with humans. However, the degree to which dogs’ unusual skills are contextually specialized is still unclear. Here, we presented dogs with a social problem that did not require them to use cooperative-communicative cues and compared their performance to that of chimpanzees to assess the extent of dogs’ capabilities relative to those of non-human primates. We tested the abilities of dogs and chimpanzees to inhibit previously learned responses by using a social and a non-social version of a reversal learning task. In contrast to previous findings in cooperative-communicative social tasks, dogs were not more skilled on the social task than the non-social task, while chimpanzees were significantly better in the social paradigm. Chimpanzees were able to inhibit their prior learning better and more quickly in the social paradigm than they were in the non-social paradigm, while dogs took more time to inhibit what they had learned in both versions of the task. These results suggest that the dogs’ sophisticated social skills in using human social cues may be relatively specialized as a result of domestication.     

Reinforcement pre-exposure enhances spatial memory formation in <Emphasis Type="Italic">Drosophila</Emphasis>

Apparently unpaired exposure to appetitive or aversive stimuli can suppress or enhance later associative learning. While the suppressive effect has been found in both vertebrate and invertebrate animals, it is not clear if the enhancing effect is restricted to the vertebrates. Additionally, whether Drosophila associative learning can be influenced in either direction is open. To address these questions, we examined the effects of pre-exposing flies to a high temperature negative reinforcer in the heat-box place-learning paradigm. We found that pre-exposing flies to an unavoidable high temperature enhanced later associative conditioning that uses mild increases in temperature. This enhancement lasts at least 20 min, does not depend on changes in the straightforward avoidance behavior of a high temperature source, and is independent of the antennal thermosensor. We thus provide an example of enhanced associative learning after unpaired exposure to a typical reinforcer in an invertebrate animal, suggesting the conservation of this component of learning.     

Are cognition and personality related in budgerigars?

In recent years, researchers have been attempting to relate differences in personality (e.g., boldness, aggressiveness, exploration tendency) to variation in cognition (performances in tasks that require learning, reasoning, attention, or memory, etc.) both theoretically and empirically. However, it is unclear on what basis personality and cognition might be associated with each other. Previous theory suggests a connection between fast–slow personality types and cognitive speed–accuracy tradeoffs. In this study, we tested this hypothesis in budgerigars and found that, in their 1st associative learning, birds with fast personality (less fearful of handling stress) were fast learners in the beginning, while slow personality individuals improved faster, but both types of birds did not differ in accuracy. However, these relationships were context-dependent. No significant relationship was found in subsequent learning tasks (reversal learning and a 2nd associative learning) in the familiar context (task setup and apparatus similar to the 1st associative learning). We then conducted a problem-solving experiment with novel setup and apparatus to test 1 possible explanation that the association between personality and cognition in the 1st associative learning might be caused by noncognitive constraint, such as fearfulness when facing novel task setup and apparatus. We found that fast individuals interacted more with the problem box and solved it, whereas the slow birds were not. We suggest that personalities can influence cognitive performances and trigger a cognitive speed-improvement tradeoff under the novel context. However, there are no consistent cognitive styles that co-varied with different personalities.     

Olfactory conditioning of proboscis activity in Drosophila melanogaster

Olfactory learning and memory processes in Drosophila have been well investigated with aversive conditioning, but appetitive conditioning has rarely been documented. Here, we report for the first time individual olfactory conditioning of proboscis activity in restrained Drosophila melanogaster. The protocol was adapted from those developed for proboscis extension conditioning in the honeybee Apis mellifera. After establishing a scale of small proboscis movements necessary to characterize responses to olfactory stimulation, we applied Pavlovian conditioning, with five trials consisting of paired presentation of a banana odour and a sucrose reward. Drosophila showed conditioned proboscis activity to the odour, with a twofold increase of percentage of responses after the first trial. No change occurred in flies experiencing unpaired presentations of the stimuli, confirming an associative basis for this form of olfactory learning. The adenylyl cyclase mutant rutabaga did not exhibit learning in this paradigm. This protocol generated at least a short-term memory of 15 min, but no significant associative memory was detected at 1 h. We also showed that learning performance was dependent on food motivation, by comparing flies subjected to different starvation regimes.     

Genetic disruptions of Drosophila Pavlovian learning leave extinction learning intact

Individuals who experience traumatic events may develop persistent posttraumatic stress disorder. Patients with this disorder are commonly treated with exposure therapy, which has had limited long‐term success. In experimental neurobiology, fear extinction is a model for exposure therapy. In this behavioral paradigm, animals are repeatedly exposed in a safe environment to the fearful stimulus, which leads to greatly reduced fear. Studying animal models of extinction already has lead to better therapeutic strategies and development of new candidate drugs. Lack of a powerful genetic model of extinction, however, has limited progress in identifying underlying molecular and genetic factors. In this study, we established a robust behavioral paradigm to study the short‐term effect (acquisition) of extinction in Drosophila melanogaster. We focused on the extinction of olfactory aversive 1‐day memory with a task that has been the main workhorse for genetics of memory in flies. Using this paradigm, we show that extinction can inhibit each of two genetically distinct forms of consolidated memory. We then used a series of single‐gene mutants with known impact on associative learning to examine the effects on extinction. We find that extinction is intact in each of these mutants, suggesting that extinction learning relies on different molecular mechanisms than does Pavlovian learning.     

Investigating the molecular mechanisms of learning and memory using Caenorhabditis elegans

Learning is an essential biological process for survival since it facilitates behavioural plasticity in response to environmental changes. This process is mediated by a wide variety of genes, mostly expressed in the nervous system. Many studies have extensively explored the molecular and cellular mechanisms underlying learning and memory. This review will focus on the advances gained through the study of the nematode Caenorhabditis elegans. C. elegans provides an excellent system to study learning because of its genetic tractability, in addition to its invariant, compact nervous system (~300 neurons) that is well-characterised at the structural level. Importantly, despite its compact nature, the nematode nervous system possesses a high level of conservation with mammalian systems. These features allow the study of genes within specific sensory-, inter- and motor neurons, facilitating the interrogation of signalling pathways that mediate learning via defined neural circuits. This review will detail how learning and memory can be studied in C. elegans through behavioural paradigms that target distinct sensory modalities. We will also summarise recent studies describing mechanisms through which key molecular and cellular pathways are proposed to affect associative and non-associative forms of learning.

     

The parasitic mite <Emphasis Type="Italic">Varroa destructor</Emphasis> affects non-associative learning in honey bee foragers, <Emphasis Type="Italic">Apis mellifera</Emphasis> L.

The parasitic mite Varroa destructor influences flight behavior, orientation and returning success of forager honeybees (Apis mellifera) infested as adults. As impaired orientation toward the nest entrance might be due to deficiency in recognition and responsiveness to stimuli in the environment, we examined effects of V. destructor on sensory responsiveness, non-associative and associative learning of honey bee foragers by using proboscis extension reaction paradigm (PER). Although infested and uninfested workers were initially equally responsive to different concentrations of sugar water, we found differences in non-associative learning. In habituation, PER to repeated sugar stimulation of the antennae occurred faster in infested foragers compared to uninfested foragers. In sensitization, infested foragers showed a lower response to an odor stimulus following sugar stimulation than non-infested foragers. Differences in non-associative paradigms were more pronounced in bees with lower responsiveness to sucrose. In conditioning learning experiments, a significant reduction in proboscis extension response was found 1 min but not 12 min after a single conditioning trial indicating that V. destructor predominantly affects the non-associative components of learning and its underlying neural and molecular processes. Jasna Kralj and Axel Brockmann have contributed equally to this study.     

Colour discrimination learning in black-handed tamarin (<Emphasis Type="Italic">Saguinus midas niger</Emphasis>)

Colour is one cue that monkeys use for perceptual segregation of targets and to identify food resources. For fruit-eating primates such as Saguinus, an accurate colour perception would be advantageous to help find ripe fruits at distance. The colour vision abilities of black-handed tamarins (Saguinus midas niger) were assessed through a discrimination learning paradigm using Munsell colour chips as stimuli. Pairs of chips were chosen from an early experiment with protan and deutan humans. The monkeys (three males and one female) were tested with stimuli of the same hue, but different brightness values, in order to make sure that discriminations were based on colour rather than brightness cues. The results showed that the female, but not the males, presented an above-chance performance for stimuli resembling hue conditions under which tamarins forage (oranges vs greens). Colour vision in S. m. niger is discussed according to the advantages and disadvantages of dichromatism in daily search for food as well as to aspects regarding polymorphism in New World monkeys.     

Reversal Learning and Associative Memory Impairments in a BACHD Rat Model for Huntington Disease

Chorea and psychiatric symptoms are hallmarks of Huntington disease (HD), a neurodegenerative disorder, genetically characterized by the presence of expanded CAG repeats (>35) in the HUNTINGTIN (HTT) gene. HD patients present psychiatric symptoms prior to the onset of motor symptoms and we recently found a similar emergence of non motor and motor deficits in BACHD rats carrying the human full length mutated HTT (97 CAG-CAA repeats). We evaluated cognitive performance in reversal learning and associative memory tests in different age cohorts of BACHD rats. Male wild type (WT) and transgenic (TG) rats between 2 and 12 months of age were tested. Learning and strategy shifting were assessed in a cross-maze test. Associative memory was evaluated in different fear conditioning paradigms (context, delay and trace). The possible confound of a fear conditioning phenotype by altered sensitivity to a ‘painful’ stimulus was assessed in a flinch-jump test. In the cross maze, 6 months old TG rats showed a mild impairment in reversal learning. In the fear conditioning tasks, 4, 6 and 12 months old TG rats showed a marked reduction in contextual fear conditioning. In addition, TG rats showed impaired delay conditioning (9 months) and trace fear conditioning (3 months). This phenotype was unlikely to be affected by a change in ‘pain’ sensitivity as WT and TG rats showed no difference in their threshold response in the flinch-jump test. Our results suggest that BACHD rats have a profound associative memory deficit and, possibly, a deficit in reversal learning as assessed in a cross maze task. The time course for the emergence of these symptoms (i.e., before the occurrence of motor symptoms) in this rat model for HD appears similar to the time course in patients. These data suggest that BACHD rats may be a useful model for preclinical drug discovery.     

Operant conditioning in Drosophila melanogaster wild-type and learning mutants with defects in the cyclic AMP metabolism

Wild-type Drosophila melanogaster and the learning mutants dunce, amnesiac and rutabaga, were tested using a new operant conditioning paradigm for single flies. All strains are able to learn to different extents, but no evidence of memory was found in the mutants amnesiac and rutabaga, while dunce has a reduced but extended memory. The relationship between this characteristic and cAMP levels are discussed. The three mutants have previously been shown, using classical conditioning paradigms to be deficient in olfactory learning and/or memory, and show reduced visual learning. The variability of the response of the mutants in the different paradigms is discussed in relation to the generality of the Aplysia model of the cellular mechanism underlying learning. In the operant conditioning paradigm described here, 93% of the wild-type flies learned to criterion. The performance of individual flies was consistent.     

Learning outdoors: male lizards show flexible spatial learning under semi-natural conditions

Spatial cognition is predicted to be a fundamental component of fitness in many lizard species, and yet some studies suggest that it is relatively slow and inflexible. However, such claims are based on work conducted using experimental designs or in artificial contexts that may underestimate their cognitive abilities. We used a biologically realistic experimental procedure (using simulated predatory attacks) to study spatial learning and its flexibility in the lizard Eulamprus quoyii in semi-natural outdoor enclosures under similar conditions to those experienced by lizards in the wild. To evaluate the flexibility of spatial learning, we conducted a reversal spatial-learning task in which positive and negative reinforcements of learnt spatial stimuli were switched. Nineteen (32%) male lizards learnt both tasks within 10 days (spatial task mean: 8.16 ± 0.69 (s.e.) and reversal spatial task mean: 10.74 ± 0.98 (s.e.) trials). We demonstrate that E. quoyii are capable of flexible spatial learning and suggest that future studies focus on a range of lizard species which differ in phylogeny and/or ecology, using biologically relevant cognitive tasks, in an effort to bridge the cognitive divide between ecto- and endotherms.     

Contextual taste cues modulate olfactory learning in C. elegans by an occasion-setting mechanism

Manipulations of context can affect learning and memory performance across species in many associative learning paradigms. Using taste cues to create distinct contexts for olfactory adaptation assays in the nematode Caenorhabditis elegans, we now show that performance in this associative learning paradigm is sensitive to context manipulations, and we investigate the mechanism(s) used for the integration of context cues in learning. One possibility is that the taste and olfactory stimuli are perceived as a combined, blended cue that the animals then associate with the unconditioned stimulus (US) in the same manner as with any other unitary conditioned stimuli (CS). Alternatively, an occasion-setting model suggests that the taste cues only define the appropriate context for olfactory memory retrieval without directly entering into the primary association. Analysis of genetic mutants demonstrated that the olfactory and context cues are sensed by distinct primary sensory neurons and that the animals' ability to use taste cues to modulate olfactory learning is independent from their ability to utilize these same taste cues for adaptation. We interpret these results as evidence for the occasion-setting mechanism in which context cues modulate primary Pavlovian association by functioning in a hierarchical manner to define the appropriate setting for memory recall.     

Induction of immediate early genes in the mouse auditory cortex after auditory cued fear conditioning to complex sounds

Immediate early genes (IEGs) are widely used as markers to delineate neuronal circuits because they show fast and transient expression induced by various behavioral paradigms. In this study, we investigated the expression of the IEGs c-fos and Arc in the auditory cortex of the mouse after auditory cued fear conditioning using quantitative polymerase chain reaction and microarray analysis. To test for the specificity of the IEG induction, we included several control groups that allowed us to test for factors other than associative learning to sounds that could lead to an induction of IEGs. We found that both c-fos and Arc showed strong and robust induction after auditory fear conditioning. However, we also observed increased expression of both genes in any control paradigm that involved shocks, even when no sounds were presented. Using mRNA microarrays and comparing the effect of the various behavioral paradigms on mRNA expression levels, we did not find genes being selectively upregulated in the auditory fear conditioned group. In summary, our results indicate that the use of IEGs to identify neuronal circuits involved specifically in processing of sound cues in the fear conditioning paradigm can be limited by the effects of the aversive unconditional stimulus and that activity levels in a particular primary sensory cortical area can be strongly influenced by stimuli mediated by other modalities.     

The Relationship Between Foraging,Learning Abilities and Neophobia in Two Species of Darwin’s Finches

The ability to unlearn a previously established association is an important component of behavioural flexibility and may vary according to species ecology. Previously, two closely related sympatric Darwin’s finches were found to differ in their learning abilities. Small tree finches (Camarhynchus parvulus) outperformed woodpecker finches (Cactospiza pallida) in reversal learning but performed worse in an operant task. We attributed this difference to the habit of woodpecker finches to engage in long bouts of energetic pecking during extractive foraging. Persistently repeating one action without reward could favour performance in operant tasks but also limit behavioural flexibility. Here, we tested whether perseverance is the reason for woodpecker finches’ depressed reversal learning performance. Two new reversal conditions allowed the disentanglement of two sources of error in reversal learning: perseverant choice of the previously rewarded stimulus and failure to respond to the previously non‐rewarded stimulus. For the within‐species comparison, we predicted that woodpecker finches should find it more difficult to learn to avoid the previously rewarded stimulus than learning to choose the previously non‐rewarded stimulus. For the species comparison, we predicted the woodpecker finches should make more errors of perseverance than small tree finches. As performance could also be influenced by reaction to novelty, we compared neophobic responses between species and related them to reversal learning proficiency. We found no significant difference in reversal learning in the predicted direction, but found a negative correlation between neophobia and reversal learning at the inter‐ and the intraspecific level, which points towards a general relationship between reaction to novelty and flexibility.     

Interval time-place learning by rats: varying reinforcement contingencies

Two experiments with rats were conducted to study interval time-place learning when the spatiotemporal contingencies of food availability were more similar to those likely to be encountered in natural environments, than those employed in prior research. In Experiment 1, food was always available on three levers on a variable ratio (VR) 35 schedule. A VR8 schedule was in effect on Lever 1 for 5 min, then on Lever 2 for 5 min, and so forth. While rats learned to restrict the majority of their responding to the lever that provided the highest density of reinforcement, they seemed to rely on a win-stay/lose-shift strategy rather than a timing strategy. In Experiment 2, the four levers provided food on variable ratios of 15, 8, 15, and 30, each for 3 min. As expected the rats learned these contingencies. A novel finding was that the rats had a spike in response rate immediately following a change from a higher to lower reinforcement density. It is concluded that rats exposed to spatiotemporal contingencies behave so as to maximize the rate of obtained reinforcement.