Evolutionarily stable learning schedules and cumulative culture in discrete generation models

Individual learning (e.g., trial-and-error) and social learning (e.g., imitation) are alternative ways of acquiring and expressing the appropriate phenotype in an environment. The optimal choice between using individual learning and/or social learning may be dictated by the life-stage or age of an organism. Of special interest is a learning schedule in which social learning precedes individual learning, because such a schedule is apparently a necessary condition for cumulative culture. Assuming two obligatory learning stages per discrete generation, we obtain the evolutionarily stable learning schedules for the three situations where the environment is constant, fluctuates between generations, or fluctuates within generations. During each learning stage, we assume that an organism may target the optimal phenotype in the current environment by individual learning, and/or the mature phenotype of the previous generation by oblique social learning. In the absence of exogenous costs to learning, the evolutionarily stable learning schedules are predicted to be either pure social learning followed by pure individual learning (“bang-bang” control) or pure individual learning at both stages (“flat” control). Moreover, we find for each situation that the evolutionarily stable learning schedule is also the one that optimizes the learned phenotype at equilibrium.     

Fast heterosynaptic learning in a robot food retrieval task inspired by the limbic system

Hebbian learning is the most prominent paradigm in correlation based learning: if pre- and postsynaptic activity coincides the weight of the synapse is strengthened. Hebbian learning however, is not stable because of an autocorrelation term which causes the weights to grow exponentially. The standard solution would be to compensate the autocorrelation term. However, in this work we present a heterosynaptic learning rule which does not have an autocorrelation term and therefore does not show the instability of Hebbian learning. Consequently our heterosynaptic learning is much more stable than the classical Hebbian learning. The performance of our learning rule is demonstrated in a model which is inspired by the limbic system where an agent has to retrieve food.     

Co-evolution of learning complexity and social foraging strategies

Variation in learning abilities within populations suggests that complex learning may not necessarily be more adaptive than simple learning. Yet, the high cost of complex learning cannot fully explain this variation without some understanding of why complex learning is too costly for some individuals but not for others. Here we propose that different social foraging strategies can favor different learning strategies (that learn the environment with high or low resolution), thereby maintaining variable learning abilities within populations. Using a genetic algorithm in an agent-based evolutionary simulation of a social foraging game (the producer-scrounger game) we demonstrate how an association evolves between a strategy based on independent search for food (playing a producer) and a complex (high resolution) learning rule, while a strategy that combines independent search and following others (playing a scrounger) evolves an association with a simple (low resolution) learning rule. The reason for these associations is that for complex learning to have an advantage, a large number of learning steps, normally not achieved by scroungers, are necessary. These results offer a general explanation for persistent variation in cognitive abilities that is based on co-evolution of learning rules and social foraging strategies.     

Critical Social Learning: A Solution to Rogers's Paradox of Nonadaptive Culture

Alan Rogers (1988) presented a game theory model of the evolution of social learning, yielding the paradoxical conclusion that social learning does not increase the fitness of a population. We expand on this model, allowing for imperfections in individual and social learning as well as incorporating a "critical social learning" strategy that tries to solve an adaptive problem first by social learning, and then by individual learning if socially acquired behavior proves unsatisfactory. This strategy always proves superior to pure social learning and typically has higher fitness than pure individual learning, providing a solution to Rogers's paradox of nonadaptive culture. Critical social learning is an evolutionarily stable strategy (ESS) unless cultural transmission is highly unfaithful, the environment is highly variable, or social learning is much more costly than individual learning. We compare the model to empirical data on social learning and on spatial variation in primate cultures and list three requirements for adaptive culture.     

Structure learning in human sequential decision-making

Studies of sequential decision-making in humans frequently find suboptimal performance relative to an ideal actor that has perfect knowledge of the model of how rewards and events are generated in the environment. Rather than being suboptimal, we argue that the learning problem humans face is more complex, in that it also involves learning the structure of reward generation in the environment. We formulate the problem of structure learning in sequential decision tasks using Bayesian reinforcement learning, and show that learning the generative model for rewards qualitatively changes the behavior of an optimal learning agent. To test whether people exhibit structure learning, we performed experiments involving a mixture of one-armed and two-armed bandit reward models, where structure learning produces many of the qualitative behaviors deemed suboptimal in previous studies. Our results demonstrate humans can perform structure learning in a near-optimal manner.     

A mixed strategy model for the emergence and intensification of social learning in a periodically changing natural environment

Based on a population genetic model of mixed strategies determined by alleles of small effect, we derive conditions for the evolution of social learning in an infinite-state environment that changes periodically over time. Each mixed strategy is defined by the probabilities that an organism will commit itself to individual learning, social learning, or innate behavior. We identify the convergent stable strategies (CSS) by a numerical adaptive dynamics method and then check the evolutionary stability (ESS) of these strategies. A strategy that is simultaneously a CSS and an ESS is called an attractive ESS (AESS). For certain parameter sets, a bifurcation diagram shows that the pure individual learning strategy is the unique AESS for short periods of environmental change, a mixed learning strategy is the unique AESS for intermediate periods, and a mixed learning strategy (with a relatively large social learning component) and the pure innate strategy are both AESS's for long periods. This result entails that, once social learning emerges during a transient era of intermediate environmental periodicity, a subsequent elongation of the period may result in the intensification of social learning, rather than a return to innate behavior.     

A mixed strategy model for the emergence and intensification of social learning in a periodically changing natural environment

Based on a population genetic model of mixed strategies determined by alleles of small effect, we derive conditions for the evolution of social learning in an infinite-state environment that changes periodically over time. Each mixed strategy is defined by the probabilities that an organism will commit itself to individual learning, social learning, or innate behavior. We identify the convergent stable strategies (CSS) by a numerical adaptive dynamics method and then check the evolutionary stability (ESS) of these strategies. A strategy that is simultaneously a CSS and an ESS is called an attractive ESS (AESS). For certain parameter sets, a bifurcation diagram shows that the pure individual learning strategy is the unique AESS for short periods of environmental change, a mixed learning strategy is the unique AESS for intermediate periods, and a mixed learning strategy (with a relatively large social learning component) and the pure innate strategy are both AESS's for long periods. This result entails that, once social learning emerges during a transient era of intermediate environmental periodicity, a subsequent elongation of the period may result in the intensification of social learning, rather than a return to innate behavior.     

An experimental comparison of human social learning strategies: payoff-biased social learning is adaptive but underused

Analytical models have identified a set of social learning strategies that are predicted to be adaptive relative to individual (asocial) learning. In the present study, human participants engaged in an ecologically valid artifact-design task with the opportunity to engage in a range of social learning strategies: payoff bias, conformity, averaging and random copying. The artifact (an arrowhead) was composed of multiple continuous and discrete attributes which jointly generated a complex multimodal adaptive landscape that likely reflects actual cultural fitness environments. Participants exhibited a mix of individual learning and payoff-biased social learning, with negligible frequencies of the other social learning strategies. This preference for payoff-biased social learning was evident from the initial trials, suggesting that participants came into the study with an intrinsic preference for this strategy. There was also a small but significant increase in the frequency of payoff-biased social learning over sessions, suggesting that strategy choice may itself be subject to learning. Frequency of payoff-biased social learning predicted both absolute and relative success in the task, especially in a multimodal (rather than unimodal) fitness environment. This effect was driven by a minority of hardcore social learners who copied the best group member on more than half of trials. These hardcore social learners were also above-average individual learners, suggesting a link between individual and social learning ability. The lower-than-expected frequency of social learning may reflect the existence of information producer–scrounger dynamics in human populations.     

Alterations in intrinsic neuronal excitability during normal aging

Normal aging subjects, including humans, have difficulty learning hippocampus-dependent tasks. For example, at least 50% of normal aging rabbits and rats fail to meet a learning criterion in trace eyeblink conditioning. Many factors may contribute to this age-related learning impairment. An important cause is the reduced intrinsic excitability observed in hippocampal pyramidal neurons from normal aging subjects, as reflected by an enlarged postburst afterhyperpolarization (AHP) and an increased spike-frequency adaptation (accommodation). In this review, we will focus on the alterations in the AHP and accommodation during learning and normal aging. We propose that age-related increases in the postburst AHP and accommodation in hippocampal pyramidal neurons play an integral role in the learning impairment observed in normal aging subjects.     

Cognitive processing and motor skill learning in motor-handicapped teenagers: Effects of learning method

This study aimed to assess the efficiency of a motor skill learning method intended to promote learning course personalization through an increase in cognitive processing deployment in motor-handicapped persons. Thirty-three secondary school students volunteered to participate in an archery motor skill learning session, 11 motor-handicapped (MH₁) and 11 able-bodied (AB) teenagers following a standard learning method, and 11 motor-handicapped teenagers following a cognitive enriched learning method (MH₂) based on the use of an individually written and illustrated document. The results showed that MH₁ displayed lower performances than AB, both in terms of the mental representations of the movements expected and performed and of efficiency of the movement. On the other hand, MH₂ performances were higher than MH₁ for all these parameters, and similar to those of AB at the end of the learning session. Personalization of the learning course allowed optimization of the learning potential in motor-handicapped teenagers to resolve the difficulties inherent to their handicap.     

Ensemble Positive Unlabeled Learning for Disease Gene Identification

An increasing number of genes have been experimentally confirmed in recent years as causative genes to various human diseases. The newly available knowledge can be exploited by machine learning methods to discover additional unknown genes that are likely to be associated with diseases. In particular, positive unlabeled learning (PU learning) methods, which require only a positive training set P (confirmed disease genes) and an unlabeled set U (the unknown candidate genes) instead of a negative training set N, have been shown to be effective in uncovering new disease genes in the current scenario. Using only a single source of data for prediction can be susceptible to bias due to incompleteness and noise in the genomic data and a single machine learning predictor prone to bias caused by inherent limitations of individual methods. In this paper, we propose an effective PU learning framework that integrates multiple biological data sources and an ensemble of powerful machine learning classifiers for disease gene identification. Our proposed method integrates data from multiple biological sources for training PU learning classifiers. A novel ensemble-based PU learning method EPU is then used to integrate multiple PU learning classifiers to achieve accurate and robust disease gene predictions. Our evaluation experiments across six disease groups showed that EPU achieved significantly better results compared with various state-of-the-art prediction methods as well as ensemble learning classifiers. Through integrating multiple biological data sources for training and the outputs of an ensemble of PU learning classifiers for prediction, we are able to minimize the potential bias and errors in individual data sources and machine learning algorithms to achieve more accurate and robust disease gene predictions. In the future, our EPU method provides an effective framework to integrate the additional biological and computational resources for better disease gene predictions.     

A quantitative dynamical systems approach to differential learning: self-organization principle and order parameter equations

Differential learning is a learning concept that assists subjects to find individual optimal performance patterns for given complex motor skills. To this end, training is provided in terms of noisy training sessions that feature a large variety of between-exercises differences. In several previous experimental studies it has been shown that performance improvement due to differential learning is higher than due to traditional learning and performance improvement due to differential learning occurs even during post-training periods. In this study we develop a quantitative dynamical systems approach to differential learning. Accordingly, differential learning is regarded as a self-organized process that results in the emergence of subject- and context-dependent attractors. These attractors emerge due to noise-induced bifurcations involving order parameters in terms of learning rates. In contrast, traditional learning is regarded as an externally driven process that results in the emergence of environmentally specified attractors. Performance improvement during post-training periods is explained as an hysteresis effect. An order parameter equation for differential learning involving a fourth-order polynomial potential is discussed explicitly. New predictions concerning the relationship between traditional and differential learning are derived.     

Plasticity in the Rat Prefrontal Cortex: Linking Gene Expression and an Operant Learning with a Computational Theory

The plasticity in the medial Prefrontal Cortex (mPFC) of rodents or lateral prefrontal cortex in non human primates (lPFC), plays a key role neural circuits involved in learning and memory. Several genes, like brain-derived neurotrophic factor (BDNF), cAMP response element binding (CREB), Synapsin I, Calcium/calmodulin-dependent protein kinase II (CamKII), activity-regulated cytoskeleton-associated protein (Arc), c-jun and c-fos have been related to plasticity processes. We analysed differential expression of related plasticity genes and immediate early genes in the mPFC of rats during learning an operant conditioning task. Incompletely and completely trained animals were studied because of the distinct events predicted by our computational model at different learning stages. During learning an operant conditioning task, we measured changes in the mRNA levels by Real-Time RT-PCR during learning; expression of these markers associated to plasticity was incremented while learning and such increments began to decline when the task was learned. The plasticity changes in the lPFC during learning predicted by the model matched up with those of the representative gene BDNF. Herein, we showed for the first time that plasticity in the mPFC in rats during learning of an operant conditioning is higher while learning than when the task is learned, using an integrative approach of a computational model and gene expression.     

Virus infection causes specific learning deficits in honeybee foragers

In both mammals and invertebrates, virus infections can impair a broad spectrum of physiological functions including learning and memory formation. In contrast to the knowledge on the conserved mechanisms underlying learning, the effects of virus infection on different aspects of learning are barely known. We use the honeybee (Apis mellifera), a well-established model system for studying learning, to investigate the impact of deformed wing virus (DWV) on learning. Injection of DWV into the haemolymph of forager leads to a RT-PCR detectable DWV signal after 3 days. The detailed behavioural analysis of DWV-infected honeybees shows an increased responsiveness to water and low sucrose concentrations, an impaired associative learning and memory formation, but intact non-associative learning like sensitization and habituation. This contradicts all present studies in non-infected bees, where increased sucrose responsiveness is linked to improved associative learning and to changes in non-associative learning. Thus, DWV seems to interfere with molecular mechanism of learning by yet unknown processes that may include viral effects on the immune system and on gene expression.     

Cognitive processing and motor skill learning in motor-handicapped teenagers: effects of learning method

This study aimed to assess the efficiency of a motor skill learning method intended to promote learning course personalization through an increase in cognitive processing deployment in motor-handicapped persons. Thirty-three secondary school students volunteered to participate in an archery motor skill learning session, 11 motor-handicapped (MH(1)) and 11 able-bodied (AB) teenagers following a standard learning method, and 11 motor-handicapped teenagers following a cognitive enriched learning method (MH(2)) based on the use of an individually written and illustrated document. The results showed that MH(1) displayed lower performances than AB, both in terms of the mental representations of the movements expected and performed and of efficiency of the movement. On the other hand, MH(2) performances were higher than MH(1) for all these parameters, and similar to those of AB at the end of the learning session. Personalization of the learning course allowed optimization of the learning potential in motor-handicapped teenagers to resolve the difficulties inherent to their handicap.     

A note on a quantitative genetic approach for modelling of differentiation tasks

Learning capacities and cognitive abilities of farm animals will become more important and they will have an impact on animal welfare and productivity. Scientific examinations of these aspects presuppose genetic models. This paper presents an approach for modelling differentiation tasks in animal learning on a quantitative genetic basis. The approach links the likelihood function, to make a correct decision, with the two-dimensional density function, capable to be to decide right. The proposed model is able to depict several situations in learning processes of populations. Its parameters characterize the theoretical course of the learning process and also the stage of learning which a population has reached. The approach can be used to estimate parameters of a learning process and of the genetic disposition to learn, to estimate breeding values of learning capacity and to fit models on population's learning capability under selection.     

Isotropic sequence order learning using a novel linear algorithm in a closed loop behavioural system

In this article, we present an isotropic algorithm for sequence order learning. Its central goal is to learn the causal relation between two (or more) inputs in order to react to the earliest incoming signal after successful learning (like in typical classical conditioning situations). We implement this algorithm in a behaving system (a robot) thereby creating a closed loop situation where the learner's actions influence its own sensor inputs to the end of creating an autonomous agent. Autonomous behaviour implies that learning goals are internally defined within the organism's capabilities. Standard learning models for sequence learning (e.g. temporal difference (TD)-learning) need an externally defined reward. This, however, is in conflict with the requirement of an implicitly defined internal goal in autonomous behaviour. Therefore, in this study we present a system in which the external reward is replaced by a reflex loop. This loop explicitly includes the environment. Every reflex loop has the inherent disadvantage, which is that its re-actions occur each time just after a reflex-eliciting sensor event and thus 'too late'. However, a reflex can serve as the internal reference for sequence order learning, which has the task of eliminating this disadvantage by creating earlier anticipatory actions. In our system learning is achieved by modifying synaptic weights of a linear neuron with a correlation based learning rule which involves the derivative of the neuron's output. All input lines are entirely isotropic. The synaptic weight change curve of this rule is strongly related to the temporal Hebb learning rule, which was found in spike timing experiments. We find that after learning the reflex loop is replaced in functional terms with an earlier anticipatory action (and pathway). In addition, we observed that the synaptic weights stabilise as soon as the reflex remains silent.     

Learning, evolvability and exploratory behaviour: extending the evolutionary reach of learning

Traditional accounts of the role of learning in evolution have concentrated upon its capacity as a source of fitness to individuals. In this paper I use a case study from invasive species biology—the role of conditioned taste aversion in mitigating the impact of cane toads on the native species of Northern Australia—to highlight a role for learning beyond this—as a source of evolvability to populations. This has two benefits. First, it highlights an otherwise under-appreciated role for learning in evolution that does not rely on social learning as an inheritance channel nor “special” evolutionary processes such as genetic accommodation (both of which many are skeptical about). Second, and more significantly, it makes clear important and interesting parallels between learning and exploratory behaviour in development. These parallels motivate the applicability of results from existing research into learning and learning evolution to our understanding the evolution of evolvability more generally.