Evolutionarily stable learning schedules and cumulative culture in discrete generation models

Individual learning (e.g., trial-and-error) and social learning (e.g., imitation) are alternative ways of acquiring and expressing the appropriate phenotype in an environment. The optimal choice between using individual learning and/or social learning may be dictated by the life-stage or age of an organism. Of special interest is a learning schedule in which social learning precedes individual learning, because such a schedule is apparently a necessary condition for cumulative culture. Assuming two obligatory learning stages per discrete generation, we obtain the evolutionarily stable learning schedules for the three situations where the environment is constant, fluctuates between generations, or fluctuates within generations. During each learning stage, we assume that an organism may target the optimal phenotype in the current environment by individual learning, and/or the mature phenotype of the previous generation by oblique social learning. In the absence of exogenous costs to learning, the evolutionarily stable learning schedules are predicted to be either pure social learning followed by pure individual learning (“bang-bang” control) or pure individual learning at both stages (“flat” control). Moreover, we find for each situation that the evolutionarily stable learning schedule is also the one that optimizes the learned phenotype at equilibrium.     

Critical Social Learning: A Solution to Rogers's Paradox of Nonadaptive Culture

Alan Rogers (1988) presented a game theory model of the evolution of social learning, yielding the paradoxical conclusion that social learning does not increase the fitness of a population. We expand on this model, allowing for imperfections in individual and social learning as well as incorporating a "critical social learning" strategy that tries to solve an adaptive problem first by social learning, and then by individual learning if socially acquired behavior proves unsatisfactory. This strategy always proves superior to pure social learning and typically has higher fitness than pure individual learning, providing a solution to Rogers's paradox of nonadaptive culture. Critical social learning is an evolutionarily stable strategy (ESS) unless cultural transmission is highly unfaithful, the environment is highly variable, or social learning is much more costly than individual learning. We compare the model to empirical data on social learning and on spatial variation in primate cultures and list three requirements for adaptive culture.     

Development of a model for whole brain learning of physiology

In this report, a model was developed for whole brain learning based on Curry's onion model. Curry described the effect of personality traits as the inner layer of learning, information-processing styles as the middle layer of learning, and environmental and instructional preferences as the outer layer of learning. The model that was developed elaborates on these layers by relating the personality traits central to learning to the different quadrants of brain preference, as described by Neethling's brain profile, as the inner layer of the onion. This layer is encircled by the learning styles that describe different information-processing preferences for each brain quadrant. For the middle layer, the different stages of Kolb's learning cycle are classified into the four brain quadrants associated with the different brain processing strategies within the information processing circle. Each of the stages of Kolb's learning cycle is also associated with a specific cognitive learning strategy. These two inner circles are enclosed by the circle representing the role of the environment and instruction on learning. It relates environmental factors that affect learning and distinguishes between face-to-face and technology-assisted learning. This model informs on the design of instructional interventions for physiology to encourage whole brain learning.     

Learning the evolutionarily stable strategy

The possibility that animals learn a “developmentally stable strategy” (DSS) (Dawkins, 1980) is an alternative in biological game theory to the idea that evolutionarily stable strategies (ESS) (Maynard Smith, 1972) are genetically determined. A learning rule is defined as a rule which assigns for every possible behaviour the probability of displaying that behaviour at each trial of a game as a function of previous payoffs. This report examines properties of the evolutionarily stable (ES) learning rule, i.e. the rule which, when adopted by a population, is uninvadable by a mutant with a different learning rule. The DSS is defined as the strategy used by individuals with the ES learning rule. With some simplifying assumptions, it is shown that the DSS is the ESS: the ES learning rule is a rule for learning ESSs. This and other properties of the ES learning rule suggested that an approximation to such a rule is the relative payoff sum (RPS) learning rule, which states that the probability of displaying a behaviour is equal to the cumulative payoff for that behaviour relative to the total sum of payoffs for the game. Residual payoffs and a memory factor are incorporated into the RPS learning rule to account for prior expectations of payoff and the decay of memory with time. Both features are adaptive. In simulations of several frequency dependent and frequency independent games using the RPS learning rule, the response of the simulated animals was consistent with the predictions of the ES learning rule. This analysis has shown how ESSs may be achieved by non-genetic means. The RPS learning rule is described in molecular terms utilizing synthesis, storage, and degradation of a substance which elicits the behavioural response. If the RPS learning rule is used by animals, it should be possible to identify within neurons substances whose synthesis is regulated by behavioural stimuli and which initiate alternative behaviours in proportion to their concentrations.     

A mixed strategy model for the emergence and intensification of social learning in a periodically changing natural environment

Based on a population genetic model of mixed strategies determined by alleles of small effect, we derive conditions for the evolution of social learning in an infinite-state environment that changes periodically over time. Each mixed strategy is defined by the probabilities that an organism will commit itself to individual learning, social learning, or innate behavior. We identify the convergent stable strategies (CSS) by a numerical adaptive dynamics method and then check the evolutionary stability (ESS) of these strategies. A strategy that is simultaneously a CSS and an ESS is called an attractive ESS (AESS). For certain parameter sets, a bifurcation diagram shows that the pure individual learning strategy is the unique AESS for short periods of environmental change, a mixed learning strategy is the unique AESS for intermediate periods, and a mixed learning strategy (with a relatively large social learning component) and the pure innate strategy are both AESS's for long periods. This result entails that, once social learning emerges during a transient era of intermediate environmental periodicity, a subsequent elongation of the period may result in the intensification of social learning, rather than a return to innate behavior.     

A mixed strategy model for the emergence and intensification of social learning in a periodically changing natural environment

Based on a population genetic model of mixed strategies determined by alleles of small effect, we derive conditions for the evolution of social learning in an infinite-state environment that changes periodically over time. Each mixed strategy is defined by the probabilities that an organism will commit itself to individual learning, social learning, or innate behavior. We identify the convergent stable strategies (CSS) by a numerical adaptive dynamics method and then check the evolutionary stability (ESS) of these strategies. A strategy that is simultaneously a CSS and an ESS is called an attractive ESS (AESS). For certain parameter sets, a bifurcation diagram shows that the pure individual learning strategy is the unique AESS for short periods of environmental change, a mixed learning strategy is the unique AESS for intermediate periods, and a mixed learning strategy (with a relatively large social learning component) and the pure innate strategy are both AESS's for long periods. This result entails that, once social learning emerges during a transient era of intermediate environmental periodicity, a subsequent elongation of the period may result in the intensification of social learning, rather than a return to innate behavior.     

Why Robots Should Be Social: Enhancing Machine Learning through Social Human-Robot Interaction

Social learning is a powerful method for cultural propagation of knowledge and skills relying on a complex interplay of learning strategies, social ecology and the human propensity for both learning and tutoring. Social learning has the potential to be an equally potent learning strategy for artificial systems and robots in specific. However, given the complexity and unstructured nature of social learning, implementing social machine learning proves to be a challenging problem. We study one particular aspect of social machine learning: that of offering social cues during the learning interaction. Specifically, we study whether people are sensitive to social cues offered by a learning robot, in a similar way to children’s social bids for tutoring. We use a child-like social robot and a task in which the robot has to learn the meaning of words. For this a simple turn-based interaction is used, based on language games. Two conditions are tested: one in which the robot uses social means to invite a human teacher to provide information based on what the robot requires to fill gaps in its knowledge (i.e. expression of a learning preference); the other in which the robot does not provide social cues to communicate a learning preference. We observe that conveying a learning preference through the use of social cues results in better and faster learning by the robot. People also seem to form a “mental model” of the robot, tailoring the tutoring to the robot’s performance as opposed to using simply random teaching. In addition, the social learning shows a clear gender effect with female participants being responsive to the robot’s bids, while male teachers appear to be less receptive. This work shows how additional social cues in social machine learning can result in people offering better quality learning input to artificial systems, resulting in improved learning performance.     

Evolution of learned strategy choice in a frequency-dependent game

In frequency-dependent games, strategy choice may be innate or learned. While experimental evidence in the producer-scrounger game suggests that learned strategy choice may be common, a recent theoretical analysis demonstrated that learning by only some individuals prevents learning from evolving in others. Here, however, we model learning explicitly, and demonstrate that learning can easily evolve in the whole population. We used an agent-based evolutionary simulation of the producer-scrounger game to test the success of two general learning rules for strategy choice. We found that learning was eventually acquired by all individuals under a sufficient degree of environmental fluctuation, and when players were phenotypically asymmetric. In the absence of sufficient environmental change or phenotypic asymmetries, the correct target for learning seems to be confounded by game dynamics, and innate strategy choice is likely to be fixed in the population. The results demonstrate that under biologically plausible conditions, learning can easily evolve in the whole population and that phenotypic asymmetry is important for the evolution of learned strategy choice, especially in a stable or mildly changing environment.     

对比度检测学习提高猫的视觉对比敏感度

对人的心理学研究结果显示,对比度检测学习可提高学习者对视觉刺激的对比敏感度,但其潜在的神经机制尚不清楚。该研究用二选一（two-alternative forced choice）方法训练3只猫（Felis catus）通过单眼进行对比度检测学习,发现每只猫对视觉刺激的对比敏感度随着训练而显著提高。该学习效果虽然对训练眼有明显的特异性,但部分学习效果可以传递给非训练眼,提示对比度检测学习可能会引起双眼信息汇聚前后的视觉中枢的神经可塑性。另外,猫视觉对比敏感度的提高主要发生在训练刺激的空间频率附近,表明对比度检测学习具有一定的空间频率选择性。该研究结果显示,猫对视觉刺激的对比度检测学习表现出与人类相似的特性,因此可以作为模式动物来研究人类学习诱导的视觉对比敏感度升高的神经机制。     

Student-centered introductory biology course: evidence for deep learning

Our interpretative study that was carried out in a science and engineering oriented university examined the ways students in an introductory biology course perceived their learning in the course that was substantially changed to allow student-centered learning. The instructional change was framed by the view of learning as a sociocultural activity as well as a cognitive process that can take place face-to-face or through online interaction. Most of the lectures were substituted with individual learning and project-based, small-group learning that lasted one month. Data were collected through interviews with students and instructors, and through observations. In the paper, we show evidence for deep learning that was associated by the students and the instructors with short-term, meaningful activities in a setting that included collaborative peer learning; and replacing most lectures by small group learning that ended in a mini-conference. Deep learning was evidenced by the ways students reflected on how they organised and applied knowledge using deep learning strategies.     

Replacing lectures by textbased flexible learning: students' performance and perceptions

This paper presents the results of an extended evaluation programme designed to test the effectiveness of text-based flexible learning as a replacement to 30 – 50% of the lectures in certain modules in conventional undergraduate courses within the School of Life Sciences at Napier University. For examinations in which students answered both types of question, marks for questions based on topics taught by flexible learning were equivalent to marks for questions based on topics taught by conventional teaching methods. Marks for examinations in which at least 50% of the answers were based on topics taught by flexible learning were equivalent to marks for examinations in which questions were based on topics taught by conventional teaching methods. Rates of achievement of a mark of 40% or more at first attempt for examinations in which at least 50% of the answers were based on topics taught by flexible learning were significantly better than rates of achievement of a mark of 40% or more at first attempt for questions based on topics taught by conventional teaching methods. Students gave positive feedback on flexible learning, both verbally and by questionnaire, and showed highly significant bias in favour of topics taught by flexible learning in their choice of questions in examinations. The flexible learning programme studied here has satisfied the various quality assurance criteria in place within the University throughout the time that it has been in operation. The evaluation has demonstrated that the textbased flexible learning programme studied here was an effective alternative to lectures.     

Learning ability and longevity: a symmetrical evolutionary trade-off in Drosophila

Learning ability can be substantially improved by artificial selection in animals ranging from Drosophila to rats. Thus these species have not used their evolutionary potential with respect to learning ability, despite intuitively expected and experimentally demonstrated adaptive advantages of learning. This suggests that learning is costly, but this notion has rarely been tested. Here we report correlated responses of life-history traits to selection for improved learning in Drosophila melanogaster. Replicate populations selected for improved learning lived on average 15% shorter than the corresponding unselected control populations. They also showed a minor reduction in fecundity late in life and possibly a minor increase in dry adult mass. Selection for improved learning had no effect on egg-to-adult viability, development rate, or desiccation resistance. Because shortened longevity was the strongest correlated response to selection for improved learning, we also measured learning ability in another set of replicate populations that had been selected for extended longevity. In a classical olfactory conditioning assay, these long-lived flies showed an almost 40% reduction in learning ability early in life. This effect disappeared with age. Our results suggest a symmetrical evolutionary trade-off between learning ability and longevity in Drosophila.     

Are There Age-Related Differences in the Ability to Learn Configural Responses?

Age is often associated with a decline in cognitive abilities that are important for maintaining functional independence, such as learning new skills. Many forms of motor learning appear to be relatively well preserved with age, while learning tasks that involve associative binding tend to be negatively affected. The current study aimed to determine whether age differences exist on a configural response learning task, which includes aspects of motor learning and associative binding. Young (M = 24 years) and older adults (M = 66.5 years) completed a modified version of a configural learning task. Given the requirement of associative binding in the configural relationships between responses, we predicted older adults would show significantly less learning than young adults. Older adults demonstrated lower performance (slower reaction time and lower accuracy). However, contrary to our prediction, older adults showed similar rates of learning as indexed by a configural learning score compared to young adults. These results suggest that the ability to acquire knowledge incidentally about configural response relationships is largely unaffected by cognitive aging. The configural response learning task provides insight into the task demands that constrain learning abilities in older adults.     

Impaired associative learning in schizophrenia: behavioral and computational studies

Associative learning is a central building block of human cognition and in large part depends on mechanisms of synaptic plasticity, memory capacity and fronto–hippocampal interactions. A disorder like schizophrenia is thought to be characterized by altered plasticity, and impaired frontal and hippocampal function. Understanding the expression of this dysfunction through appropriate experimental studies, and understanding the processes that may give rise to impaired behavior through biologically plausible computational models will help clarify the nature of these deficits. We present a preliminary computational model designed to capture learning dynamics in healthy control and schizophrenia subjects. Experimental data was collected on a spatial-object paired-associate learning task. The task evinces classic patterns of negatively accelerated learning in both healthy control subjects and patients, with patients demonstrating lower rates of learning than controls. Our rudimentary computational model of the task was based on biologically plausible assumptions, including the separation of dorsal/spatial and ventral/object visual streams, implementation of rules of learning, the explicit parameterization of learning rates (a plausible surrogate for synaptic plasticity), and learning capacity (a plausible surrogate for memory capacity). Reductions in learning dynamics in schizophrenia were well-modeled by reductions in learning rate and learning capacity. The synergy between experimental research and a detailed computational model of performance provides a framework within which to infer plausible biological bases of impaired learning dynamics in schizophrenia.     

基于微信的“微生物遗传育种实验”混合式教学模式探究

随着互联网的飞速发展,传统课堂教学与互联网相结合的混合式教学模式越来越受到人们的关注。微信作为使用最广泛的即时通讯软件,其公众号功能非常适合作为移动学习的平台。本文介绍了将微信应用到“微生物遗传育种实验”的教学实践,探索线上和线下结合的混合式教学模式。以“绿色荧光蛋白(green fluorescent protein, GFP)的基因定点突变实验”为例,从教学设计、建立公众号及推送素材、课前预习、课堂学习、课后复习及反馈等5个方面详细介绍混合式教学过程。GFP基因定点突变实验在引物上引入一个GFP突变位点(Y66H),以质粒pGFPuv为模板,经PCR扩增后,以DpnⅠ消化原始质粒,并转化大肠杆菌筛选蓝色荧光蛋白突变株。采用微信与课堂教学结合的模式,既方便学生与老师交流互动,又有利于学生利用碎片化时间学习,使得“教与学”更加顺畅。实践证明,这种混合式教学模式深受学生喜爱,增强了学生学习兴趣与学习自主性,显著提高了教学效果。     

Co-evolution of learning complexity and social foraging strategies

Variation in learning abilities within populations suggests that complex learning may not necessarily be more adaptive than simple learning. Yet, the high cost of complex learning cannot fully explain this variation without some understanding of why complex learning is too costly for some individuals but not for others. Here we propose that different social foraging strategies can favor different learning strategies (that learn the environment with high or low resolution), thereby maintaining variable learning abilities within populations. Using a genetic algorithm in an agent-based evolutionary simulation of a social foraging game (the producer-scrounger game) we demonstrate how an association evolves between a strategy based on independent search for food (playing a producer) and a complex (high resolution) learning rule, while a strategy that combines independent search and following others (playing a scrounger) evolves an association with a simple (low resolution) learning rule. The reason for these associations is that for complex learning to have an advantage, a large number of learning steps, normally not achieved by scroungers, are necessary. These results offer a general explanation for persistent variation in cognitive abilities that is based on co-evolution of learning rules and social foraging strategies.     

Learning, climate and the evolution of cultural capacity

Patterns of environmental variation influence the utility, and thus evolution, of different learning strategies. I use stochastic, individual-based evolutionary models to assess the relative advantages of 15 different learning strategies (genetic determination, individual learning, vertical social learning, horizontal/oblique social learning, and contingent combinations of these) when competing in variable environments described by 1/f noise. When environmental variation has little effect on fitness, then genetic determinism persists. When environmental variation is large and equal over all time-scales ("white noise") then individual learning is adaptive. Social learning is advantageous in "red noise" environments when variation over long time-scales is large. Climatic variability increases with time-scale, so that short-lived organisms should be able to rely largely on genetic determination. Thermal climates usually are insufficiently red for social learning to be advantageous for species whose fitness is very determined by temperature. In contrast, population trajectories of many species, especially large mammals and aquatic carnivores, are sufficiently red to promote social learning in their predators. The ocean environment is generally redder than that on land. Thus, while individual learning should be adaptive for many longer-lived organisms, social learning will often be found in those dependent on the populations of other species, especially if they are marine. This provides a potential explanation for the evolution of a prevalence of social learning, and culture, in humans and cetaceans.     

Learning with incomplete information and the mathematical structure behind it

We investigate the problem of learning with incomplete information as exemplified by learning with delayed reinforcement. We study a two phase learning scenario in which a phase of Hebbian associative learning based on momentary internal representations is supplemented by an ‘unlearning’ phase depending on a graded reinforcement signal. The reinforcement signal quantifies the success-rate globally for a number of learning steps in phase one, and ‘unlearning’ is indiscriminate with respect to associations learnt in that phase. Learning according to this model is studied via simulations and analytically within a student–teacher scenario for both single layer networks and, for a committee machine. Success and speed of learning depend on the ratio λ of the learning rates used for the associative Hebbian learning phase and for the unlearning-correction in response to the reinforcement signal, respectively. Asymptotically perfect generalization is possible only, if this ratio exceeds a critical value λ _c , in which case the generalization error exhibits a power law decay with the number of examples seen by the student, with an exponent that depends in a non-universal manner on the parameter λ. We find these features to be robust against a wide spectrum of modifications of microscopic modelling details. Two illustrative applications—one of a robot learning to navigate a field containing obstacles, and the problem of identifying a specific component in a collection of stimuli—are also provided.     

The neural basis of following advice

Learning by following explicit advice is fundamental for human cultural evolution, yet the neurobiology of adaptive social learning is largely unknown. Here, we used simulations to analyze the adaptive value of social learning mechanisms, computational modeling of behavioral data to describe cognitive mechanisms involved in social learning, and model-based functional magnetic resonance imaging (fMRI) to identify the neurobiological basis of following advice. One-time advice received before learning had a sustained influence on people's learning processes. This was best explained by social learning mechanisms implementing a more positive evaluation of the outcomes from recommended options. Computer simulations showed that this "outcome-bonus" accumulates more rewards than an alternative mechanism implementing higher initial reward expectation for recommended options. fMRI results revealed a neural outcome-bonus signal in the septal area and the left caudate. This neural signal coded rewards in the absence of advice, and crucially, it signaled greater positive rewards for positive and negative feedback after recommended rather than after non-recommended choices. Hence, our results indicate that following advice is intrinsically rewarding. A positive correlation between the model's outcome-bonus parameter and amygdala activity after positive feedback directly relates the computational model to brain activity. These results advance the understanding of social learning by providing a neurobiological account for adaptive learning from advice.     

How Does Our Motor System Determine Its Learning Rate?

Motor learning is driven by movement errors. The speed of learning can be quantified by the learning rate, which is the proportion of an error that is corrected for in the planning of the next movement. Previous studies have shown that the learning rate depends on the reliability of the error signal and on the uncertainty of the motor system’s own state. These dependences are in agreement with the predictions of the Kalman filter, which is a state estimator that can be used to determine the optimal learning rate for each movement such that the expected movement error is minimized. Here we test whether not only the average behaviour is optimal, as the previous studies showed, but if the learning rate is chosen optimally in every individual movement. Subjects made repeated movements to visual targets with their unseen hand. They received visual feedback about their endpoint error immediately after each movement. The reliability of these error-signals was varied across three conditions. The results are inconsistent with the predictions of the Kalman filter because correction for large errors in the beginning of a series of movements to a fixed target was not as fast as predicted and the learning rates for the extent and the direction of the movements did not differ in the way predicted by the Kalman filter. Instead, a simpler model that uses the same learning rate for all movements with the same error-signal reliability can explain the data. We conclude that our brain does not apply state estimation to determine the optimal planning correction for every individual movement, but it employs a simpler strategy of using a fixed learning rate for all movements with the same level of error-signal reliability.