A mixed strategy model for the emergence and intensification of social learning in a periodically changing natural environment

Based on a population genetic model of mixed strategies determined by alleles of small effect, we derive conditions for the evolution of social learning in an infinite-state environment that changes periodically over time. Each mixed strategy is defined by the probabilities that an organism will commit itself to individual learning, social learning, or innate behavior. We identify the convergent stable strategies (CSS) by a numerical adaptive dynamics method and then check the evolutionary stability (ESS) of these strategies. A strategy that is simultaneously a CSS and an ESS is called an attractive ESS (AESS). For certain parameter sets, a bifurcation diagram shows that the pure individual learning strategy is the unique AESS for short periods of environmental change, a mixed learning strategy is the unique AESS for intermediate periods, and a mixed learning strategy (with a relatively large social learning component) and the pure innate strategy are both AESS's for long periods. This result entails that, once social learning emerges during a transient era of intermediate environmental periodicity, a subsequent elongation of the period may result in the intensification of social learning, rather than a return to innate behavior.     

Do social learning and conformist bias coevolve? Henrich and Boyd revisited

We studied the coevolution of social learning and conformist bias in a modified version of the Henrich and Boyd [1998. The evolution of conformist transmission and the emergence of between-group differences. Evol. Hum. Behav. 19, 215–241] model that nevertheless preserves its essential features. The convergent stable strategies (CSS) are identified by a numerical adaptive dynamics method and then checked for evolutionary stability. A strategy that is simultaneously a CSS and an ESS is called an attractive evolutionarily stable strategy (AESS). Our main findings are as follows. First, the AESS reliance on social learning is monotone increasing in the fixed interval between environmental changes and monotone decreasing in the quality of environmental information. Second, the AESS strength of conformist bias is monotone non-increasing in the fixed interval between environmental changes and monotone non-decreasing in the quality of environmental information. The first observation is in agreement with Henrich and Boyd (1998), but the second is in direct contradiction. In addition, we conducted Monte Carlo simulations as in Henrich and Boyd (1998), which supported our findings. We believe that the reason for the discrepancy with regard to the strength of conformist bias is that Henrich and Boyd (1998) did not allow a sufficient number of iterations for true convergence to occur. In conclusion, the conditions favoring a heavy reliance on social learning are not the same as those favoring a strong conformist bias.     

Do social learning and conformist bias coevolve? Henrich and Boyd revisited

We studied the coevolution of social learning and conformist bias in a modified version of the Henrich and Boyd [1998. The evolution of conformist transmission and the emergence of between-group differences. Evol. Hum. Behav. 19, 215-241] model that nevertheless preserves its essential features. The convergent stable strategies (CSS) are identified by a numerical adaptive dynamics method and then checked for evolutionary stability. A strategy that is simultaneously a CSS and an ESS is called an attractive evolutionarily stable strategy (AESS). Our main findings are as follows. First, the AESS reliance on social learning is monotone increasing in the fixed interval between environmental changes and monotone decreasing in the quality of environmental information. Second, the AESS strength of conformist bias is monotone non-increasing in the fixed interval between environmental changes and monotone non-decreasing in the quality of environmental information. The first observation is in agreement with Henrich and Boyd (1998), but the second is in direct contradiction. In addition, we conducted Monte Carlo simulations as in Henrich and Boyd (1998), which supported our findings. We believe that the reason for the discrepancy with regard to the strength of conformist bias is that Henrich and Boyd (1998) did not allow a sufficient number of iterations for true convergence to occur. In conclusion, the conditions favoring a heavy reliance on social learning are not the same as those favoring a strong conformist bias.     

Revisiting matrix games: the concept of neighborhood invader strategies

We extend the concept of neighborhood invader strategy (NIS) to finite-dimensional matrix games and compare this concept to the evolutionarily stable strategy (ESS) concept. We show that these two concepts are not equivalent in general. Just as ESS's may not be unique, NIS's may also not be unique. However, if there is an ESS and a NIS then these strategies must be the same. We show that an ESNIS (an ESS and NIS) for any matrix game is unique and that a mixed ESS with full support is a NIS. Thus a mixed ESS with full support is not invadable by any pure or mixed strategy and it can invade any pure or mixed strategy. An ESS which is an ESNIS, therefore, has better chance of being established evolutionarily through dynamic selection.     

Critical Social Learning: A Solution to Rogers's Paradox of Nonadaptive Culture

Alan Rogers (1988) presented a game theory model of the evolution of social learning, yielding the paradoxical conclusion that social learning does not increase the fitness of a population. We expand on this model, allowing for imperfections in individual and social learning as well as incorporating a "critical social learning" strategy that tries to solve an adaptive problem first by social learning, and then by individual learning if socially acquired behavior proves unsatisfactory. This strategy always proves superior to pure social learning and typically has higher fitness than pure individual learning, providing a solution to Rogers's paradox of nonadaptive culture. Critical social learning is an evolutionarily stable strategy (ESS) unless cultural transmission is highly unfaithful, the environment is highly variable, or social learning is much more costly than individual learning. We compare the model to empirical data on social learning and on spatial variation in primate cultures and list three requirements for adaptive culture.     

Imitation or innovation? Unselective mixed strategies can provide a better solution

Current theory about the evolution of social learning in a changing environment predicts the emergence of mixed strategies that rely on some selective combination of social and asocial learning. However, the results of a recent tournament of social learning strategies [Rendell et al. Science 328(5975):208?C213, 2010] suggest that the success relies almost entirely on copying to learn behavior. Those authors conclude that mixed strategies are vulnerable to invasion by individuals using social learning strategies alone. Here we perform a competition using unselective strategies that differ only in the degree of social versus asocial learning. We show that, under the same conditions of the aforementioned tournament, a pure social learning strategy can be invaded by an unselectively mixed strategy and attain an equilibrium where the latter is majority. Although existing theory suggests that copying other individuals unselectively is not adaptive, we show that, at this equilibrium, the average individual fitness of the population is higher than for a population of pure asocial learners, overcoming Rogers?? paradox in finite populations.     

Evolution of learning in fluctuating environments: when selection favors both social and exploratory individual learning

Cumulative cultural change requires organisms that are capable of both exploratory individual learning and faithful social learning. In our model, an organism's phenotype is initially determined innately (by its genotypic value) or by social learning (copying a phenotype from the parental generation), and then may or may not be modified by individual learning (exploration around the initial phenotype). The environment alternates periodically between two states, each defined as a certain range of phenotypes that can survive. These states may overlap, in which case the same phenotype can survive in both states, or they may not. We find that a joint social and exploratory individual learning strategy-the strategy that supports cumulative culture-is likely to spread when the environmental states do not overlap. In particular, when the environmental states are contiguous and mutation is allowed among the genotypic values, this strategy will spread in either moderately or highly stable environments, depending on the exact nature of the individual learning applied. On the other hand, natural selection often favors a social learning strategy without exploration when the environmental states overlap. We find only partial support for the "consensus" view, which holds that individual learning, social learning, and innate determination of behavior will evolve at short, intermediate, and long environmental periodicities, respectively.     

Evolution of social learning: a mathematical analysis

Social learning is an important ability seen in a wide range of animals including humans. It has been argued that individual learning, social learning, and innate determination of behavior are favored by natural selection when environmental changes occur at short, intermediate, and long intervals, respectively. Only recently, however, has the hypothesis been examined by means of mathematical models. In this paper, we construct a simple model in which each organism uses one of three genetically determined strategies--it is an individual learner, a social learner or an "innate"--and the three types of organisms are in direct competition with each other. A reduced model, involving only the individual learners and innates, is effectively linear, and we show that by solving the eigenvalue problem of this reduced system we arrive at a good approximation to the global dynamics of the full model. We also study the effect of stochastic environmental changes and reversible mutations among the three strategies. Our results are consistent with the predictions of previous studies. In addition, we identify a critical level of environmental constancy below which only individual and social learners are present.     

Why do some nectar foragers perch and others hover while probing flowers?

Diurnal hawkmoths, Hemaris fuciformis, and bumblebees, Bombus pasquorum, were observed foraging for nectar in flowers of Viscaria vulgaris. The hawkmoths hovered in front of the flowers, while the bees perched on them. The hawkmoths had a faster probing rate than the bees, and consequently also had higher gross and net rates of energy gain. A model is presented that shows that hovering only yields a higher net rate of energy gain (NREG) than perching when nectar volumes are high due to low competition for the resource. The difference in NREG of perchers and hoverers decreases with an increase of competition, and eventually perching yields the highest NREG. This is an effect of the higher cost of hovering. The results suggest that hovering can only evolve as a pure evolutionarily stable strategy (ESS) if competition is reduced, for example by co-evolutionary specializations with plants. The possibility that it has evolved as a mixed ESS (i.e. individuals can both hover and perch depending on the resource level) is discussed. The evolution of optimal foraging strategies is discussed, and it is pointed out that the rate of gain of an animal is independent of the strategy used when all competing foragers use the same strategy, but competitively superior strategies will nevertheless evolve because they are ESSs. Competition between strategies with different energy costs are special, because resource availability determines which strategy is competitively superior. A high-cost strategy can only evolve as a pure ESS at high resource levels, or as a mixed ESS at intermediate levels.     

Group level effects of social versus individual learning

We study the effects of learning by imitating others within the framework of an iterated game in which the members of two complementary populations interact via random pairing at each round. This allows us to compare both the fitness of different strategies within a population and the performance of populations in which members have access to different types of strategies. Previous studies reveal some emergent dynamics at the population level, when players learn individually. We here investigate a different mechanism in which players can choose between two different learning strategies, individual or social. Imitating behavior can spread within a mixed population, with the frequency of imitators varying over generation time. When compared to a pure population with solely individual learners, a mixed population with both individual and social learners can do better, independently of the precise learning scheme employed. We can then search for the best imitating strategy. Imitating the neighbor with the highest payoff turns out to be consistently superior. This is in agreement with findings in experimental and model studies that have been carried out in different settings.     

The evolution of intermittent breeding

A central issue in life history theory is how organisms trade off current and future reproduction. A variety of organisms exhibit intermittent breeding, meaning sexually mature adults will skip breeding opportunities between reproduction attempts. It’s thought that intermittent breeding occurs when reproduction incurs an extra cost in terms of survival, energy, or recovery time. We have developed a matrix population model for intermittent breeding, and use adaptive dynamics to determine under what conditions individuals should breed at every opportunity, and under what conditions they should skip some breeding opportunities (and if so, how many). We also examine the effect of environmental stochasticity on breeding behavior. We find that the evolutionarily stable strategy (ESS) for breeding behavior depends on an individual’s expected growth and mortality, and that the conditions for skipped breeding depend on the type of reproductive cost incurred (survival, energy, recovery time). In constant environments there is always a pure ESS, however environmental stochasticity and deterministic population fluctuations can both select for a mixed ESS. Finally, we compare our model results to patterns of intermittent breeding in species from a range of taxonomic groups.     

The evolution of conformist transmission in social learning when the environment changes periodically

Conformity is often observed in human social learning. Social learners preferentially imitate the majority or most common behavior in many situations, though the strength of conformity varies with the situation. Why has such a psychological tendency evolved? I investigate this problem by extending a standard model of social learning evolution with infinite environmental states (Feldman, M.W., Aoki, K., Kumm, J., 1996. Individual versus social learning: evolutionary analysis in a fluctuating environment. Anthropol. Sci. 104, 209-231) to include conformity bias. I mainly focus on the relationship between the strength of conformity bias that evolves and environmental stability, which is one of the most important factors in the evolution of social learning. Using the evolutionarily stable strategy (ESS) approach, I show that conformity always evolves when environmental stability and the cost of adopting a wrong behavior are small, though environmental stability and the cost of individual learning both negatively affect the strength of conformity.     

Multi-species evolutionary dynamics

Dynamical attainability of an evolutionarily stable strategy (ESS) through the process of mutations and natural selection has mostly been addressed through the use of the continuously stable strategy (CSS) concept for species evolutionary games in which strategies are drawn from a continuum, and by the adaptive trait dynamics method. We address the issue of dynamical attainability of an ESS in coevolving species through the use of the concept of an ESNIS. It is shown that the definition of an ESNIS coalition for coevolving species is not in general equivalent to other definitions for CSS given in the literature. We show under some additional conditions that, in a dynamic system which involves the strategies of a dimorphic ESNIS coalition and at most two strategies that are not members of ESNIS coalition, the ESNIS coalition will emerge as the winner. In addition an ESNIS will be approached because of the invasion structure of strategies in its neighborhood. This proves that under the above conditions an ESNIS has a better chance of being attained than a strategy coalition which is a CSS. The theory developed is applied to a class of coevolutionary game models with Lotka–Volterra type interactions and we show that for such models, an ESS coalition will be dynamically attainable through mutations and natural selection if the ESS coalition is also an ESNIS coalition.Co-ordinating editor: Metz     

Evolutionary matrix games and optimization theory

An evolutionarily stable strategy (ESS) is only required to be capable of resisting invasion by rare mutant strategies. In contrast, an absolute invader strategy (AIS) is a rare mutant strategy that can invade any established strategy. We show that the predictions of the outcome of evolution made by optimization models are compatible with those made by the classical expected payoff comparisons in matrix games. We also show that if a matrix game has an AIS that AIS is unique and is also an ESS. But an ESS need not be an AIS. In pure-strategy submodels, an AIS need not be unique. An AIS of a matrix game has global asymptotic stability property in the game dynamics which involve only pure strategies including the AIS.     

Fixation of Strategies for an Evolutionary Game in Finite Populations

A stochastic evolutionary dynamics of two strategies given by 2x 2 matrix games is studied in finite populations. We focus on stochastic properties of fixation: how a strategy represented by a single individual wins over the entire population. The process is discussed in the framework of a random walk with site dependent hopping rates. The time of fixation is found to be identical for both strategies in any particular game. The asymptotic behavior of the fixation time and fixation probabilities in the large population size limit is also discussed. We show that fixation is fast when there is at least one pure evolutionary stable strategy (ESS) in the infinite population size limit, while fixation is slow when the ESS is the coexistence of the two strategies.     

An ESS maximum principle for matrix games

Previous work has demonstrated that for games defined by differential or difference equations with a continuum of strategies, there exists a G-function, related to individual fitness, that must take on a maximum with respect to a virtual variable v whenever v is one of the vectors in the coalition of vectors which make up the evolutionarily stable strategy (ESS). This result, called the ESS maximum principle, is quite useful in determining candidates for an ESS. This principle is reformulated here, so that it may be conveniently applied to matrix games. In particular, we define a matrix game to be one in which fitness is expressed in terms of strategy frequencies and a matrix of expected payoffs. It is shown that the G-function in the matrix game setting must again take on a maximum value at all the strategies which make up the ESS coalition vector. The reformulated maximum principle is applicable to both bilinear and nonlinear matrix games. One advantage in employing this principle to solve the traditional bilinear matrix game is that the same G-function is used to find both pure and mixed strategy solutions by simply specifying an appropriate strategy space. Furthermore we show how the theory may be used to solve matrix games which are not in the usual bilinear form. We examine in detail two nonlinear matrix games: the game between relatives and the sex ratio game. In both of these games an ESS solution is determined. These examples not only illustrate the usefulness of this approach to finding solutions to an expanded class of matrix games, but aids in understanding the nature of the ESS as well.     

Properties of a mixed ESS candidate in continuous strategy sets

An evolutionarily stable strategy (ESS) is a strategy that if almost all members of the population adopt, then this population cannot be invaded by any mutant strategy. An ESS is not necessarily a possible end point of the evolutionary process. Moreover, there are cases where the population evolves towards a strategy that is not an ESS. This paper studies the properties of a unique mixed ESS candidate in a continuous time animal conflict. A member of a group sized three finds itself at risk and needs the assistance of another group member to be saved. In this conflict, a player's strategy is to choose the probability distribution of the interval between the beginning of the game and the moment it assists the player which is at risk. We first assume that a player is only allowed to choose an exponential distribution, and show that in this case the ESS candidate is an attracting ESS; the population will always evolve towards this strategy, and once it is adopted by most members of the population it cannot be invaded by mutant strategies. Then, we extend the strategy sets and allow a player to choose any continuous distribution. We show that although this ESS candidate may no longer be an ESS, under fairly general conditions the population will tend towards it. This is done by characterizing types of strategies that if established in the population, can be invaded by this ESS candidate, and by presenting possible paths of transition from other types of common strategies to this ESS candidate.     

Variability in diapause duration in the chestnut weevil: mixed ESS,genetic polymorphism or bet‐hedging?

Within-generation variability in diapause duration can be viewed either as a mixed Evolutionary Stable Strategy (ESS), a genetic polymorphism of pure strategies, or as bet-hedging. Diapause variability expressed by a single genotype that maximizes mean geometric fitness at the cost of mean arithmetic fitness is a bet-hedging strategy. Bet-hedging differs from mixed ESS and stable genetic polymorphism of pure strategies because in these latter the expected pay-offs for all phenotypes are equal. In insects, individuals with a prolonged diapause (long cycle) lose at least one reproductive opportunity and suffer lower survival before reproduction than those with a short diapause (short cycle). If long-cycle individuals compensate this cost by better adult performance, the compensation leads to a trade-off which could result in mixed ESS or genetic polymorphism of pure strategies since the overall fitness of the two morphs may be similar. In this paper, we show that in the chestnut weevil Curculio elephas adult performance, measured as sex ratio, longevity, weight, and realized fecundity of females, are similar in individuals emerged after one and two years. Long-cycle morphs emerge slightly before short-cycle ones but this eventual advantage for fertility probably does not compensate higher larval mortality and missed reproductive opportunity in long-cycle phenotypes. Therefore, the cost associated with prolonged diapause cannot be completely compensated for by a better adult performance. From these results, and previous data, we conclude that variability in diapause duration cycle is better explained as bet-hedging than mixed ESS or genetic polymorphism of pure strategies.     

The effect of social learning in a small population facing environmental change: an agent-based simulation

Learning is defined as behavioral modification due to experience, social or asocial. Social learning might be less costly than asocial learning and allow the rapid accumulation of learned traits across generations. However, the benefits of social learning in a small population of individuals relying on local interactions and experiencing environmental change are not well understood yet. In this study, we used agent-based simulations to address this issue by comparing the performance of social learning to asocial learning and innate behavior, in both a static and a changing environment. Learning was modeled using neural networks, and innate behavior was modeled using genetically coded behaviors. The performance of 10 mobile simulated agents was measured under three environmental scenarios: static, abrupt change and gradual change. We found that social learning allows for a better performance (in terms of survival) than asocial learning in static and abrupt-change scenarios. In contrast, when changes are gradual, social learning delays achieving the correct alternative, while asocial learning facilitates innovation; interestingly, a mixed population (social and asocial learners) performs the best.     

Game theory and evolution: finite size and absolute fitness measures

This article is concerned with the characterization and existence of evolutionarily stable strategies (ESS) in Games against Nature, a class of models described by finite size populations and absolute fitness measures. We address these problems in terms of a new formalism which revolves around the concept evolutionary entropy, a measure of the diversity of options associated with a strategy pure - strategies have zero entropy, mixed strategies positive entropy. We invoke this formalism to show that ESS are characterized by extremal states of entropy. We illustrate this characterization of ESS by an analysis of the evolution of the sex ratio and the evolution of seed size.