sOAR: a tool for modelling optimal animal life‐history strategies in cyclic environments

Periodic environments determine the life cycle of many animals across the globe and the timing of important life history events, such as reproduction and migration. These adaptive behavioural strategies are complex and can only be fully understood (and predicted) within the framework of natural selection in which species adopt evolutionary stable strategies. We present sOAR, a powerful and user‐friendly implementation of the well‐established framework of optimal annual routine modelling. It allows determining optimal animal life history strategies under cyclic environmental conditions using stochastic dynamic programming. It further includes the simulation of population dynamics under the optimal strategy. sOAR provides an important tool for theoretical studies on the behavioural and evolutionary ecology of animals. It is especially suited for studying bird migration. In particular, we integrated options to differentiate between costs of active and passive flight into the optimal annual routine modelling framework, as well as options to consider periodic wind conditions affecting flight energetics. We provide an illustrative example of sOAR where food supply in the wintering habitat of migratory birds significantly alters the optimal timing of migration. sOAR helps improving our understanding of how complex behaviours evolve and how behavioural decisions are constrained by internal and external factors experienced by the animal. Such knowledge is crucial for anticipating potential species’ response to global environmental change.     

The ideal free distribution as an evolutionarily stable strategy

We examine the evolutionary stability of strategies for dispersal in heterogeneous patchy environments or for switching between discrete states (e.g. defended and undefended) in the context of models for population dynamics or species interactions in either continuous or discrete time. There have been a number of theoretical studies that support the view that in spatially heterogeneous but temporally constant environments there will be selection against unconditional, i.e. random, dispersal, but there may be selection for certain types of dispersal that are conditional in the sense that dispersal rates depend on environmental factors. A particular type of dispersal strategy that has been shown to be evolutionarily stable in some settings is balanced dispersal, in which the equilibrium densities of organisms on each patch are the same whether there is dispersal or not. Balanced dispersal leads to a population distribution that is ideal free in the sense that at equilibrium all individuals have the same fitness and there is no net movement of individuals between patches or states. We find that under rather general assumptions about the underlying population dynamics or species interactions, only such ideal free strategies can be evolutionarily stable. Under somewhat more restrictive assumptions (but still in considerable generality), we show that ideal free strategies are indeed evolutionarily stable. Our main mathematical approach is invasibility analysis using methods from the theory of ordinary differential equations and nonnegative matrices. Our analysis unifies and extends previous results on the evolutionary stability of dispersal or state-switching strategies.     

The ideal free distribution as an evolutionarily stable strategy

We examine the evolutionary stability of strategies for dispersal in heterogeneous patchy environments or for switching between discrete states (e.g. defended and undefended) in the context of models for population dynamics or species interactions in either continuous or discrete time. There have been a number of theoretical studies that support the view that in spatially heterogeneous but temporally constant environments there will be selection against unconditional, i.e. random, dispersal, but there may be selection for certain types of dispersal that are conditional in the sense that dispersal rates depend on environmental factors. A particular type of dispersal strategy that has been shown to be evolutionarily stable in some settings is balanced dispersal, in which the equilibrium densities of organisms on each patch are the same whether there is dispersal or not. Balanced dispersal leads to a population distribution that is ideal free in the sense that at equilibrium all individuals have the same fitness and there is no net movement of individuals between patches or states. We find that under rather general assumptions about the underlying population dynamics or species interactions, only such ideal free strategies can be evolutionarily stable. Under somewhat more restrictive assumptions (but still in considerable generality), we show that ideal free strategies are indeed evolutionarily stable. Our main mathematical approach is invasibility analysis using methods from the theory of ordinary differential equations and nonnegative matrices. Our analysis unifies and extends previous results on the evolutionary stability of dispersal or state-switching strategies.     

Mutation Rate Evolution in Replicator Dynamics

The mutation rate of an organism is itself evolvable. In stable environments, if faithful replication is costless, theory predicts that mutation rates will evolve to zero. However, positive mutation rates can evolve in novel or fluctuating environments, as analytical and empirical studies have shown. Previous work on this question has focused on environments that fluctuate independently of the evolving population. Here we consider fluctuations that arise from frequency-dependent selection in the evolving population itself. We investigate how the dynamics of competing traits can induce selective pressure on the rates of mutation between these traits. To address this question, we introduce a theoretical framework combining replicator dynamics and adaptive dynamics. We suppose that changes in mutation rates are rare, compared to changes in the traits under direct selection, so that the expected evolutionary trajectories of mutation rates can be obtained from analysis of pairwise competition between strains of different rates. Depending on the nature of frequency-dependent trait dynamics, we demonstrate three possible outcomes of this competition. First, if trait frequencies are at a mutation–selection equilibrium, lower mutation rates can displace higher ones. Second, if trait dynamics converge to a heteroclinic cycle—arising, for example, from “rock-paper-scissors” interactions—mutator strains succeed against non-mutators. Third, in cases where selection alone maintains all traits at positive frequencies, zero and nonzero mutation rates can coexist indefinitely. Our second result suggests that relatively high mutation rates may be observed for traits subject to cyclical frequency-dependent dynamics.     

Evolutionary game dynamics with non-uniform interaction rates

The classical setting of evolutionary game theory, the replicator equation, assumes uniform interaction rates. The rate at which individuals meet and interact is independent of their strategies. Here we extend this framework by allowing the interaction rates to depend on the strategies. This extension leads to non-linear fitness functions. We show that a strict Nash equilibrium remains uninvadable for non-uniform interaction rates, but the conditions for evolutionary stability need to be modified. We analyze all games between two strategies. If the two strategies coexist or exclude each other, then the evolutionary dynamics do not change qualitatively, only the location of the equilibrium point changes. If, however, one strategy dominates the other in the classical setting, then the introduction of non-uniform interaction rates can lead to a pair of interior equilibria. For the Prisoner's Dilemma, non-uniform interaction rates allow the coexistence between cooperators and defectors. For the snowdrift game, non-uniform interaction rates change the equilibrium frequency of cooperators.     

Evolutionary games and two species population dynamics

Competition between species has long been modeled by population dynamics based on total numbers of each species. Recently, the evolution of strategy frequencies has been used successfully for competition models between individuals. In this paper, we illustrate that these two views of competition are compatible. It is shown that the rate of intra and interspecific competitions between individuals largely determines the population dynamics. Competition models over a single common resource and predator-prey models are developed from this individual competition approach. In particular, the equilibrium strategies in a co-evolving predator-prey system are shown to be more stable than the predicted strategy cycling of standard evolutionary game theory.     

Population dynamics with a stable efficient equilibrium

We propose a game-theoretic dynamics of a population of replicating individuals. It consists of two parts: the standard replicator one and a migration between two different habitats. We consider symmetric two-player games with two evolutionarily stable strategies: the efficient one in which the population is in a state with a maximal payoff and the risk-dominant one where players are averse to risk. We show that for a large range of parameters of our dynamics, even if the initial conditions in both habitats are in the basin of attraction of the risk-dominant equilibrium (with respect to the standard replication dynamics without migration), in the long run most individuals play the efficient strategy.     

Evolutionary Behaviour,Trade-Offs and Cyclic and Chaotic Population Dynamics

Many studies of the evolution of life-history traits assume that the underlying population dynamical attractor is stable point equilibrium. However, evolutionary outcomes can change significantly in different circumstances. We present an analysis based on adaptive dynamics of a discrete-time demographic model involving a trade-off whose shape is also an important determinant of evolutionary behaviour. We derive an explicit expression for the fitness in the cyclic region and consequently present an adaptive dynamic analysis which is algebraic. We do this fully in the region of 2-cycles and (using a symbolic package) almost fully for 4-cycles. Simulations illustrate and verify our results. With equilibrium population dynamics, trade-offs with accelerating costs produce a continuously stable strategy (CSS) whereas trade-offs with decelerating costs produce a non-ES repellor. The transition to 2-cycles produces a discontinuous change: the appearance of an intermediate region in which branching points occur. The size of this region decreases as we move through the region of 2-cycles. There is a further discontinuous fall in the size of the branching region during the transition to 4-cycles. We extend our results numerically and with simulations to higher-period cycles and chaos. Simulations show that chaotic population dynamics can evolve from equilibrium and vice-versa.     

Probabilistic participation in public goods games

Voluntary participation in public goods games (PGGs) has turned out to be a simple but effective mechanism for promoting cooperation under full anonymity. Voluntary participation allows individuals to adopt a risk-aversion strategy, termed loner. A loner refuses to participate in unpromising public enterprises and instead relies on a small but fixed pay-off. This system leads to a cyclic dominance of three pure strategies, cooperators, defectors and loners, but at the same time, there remain two considerable restrictions: the addition of loners cannot stabilize the dynamics and the time average pay-off for each strategy remains equal to the pay-off of loners. Here, we introduce probabilistic participation in PGGs from the standpoint of diversification of risk, namely simple mixed strategies with loners, and prove the existence of a dynamical regime in which the restrictions ono longer hold. Considering two kinds of mixed strategies associated with participants (cooperators or defectors) and non-participants (loners), we can recover all basic evolutionary dynamics of the two strategies: dominance; coexistence; bistability; and neutrality, as special cases depending on pairs of probabilities. Of special interest is that the expected pay-off of each mixed strategy exceeds the pay-off of loners at some interior equilibrium in the coexistence region.     

Continuously stable strategies as evolutionary branching points

Evolutionary branching points are a paradigmatic feature of adaptive dynamics, because they are potential starting points for adaptive diversification. The antithesis to evolutionary branching points are continuously stable strategies (CSS's), which are convergent stable and evolutionarily stable equilibrium points of the adaptive dynamics and hence are thought to represent endpoints of adaptive processes. However, this assessment is based on situations in which the invasion fitness function determining the adaptive dynamics have non-zero second derivatives at CSS. Here we show that the scope of evolutionary branching can increase if the invasion fitness function vanishes to higher than first order at CSS. Using classical models for frequency-dependent competition, we show that if the invasion fitness vanishes to higher orders, a CSS may be the starting point for evolutionary branching. Thus, when invasion fitness functions vanish to higher than first order at equilibrium points of the adaptive dynamics, evolutionary diversification can occur even after convergence to an evolutionarily stable strategy.     

Forgiver Triumphs in Alternating Prisoner's Dilemma

Cooperative behavior, where one individual incurs a cost to help another, is a wide spread phenomenon. Here we study direct reciprocity in the context of the alternating Prisoner''s Dilemma. We consider all strategies that can be implemented by one and two-state automata. We calculate the payoff matrix of all pairwise encounters in the presence of noise. We explore deterministic selection dynamics with and without mutation. Using different error rates and payoff values, we observe convergence to a small number of distinct equilibria. Two of them are uncooperative strict Nash equilibria representing always-defect (ALLD) and Grim. The third equilibrium is mixed and represents a cooperative alliance of several strategies, dominated by a strategy which we call Forgiver. Forgiver cooperates whenever the opponent has cooperated; it defects once when the opponent has defected, but subsequently Forgiver attempts to re-establish cooperation even if the opponent has defected again. Forgiver is not an evolutionarily stable strategy, but the alliance, which it rules, is asymptotically stable. For a wide range of parameter values the most commonly observed outcome is convergence to the mixed equilibrium, dominated by Forgiver. Our results show that although forgiving might incur a short-term loss it can lead to a long-term gain. Forgiveness facilitates stable cooperation in the presence of exploitation and noise.     

Reactive strategies in indirect reciprocity

Evolution of reactive strategy of indirect reciprocity is discussed, where individuals interact with others through the one-shot Prisoner's Dilemma game, changing their partners in every round. We investigate all of the reactive strategies that are stochastic, including deterministic ones as special cases. First we study adaptive dynamics of reactive strategies by assuming monomorphic population. Results are very similar to the corresponding evolutionary dynamics of direct reciprocity. The discriminating strategy, which prescribes cooperation only with those who cooperated in the previous round, cannot be an outcome of the evolution. Next we examine the case where the population includes a diversity of strategies. We find that only the mean 'discriminatoriness' in the population is the parameter that affects the evolutionary dynamics. The discriminating strategy works as a promoter of cooperation there. However, it is again not the end point of the evolution. This is because retaliatory defection, which was prescribed by the discriminating strategy, is regarded as another defection toward the society. These results caution that we have to reconsider the role of retaliatory defection much more carefully.     

Bistability of Evolutionary Stable Vaccination Strategies in the Reinfection SIRI Model

We use the reinfection SIRI epidemiological model to analyze the impact of education programs and vaccine scares on individuals decisions to vaccinate or not. The presence of the reinfection provokes the novelty of the existence of three Nash equilibria for the same level of the morbidity relative risk instead of a single Nash equilibrium as occurs in the SIR model studied by Bauch and Earn (PNAS 101:13391–13394, 2004). The existence of three Nash equilibria, with two of them being evolutionary stable, introduces two scenarios with relevant and opposite features for the same level of the morbidity relative risk: the low-vaccination scenario corresponding to the evolutionary stable vaccination strategy, where individuals will vaccinate with a low probability; and the high-vaccination scenario corresponding to the evolutionary stable vaccination strategy, where individuals will vaccinate with a high probability. We introduce the evolutionary vaccination dynamics for the SIRI model and we prove that it is bistable. The bistability of the evolutionary dynamics indicates that the damage provoked by false scares on the vaccination perceived morbidity risks can be much higher and much more persistent than in the SIR model. Furthermore, the vaccination education programs to be efficient they need to implement a mechanism to suddenly increase the vaccination coverage level.     

The evolutionary language game.

We explore how evolutionary game dynamics have to be modified to accomodate a mathematical framework for the evolution of language. In particular, we are interested in the evolution of vocabulary, that is associations between signals and objects. We assume that successful communication contributes to biological fitness: individuals who communicate well leave more offspring. Children inherit from their parents a strategy for language learning (a language acquisition device). We consider three mechanisms whereby language is passed from one generation to the next: (i) parental learning: children learn the language of their parents; (ii) role model learning: children learn the language of individuals with a high payoff; and (iii) random learning: children learn the language of randomly chosen individuals. We show that parental and role model learning outperform random learning. Then we introduce mistakes in language learning and study how this process changes language over time. Mistakes increase the overall efficacy of parental and role model learning: in a world with errors evolutionary adaptation is more efficient. Our model also provides a simple explanation why homonomy is common while synonymy is rare.     

Ecological stability,evolutionary stability and the ESS maximum principle

Summary Since the fitness of each individual organism in a biological community may be affected by the strategies of all other individuals in the community, the essential element of a game exists. This game is an evolutionary game where the individual organisms (players) inherit their strategies from continuous play of the game through time. Here, the strategies are assumed to be constants associated with certain adaptive parameters (such as sunlight conversion efficiency for plants or body length in animals) in a set of differential equations which describe the population dynamics of the community. By means of natural selection, these parameters will evolve to a set of strategy values that natural selection, by itself, can no longer modify, i.e. an evolutionarily stable strategy (ESS). For a given class of models, it is possible to predict the outcome of this evolutionary process by determining ESSs using an ESS maximum principle. However, heretofore, the proof of this principle has been based on a limited set of conditions. Herein, we generalize the proof by removing certain restrictions and use instead the concept of an ecological stable equilibrium (ESE). Individuals in a biological community will be at an ESE if fixing the strategies used by the individuals results in stable population densities subject to perturbations in those densities. We present both necessary and sufficient conditions for an ESE to exist and then use the ESE concept to provide a very simple proof of the ESS maximum principle (which is a necessary condition for an ESS). A simple example is used to illustrate the difference between a strategy that maximizes fitness and one that satisfies the ESS maximum principle. In general they are different. We also look for ESEs in Lotka—Volterra competition and use the maximum principle to determine when an ESE will be an ESS. Finally, we examine the applicability of these ideas to matrix games.     

Host-parasite dynamics and the evolution of host immunity and parasite fecundity strategies

We explore evolutionarily stable co-evolution of host-macroparasite interactions in a discrete-time two-species population dynamics model, in which the dynamics may be stable, cyclic or chaotic. The macroparasites are assumed to harm host individuals through decreased reproductive output. Hosts may develop costly immune responses to defend themselves against parasites. Parasites compete with conspecifics by adjusting their fecundities. Overall, the presence of both parasites and the immune response in hosts produces more stable dynamics and lower host population sizes than that observed in the absence of the parasites. In our evolutionary analyses, we show that maximum parasite fecundity is always an evolutionarily stable strategy (ESS), irrespective of the type of population interaction, and that maximum parasite fecundity generally induces a minimum parasite population size through over-exploitation of the host. Phenotypic polymorphisms with respect to immunity in the host species are common and expected in ESS host strategies: the benefits of immunication depend on the frequency of the immune hosts in the population. In particular, the steady-state proportions of immune hosts depend, in addition to all the parameters of the parasite dynamics only on the cost of immunity and on the virulence of parasites in susceptible hosts. The implicit ecological dynamics of the host-parasite interaction affect the proportion of immune host individuals in the population. Furthermore, when changes in certain population parameters cause the dynamics of the host-parasite interaction to move from stability to cyclicity and then to chaos, the proportion of immune hosts tends to decrease; however, we also detected counter-examples to this result. As a whole, incorporating immunological and genetic aspects, as well as life-history trade-offs, into host-macroparasite dynamics produces a rich extension to the patterns observed in the models of ecological interactions and epidemics, and deserves more attention than is currently the case.     

Nash equilibria for an evolutionary language game

We study an evolutionary language game that describes how signals become associated with meaning. In our context, a language, L, is described by two matrices: the P matrix contains the probabilities that for a speaker certain objects are associated with certain signals, while the Q matrix contains the probabilities that for a listener certain signals are associated with certain objects. We define the payoff in our evolutionary language game as the total amount of information exchanged between two individuals. We give a formal classification of all languages, L(P, Q), describing the conditions for Nash equilibria and evolutionarily stable strategies (ESS). We describe an algorithm for generating all languages that are Nash equilibria. Finally, we show that starting from any random language, there exists an evolutionary trajectory using selection and neutral drift that ends up with a strategy that is a strict Nash equilibrium (or very close to a strict Nash equilibrium). Received: 1 March 2000 / Published online: 3 August 2000     

Evolutionary dynamics of continuous strategy games on graphs and social networks under weak selection

Understanding the emergence of cooperation among selfish individuals has been a long-standing puzzle, which has been studied by a variety of game models. Most previous studies presumed that interactions between individuals are discrete, but it seems unrealistic in real systems. Recently, there are increasing interests in studying game models with a continuous strategy space. Existing research work on continuous strategy games mainly focuses on well-mixed populations. Especially, little theoretical work has been conducted on their evolutionary dynamics in a structured population. In the previous work (Zhong et al., BioSystems, 2012), we showed that under strong selection, continuous and discrete strategies have significantly different equilibrium and game dynamics in spatially structured populations. In this paper, we further study evolutionary dynamics of continuous strategy games under weak selection in structured populations. By using the fixation probability based stochastic dynamics, we derive exact conditions of natural selection favoring cooperation for the death–birth updating scheme. We also present a network gain decomposition of the game equilibrium, which might provide a new view of the network reciprocity in a quantitative way. Finally, we make a detailed comparison between games using discrete and continuous strategies. As compared to the former, we find that for the latter (i) the same selection conditions are derived for the general 2 × 2 game; especially, the rule b/c > k in a simplified Prisoner's Dilemma is valid as well; however, (ii) for a coordination game, interestingly, the risk-dominant strategy is disfavored. Numerical simulations have also been conducted to validate our results.     

Aggregation as an antipredator strategy in the rock-paper-scissors model

We study a nonhierarchical tritrophic system, whose predator-prey interactions are described by the rock-paper-scissors game rules. In our stochastic simulations, individuals may move strategically towards the direction with more conspecifics to form clumps instead of moving aimlessly on the lattice. Considering that the conditioning to move gregariously depends on the organism's physical and cognitive abilities, we introduce a maximum distance an individual can perceive the environment and a minimum conditioning level to perform the gregarious movement. We investigate the pattern formation and compute the average size of the single-species spatial domains emerging from the grouping behaviour. The results reveal that the defence tactic reduces the predation risk significantly, being more profitable if individuals perceive further distances, thus creating bigger groups. Our outcomes show that the species with more conditioned organisms dominate the cyclic spatial game, controlling most of the territory. On the other hand, the species with fewer individuals ready to perform aggregation strategy gives its predator the chance to fill the more significant fraction of the grid. The spatial interactions assumed in our numerical experiments constitute a data set that may help biologists and data scientists understand how local interactions influence ecosystem dynamics.     

On the Sympatric Evolution and Evolutionary Stability of Coexistence by Relative Nonlinearity of Competition

If two species exhibit different nonlinear responses to a single shared resource, and if each species modifies the resource dynamics such that this favors its competitor, they may stably coexist. This coexistence mechanism, known as relative nonlinearity of competition, is well understood theoretically, but less is known about its evolutionary properties and its prevalence in real communities. We address this challenge by using adaptive dynamics theory and individual-based simulations to compare community stabilization and evolutionary stability of species that coexist by relative nonlinearity. In our analysis, evolution operates on the species'' density-compensation strategies, and we consider a trade-off between population growth rates at high and low resource availability. We confirm previous findings that, irrespective of the particular model of density dependence, there are many combinations of overcompensating and undercompensating density-compensation strategies that allow stable coexistence by relative nonlinearity. However, our analysis also shows that most of these strategy combinations are not evolutionarily stable and will be outcompeted by an intermediate density-compensation strategy. Only very specific trade-offs lead to evolutionarily stable coexistence by relative nonlinearity. As we find no reason why these particular trade-offs should be common in nature, we conclude that the sympatric evolution and evolutionary stability of relative nonlinearity, while possible in principle, seems rather unlikely. We speculate that this may, at least in part, explain why empirical demonstrations of this coexistence mechanism are rare, noting, however, that the difficulty to detect relative nonlinearity in the field is an equally likely explanation for the current lack of empirical observations, and that our results are limited to communities with non-overlapping generations and constant resource supply. Our study highlights the need for combining ecological and evolutionary perspectives for gaining a better understanding of community assembly and biogeographic patterns.