Local Replicator Dynamics: A Simple Link Between Deterministic and Stochastic Models of Evolutionary Game Theory

Classical replicator dynamics assumes that individuals play their games and adopt new strategies on a global level: Each player interacts with a representative sample of the population and if a strategy yields a payoff above the average, then it is expected to spread. In this article, we connect evolutionary models for infinite and finite populations: While the population itself is infinite, interactions and reproduction occurs in random groups of size N. Surprisingly, the resulting dynamics simplifies to the traditional replicator system with a slightly modified payoff matrix. The qualitative results, however, mirror the findings for finite populations, in which strategies are selected according to a probabilistic Moran process. In particular, we derive a one-third law that holds for any population size. In this way, we show that the deterministic replicator equation in an infinite population can be used to study the Moran process in a finite population and vice versa. We apply the results to three examples to shed light on the evolution of cooperation in the iterated prisoner’s dilemma, on risk aversion in coordination games and on the maintenance of dominated strategies.     

Network fluctuations hinder cooperation in evolutionary games

In this paper we study the influence of random network fluctuations on the behavior of evolutionary games on Barabási-Albert networks. This network class has been shown to promote cooperation on social dilemmas such as the Prisoner's Dilemma and the Snowdrift games when the population network is fixed. Here we introduce exogenous random fluctuations of the network links through several noise models, and we investigate the evolutionary dynamics comparing them with the known static network case. The results we obtain show that even a moderate amount of random noise on the network links causes a significant loss of cooperation, to the point that cooperation vanishes altogether in the Prisoner's Dilemma when the noise rate is the same as the agents' strategy revision rate. The results appear to be robust since they are essentially the same whatever the type of the exogenous noise. Besides, it turns out that random network noise is more important than strategy noise in suppressing cooperation. Thus, even in the more favorable situation of accumulated payoff in which links have no cost, the mere presence of random external network fluctuations act as a powerful limitation to the attainment of high levels of cooperation.     

Cooperation driven by mutations in multi-person Prisoner's Dilemma

The n-person Prisoner's Dilemma is a widely used model for populations where individuals interact in groups. The evolutionary stability of populations has been analysed in the literature for the case where mutations in the population may be considered as isolated events. For this case, and assuming simple trigger strategies and many iterations per game, we analyse the rate of convergence to the evolutionarily stable populations. We find that for some values of the payoff parameters of the Prisoner's Dilemma this rate is so low that the assumption, that mutations in the population are infrequent on that time-scale, is unreasonable. Furthermore, the problem is compounded as the group size is increased. In order to address this issue, we derive a deterministic approximation of the evolutionary dynamics with explicit, stochastic mutation processes, valid when the population size is large. We then analyse how the evolutionary dynamics depends on the following factors: mutation rate, group size, the value of the payoff parameters, and the structure of the initial population. In order to carry out the simulations for groups of more than just a few individuals, we derive an efficient way of calculating the fitness values. We find that when the mutation rate per individual and generation is very low, the dynamics is characterized by populations which are evolutionarily stable. As the mutation rate is increased, other fixed points with a higher degree of cooperation become stable. For some values of the payoff parameters, the system is characterized by (apparently) stable limit cycles dominated by cooperative behaviour. The parameter regions corresponding to high degree of cooperation grow in size with the mutation rate, and in number with the group size. For some parameter values, we find more than one stable fixed point, corresponding to different structures of the initial population.     

Adaptive Dynamics of Altruistic Cooperation in a Metapopulation: Evolutionary Emergence of Cooperators and Defectors or Evolutionary Suicide?

We investigate the evolution of public goods cooperation in a metapopulation model with small local populations, where altruistic cooperation can evolve due to assortment and kin selection, and the evolutionary emergence of cooperators and defectors via evolutionary branching is possible. Although evolutionary branching of cooperation has recently been demonstrated in the continuous snowdrift game and in another model of public goods cooperation, the required conditions on the cost and benefit functions are rather restrictive, e.g., altruistic cooperation cannot evolve in a defector population. We also observe selection for too low cooperation, such that the whole metapopulation goes extinct and evolutionary suicide occurs. We observed intuitive effects of various parameters on the numerical value of the monomorphic singular strategy. Their effect on the final coexisting cooperator–defector pair is more complex: changes expected to increase cooperation decrease the strategy value of the cooperator. However, at the same time the population size of the cooperator increases enough such that the average strategy does increase. We also extend the theory of structured metapopulation models by presenting a method to calculate the fitness gradient in a general class of metapopulation models, and try to make a connection with the kin selection approach.     

Coveting thy neighbors fitness as a means to resolve social dilemmas

In spatial evolutionary games the fitness of each individual is traditionally determined by the payoffs it obtains upon playing the game with its neighbors. Since defection yields the highest individual benefits, the outlook for cooperators is gloomy. While network reciprocity promotes collaborative efforts, chances of averting the impending social decline are slim if the temptation to defect is strong. It is, therefore, of interest to identify viable mechanisms that provide additional support for the evolution of cooperation. Inspired by the fact that the environment may be just as important as inheritance for individual development, we introduce a simple switch that allows a player to either keep its original payoff or use the average payoff of all its neighbors. Depending on which payoff is higher, the influence of either option can be tuned by means of a single parameter. We show that, in general, taking into account the environment promotes cooperation. Yet coveting the fitness of one's neighbors too strongly is not optimal. In fact, cooperation thrives best only if the influence of payoffs obtained in the traditional way is equal to that of the average payoff of the neighborhood. We present results for the prisoner's dilemma and the snowdrift game, for different levels of uncertainty governing the strategy adoption process, and for different neighborhood sizes. Our approach outlines a viable route to increased levels of cooperative behavior in structured populations, but one that requires a thoughtful implementation.     

The ecology of cooperative breeding behaviour

Ecology is a fundamental driving force for the evolutionary transition from solitary living to breeding cooperatively in groups. However, the fact that both benign and harsh, as well as stable and fluctuating, environments can favour the evolution of cooperative breeding behaviour constitutes a paradox of environmental quality and sociality. Here, we propose a new model – the dual benefits framework – for resolving this paradox. Our framework distinguishes between two categories of grouping benefits – resource defence benefits that derive from group‐defended critical resources and collective action benefits that result from social cooperation among group members – and uses insider–outsider conflict theory to simultaneously consider the interests of current group members (insiders) and potential joiners (outsiders) in determining optimal group size. We argue that the different grouping benefits realised from resource defence and collective action profoundly affect insider–outsider conflict resolution, resulting in predictable differences in the per capita productivity, stable group size, kin structure and stability of the social group. We also suggest that different types of environmental variation (spatial vs. temporal) select for societies that form because of the different grouping benefits, thus helping to resolve the paradox of why cooperative breeding evolves in such different types of environments.     

Evidence of helping behavior in a free-ranging population of communally breeding warthogs

Cooperative breeding societies are defined by the presence of helpers. Defining helping behavior in cooperatively breeding mammals has been difficult because lactation limits the ability of individuals to provision non-genetic young. As a consequence, “helping” behavior has frequently included predator and conspecific defense and thermoregulation. However, these behaviors are often associated with the benefits of group living and their expression may not warrant a species’ classification as a cooperative breeder (e.g., many ungulates and pinnipeds). In this study, we examine cooperative breeding behavior in the common warthog, Phacochoerus africanus. Warthogs exhibit substantial variation in breeding strategies and females will raise their young alone or in association with other females. The size of warthog groups varies throughout the year and we investigate fission and fusion of individual breeding groups to elucidate the costs and benefits of adopting different reproductive strategies. We found that the cohesion of female groups was related to parturition suggesting that there are benefits to sociality that are related to the production and care of offspring. Additionally, we found that reproductively-aged group members will help other group members by both babysitting and adopting the group’s offspring indicating active selection for cooperation. We did not witness any incidences of yearling group members exhibiting these behaviors indicating differential trade-offs to cooperation possibly related to the helper’s age/experience.     

Synergy and discounting of cooperation in social dilemmas

The emergence and maintenance of cooperation by natural selection is an enduring conundrum in evolutionary biology, which has been studied using a variety of game theoretical models inspired by different biological situations. The most widely studied games are the Prisoner's Dilemma, the Snowdrift game and by-product mutualism for pairwise interactions, as well as Public Goods games in larger groups of interacting individuals. Here, we present a general framework for cooperation in social dilemmas in which all the traditional scenarios can be recovered as special cases. In social dilemmas, cooperators provide a benefit to the group at some cost, while defectors exploit the group by reaping the benefits without bearing the costs of cooperation. Using the concepts of discounting and synergy for describing how benefits accumulate when more than one cooperator is present in a group of interacting individuals, we recover the four basic scenarios of evolutionary dynamics given by (i) dominating defection, (ii) coexistence of defectors and cooperators, (iii) dominating cooperation and (iv) bi-stability, in which cooperators and defectors cannot invade each other. Generically, for groups of three or more interacting individuals further, more complex, dynamics can occur. Our framework provides the first unifying approach to model cooperation in different kinds of social dilemmas.     

Differential susceptibility to food stress in neonates of sexual and asexual mollies (<Emphasis Type="Italic">Poecilia</Emphasis>, Poeciliidae)

The maintenance of sex is still an evolutionary puzzle given its immediate costs. Stably coexisting complexes of asexually and sexually reproducing forms allow to study mechanisms that balance the costs and benefits of both asexual and sexual reproduction. Here, we tested whether coexisting asexual and sexual fish of the genus Poecilia differed in neonate mortality when exposed to environmental stress in the form of fluctuating temperatures and food deprivation. We find that asexual Amazon mollies, Poecilia formosa, are significantly more sensitive to food stress than their sexual relative Poecilia latipinna, but both are equally unaffected by variable temperatures. Differences in the susceptibility to environmental stress may contribute to diminishing the asexuals’ benefits of a higher intrinsic population growth rate and thus mediate stable coexistence of the two reproductive forms.     

The different limits of weak selection and the evolutionary dynamics of finite populations

Evolutionary theory often resorts to weak selection, where different individuals have very similar fitness. Here, we relate two ways to introduce weak selection. The first considers evolutionary games described by payoff matrices with similar entries. This approach has recently attracted a lot of interest in the context of evolutionary game dynamics in finite populations. The second way to introduce weak selection is based on small distances in phenotype space and is a standard approach in kin-selection theory. Whereas both frameworks are interchangeable for constant fitness, frequency-dependent selection shows significant differences between them. We point out the difference between both limits of weak selection and discuss the condition under which the differences vanish. It turns out that this condition is fulfilled by the popular parametrization of the prisoner's dilemma in benefits and costs. However, for general payoff matrices differences between the two frameworks prevail.     

The continuous prisoner's dilemma: I. linear reactive strategies.

We present a general model for the Prisoner's Dilemma in which variable degrees of cooperation are possible, and payoffs are scaled accordingly. We describe a continuous strategy space, and divide this space into strategy families. We derive the payoff function for these families analytically, and study the evolutionary outcome when a wide range of strategies play against each other. Our results show that the initial degree of cooperation offered by a strategy is a decisive factor for evolutionary robustness: the most successful strategies in our model offer full cooperation as an initial move, but thereafter cooperate fully only if their opponent does the same. These strategies gradually raise the stakes when playing a strategy which is initially reticent to cooperate, but differ from the strategies predicted by other continuous models in that they are not only generous, but are also consistently optimistic and uncompromising.     

Finite populations choose an optimal language

This paper studies the evolution of a proto-language in a finite population under the frequency-dependent Moran process. A proto-language can be seen as a collection of concept-to-sign mappings. An efficient proto-language is a bijective mapping from objects of communication to used signs and vice versa. Based on the comparison of fixation probabilities, a method for deriving conditions of evolutionary stability in a finite population [Nowak et al., 2004. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 246-650], it is shown that efficient proto-languages are the only strategies that are protected by selection, which means that no mutant strategy can have a fixation probability that is greater than the inverse population size. In passing, the paper provides interesting results about the comparison of fixation probabilities as well as Maynard Smith's notion of evolutionary stability for finite populations [Maynard Smith, 1988. Can a mixed strategy be stable in a finite population? J. Theor. Biol. 130, 247-251] that are generally true for games with a symmetric payoff function.     

Divergent evolution of dispersal in a heterogeneous landscape

The evolution of dispersal is investigated in a landscape of many patches with fluctuating carrying capacities and spatial heterogeneity in temporal fluctuations. Although asynchronous temporal fluctuations select for dispersal, spatial heterogeneity in the distribution of fluctuating environmental variables selects against it. We find evolutionary branching in dispersal rate leading to the evolutionarily stable coexistence of a high- and a low-dispersal phenotype. We study how the opposing forces of selection for and against dispersal change with the relative size and the environmental qualities of the source and sink habitats. Our results suggest that the evolution of dispersal dimorphism could be a first step towards speciation and local adaptation.     

A simple rule for the evolution of contingent cooperation in large groups

Humans cooperate in large groups of unrelated individuals, and many authors have argued that such cooperation is sustained by contingent reward and punishment. However, such sanctioning systems can also stabilize a wide range of behaviours, including mutually deleterious behaviours. Moreover, it is very likely that large-scale cooperation is derived in the human lineage. Thus, understanding the evolution of mutually beneficial cooperative behaviour requires knowledge of when strategies that support such behaviour can increase when rare. Here, we derive a simple formula that gives the relatedness necessary for contingent cooperation in n-person iterated games to increase when rare. This rule applies to a wide range of pay-off functions and assumes that the strategies supporting cooperation are based on the presence of a threshold fraction of cooperators. This rule suggests that modest levels of relatedness are sufficient for invasion by strategies that make cooperation contingent on previous cooperation by a small fraction of group members. In contrast, only high levels of relatedness allow the invasion by strategies that require near universal cooperation. In order to derive this formula, we introduce a novel methodology for studying evolution in group structured populations including local and global group-size regulation and fluctuations in group size.     

Evolutionary games and population dynamics: maintenance of cooperation in public goods games

The emergence and abundance of cooperation in nature poses a tenacious and challenging puzzle to evolutionary biology. Cooperative behaviour seems to contradict Darwinian evolution because altruistic individuals increase the fitness of other members of the population at a cost to themselves. Thus, in the absence of supporting mechanisms, cooperation should decrease and vanish, as predicted by classical models for cooperation in evolutionary game theory, such as the Prisoner's Dilemma and public goods games. Traditional approaches to studying the problem of cooperation assume constant population sizes and thus neglect the ecology of the interacting individuals. Here, we incorporate ecological dynamics into evolutionary games and reveal a new mechanism for maintaining cooperation. In public goods games, cooperation can gain a foothold if the population density depends on the average population payoff. Decreasing population densities, due to defection leading to small payoffs, results in smaller interaction group sizes in which cooperation can be favoured. This feedback between ecological dynamics and game dynamics can generate stable coexistence of cooperators and defectors in public goods games. However, this mechanism fails for pairwise Prisoner's Dilemma interactions and the population is driven to extinction. Our model represents natural extension of replicator dynamics to populations of varying densities.     

Transforming the dilemma

How does natural selection lead to cooperation between competing individuals? The Prisoner's Dilemma captures the essence of this problem. Two players can either cooperate or defect. The payoff for mutual cooperation, R, is greater than the payoff for mutual defection, P. But a defector versus a cooperator receives the highest payoff, T, where as the cooperator obtains the lowest payoff, S. Hence, the Prisoner's Dilemma is defined by the payoff ranking T > R > P > S . In a well‐mixed population, defectors always have a higher expected payoff than cooperators, and therefore natural selection favors defectors. The evolution of cooperation requires specific mechanisms. Here we discuss five mechanisms for the evolution of cooperation: direct reciprocity, indirect reciprocity, kin selection, group selection, and network reciprocity (or graph selection). Each mechanism leads to a transformation of the Prisoner's Dilemma payoff matrix. From the transformed matrices, we derive the fundamental conditions for the evolution of cooperation. The transformed matrices can be used in standard frameworks of evolutionary dynamics such as the replicator equation or stochastic processes of game dynamics in finite populations.     

What pair formation can do to the battle of the sexes: towards more realistic game dynamics

In the various dynamic models of Dawkin's Battle of the Sexes, payoff matrices serve as the basic ingredients for the specification of a game-dynamic model. Here I model the sex war mechanistically, by expressing the costs of raising the offspring and performing a prolonged courtship via a time delay for the corresponding individuals, instead of via payoff matrices. During such a time delay an individual is not able to have new matings. Only after the delay has occurred, an individual (and its offspring) appears on the mating market again. From these assumptions I derive a pair-formation submodel, and a system of delay-differential equations describing the dynamics of the game. By a time-scale argument, I obtain an approximation of this system by means of a much simpler system of ordinary differential equations. Analysis of this simplified system shows that the model can give rise to two non-trivial asymptotically stable equilibrium points: an interior equilibrium where both female strategies and both male strategies are present, and a boundary equilibrium where only one of the female strategies and both male strategies are present. This behaviour is qualitatively different from that of models of the battle of the sexes formulated in the traditional framework of game-dynamic equations. In other words, the addition of a most elementary further assumption about individual life history fundamentally changes the model predictions. These results show that in analysing evolutionary games one should pay careful attention to the specific mechanisms involved in the conflict. In general, I advocate deriving simple models for evolutionary games, starting from more complex, mechanistic building blocks. The wide-spread method of modelling games at a high phenomenological level, through payoff matrices, can be misleading.     

Adaptive dynamics of cooperation may prevent the coexistence of defectors and cooperators and even cause extinction

It has recently been demonstrated that ecological feedback mechanisms can facilitate the emergence and maintenance of cooperation in public goods interactions: the replicator dynamics of defectors and cooperators can result, for example, in the ecological coexistence of cooperators and defectors. Here we show that these results change dramatically if cooperation strategy is not fixed but instead is a continuously varying trait under natural selection. For low values of the factor with which the value of resources is multiplied before they are shared among all participants, evolution will always favour lower cooperation strategies until the population falls below an Allee threshold and goes extinct, thus evolutionary suicide occurs. For higher values of the factor, there exists a unique evolutionarily singular strategy, which is convergence stable. Because the fitness function is linear with respect to the strategy of the mutant, this singular strategy is neutral against mutant invasions. This neutrality disappears if a nonlinear functional response in receiving benefits is assumed. For strictly concave functional responses, singular strategies become uninvadable. Evolutionary branching, which could result in the evolutionary emergence of cooperators and defectors, can occur only with locally convex functional responses, but we illustrate that it can also result in coevolutionary extinction.     

The replicator equation on graphs

We study evolutionary games on graphs. Each player is represented by a vertex of the graph. The edges denote who meets whom. A player can use any one of n strategies. Players obtain a payoff from interaction with all their immediate neighbors. We consider three different update rules, called 'birth-death', 'death-birth' and 'imitation'. A fourth update rule, 'pairwise comparison', is shown to be equivalent to birth-death updating in our model. We use pair approximation to describe the evolutionary game dynamics on regular graphs of degree k. In the limit of weak selection, we can derive a differential equation which describes how the average frequency of each strategy on the graph changes over time. Remarkably, this equation is a replicator equation with a transformed payoff matrix. Therefore, moving a game from a well-mixed population (the complete graph) onto a regular graph simply results in a transformation of the payoff matrix. The new payoff matrix is the sum of the original payoff matrix plus another matrix, which describes the local competition of strategies. We discuss the application of our theory to four particular examples, the Prisoner's Dilemma, the Snow-Drift game, a coordination game and the Rock-Scissors-Paper game.     

Evolution of cooperation with shared costs and benefits

The quest to determine how cooperation evolves can be based on evolutionary game theory, in spite of the fact that evolutionarily stable strategies (ESS) for most non-zero-sum games are not cooperative. We analyse the evolution of cooperation for a family of evolutionary games involving shared costs and benefits with a continuum of strategies from non-cooperation to total cooperation. This cost-benefit game allows the cooperator to share in the benefit of a cooperative act, and the recipient to be burdened with a share of the cooperator's cost. The cost-benefit game encompasses the Prisoner's Dilemma, Snowdrift game and Partial Altruism. The models produce ESS solutions of total cooperation, partial cooperation, non-cooperation and coexistence between cooperation and non-cooperation. Cooperation emerges from an interplay between the nonlinearities in the cost and benefit functions. If benefits increase at a decelerating rate and costs increase at an accelerating rate with the degree of cooperation, then the ESS has an intermediate level of cooperation. The game also exhibits non-ESS points such as unstable minima, convergent-stable minima and unstable maxima. The emergence of cooperative behaviour in this game represents enlightened self-interest, whereas non-cooperative solutions illustrate the Tragedy of the Commons. Games having either a stable maximum or a stable minimum have the property that small changes in the incentive structure (model parameter values) or culture (starting frequencies of strategies) result in correspondingly small changes in the degree of cooperation. Conversely, with unstable maxima or unstable minima, small changes in the incentive structure or culture can result in a switch from non-cooperation to total cooperation (and vice versa). These solutions identify when human or animal societies have the potential for cooperation and whether cooperation is robust or fragile.