The evolution of n-player cooperation-threshold games and ESS bifurcations

An evolutionary game of individuals cooperating to obtain a collective benefit is here modelled as an n-player Prisoner's Dilemma game. With reference to biological situations, such as group foraging, we introduce a threshold condition in the number of cooperators required to obtain the collective benefit. In the simplest version, a three-player game, complex behaviour appears as the replicator dynamics exhibits a catastrophic event separating a parameter region allowing for coexistence of cooperators and defectors and a region of pure defection. Cooperation emerges through an ESS bifurcation, and cooperators only thrive beyond a critical point in cost-benefit space. Moreover, a repelling fixed point of the dynamics acts as a barrier to the introduction of cooperation in defecting populations. The results illustrate the qualitative difference between two-player games and multiple player games and thus the limitations to the generality of conclusions from two-player games. We present a procedure to find the evolutionarily stable strategies in any n-player game with cost and benefit depending on the number of cooperators. This was previously done by Motro [1991. Co-operation and defection: playing the field and the ESS. J. Theor. Biol. 151, 145-154] in the special cases of convex and concave benefit functions and constant cost.     

Evolutionary Stability in the Asymmetric Volunteer's Dilemma

It is often assumed that in public goods games, contributors are either strong or weak players and each individual has an equal probability of exhibiting cooperation. It is difficult to explain why the public good is produced by strong individuals in some cooperation systems, and by weak individuals in others. Viewing the asymmetric volunteer''s dilemma game as an evolutionary game, we find that whether the strong or the weak players produce the public good depends on the initial condition (i.e., phenotype or initial strategy of individuals). These different evolutionarily stable strategies (ESS) associated with different initial conditions, can be interpreted as the production modes of public goods of different cooperation systems. A further analysis revealed that the strong player adopts a pure strategy but mixed strategies for the weak players to produce the public good, and that the probability of volunteering by weak players decreases with increasing group size or decreasing cost-benefit ratio. Our model shows that the defection probability of a “strong” player is greater than the “weak” players in the model of Diekmann (1993). This contradicts Selten''s (1980) model that public goods can only be produced by a strong player, is not an evolutionarily stable strategy, and will therefore disappear over evolutionary time. Our public good model with ESS has thus extended previous interpretations that the public good can only be produced by strong players in an asymmetric game.     

A theory for the evolutionary game

We present an evolutionary game theory. This theory differs in several respects from current theories related to Maynard Smith's pioneering work on evolutionary stable strategies (ESS). Most current work deals with two person matrix games. For these games the strategy set is finite. We consider evolutionary games which are defined over a continuous strategy set and which permit any number of players. Matrix games are included as a bilinear continuous game. However, under our definition, such games will not posses an ESS on the interior of the strategy set. We extend previous work on continuous games by developing an ESS definition which permits the ESS to be composed of a coalition of several strategies. This definition requires that the coalition must not only be stable with respect to perturbations in strategy frequencies which comprise the coalition, but the coalition must also satisfy the requirement that no mutant strategies can invade. Ecological processes are included in the model by explicitly considering population size and density dependent selection.     

Differential games in evolutionary theory: Height growth strategies of trees

Differential game theory is applied to the analysis of evolutionarily stable strategies (ESS) in this article. A general form for the evolutionary differential game is introduced in the case of intra-specific competition, and a connection between the ESS and the mathematical Nash solution concept is indicated. A dynamic ESS is found for the height growth strategies of trees. A hierarchical model is introduced to account for different time constants in simultaneous selection processes. Differential evolutionary games are compared with static evolutionary games utilizing the hierarchical approach.     

Unpredictability induced by unfocused games in evolutionary game dynamics

Evolutionary game theory is a basis of replicator systems and has applications ranging from animal behavior and human language to ecosystems and other hierarchical network systems. Most studies in evolutionary game dynamics have focused on a single game, but, in many situations, we see that many games are played simultaneously. We construct a replicator equation with plural games by assuming that a reward of a player is a simple summation of the reward of each game. Even if the numbers of the strategies of the games are different, its dynamics can be described in one replicator equation. We here show that when players play several games at the same time, the fate of a single game cannot be determined without knowing the structures of the whole other games. The most absorbing fact is that even if a single game has a ESS (evolutionary stable strategy), the relative frequencies of strategies in the game does not always converge to the ESS point when other games are played simultaneously.     

Simulating natural selection as a culling mechanism on finite populations with the hawk-dove game

The behaviors of individuals and species are often explained in terms of evolutionary stable strategies (ESSs). The analysis of ESSs determines which, if any, combinations of behaviors cannot be invaded by alternative strategies. Two assumptions required to generate an ESS (i.e., an infinite population and payoffs described only on the average) do not hold under natural conditions. Previous experiments indicated that under more realistic conditions of finite populations and stochastic payoffs, populations may evolve in trajectories that are unrelated to an ESS, even in very simple games. The simulations offered here extend earlier research by employing truncation selection with random parental selection in a hawk-dove game. Payoffs are determined in pairwise contests using either the expected outcome, or the result of a random variable. In each case, however, the mean fraction of hawks over many generations and across many independent trials does not conform to the expected ESS. Implications of these results and philosophical underpinnings of ESS theory are offered.     

Fixation of Strategies for an Evolutionary Game in Finite Populations

A stochastic evolutionary dynamics of two strategies given by 2x 2 matrix games is studied in finite populations. We focus on stochastic properties of fixation: how a strategy represented by a single individual wins over the entire population. The process is discussed in the framework of a random walk with site dependent hopping rates. The time of fixation is found to be identical for both strategies in any particular game. The asymptotic behavior of the fixation time and fixation probabilities in the large population size limit is also discussed. We show that fixation is fast when there is at least one pure evolutionary stable strategy (ESS) in the infinite population size limit, while fixation is slow when the ESS is the coexistence of the two strategies.     

An ESS maximum principle for matrix games

Previous work has demonstrated that for games defined by differential or difference equations with a continuum of strategies, there exists a G-function, related to individual fitness, that must take on a maximum with respect to a virtual variable v whenever v is one of the vectors in the coalition of vectors which make up the evolutionarily stable strategy (ESS). This result, called the ESS maximum principle, is quite useful in determining candidates for an ESS. This principle is reformulated here, so that it may be conveniently applied to matrix games. In particular, we define a matrix game to be one in which fitness is expressed in terms of strategy frequencies and a matrix of expected payoffs. It is shown that the G-function in the matrix game setting must again take on a maximum value at all the strategies which make up the ESS coalition vector. The reformulated maximum principle is applicable to both bilinear and nonlinear matrix games. One advantage in employing this principle to solve the traditional bilinear matrix game is that the same G-function is used to find both pure and mixed strategy solutions by simply specifying an appropriate strategy space. Furthermore we show how the theory may be used to solve matrix games which are not in the usual bilinear form. We examine in detail two nonlinear matrix games: the game between relatives and the sex ratio game. In both of these games an ESS solution is determined. These examples not only illustrate the usefulness of this approach to finding solutions to an expanded class of matrix games, but aids in understanding the nature of the ESS as well.     

Evolution of cooperation under -person snowdrift games

In the animal world, performing a given task which is beneficial to an entire group requires the cooperation of several individuals of that group who often share the workload required to perform the task. The mathematical framework to study the dynamics of collective action is game theory. Here we study the evolutionary dynamics of cooperators and defectors in a population in which groups of individuals engage in N-person, non-excludable public goods games. We explore an N-person generalization of the well-known two-person snowdrift game. We discuss both the case of infinite and finite populations, taking explicitly into consideration the possible existence of a threshold above which collective action is materialized. Whereas in infinite populations, an N-person snowdrift game (NSG) leads to a stable coexistence between cooperators and defectors, the introduction of a threshold leads to the appearance of a new interior fixed point associated with a coordination threshold. The fingerprints of the stable and unstable interior fixed points still affect the evolutionary dynamics in finite populations, despite evolution leading the population inexorably to a monomorphic end-state. However, when the group size and population size become comparable, we find that spite sets in, rendering cooperation unfeasible.     

感潮河网区环境合作博弈模型及实证

区域决策者往往要选择合适的排污策略,使自己的经济和环境收益最大。当多个决策者进行策略选择时,就形成了区域环境博弈格局。针对感潮河网区排污行为与环境质量具有互相影响的特点,考虑了税收收益、治理成本和环境损失等因素,建立了河网区环境非合作博弈模型和合作博弈模型。非合作博弈的Nash均衡表明,在非合作局面下,区域决策者仅仅考虑最大化自己的收益,选择的排污策略与其他参与者无关。在合作博弈分析中,对大联盟、子联盟博弈进行了分析求解。通过计算所有可能联盟博弈的特征函数,使用常用的Shapley值法进行合作收益公平分配。研究表明,在大联盟合作局面下,区域决策者的策略选择考虑了全局收益的最大化,选择的排污策略与所有参与者的环境损失参数、环境影响参数有关。结合感潮河网区案例对模型进行了验证,研究了3个区域非合作博弈状态和合作博弈状态下的排放量和收益,并使用Shapley值法对合作收益进行分配。对比非合作博弈与合作博弈,合作后3个区域排污量分别比合作前减少了17.98%、15.36%、5.55%。合作收益分别增加了2.17%、3.21%、1.25%。环境质量分别提高了14.24%、13.33%、10.52%。这说明合作局面有利于降低污染排放,分配后的环境合作收益大于非合作收益,河网区环境合作是多赢的。     

Consequences of fluctuating group size for the evolution of cooperation

Studies of cooperation have traditionally focused on discrete games such as the well-known prisoner’s dilemma, in which players choose between two pure strategies: cooperation and defection. Increasingly, however, cooperation is being studied in continuous games that feature a continuum of strategies determining the level of cooperative investment. For the continuous snowdrift game, it has been shown that a gradually evolving monomorphic population may undergo evolutionary branching, resulting in the emergence of a defector strategy that coexists with a cooperator strategy. This phenomenon has been dubbed the ‘tragedy of the commune’. Here we study the effects of fluctuating group size on the tragedy of the commune and derive analytical conditions for evolutionary branching. Our results show that the effects of fluctuating group size on evolutionary dynamics critically depend on the structure of payoff functions. For games with additively separable benefits and costs, fluctuations in group size make evolutionary branching less likely, and sufficiently large fluctuations in group size can always turn an evolutionary branching point into a locally evolutionarily stable strategy. For games with multiplicatively separable benefits and costs, fluctuations in group size can either prevent or induce the tragedy of the commune. For games with general interactions between benefits and costs, we derive a general classification scheme based on second derivatives of the payoff function, to elucidate when fluctuations in group size help or hinder cooperation.     

Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game

Understanding the mechanisms that can lead to the evolution of cooperation through natural selection is a core problem in biology. Among the various attempts at constructing a theory of cooperation, game theory has played a central role. Here, we review models of cooperation that are based on two simple games: the Prisoner's Dilemma, and the Snowdrift game. Both games are two‐person games with two strategies, to cooperate and to defect, and both games are social dilemmas. In social dilemmas, cooperation is prone to exploitation by defectors, and the average payoff in populations at evolutionary equilibrium is lower than it would be in populations consisting of only cooperators. The difference between the games is that cooperation is not maintained in the Prisoner's Dilemma, but persists in the Snowdrift game at an intermediate frequency. As a consequence, insights gained from studying extensions of the two games differ substantially. We review the most salient results obtained from extensions such as iteration, spatial structure, continuously variable cooperative investments, and multi‐person interactions. Bridging the gap between theoretical and empirical research is one of the main challenges for future studies of cooperation, and we conclude by pointing out a number of promising natural systems in which the theory can be tested experimentally.     

The importance of mechanisms for the evolution of cooperation

Studies aimed at explaining the evolution of phenotypic traits have often solely focused on fitness considerations, ignoring underlying mechanisms. In recent years, there has been an increasing call for integrating mechanistic perspectives in evolutionary considerations, but it is not clear whether and how mechanisms affect the course and outcome of evolution. To study this, we compare four mechanistic implementations of two well-studied models for the evolution of cooperation, the Iterated Prisoner''s Dilemma (IPD) game and the Iterated Snowdrift (ISD) game. Behavioural strategies are either implemented by a 1 : 1 genotype–phenotype mapping or by a simple neural network. Moreover, we consider two different scenarios for the effect of mutations. The same set of strategies is feasible in all four implementations, but the probability that a given strategy arises owing to mutation is largely dependent on the behavioural and genetic architecture. Our individual-based simulations show that this has major implications for the evolutionary outcome. In the ISD, different evolutionarily stable strategies are predominant in the four implementations, while in the IPD each implementation creates a characteristic dynamical pattern. As a consequence, the evolved average level of cooperation is also strongly dependent on the underlying mechanism. We argue that our findings are of general relevance for the evolution of social behaviour, pleading for the integration of a mechanistic perspective in models of social evolution.     

On Nash Equilibrium and Evolutionarily Stable States That Are Not Characterised by the Folk Theorem

In evolutionary game theory, evolutionarily stable states are characterised by the folk theorem because exact solutions to the replicator equation are difficult to obtain. It is generally assumed that the folk theorem, which is the fundamental theory for non-cooperative games, defines all Nash equilibria in infinitely repeated games. Here, we prove that Nash equilibria that are not characterised by the folk theorem do exist. By adopting specific reactive strategies, a group of players can be better off by coordinating their actions in repeated games. We call it a type-k equilibrium when a group of k players coordinate their actions and they have no incentive to deviate from their strategies simultaneously. The existence and stability of the type-k equilibrium in general games is discussed. This study shows that the sets of Nash equilibria and evolutionarily stable states have greater cardinality than classic game theory has predicted in many repeated games.     

Spatial effects in social dilemmas

Social dilemmas and the evolutionary conundrum of cooperation are traditionally studied through various kinds of game theoretical models such as the prisoner's dilemma, public goods games, snowdrift games or by-product mutualism. All of them exemplify situations which are characterized by different degrees of conflicting interests between the individuals and the community. In groups of interacting individuals, cooperators produce a common good benefitting the entire group at some cost to themselves, whereas defectors attempt to exploit the resource by avoiding the costly contributions. Based on synergistic or discounted accumulation of cooperative benefits a unifying theoretical framework was recently introduced that encompasses all games that have traditionally been studied separately (Hauert, Michor, Nowak, Doebeli, 2005. Synergy and discounting of cooperation in social dilemmas. J. Theor. Biol., in press.). Within this framework we investigate the effects of spatial structure with limited local interactions on the evolutionary fate of cooperators and defectors. The quantitative effects of space turn out to be quite sensitive to the underlying microscopic update mechanisms but, more general, we demonstrate that in prisoner's dilemma type interactions spatial structure benefits cooperation-although the parameter range is quite limited-whereas in snowdrift type interactions spatial structure may be beneficial too, but often turns out to be detrimental to cooperation.     

On evolutionarily stable sets

As an extension of the concept of an evolutionarily stable strategy (ESS) evolutionarily stable sets are introduced, i.e. sets of equilibrium strategies (EQS) which have much of the properties of an ESS. They are primarily used with evolutionary game models that allow a continuum of EQSs, none of which can be an ESS, but also include common ESSs as a special case. For a large class even of nonlinear models it can be shown that the standard dynamics converge towards some equilibrium point in an ES set if started within a neighbourhood of the set. Important applications of ES sets include e.g. mixed-strategist models and evolutionary game models in sexual populations.     

Emergence and resilience of cooperation in the spatial prisoner's dilemma via a reward mechanism

We study the problem of the emergence of cooperation in the spatial Prisoner's Dilemma. The pioneering work by Nowak and May [1992. Evolutionary games and spatial chaos. Nature 415, 424-426] showed that large initial populations of cooperators can survive and sustain cooperation in a square lattice with imitate-the-best evolutionary dynamics. We revisit this problem in a cost-benefit formulation suitable for a number of biological applications. We show that if a fixed-amount reward is established for cooperators to share, a single cooperator can invade a population of defectors and form structures that are resilient to re-invasion even if the reward mechanism is turned off. We discuss analytically the case of the invasion by a single cooperator and present agent-based simulations for small initial fractions of cooperators. Large cooperation levels, in the sustainability range, are found. In the conclusions we discuss possible applications of this model as well as its connections with other mechanisms proposed to promote the emergence of cooperation.     

The evolution of payoff matrices: providing incentives to cooperate

Most of the work in evolutionary game theory starts with a model of a social situation that gives rise to a particular payoff matrix and analyses how behaviour evolves through natural selection. Here, we invert this approach and ask, given a model of how individuals behave, how the payoff matrix will evolve through natural selection. In particular, we ask whether a prisoner's dilemma game is stable against invasions by mutant genotypes that alter the payoffs. To answer this question, we develop a two-tiered framework with goal-oriented dynamics at the behavioural time scale and a diploid population genetic model at the evolutionary time scale. Our results are two-fold: first, we show that the prisoner's dilemma is subject to invasions by mutants that provide incentives for cooperation to their partners, and that the resulting game is a coordination game similar to the hawk-dove game. Second, we find that for a large class of mutants and symmetric games, a stable genetic polymorphism will exist in the locus determining the payoff matrix, resulting in a complex pattern of behavioural diversity in the population. Our results highlight the importance of considering the evolution of payoff matrices to understand the evolution of animal social systems.     

Can transitive inference evolve in animals playing the hawk-dove game?

What should an individual do if there are no reliable cues to the strength of a competitor when fighting with it for resources? We herein examine the evolutionarily stable strategy (ESS) in the hawk-dove game, if the opponent's resource-holding potential (RHP) can only indirectly be inferred from the outcome of past interactions in the population. The strategies we examined include the classical mixed strategy in which no information on past games is utilized, the 'imprinting' strategy in which a player increases/decreases its aggressiveness if it wins/loses a game, the 'immediate inference' strategy in which a player can infer the strength of those opponents it fought before, and the 'transitive inference' strategy in which a player can infer the strength of a new opponent through a third party with which both players have fought before. Invasibility analysis for each pair of strategies revealed that (i) the transitive-inference strategy can always invade the mixed strategy and the imprinting strategy, and itself refuses invasion by these strategies; (ii) the largest advantage for transitive inference is achieved when the number of games played per individual in one generation is small and when the cost of losing an escalated game is large; (iii) the immediate inference, rather than the transitive inference, can be an ESS if the cost of fighting is small; (iv) a strong linear ranking is established in the population of transitive-inference strategists, though it does not perfectly correlate to the ranking by actual RHPs. We found that the advantage of the transitive inference is not in its ability to correct a misassessment (it is actually the worst in doing so), but in the ability of quickly lining up either incorrect or correct assessments to form a linear dominance hierarchy.     

Stochastic evolutionary dynamics of direct reciprocity

Evolutionary game theory is the study of frequency-dependent selection. The success of an individual depends on the frequencies of strategies that are used in the population. We propose a new model for studying evolutionary dynamics in games with a continuous strategy space. The population size is finite. All members of the population use the same strategy. A mutant strategy is chosen from some distribution over the strategy space. The fixation probability of the mutant strategy in the resident population is calculated. The new mutant takes over the population with this probability. In this case, the mutant becomes the new resident. Otherwise, the existing resident remains. Then, another mutant is generated. These dynamics lead to a stationary distribution over the entire strategy space. Our new approach generalizes classical adaptive dynamics in three ways: (i) the population size is finite; (ii) mutants can be drawn non-locally and (iii) the dynamics are stochastic. We explore reactive strategies in the repeated Prisoner''s Dilemma. We perform ‘knock-out experiments’ to study how various strategies affect the evolution of cooperation. We find that ‘tit-for-tat’ is a weak catalyst for the emergence of cooperation, while ‘always cooperate’ is a strong catalyst for the emergence of defection. Our analysis leads to a new understanding of the optimal level of forgiveness that is needed for the evolution of cooperation under direct reciprocity.