Mistakes allow evolutionary stability in the repeated prisoner's dilemma game

The repeated prisoner's dilemma game has been widely used in analyses of the evolution of reciprocal altruism. Recently it was shown that no pure strategy could be evolutionarily stable in the repeated prisoner's dilemma. Here I show that if there is always some probability that individuals will make a mistake, then a pure strategy can be evolutionarily stable provided that it is "strong perfect equilibria" against itself. To be a strong perfect equilibrium against itself, a strategy must be the best response to itself after every possible sequence of behavior. I show that both unconditional defection and a modified version of tit-for-tat have this property.     

Properties of a mixed ESS candidate in continuous strategy sets

An evolutionarily stable strategy (ESS) is a strategy that if almost all members of the population adopt, then this population cannot be invaded by any mutant strategy. An ESS is not necessarily a possible end point of the evolutionary process. Moreover, there are cases where the population evolves towards a strategy that is not an ESS. This paper studies the properties of a unique mixed ESS candidate in a continuous time animal conflict. A member of a group sized three finds itself at risk and needs the assistance of another group member to be saved. In this conflict, a player's strategy is to choose the probability distribution of the interval between the beginning of the game and the moment it assists the player which is at risk. We first assume that a player is only allowed to choose an exponential distribution, and show that in this case the ESS candidate is an attracting ESS; the population will always evolve towards this strategy, and once it is adopted by most members of the population it cannot be invaded by mutant strategies. Then, we extend the strategy sets and allow a player to choose any continuous distribution. We show that although this ESS candidate may no longer be an ESS, under fairly general conditions the population will tend towards it. This is done by characterizing types of strategies that if established in the population, can be invaded by this ESS candidate, and by presenting possible paths of transition from other types of common strategies to this ESS candidate.     

Evolutionarily stable strategies for a finite population and a variable contest size

This paper presents a generalization of Maynard Smith's concept of an evolutionarily stable strategy (ESS) to cover the cases of a finite population and a variable contest size. Both equilibrium and stability conditions are analysed. The standard Maynard Smith ESS with an infinite population and a contest size of two (pairwise contests) is shown to be a special case of this generalized ESS. An important implication of the generalized ESS is that in finite populations the behaviour of an ESS player is "spiteful", in the sense that an ESS player acts not only to increase his payoff but also to decrease the payoffs of his competitors. The degree of this "spiteful" behaviour is shown to increase with a decrease in the population size, and so is most likely to be observed in small populations. The paper concludes with an extended example: a symmetric two-pure-strategies two-player game for a finite population. It is shown that a mixed strategy ESS is globally stable against invasion by any one type of mutant strategist. The condition for the start of simultaneous invasion by two types of mutant is also given.     

Evolutionary escape from the prisoner's dilemma

The classic prisoner's dilemma model of game theory is modified by introducing occasional variations on the options available to players. Mutation and selection of game options reliably change the game matrix, gradually, from a prisoner's dilemma game into a byproduct mutualism one, in which cooperation is stable, and "temptation to defect" is replaced by temptation to cooperate. This result suggests that when there are many different potential ways of interacting, exploring those possibilities may make escape from prisoner's dilemmas a common outcome in the world. A consequence is that persistent prisoner's dilemma structures may be less common than one might otherwise expect.     

Co-operation and defection: playing the field and the ESS

If food is patchily dispersed, food clumps being very rich, but rare and hard to find, each individual in a foraging flock then faces an evident dilemma: whether to co-operate and participate in the search, thus enhancing the rate by which rich patches are discovered, or to defect and let others do the searching, thus avoiding any possible expenditures and risks involved in the search (but enjoying the abundant resources once a rich patch is discovered). This conflict (and its possible solution) is treated as an example in the analysis of the synergistic n-player game presented in this paper. After deriving conditions for the existence of a mixed ESS in such games, the evolutionary stability of the mixed strategy against invasions by pure strategists, in particular against invasions by recognizable defectors, is analyzed. Whereas in any "degenerating" mixed-strategy model a recognizable defector can invade and spread, a "non-degenerating" model can sometimes yield a mixed ESS which is immune to such invasions.     

The evolution of payoff matrices: providing incentives to cooperate

Most of the work in evolutionary game theory starts with a model of a social situation that gives rise to a particular payoff matrix and analyses how behaviour evolves through natural selection. Here, we invert this approach and ask, given a model of how individuals behave, how the payoff matrix will evolve through natural selection. In particular, we ask whether a prisoner's dilemma game is stable against invasions by mutant genotypes that alter the payoffs. To answer this question, we develop a two-tiered framework with goal-oriented dynamics at the behavioural time scale and a diploid population genetic model at the evolutionary time scale. Our results are two-fold: first, we show that the prisoner's dilemma is subject to invasions by mutants that provide incentives for cooperation to their partners, and that the resulting game is a coordination game similar to the hawk-dove game. Second, we find that for a large class of mutants and symmetric games, a stable genetic polymorphism will exist in the locus determining the payoff matrix, resulting in a complex pattern of behavioural diversity in the population. Our results highlight the importance of considering the evolution of payoff matrices to understand the evolution of animal social systems.     

The continuous prisoner's dilemma and the evolution of cooperation through reciprocal altruism with variable investment

Understanding the evolutionary origin and persistence of cooperative behavior is a fundamental biological problem. The standard "prisoner's dilemma," which is the most widely adopted framework for studying the evolution of cooperation through reciprocal altruism between unrelated individuals, does not allow for varying degrees of cooperation. Here we study the continuous iterated prisoner's dilemma, in which cooperative investments can vary continuously in each round. This game has been previously considered for a class of reactive strategies in which current investments are based on the partner's previous investment. In the standard iterated prisoner's dilemma, such strategies are inferior to strategies that take into account both players' previous moves, as is exemplified by the evolutionary dominance of "Pavlov" over "tit for tat." Consequently, we extend the analysis of the continuous prisoner's dilemma to a class of strategies in which current investments depend on previous payoffs and, hence, on both players' previous investments. We show, both analytically and by simulation, that payoff-based strategies, which embody the intuitively appealing idea that individuals invest more in cooperative interactions when they profit from these interactions, provide a natural explanation for the gradual evolution of cooperation from an initially noncooperative state and for the maintenance of cooperation thereafter.     

THE EVOLUTION OF STRATEGY VARIATION: WILL AN ESS EVOLVE?

Evolutionarily stable strategy (ESS) models are widely viewed as predicting the strategy of an individual that when monomorphic or nearly so prevents a mutant with any other strategy from entering the population. In fact, the prediction of some of these models is ambiguous when the predicted strategy is "mixed", as in the case of a sex ratio, which may be regarded as a mixture of the subtraits "produce a daughter" and "produce a son." Some models predict only that such a mixture be manifested by the population as a whole, that is, as an "evolutionarily stable state"; consequently, strategy monomorphism or polymorphism is consistent with the prediction. The hawk-dove game and the sex-ratio game in a panmictic population are models that make such a "degenerate" prediction. We show here that the incorporation of population finiteness into degenerate models has effects for and against the evolution of a monomorphism (an ESS) that are of equal order in the population size, so that no one effect can be said to predominate. Therefore, we used Monte Carlo simulations to determine the probability that a finite population evolves to an ESS as opposed to a polymorphism. We show that the probability that an ESS will evolve is generally much less than has been reported and that this probability depends on the population size, the type of competition among individuals, and the number of and distribution of strategies in the initial population. We also demonstrate how the strength of natural selection on strategies can increase as population size decreases. This inverse dependency underscores the incorrectness of Fisher's and Wright's assumption that there is just one qualitative relationship between population size and the intensity of natural selection.     

Are there really no evolutionarily stable strategies in the iterated prisoner's dilemma?

The evolutionary form of the iterated prisoner's dilemma (IPD) is a repeated game where players strategically choose whether to cooperate with or exploit opponents and reproduce in proportion to game success. It has been widely used to study the evolution of cooperation among selfish agents. In the past 15 years, researchers proved over a series of papers that there is no evolutionarily stable strategy (ESS) in the IPD when players maintain long-term relationships. This makes it difficult to make predictions about what strategies can actually persist as prevalent in a population over time. Here, we show that this no ESS finding may be a mathematical technicality, relying on implausible players who are "too perfect" in that their probability of cooperating on any move is arbitrarily close to either 0 or 1. Specifically, in the no ESS proof, all strategies were allowed, meaning that after a strategy X experiences any history H, X cooperates with an unrestricted probability p (X, H) where 0< or =p (X, H)< or =1. Here, we restrict strategies to the set S in which X is a member of S [corrected] if after any H, X cooperates with a restricted probability p (X, H) where e< or =p (X, H)< or =1-e and 0相似文献   

A theory for the evolutionary game

We present an evolutionary game theory. This theory differs in several respects from current theories related to Maynard Smith's pioneering work on evolutionary stable strategies (ESS). Most current work deals with two person matrix games. For these games the strategy set is finite. We consider evolutionary games which are defined over a continuous strategy set and which permit any number of players. Matrix games are included as a bilinear continuous game. However, under our definition, such games will not posses an ESS on the interior of the strategy set. We extend previous work on continuous games by developing an ESS definition which permits the ESS to be composed of a coalition of several strategies. This definition requires that the coalition must not only be stable with respect to perturbations in strategy frequencies which comprise the coalition, but the coalition must also satisfy the requirement that no mutant strategies can invade. Ecological processes are included in the model by explicitly considering population size and density dependent selection.     

An escape from ‘the prisoner's dilemma‘

Conventional escapes from the paradox that noncooperation between two organisms may be rational, even when cooperation would yield a higher reward to each, are based on the mechanism of reciprocity; but an analytical model of foraging among oviposition sites reveals a more immediate rationale, namely, equivalence of selfishness and altruism. The resulting game is unconditionally the prisoner's dilemma if the players have perfect recognition; however, in the absence thereof and for three different parameter regimes, it yields either the prisoner's dilemma, or two evolutionarily stable strategies, or a unique cooperative ESS. Thus unrecognition can favor cooperation; and environments can exist in which cooperation persists, or even invades, without reciprocity. The results suggest that different mechanisms for cooperation may operate at different levels of neural complexity.     

Nash equilibrium and evolutionary stability in large- and finite-population "playing the field" models

This paper studies the correspondence between Nash equilibrium and evolutionary stability in large- and finite-population "playing the field" models. Whenever the fitness function is sufficiently continuous, any large-population ESS corresponds to a symmetric Nash equilibrium in the game that describes the simultaneous interaction of the individuals in the population, and any strict, symmetric Nash equilibrium in that game corresponds to a large-population ESS. This correspondence continues to hold, approximately, in finite populations; and it holds exactly for strict pure-strategy equilibria in sufficiently large finite populations. By contrast, a sequence of (mixed-strategy) finite-population ESSs can converge, as the population grows, to a limit that is not a large-population ESS, and a large-population ESS need not be the limit of any sequence of finite-population ESSs.     

The structure of mutations and the evolution of cooperation

Evolutionary game dynamics in finite populations assumes that all mutations are equally likely, i.e., if there are n strategies a single mutation can result in any strategy with probability 1/n. However, in biological systems it seems natural that not all mutations can arise from a given state. Certain mutations may be far away, or even be unreachable given the current composition of an evolving population. These distances between strategies (or genotypes) define a topology of mutations that so far has been neglected in evolutionary game theory. In this paper we re-evaluate classic results in the evolution of cooperation departing from the assumption of uniform mutations. We examine two cases: the evolution of reciprocal strategies in a repeated prisoner's dilemma, and the evolution of altruistic punishment in a public goods game. In both cases, alternative but reasonable mutation kernels shift known results in the direction of less cooperation. We therefore show that assuming uniform mutations has a substantial impact on the fate of an evolving population. Our results call for a reassessment of the "model-less" approach to mutations in evolutionary dynamics.     

Evolutionary matrix games and optimization theory

An evolutionarily stable strategy (ESS) is only required to be capable of resisting invasion by rare mutant strategies. In contrast, an absolute invader strategy (AIS) is a rare mutant strategy that can invade any established strategy. We show that the predictions of the outcome of evolution made by optimization models are compatible with those made by the classical expected payoff comparisons in matrix games. We also show that if a matrix game has an AIS that AIS is unique and is also an ESS. But an ESS need not be an AIS. In pure-strategy submodels, an AIS need not be unique. An AIS of a matrix game has global asymptotic stability property in the game dynamics which involve only pure strategies including the AIS.     

Evolutionary cheating in Escherichia coli stationary phase cultures

Starved cultures of Escherichia coli are highly dynamic, undergoing frequent population shifts. The shifts result from the spread of mutants able to grow under conditions that impose growth arrest on the ancestral population. To analyze competitive interactions underlying this dynamic we measured the survival of a typical mutant and the wild type during such population shifts. Here we show that the survival advantage of the mutant at any given time during a takeover is inversely dependent on its frequency in the population, its growth adversely affects the survival of the wild type, and its ability to survive in stationary phase at fixation is lower than that of its ancestor. These mutants do not enter, or exit early, the nondividing stationary-phase state, cooperatively maintained by the wild type. Thus they end up overrepresented as compared to their initial frequency at the onset of the stationary phase, and subsequently they increase disproportionately their contribution in terms of progeny to the succeeding generation in the next growth cycle, which is a case of evolutionary cheating. If analyzed through the game theory framework, these results might be explained by the prisoner's dilemma type of conflict, which predicts that selfish defection is favored over cooperation.     

What initially brought about communications?

This paper reports an intelligent agent equipped with two-layer finite state machines (FSMs) that can communicate by turning lighting on and off, leading to social cooperation that solves the dilemma situation, modeled by a one-shot 2x2 game. This communication between two gaming agents can be observed in hero- and leader-type dilemma games, where alternating reciprocity, repeating cooperation (C)-defeat (D) after D-C, is the equal pareto optimum instead of a sequence of mutual cooperation that is the equal pareto optimum for a prisoner's dilemma (PD) game.     

On the iterated prisoner's dilemma in a finite population

Two standard assumptions in analytical work on the iterated prisoner's dilemma are that the population is infinite, and that opponents—though randomly selected—are fixed for the duration of the game. This paper explores the consequences of relaxing both assumptions. It is shown in particular that if opponents are drawn at random throughout the game, then stable cooperation via reciprocity requires both that the probability of a further interaction be sufficiently high—higher than when opponents are fixed—and that the population not exceed a certain critical size, which depends on the probability of further interaction.     

Win-stay-lose-learn promotes cooperation in the spatial prisoner's dilemma game

Holding on to one's strategy is natural and common if the later warrants success and satisfaction. This goes against widespread simulation practices of evolutionary games, where players frequently consider changing their strategy even though their payoffs may be marginally different than those of the other players. Inspired by this observation, we introduce an aspiration-based win-stay-lose-learn strategy updating rule into the spatial prisoner's dilemma game. The rule is simple and intuitive, foreseeing strategy changes only by dissatisfied players, who then attempt to adopt the strategy of one of their nearest neighbors, while the strategies of satisfied players are not subject to change. We find that the proposed win-stay-lose-learn rule promotes the evolution of cooperation, and it does so very robustly and independently of the initial conditions. In fact, we show that even a minute initial fraction of cooperators may be sufficient to eventually secure a highly cooperative final state. In addition to extensive simulation results that support our conclusions, we also present results obtained by means of the pair approximation of the studied game. Our findings continue the success story of related win-stay strategy updating rules, and by doing so reveal new ways of resolving the prisoner's dilemma.     

Game dynamics and evolutionary transitions

An evolutionary model based on the Taylor-Jonker game dynamics is presented. A set of strategies is compatible if there exists a dynamical equilibrium between its members and there is an evolutionary transition to another compatible set if new mutant strategies bring about a passage to another equilibrium. We apply these concepts to supergame strategies, which play repeatedly a given matrix game and at each time step choose their pure strategy according to the preceding moves of the opponent. We investigate the patterns of evolution in zero-sum games, games of partnership, the prisoner's dilemma and the hawkdove game.     

Replicator dynamics for optional public good games

The public goods game represents a straightforward generalization of the prisoner's dilemma to an arbitrary number of players. Since the dominant strategy is to defect, both classical and evolutionary game theory predict the asocial outcome that no player contributes to the public goods. In contrast to the compulsory public goods game, optional participation provides a natural way to avoid deadlocks in the state of mutual defection. The three resulting strategies--collaboration or defection in the public goods game, as well as not joining at all--are studied by means of a replicator dynamics, which can be completely analysed in spite of the fact that the payoff terms are nonlinear. If cooperation is valuable enough, the dynamics exhibits a rock-scissors-paper type of cycling between the three strategies, leading to sizeable average levels of cooperation in the population. Thus, voluntary participation makes cooperation feasible. But for each strategy, the average payoff value remains equal to the earnings of those not participating in the public goods game.