The good, the bad and the discriminator--errors in direct and indirect reciprocity

This paper presents, in a series of simple diagrams, concise results about the replicator dynamics of direct and indirect reciprocity. We consider repeated interactions between donors and recipients, and analyse the relationship between three basic strategies for the donor: unconditional cooperation, all-out defection, and conditional cooperation. In other words, we investigate the competition of discriminating and indiscriminating altruists with defectors. Discriminators and defectors form a bistable community, and hence a population of discriminators cannot be invaded by defectors. But unconditional altruists can invade a discriminating population and 'soften it up' for a subsequent invasion by defectors. The resulting dynamics exhibits various forms of rock-paper-scissors cycles and depends in subtle ways on noise, in the form of errors in implementation. The probability for another round (in the case of direct reciprocity), and information about the co-player (in the case of indirect reciprocity), add further elements to the ecology of reciprocation.     

Fixation probabilities in evolutionary game dynamics with a two-strategy game in finite diploid populations

Fixation processes in evolutionary game dynamics in finite diploid populations are investigated. Traditionally, frequency dependent evolutionary dynamics is modeled as deterministic replicator dynamics. This implies that the infinite size of the population is assumed implicitly. In nature, however, population sizes are finite. Recently, stochastic processes in finite populations have been introduced in order to study finite size effects in evolutionary game dynamics. One of the most significant studies on evolutionary dynamics in finite populations was carried out by Nowak et al. which describes “one-third law” [Nowak, et al., 2004. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 646-650]. It states that under weak selection, if the fitness of strategy α is greater than that of strategy β when α has a frequency , strategy α fixates in a β-population with selective advantage. In their study, it is assumed that the inheritance of strategies is asexual, i.e. the population is haploid. In this study, we apply their framework to a diploid population that plays a two-strategy game with two ESSs (a bistable game). The fixation probability of a mutant allele in this diploid population is derived. A “three-tenth law” for a completely recessive mutant allele and a “two-fifth law” for a completely dominant mutant allele are found; other cases are also discussed.     

Evolutionary game dynamics in finite populations with strong selection and weak mutation

We study stochastic game dynamics in finite populations. To this end we extend the classical Moran process to incorporate frequency-dependent selection and mutation. For 2 x 2 games, we give a complete analysis of the long-run behavior when mutation rates are small. For 3 x 3 coordination games, we provide a simple rule to determine which strategy will be selected in large populations. The expected motion in our model resembles the standard replicator dynamics when the population is large, but is qualitatively different when the population is small. Our analysis shows that even in large finite populations the behavior of a replicator-like system can be different from that of the standard replicator dynamics. As an application, we consider selective language dynamics. We determine which language will be spoken in finite large populations. The results have an intuitive interpretation but would not be expected from an analysis of the replicator dynamics.     

Evolutionary graph theory: breaking the symmetry between interaction and replacement

We study evolutionary dynamics in a population whose structure is given by two graphs: the interaction graph determines who plays with whom in an evolutionary game; the replacement graph specifies the geometry of evolutionary competition and updating. First, we calculate the fixation probabilities of frequency dependent selection between two strategies or phenotypes. We consider three different update mechanisms: birth-death, death-birth and imitation. Then, as a particular example, we explore the evolution of cooperation. Suppose the interaction graph is a regular graph of degree h, the replacement graph is a regular graph of degree g and the overlap between the two graphs is a regular graph of degree l. We show that cooperation is favored by natural selection if b/c>hg/l. Here, b and c denote the benefit and cost of the altruistic act. This result holds for death-birth updating, weak-selection and large population size. Note that the optimum population structure for cooperators is given by maximum overlap between the interaction and the replacement graph (g=h=l), which means that the two graphs are identical. We also prove that a modified replicator equation can describe how the expected values of the frequencies of an arbitrary number of strategies change on replacement and interaction graphs: the two graphs induce a transformation of the payoff matrix.     

Evolutionary game dynamics in a finite asymmetric two-deme population and emergence of cooperation

We consider evolutionary game dynamics in a finite population subdivided into two demes with both unequal deme sizes and different migration rates. Assuming viability differences in the population according to a linear game within each deme as a result of pairwise interactions, we specify conditions for weak selection favoring a mutant strategy to go to fixation, under the structured-coalescent assumptions, and their connections with evolutionary stability concepts. In the framework of the Iterated Prisoner's Dilemma with strategy ‘tit-for-tat’ as mutant strategy and ‘always defect’ as resident strategy, we deduce a condition under which the emergence of cooperation is favored by selection, when the game matrix is the same in both demes. We show how this condition extends the one-third law for a panmictic population and when an asymmetry in the spatial structure of a two-deme population facilitates the emergence of the cooperative tit-for-tat strategy in comparison with both its symmetric and panmictic population structure counterparts. We find that the condition is less stringent in the asymmetric scenario versus the symmetric scenario if both the fraction of the population in the deme where the mutant was initially introduced, and the expected proportion of migrant offspring in this deme among all migrant offspring after population regulation, are smaller than, or equal to, , provided they are not too small. On the other hand, the condition is less stringent than the one-third law, which holds in the panmictic case, if the latter proportion remains not too close to 1.     

Tit-for-tat or win-stay, lose-shift?

The repeated Prisoner's Dilemma is usually known as a story of tit-for-tat (TFT). This remarkable strategy has won both of Robert Axelrod's tournaments. TFT does whatever the opponent has done in the previous round. It will cooperate if the opponent has cooperated, and it will defect if the opponent has defected. But TFT has two weaknesses: (i) it cannot correct mistakes (erroneous moves) and (ii) a population of TFT players is undermined by random drift when mutant strategies appear which play always-cooperate (ALLC). Another equally simple strategy called 'win-stay, lose-shift' (WSLS) has neither of these two disadvantages. WSLS repeats the previous move if the resulting payoff has met its aspiration level and changes otherwise. Here, we use a novel approach of stochastic evolutionary game dynamics in finite populations to study mutation-selection dynamics in the presence of erroneous moves. We compare four strategies: always-defect (ALLD), ALLC, TFT and WSLS. There are two possible outcomes: if the benefit of cooperation is below a critical value then ALLD is selected; if the benefit of cooperation is above this critical value then WSLS is selected. TFT is never selected in this evolutionary process, but lowers the selection threshold for WSLS.     

Mutation in evolutionary games can increase average fitness at equilibrium

We study game dynamical interactions between two strategies, A and B, and analyse whether the average fitness of the population at equilibrium can be increased by adding mutation from A to B. Classifying all two by two games with payoff matrix [(a,b),(c,d)], we show that mutation from A to B enhances the average fitness of the whole population (i) if both a and d are less than (b + c)/2 and (ii) if c is less than b. Furthermore, we study conditions for maximizing the productivity of strategy A, and we analyse the effect of mutations in both directions. Depending on the biological system, a mutation in an evolutionary game can be interpreted as a genetic alteration, a cellular differentiation, a change in gene expression, an accidental or deliberate modification in cultural transmission, or a learning error. In a cultural context, our results indicate that the equilibrium payoff of the population can be increased if players sometimes choose the strategy with lower payoff. In a genetic context, we have shown that for frequency-dependent selection mutation can enhance the average fitness of the population at equilibrium.     

Evolutionary matrix games and optimization theory

An evolutionarily stable strategy (ESS) is only required to be capable of resisting invasion by rare mutant strategies. In contrast, an absolute invader strategy (AIS) is a rare mutant strategy that can invade any established strategy. We show that the predictions of the outcome of evolution made by optimization models are compatible with those made by the classical expected payoff comparisons in matrix games. We also show that if a matrix game has an AIS that AIS is unique and is also an ESS. But an ESS need not be an AIS. In pure-strategy submodels, an AIS need not be unique. An AIS of a matrix game has global asymptotic stability property in the game dynamics which involve only pure strategies including the AIS.     

Population dynamics with a stable efficient equilibrium

We propose a game-theoretic dynamics of a population of replicating individuals. It consists of two parts: the standard replicator one and a migration between two different habitats. We consider symmetric two-player games with two evolutionarily stable strategies: the efficient one in which the population is in a state with a maximal payoff and the risk-dominant one where players are averse to risk. We show that for a large range of parameters of our dynamics, even if the initial conditions in both habitats are in the basin of attraction of the risk-dominant equilibrium (with respect to the standard replication dynamics without migration), in the long run most individuals play the efficient strategy.     

Cross entropy minimization in uninvadable states of complex populations

Selection is often. viewed as a process that maximizes the average fitness of a population. However, there are often constraints even on the phenotypic level which may prevent fitness optimization. Consequently, in evolutionary game theory, models of frequency dependent selection are investigated, which focus on equilibrium states that are characterized by stability (or uninvadability) rather than by optimality. The aim of this article is to show that nevertheless there is a biologically meaningful quantity, namely cross (fitness) entropy, which is optimized during the course of evolution: a dynamical model adapted to evolutionary games is presented which has the property that relative entropy decreases monotonically, if the state of a (complex) population is close to an uninvadable state. This result may be interpreted as if evolution has an order stabilizing effect.     

Evolutionary stability and quasi-stationary strategy in stochastic evolutionary game dynamics

Stochastic evolutionary game dynamics for finite populations has recently been widely explored in the study of evolutionary game theory. It is known from the work of Traulsen et al. [2005. Phys. Rev. Lett. 95, 238701] that the stochastic evolutionary dynamics approaches the deterministic replicator dynamics in the limit of large population size. However, sometimes the limiting behavior predicted by the stochastic evolutionary dynamics is not quite in agreement with the steady-state behavior of the replicator dynamics. This paradox inspired us to give reasonable explanations of the traditional concept of evolutionarily stable strategy (ESS) in the context of finite populations. A quasi-stationary analysis of the stochastic evolutionary game dynamics is put forward in this study and we present a new concept of quasi-stationary strategy (QSS) for large but finite populations. It is shown that the consistency between the QSS and the ESS implies that the long-term behavior of the replicator dynamics can be predicted by the quasi-stationary behavior of the stochastic dynamics. We relate the paradox to the time scales and find that the contradiction occurs only when the fixation time scale is much longer than the quasi-stationary time scale. Our work may shed light on understanding the relationship between the deterministic and stochastic methods of modeling evolutionary game dynamics.     

The donation game with roles played between relatives

We consider a social game with two choices, played between two relatives, where roles are assigned to individuals so that the interaction is asymmetric. Behaviour in each of the two roles is determined by a separate genetic locus. Such asymmetric interactions between relatives, in which individuals occupy different behavioural contexts, may occur in nature, for example between adult parents and juvenile offspring. The social game considered is known to be equivalent to a donation game with non-additive payoffs, and has previously been analysed for the single locus case, both for discrete and continuous strategy traits. We present an inclusive fitness analysis of the discrete trait game with roles and recover equilibrium conditions including fixation of selfish or altruistic behaviour under both behavioural contexts, or fixation of selfish behaviour under one context and altruistic behaviour under the other context. These equilibrium solutions assume that the payoff matrices under each behavioural context are identical. The equilibria possible do depend crucially, however, on the deviation from payoff additivity that occurs when both interacting individuals act altruistically.     

Evolutionary game dynamics in a Wright-Fisher process

Evolutionary game dynamics in finite populations can be described by a frequency dependent, stochastic Wright-Fisher process. We consider a symmetric game between two strategies, A and B. There are discrete generations. In each generation, individuals produce offspring proportional to their payoff. The next generation is sampled randomly from this pool of offspring. The total population size is constant. The resulting Markov process has two absorbing states corresponding to homogeneous populations of all A or all B. We quantify frequency dependent selection by comparing the absorption probabilities to the corresponding probabilities under random drift. We derive conditions for selection to favor one strategy or the other by using the concept of total positivity. In the limit of weak selection, we obtain the 1/3 law: if A and B are strict Nash equilibria then selection favors replacement of B by A, if the unstable equilibrium occurs at a frequency of A which is less than 1/3.     

The different limits of weak selection and the evolutionary dynamics of finite populations

Evolutionary theory often resorts to weak selection, where different individuals have very similar fitness. Here, we relate two ways to introduce weak selection. The first considers evolutionary games described by payoff matrices with similar entries. This approach has recently attracted a lot of interest in the context of evolutionary game dynamics in finite populations. The second way to introduce weak selection is based on small distances in phenotype space and is a standard approach in kin-selection theory. Whereas both frameworks are interchangeable for constant fitness, frequency-dependent selection shows significant differences between them. We point out the difference between both limits of weak selection and discuss the condition under which the differences vanish. It turns out that this condition is fulfilled by the popular parametrization of the prisoner's dilemma in benefits and costs. However, for general payoff matrices differences between the two frameworks prevail.     

Evolutionary stability conditions for signaling games with costly signals

The paper investigates the class of signaling games with the following properties: (a) the interests of sender and receiver coincide, (b) different signals incur differential costs, and (c) different events (meanings/types) have different probabilities. Necessary and sufficient conditions are presented for a profile to be evolutionarily stable and neutrally stable, and for a set of profiles to be an evolutionarily stable set.The main finding is that a profile belongs to some evolutionarily stable set if and only if a maximal number of events can be reliably communicated. Furthermore, it is shown that under the replicator dynamics, a set of states with a positive measure is attracted to “sub-optimal” equilibria that do not belong to any asymptotically stable set.     

Evolutionary stable strategies and game dynamics for n-person games

This note contains a generalization of the definition of an evolutionary stable strategy and of the corresponding game dynamics from 2-person to n-person games. This broader framework also allows modelling of several interacting populations or of populations containing different types of individuals, for example males and females.     

Participation costs can suppress the evolution of upstream reciprocity

Indirect reciprocity, one of the many mechanisms proposed to explain the evolution of cooperation, is the idea that altruistic actions can be rewarded by third parties. Upstream or generalized reciprocity is one type of indirect reciprocity in which individuals help someone if they have been helped by somebody else in the past. Although empirically found to be at work in humans, the evolution of upstream reciprocity is difficult to explain from a theoretical point of view. A recent model of upstream reciprocity, first proposed by Nowak and Roch (2007) and further analyzed by Iwagami and Masuda (2010), shows that while upstream reciprocity alone does not lead to the evolution of cooperation, it can act in tandem with mechanisms such as network reciprocity and increase the total level of cooperativity in the population. We argue, however, that Nowak and Roch's model systematically leads to non-uniform interaction rates, where more cooperative individuals take part in more games than less cooperative ones. As a result, the critical benefit-to-cost ratios derived under this model in previous studies are not invariant with respect to the addition of participation costs. We show that accounting for these costs can hinder and even suppress the evolution of upstream reciprocity, both for populations with non-random encounters and graph-structured populations.     

Evolutionary dynamics of finite populations in games with polymorphic fitness equilibria

The hawk-dove (HD) game, as defined by Maynard Smith [1982. Evolution and the Theory of Games. Cambridge University Press, Cambridge], allows for a polymorphic fitness equilibrium (PFE) to exist between its two pure strategies; this polymorphism is the attractor of the standard replicator dynamics [Taylor, P.D., Jonker, L., 1978. Evolutionarily stable strategies and game dynamics. Math. Biosci. 40, 145-156; Hofbauer, J., Sigmund, K., 1998. Evolutionary Games and Population Dynamics. Cambridge University Press, Cambridge] operating on an infinite population of pure-strategists. Here, we consider stochastic replicator dynamics, operating on a finite population of pure-strategists playing games similar to HD; in particular, we examine the transient behavior of the system, before it enters an absorbing state due to sampling error. Though stochastic replication prevents the population from fixing onto the PFE, selection always favors the under-represented strategy. Thus, we may naively expect that the mean population state (of the pre-absorption transient) will correspond to the PFE. The empirical results of Fogel et al. [1997. On the instability of evolutionary stable states. BioSystems 44, 135-152] show that the mean population state, in fact, deviates from the PFE with statistical significance. We provide theoretical results that explain their observations. We show that such deviation away from the PFE occurs when the selection pressures that surround the fitness-equilibrium point are asymmetric. Further, we analyze a Markov model to prove that a finite population will generate a distribution over population states that equilibrates selection-pressure asymmetry; the mean of this distribution is generally not the fitness-equilibrium state.     

Stochastic sampling of interaction partners versus deterministic payoff assignment

Evolutionary game dynamics describes how successful strategies spread in a population. In well-mixed populations, the usual assumption, e.g. underlying the replicator dynamics, is that individuals obtain a payoff from interactions with a representative sample of the population. This determines their fitness. Here, we analyze a situation in which payoffs are obtained through a single interaction, so that individuals of the same type can have different payoffs. We show analytically that for weak selection, this scenario is identical to the usual approach in which an individual interacts with the whole population. For strong selection, however, differences arise that are reflected in the fixation probabilities and lead to deviating evolutionary dynamics.     

Evolution of cooperation with shared costs and benefits

The quest to determine how cooperation evolves can be based on evolutionary game theory, in spite of the fact that evolutionarily stable strategies (ESS) for most non-zero-sum games are not cooperative. We analyse the evolution of cooperation for a family of evolutionary games involving shared costs and benefits with a continuum of strategies from non-cooperation to total cooperation. This cost-benefit game allows the cooperator to share in the benefit of a cooperative act, and the recipient to be burdened with a share of the cooperator's cost. The cost-benefit game encompasses the Prisoner's Dilemma, Snowdrift game and Partial Altruism. The models produce ESS solutions of total cooperation, partial cooperation, non-cooperation and coexistence between cooperation and non-cooperation. Cooperation emerges from an interplay between the nonlinearities in the cost and benefit functions. If benefits increase at a decelerating rate and costs increase at an accelerating rate with the degree of cooperation, then the ESS has an intermediate level of cooperation. The game also exhibits non-ESS points such as unstable minima, convergent-stable minima and unstable maxima. The emergence of cooperative behaviour in this game represents enlightened self-interest, whereas non-cooperative solutions illustrate the Tragedy of the Commons. Games having either a stable maximum or a stable minimum have the property that small changes in the incentive structure (model parameter values) or culture (starting frequencies of strategies) result in correspondingly small changes in the degree of cooperation. Conversely, with unstable maxima or unstable minima, small changes in the incentive structure or culture can result in a switch from non-cooperation to total cooperation (and vice versa). These solutions identify when human or animal societies have the potential for cooperation and whether cooperation is robust or fragile.