Evolutionary game dynamics in finite populations with strong selection and weak mutation

We study stochastic game dynamics in finite populations. To this end we extend the classical Moran process to incorporate frequency-dependent selection and mutation. For 2 x 2 games, we give a complete analysis of the long-run behavior when mutation rates are small. For 3 x 3 coordination games, we provide a simple rule to determine which strategy will be selected in large populations. The expected motion in our model resembles the standard replicator dynamics when the population is large, but is qualitatively different when the population is small. Our analysis shows that even in large finite populations the behavior of a replicator-like system can be different from that of the standard replicator dynamics. As an application, we consider selective language dynamics. We determine which language will be spoken in finite large populations. The results have an intuitive interpretation but would not be expected from an analysis of the replicator dynamics.     

Evolutionary game dynamics in finite populations

We introduce a model of stochastic evolutionary game dynamics in finite populations which is similar to the familiar replicator dynamics for infinite populations. Our focus is on the conditions for selection favoring the invasion and/or fixation of new phenotypes. For infinite populations, there are three generic selection scenarios describing evolutionary game dynamics among two strategies. For finite populations, there are eight selection scenarios. For a fixed payoff matrix a number of these scenarios can occur for different population sizes. We discuss several examples with unexpected behavior.     

Equilibrium selection in evolutionary games with random matching of players

We discuss stochastic dynamics of populations of individuals playing games. Our models possess two evolutionarily stable strategies: an efficient one, where a population is in a state with the maximal payoff (fitness) and a risk-dominant one, where players are averse to risks. We assume that individuals play with randomly chosen opponents (they do not play against average strategies as in the standard replicator dynamics). We show that the long-run behavior of a population depends on its size and the mutation level.     

Evolutionary games on networks and payoff invariance under replicator dynamics

The commonly used accumulated payoff scheme is not invariant with respect to shifts of payoff values when applied locally in degree-inhomogeneous population structures. We propose a suitably modified payoff scheme and we show both formally and by numerical simulation, that it leaves the replicator dynamics invariant with respect to affine transformations of the game payoff matrix. We then show empirically that, using the modified payoff scheme, an interesting amount of cooperation can be reached in three paradigmatic non-cooperative two-person games in populations that are structured according to graphs that have a marked degree inhomogeneity, similar to actual graphs found in society. The three games are the Prisoner’s Dilemma, the Hawks-Doves and the Stag-Hunt. This confirms previous important observations that, under certain conditions, cooperation may emerge in such network-structured populations, even though standard replicator dynamics for mixing populations prescribes equilibria in which cooperation is totally absent in the Prisoner’s Dilemma, and it is less widespread in the other two games.     

Stochastic evolutionary dynamics of bimatrix games

Evolutionary game dynamics of two-player asymmetric games in finite populations is studied. We consider two roles in the game, roles α and β. α-players and β-players interact and gain payoffs. The game is described by a pair of matrices, which is called bimatrix. One's payoff in the game is interpreted as its fecundity, thus strategies are subject to natural selection. In addition, strategies can randomly mutate to others. We formulate a stochastic evolutionary game dynamics of bimatrix games as a frequency-dependent Moran process with mutation. We analytically derive the stationary distribution of strategies under weak selection. Our result provides a criterion for equilibrium selection in general bimatrix games.     

Finite populations choose an optimal language

This paper studies the evolution of a proto-language in a finite population under the frequency-dependent Moran process. A proto-language can be seen as a collection of concept-to-sign mappings. An efficient proto-language is a bijective mapping from objects of communication to used signs and vice versa. Based on the comparison of fixation probabilities, a method for deriving conditions of evolutionary stability in a finite population [Nowak et al., 2004. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 246-650], it is shown that efficient proto-languages are the only strategies that are protected by selection, which means that no mutant strategy can have a fixation probability that is greater than the inverse population size. In passing, the paper provides interesting results about the comparison of fixation probabilities as well as Maynard Smith's notion of evolutionary stability for finite populations [Maynard Smith, 1988. Can a mixed strategy be stable in a finite population? J. Theor. Biol. 130, 247-251] that are generally true for games with a symmetric payoff function.     

Mutation-selection equilibrium in games with multiple strategies

In evolutionary games the fitness of individuals is not constant but depends on the relative abundance of the various strategies in the population. Here we study general games among n strategies in populations of large but finite size. We explore stochastic evolutionary dynamics under weak selection, but for any mutation rate. We analyze the frequency dependent Moran process in well-mixed populations, but almost identical results are found for the Wright-Fisher and Pairwise Comparison processes. Surprisingly simple conditions specify whether a strategy is more abundant on average than 1/n, or than another strategy, in the mutation-selection equilibrium. We find one condition that holds for low mutation rate and another condition that holds for high mutation rate. A linear combination of these two conditions holds for any mutation rate. Our results allow a complete characterization of n×n games in the limit of weak selection.     

Evolution of cooperation under -person snowdrift games

In the animal world, performing a given task which is beneficial to an entire group requires the cooperation of several individuals of that group who often share the workload required to perform the task. The mathematical framework to study the dynamics of collective action is game theory. Here we study the evolutionary dynamics of cooperators and defectors in a population in which groups of individuals engage in N-person, non-excludable public goods games. We explore an N-person generalization of the well-known two-person snowdrift game. We discuss both the case of infinite and finite populations, taking explicitly into consideration the possible existence of a threshold above which collective action is materialized. Whereas in infinite populations, an N-person snowdrift game (NSG) leads to a stable coexistence between cooperators and defectors, the introduction of a threshold leads to the appearance of a new interior fixed point associated with a coordination threshold. The fingerprints of the stable and unstable interior fixed points still affect the evolutionary dynamics in finite populations, despite evolution leading the population inexorably to a monomorphic end-state. However, when the group size and population size become comparable, we find that spite sets in, rendering cooperation unfeasible.     

Single-Crossover Dynamics: Finite versus Infinite Populations

Populations evolving under the joint influence of recombination and resampling (traditionally known as genetic drift) are investigated. First, we summarize and adapt a deterministic approach, as valid for infinite populations, which assumes continuous time and single crossover events. The corresponding nonlinear system of differential equations permits a closed solution, both in terms of the type frequencies and via linkage disequilibria of all orders. To include stochastic effects, we then consider the corresponding finite-population model, the Moran model with single crossovers, and examine it both analytically and by means of simulations. Particular emphasis is on the connection with the deterministic solution. If there is only recombination and every pair of recombined offspring replaces their pair of parents (i.e., there is no resampling), then the expected type frequencies in the finite population, of arbitrary size, equal the type frequencies in the infinite population. If resampling is included, the stochastic process converges, in the infinite-population limit, to the deterministic dynamics, which turns out to be a good approximation already for populations of moderate size.     

Transforming the dilemma

How does natural selection lead to cooperation between competing individuals? The Prisoner's Dilemma captures the essence of this problem. Two players can either cooperate or defect. The payoff for mutual cooperation, R, is greater than the payoff for mutual defection, P. But a defector versus a cooperator receives the highest payoff, T, where as the cooperator obtains the lowest payoff, S. Hence, the Prisoner's Dilemma is defined by the payoff ranking T > R > P > S . In a well‐mixed population, defectors always have a higher expected payoff than cooperators, and therefore natural selection favors defectors. The evolution of cooperation requires specific mechanisms. Here we discuss five mechanisms for the evolution of cooperation: direct reciprocity, indirect reciprocity, kin selection, group selection, and network reciprocity (or graph selection). Each mechanism leads to a transformation of the Prisoner's Dilemma payoff matrix. From the transformed matrices, we derive the fundamental conditions for the evolution of cooperation. The transformed matrices can be used in standard frameworks of evolutionary dynamics such as the replicator equation or stochastic processes of game dynamics in finite populations.     

Fixation probabilities in evolutionary game dynamics with a two-strategy game in finite diploid populations

Fixation processes in evolutionary game dynamics in finite diploid populations are investigated. Traditionally, frequency dependent evolutionary dynamics is modeled as deterministic replicator dynamics. This implies that the infinite size of the population is assumed implicitly. In nature, however, population sizes are finite. Recently, stochastic processes in finite populations have been introduced in order to study finite size effects in evolutionary game dynamics. One of the most significant studies on evolutionary dynamics in finite populations was carried out by Nowak et al. which describes “one-third law” [Nowak, et al., 2004. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 646-650]. It states that under weak selection, if the fitness of strategy α is greater than that of strategy β when α has a frequency , strategy α fixates in a β-population with selective advantage. In their study, it is assumed that the inheritance of strategies is asexual, i.e. the population is haploid. In this study, we apply their framework to a diploid population that plays a two-strategy game with two ESSs (a bistable game). The fixation probability of a mutant allele in this diploid population is derived. A “three-tenth law” for a completely recessive mutant allele and a “two-fifth law” for a completely dominant mutant allele are found; other cases are also discussed.     

Evolutionary games and population dynamics: maintenance of cooperation in public goods games

The emergence and abundance of cooperation in nature poses a tenacious and challenging puzzle to evolutionary biology. Cooperative behaviour seems to contradict Darwinian evolution because altruistic individuals increase the fitness of other members of the population at a cost to themselves. Thus, in the absence of supporting mechanisms, cooperation should decrease and vanish, as predicted by classical models for cooperation in evolutionary game theory, such as the Prisoner's Dilemma and public goods games. Traditional approaches to studying the problem of cooperation assume constant population sizes and thus neglect the ecology of the interacting individuals. Here, we incorporate ecological dynamics into evolutionary games and reveal a new mechanism for maintaining cooperation. In public goods games, cooperation can gain a foothold if the population density depends on the average population payoff. Decreasing population densities, due to defection leading to small payoffs, results in smaller interaction group sizes in which cooperation can be favoured. This feedback between ecological dynamics and game dynamics can generate stable coexistence of cooperators and defectors in public goods games. However, this mechanism fails for pairwise Prisoner's Dilemma interactions and the population is driven to extinction. Our model represents natural extension of replicator dynamics to populations of varying densities.     

Imitation, internal absorption and the reversal of local drift in stochastic evolutionary games

Evolutionary game dynamics in finite populations is typically subject to noise, inducing effects which are not present in deterministic systems, including fixation and extinction. In the first part of this paper we investigate the phenomenon of drift reversal in finite populations, taking into account that drift is a local quantity in strategy space. Secondly, we study a simple imitation dynamics, and show that it can lead to fixation at internal mixed-strategy fixed points even in finite populations. Imitation in infinite populations is adequately described by conventional replicator dynamics, and these equations are known to have internal fixed points. Internal absorption in finite populations on the other hand is a novel dynamic phenomenon. Due to an outward drift in finite populations this type of dynamic arrest is not found in other commonly studied microscopic dynamics, not even in those with the same deterministic replicator limit as imitation.     

Fixation of Strategies for an Evolutionary Game in Finite Populations

A stochastic evolutionary dynamics of two strategies given by 2x 2 matrix games is studied in finite populations. We focus on stochastic properties of fixation: how a strategy represented by a single individual wins over the entire population. The process is discussed in the framework of a random walk with site dependent hopping rates. The time of fixation is found to be identical for both strategies in any particular game. The asymptotic behavior of the fixation time and fixation probabilities in the large population size limit is also discussed. We show that fixation is fast when there is at least one pure evolutionary stable strategy (ESS) in the infinite population size limit, while fixation is slow when the ESS is the coexistence of the two strategies.     

Evolutionary dynamics of finite populations in games with polymorphic fitness equilibria

The hawk-dove (HD) game, as defined by Maynard Smith [1982. Evolution and the Theory of Games. Cambridge University Press, Cambridge], allows for a polymorphic fitness equilibrium (PFE) to exist between its two pure strategies; this polymorphism is the attractor of the standard replicator dynamics [Taylor, P.D., Jonker, L., 1978. Evolutionarily stable strategies and game dynamics. Math. Biosci. 40, 145-156; Hofbauer, J., Sigmund, K., 1998. Evolutionary Games and Population Dynamics. Cambridge University Press, Cambridge] operating on an infinite population of pure-strategists. Here, we consider stochastic replicator dynamics, operating on a finite population of pure-strategists playing games similar to HD; in particular, we examine the transient behavior of the system, before it enters an absorbing state due to sampling error. Though stochastic replication prevents the population from fixing onto the PFE, selection always favors the under-represented strategy. Thus, we may naively expect that the mean population state (of the pre-absorption transient) will correspond to the PFE. The empirical results of Fogel et al. [1997. On the instability of evolutionary stable states. BioSystems 44, 135-152] show that the mean population state, in fact, deviates from the PFE with statistical significance. We provide theoretical results that explain their observations. We show that such deviation away from the PFE occurs when the selection pressures that surround the fitness-equilibrium point are asymmetric. Further, we analyze a Markov model to prove that a finite population will generate a distribution over population states that equilibrates selection-pressure asymmetry; the mean of this distribution is generally not the fitness-equilibrium state.     

Stochastic sampling of interaction partners versus deterministic payoff assignment

Evolutionary game dynamics describes how successful strategies spread in a population. In well-mixed populations, the usual assumption, e.g. underlying the replicator dynamics, is that individuals obtain a payoff from interactions with a representative sample of the population. This determines their fitness. Here, we analyze a situation in which payoffs are obtained through a single interaction, so that individuals of the same type can have different payoffs. We show analytically that for weak selection, this scenario is identical to the usual approach in which an individual interacts with the whole population. For strong selection, however, differences arise that are reflected in the fixation probabilities and lead to deviating evolutionary dynamics.     

Population dynamics with a stable efficient equilibrium

We propose a game-theoretic dynamics of a population of replicating individuals. It consists of two parts: the standard replicator one and a migration between two different habitats. We consider symmetric two-player games with two evolutionarily stable strategies: the efficient one in which the population is in a state with a maximal payoff and the risk-dominant one where players are averse to risk. We show that for a large range of parameters of our dynamics, even if the initial conditions in both habitats are in the basin of attraction of the risk-dominant equilibrium (with respect to the standard replication dynamics without migration), in the long run most individuals play the efficient strategy.     

Natural selection and social preferences

A large number of individuals are randomly matched into groups, where each group plays a finite symmetric game. Individuals breed true. The expected number of surviving offspring depends on own material payoff, but may also, due to cooperative breeding and/or reproductive competition, depend on the material payoffs to other group members. The induced population dynamic is equivalent with the replicator dynamic for a game with payoffs derived from those in the original game. We apply this selection dynamic to a number of examples, including prisoners' dilemma games with and without a punishment option, coordination games, and hawk-dove games. For each of these, we compare the outcomes with those obtained under the standard replicator dynamic. By way of a revealed-preference argument, our selection dynamic can explain certain "altruistic" and "spiteful" behaviors that are consistent with individuals having social preferences.     

Evolutionary stability for large populations

We present a revision of Maynard Smith's evolutionary stability criteria for populations which are very large (though technically finite) and of unknown size. We call this the large population ESS, as distinct from Maynard Smith's infinite population ESS and Schaffer's finite population ESS. Building on Schaffer's finite population model, we define the large population ESS as a strategy which cannot be invaded by any finite number of mutants, as long as the population size is sufficiently large. The large population ESS is not equivalent to the infinite population ESS: we give examples of games in which a large population ESS exists but an infinite population ESS does not, and vice versa. Our main contribution is a simple set of two criteria for a large population ESS, which are similar (but not identical) to those originally proposed by Maynard Smith for infinite populations.     

Adaptive Dynamics of Extortion and Compliance

Direct reciprocity is a mechanism for the evolution of cooperation. For the iterated prisoner’s dilemma, a new class of strategies has recently been described, the so-called zero-determinant strategies. Using such a strategy, a player can unilaterally enforce a linear relationship between his own payoff and the co-player’s payoff. In particular the player may act in such a way that it becomes optimal for the co-player to cooperate unconditionally. In this way, a player can manipulate and extort his co-player, thereby ensuring that the own payoff never falls below the co-player’s payoff. However, using a compliant strategy instead, a player can also ensure that his own payoff never exceeds the co-player’s payoff. Here, we use adaptive dynamics to study when evolution leads to extortion and when it leads to compliance. We find a remarkable cyclic dynamics: in sufficiently large populations, extortioners play a transient role, helping the population to move from selfish strategies to compliance. Compliant strategies, however, can be subverted by altruists, which in turn give rise to selfish strategies. Whether cooperative strategies are favored in the long run critically depends on the size of the population; we show that cooperation is most abundant in large populations, in which case average payoffs approach the social optimum. Our results are not restricted to the case of the prisoners dilemma, but can be extended to other social dilemmas, such as the snowdrift game. Iterated social dilemmas in large populations do not lead to the evolution of strategies that aim to dominate their co-player. Instead, generosity succeeds.