The different limits of weak selection and the evolutionary dynamics of finite populations

Evolutionary theory often resorts to weak selection, where different individuals have very similar fitness. Here, we relate two ways to introduce weak selection. The first considers evolutionary games described by payoff matrices with similar entries. This approach has recently attracted a lot of interest in the context of evolutionary game dynamics in finite populations. The second way to introduce weak selection is based on small distances in phenotype space and is a standard approach in kin-selection theory. Whereas both frameworks are interchangeable for constant fitness, frequency-dependent selection shows significant differences between them. We point out the difference between both limits of weak selection and discuss the condition under which the differences vanish. It turns out that this condition is fulfilled by the popular parametrization of the prisoner's dilemma in benefits and costs. However, for general payoff matrices differences between the two frameworks prevail.     

Nonlinear frequency-dependent selection at a single locus with two alleles and two phenotypes

 The paper investigates the discrete frequency dynamics of two phenotype diploid models where genotypic fitness is an exponential function of the expected payoff in the matrix game. Phenotypic and genotypic equilibria are defined and their stability compared to frequency-dependent selection models based on linear fitness when there are two possible phenotypes in the population. In particular, it is shown that stable equilibria of both types can exist in the same nonlinear model. It is also shown that period-doubling bifurcations emerge when there is sufficient selection in favor of interactions between different phenotypes. Received: 22 October 1998     

Maynard Smith,optimization, and evolution

Maynard Smith’s defenses of adaptationism and of the value of optimization theory in evolutionary biology are both criticized. His defense does not adequately respond to the criticism of adaptationism by Gould and Lewontin. It is also argued here that natural selection cannot be interpreted as an optimization process if the objective function to be optimized is either (i) interpretable as a fitness, or (ii) correlated with the mean population fitness. This result holds even if fitnesses are frequency-independent; the problem is further exacerbated in the frequency-dependent context modeled by evolutionary game theory. However, Eshel and Feldman’s new results on “long-term” evolution may provide some hope for the continuing relevance of the game-theoretic framework. These arguments also demonstrate the irrelevance of attempts by Intelligent Design creationists to use computational limits on optimization algorithms as evidence against evolutionary theory. It is pointed out that adaptation, natural selection, and optimization are not equivalent processes in the context of biological evolution. It is a pleasure to dedicate this paper to the memory of John Maynard Smith. Thanks are due to James Justus and Samir Okasha for comments on an earlier draft.     

MATHEMATICS OF KIN‐ AND GROUP‐SELECTION: FORMALLY EQUIVALENT?

Evolutionary game theory is a general mathematical framework that describes the evolution of social traits. This framework forms the basis of many multilevel selection models and is also frequently used to model evolutionary dynamics on networks. Kin selection, which was initially restricted to describe social interactions between relatives, has also led to a broader mathematical approach, inclusive fitness, that can not only describe social evolution among relatives, but also in group structured populations or on social networks. It turns out that the underlying mathematics of game theory is fundamentally different from the approach of inclusive fitness. Thus, both approaches—evolutionary game theory and inclusive fitness—can be helpful to understand the evolution of social traits in group structured or spatially extended populations.     

Nash equilibria for an evolutionary language game

We study an evolutionary language game that describes how signals become associated with meaning. In our context, a language, L, is described by two matrices: the P matrix contains the probabilities that for a speaker certain objects are associated with certain signals, while the Q matrix contains the probabilities that for a listener certain signals are associated with certain objects. We define the payoff in our evolutionary language game as the total amount of information exchanged between two individuals. We give a formal classification of all languages, L(P, Q), describing the conditions for Nash equilibria and evolutionarily stable strategies (ESS). We describe an algorithm for generating all languages that are Nash equilibria. Finally, we show that starting from any random language, there exists an evolutionary trajectory using selection and neutral drift that ends up with a strategy that is a strict Nash equilibrium (or very close to a strict Nash equilibrium). Received: 1 March 2000 / Published online: 3 August 2000     

Mutation in evolutionary games can increase average fitness at equilibrium

We study game dynamical interactions between two strategies, A and B, and analyse whether the average fitness of the population at equilibrium can be increased by adding mutation from A to B. Classifying all two by two games with payoff matrix [(a,b),(c,d)], we show that mutation from A to B enhances the average fitness of the whole population (i) if both a and d are less than (b + c)/2 and (ii) if c is less than b. Furthermore, we study conditions for maximizing the productivity of strategy A, and we analyse the effect of mutations in both directions. Depending on the biological system, a mutation in an evolutionary game can be interpreted as a genetic alteration, a cellular differentiation, a change in gene expression, an accidental or deliberate modification in cultural transmission, or a learning error. In a cultural context, our results indicate that the equilibrium payoff of the population can be increased if players sometimes choose the strategy with lower payoff. In a genetic context, we have shown that for frequency-dependent selection mutation can enhance the average fitness of the population at equilibrium.     

Stochastic Fluctuations Through Intrinsic Noise in Evolutionary Game Dynamics

A one-step (birth–death) process is used to investigate stochastic noise in an elementary two-phenotype evolutionary game model based on a payoff matrix. In this model, we assume that the population size is finite but not fixed and that all individuals have, in addition to the frequency-dependent fitness given by the evolutionary game, the same background fitness that decreases linearly in the total population size. Although this assumption guarantees population extinction is a globally attracting absorbing barrier of the Markov process, sample trajectories do not illustrate this result even for relatively small carrying capacities. Instead, the observed persistent transient behavior can be analyzed using the steady-state statistics (i.e., mean and variance) of a stochastic model for intrinsic noise that assumes the population does not go extinct. It is shown that there is good agreement between the theory of these statistics and the simulation results. Furthermore, the ESS of the evolutionary game can be used to predict the mean steady state.     

Evolutionary dynamics in structured populations

Evolutionary dynamics shape the living world around us. At the centre of every evolutionary process is a population of reproducing individuals. The structure of that population affects evolutionary dynamics. The individuals can be molecules, cells, viruses, multicellular organisms or humans. Whenever the fitness of individuals depends on the relative abundance of phenotypes in the population, we are in the realm of evolutionary game theory. Evolutionary game theory is a general approach that can describe the competition of species in an ecosystem, the interaction between hosts and parasites, between viruses and cells, and also the spread of ideas and behaviours in the human population. In this perspective, we review the recent advances in evolutionary game dynamics with a particular emphasis on stochastic approaches in finite sized and structured populations. We give simple, fundamental laws that determine how natural selection chooses between competing strategies. We study the well-mixed population, evolutionary graph theory, games in phenotype space and evolutionary set theory. We apply these results to the evolution of cooperation. The mechanism that leads to the evolution of cooperation in these settings could be called ‘spatial selection’: cooperators prevail against defectors by clustering in physical or other spaces.     

Stochastic sampling of interaction partners versus deterministic payoff assignment

Evolutionary game dynamics describes how successful strategies spread in a population. In well-mixed populations, the usual assumption, e.g. underlying the replicator dynamics, is that individuals obtain a payoff from interactions with a representative sample of the population. This determines their fitness. Here, we analyze a situation in which payoffs are obtained through a single interaction, so that individuals of the same type can have different payoffs. We show analytically that for weak selection, this scenario is identical to the usual approach in which an individual interacts with the whole population. For strong selection, however, differences arise that are reflected in the fixation probabilities and lead to deviating evolutionary dynamics.     

The evolution of payoff matrices: providing incentives to cooperate

Most of the work in evolutionary game theory starts with a model of a social situation that gives rise to a particular payoff matrix and analyses how behaviour evolves through natural selection. Here, we invert this approach and ask, given a model of how individuals behave, how the payoff matrix will evolve through natural selection. In particular, we ask whether a prisoner's dilemma game is stable against invasions by mutant genotypes that alter the payoffs. To answer this question, we develop a two-tiered framework with goal-oriented dynamics at the behavioural time scale and a diploid population genetic model at the evolutionary time scale. Our results are two-fold: first, we show that the prisoner's dilemma is subject to invasions by mutants that provide incentives for cooperation to their partners, and that the resulting game is a coordination game similar to the hawk-dove game. Second, we find that for a large class of mutants and symmetric games, a stable genetic polymorphism will exist in the locus determining the payoff matrix, resulting in a complex pattern of behavioural diversity in the population. Our results highlight the importance of considering the evolution of payoff matrices to understand the evolution of animal social systems.     

Strategy Selection in Evolutionary Game Dynamics on Group Interaction Networks

Evolutionary game theory provides an appropriate tool for investigating the competition and diffusion of behavioral traits in biological or social populations. A core challenge in evolutionary game theory is the strategy selection problem: Given two strategies, which one is favored by the population? Recent studies suggest that the answer depends not only on the payoff functions of strategies but also on the interaction structure of the population. Group interactions are one of the fundamental interactive modes within populations. This work aims to investigate the strategy selection problem in evolutionary game dynamics on group interaction networks. In detail, the strategy selection conditions are obtained for some typical networks with group interactions. Furthermore, the obtained conditions are applied to investigate selection between cooperation and defection in populations. The conditions for evolution of cooperation are derived for both the public goods game and volunteer’s dilemma game. Numerical experiments validate the above analytical results.     

Mutation-selection equilibrium in games with mixed strategies

We develop a new method for studying stochastic evolutionary game dynamics of mixed strategies. We consider the general situation: there are n pure strategies whose interactions are described by an n×n payoff matrix. Players can use mixed strategies, which are given by the vector (p₁,…,p_n). Each entry specifies the probability to use the corresponding pure strategy. The sum over all entries is one. Therefore, a mixed strategy is a point in the simplex S_n. We study evolutionary dynamics in a well-mixed population of finite size. Individuals reproduce proportional to payoff. We consider the case of weak selection, which means the payoff from the game is only a small contribution to overall fitness. Reproduction can be subject to mutation; a mutant adopts a randomly chosen mixed strategy. We calculate the average abundance of every mixed strategy in the stationary distribution of the mutation-selection process. We find the crucial conditions that specify if a strategy is favored or opposed by selection. One condition holds for low mutation rate, another for high mutation rate. The result for any mutation rate is a linear combination of those two. As a specific example we study the Hawk-Dove game. We prove general statements about the relationship between games with pure and with mixed strategies.     

Genetic versus phenotypic models of selection: can genetics be neglected in a long-term perspective?

Game theoretical concepts in evolutionary biology have been criticized by populations geneticists, because they neglect such crucial aspects as the mating system or the mode of inheritance. In fact, the dynamics of natural selection does not necessarily lead to a fitness maximum or an ESS if genetic constraints are taken into account. Yet, it may be premature to conclude that game theoretical concepts do not have a dynamical justification. The new paradigm of long-term evolution postulates that genetic constraints, which may be dominant in a short-term perspective, will in the long run disappera in the face of the ongoing influx of mutations. Two basic results (see Hammerstein; this issue) seem to reconcile the dynamical approach of long-term population genetics with the static approach of evolutionary game theory: (1) only populations at local fitness optima (Nash strategies) can be long-term stable; and (2) in monomorphic populations, evolutionary stability is necessary and sufficient to ensure long-term dynamic stability. The present paper has a double purpose. On the one hand, it is demonstrated by fairly general arguments that the scope of the results mentioned above extends to non-linear frequency dependent selection, to multiple loci, and to quite general mating systems. On the other hand, some limitations of the theory of long-term evolution will also be stressed: (1) there is little hope for a game theoretical characterization of stability in polymorphic populations; (2) many interesting systems do not admit long-term stable equilibria; and (3) even if a long-term stable equilibrium exists, it is not at all clear whether and how it is attainable by a series of gene substition events.     

Does evolution lead to maximizing behavior?

A long‐standing question in biology and economics is whether individual organisms evolve to behave as if they were striving to maximize some goal function. We here formalize this “as if” question in a patch‐structured population in which individuals obtain material payoffs from (perhaps very complex multimove) social interactions. These material payoffs determine personal fitness and, ultimately, invasion fitness. We ask whether individuals in uninvadable population states will appear to be maximizing conventional goal functions (with population‐structure coefficients exogenous to the individual's behavior), when what is really being maximized is invasion fitness at the genetic level. We reach two broad conclusions. First, no simple and general individual‐centered goal function emerges from the analysis. This stems from the fact that invasion fitness is a gene‐centered multigenerational measure of evolutionary success. Second, when selection is weak, all multigenerational effects of selection can be summarized in a neutral type‐distribution quantifying identity‐by‐descent between individuals within patches. Individuals then behave as if they were striving to maximize a weighted sum of material payoffs (own and others). At an uninvadable state it is as if individuals would freely choose their actions and play a Nash equilibrium of a game with a goal function that combines self‐interest (own material payoff), group interest (group material payoff if everyone does the same), and local rivalry (material payoff differences).     

Stabilizing selection of protein function and distribution of selection coefficient among sites

In this study, I take a new approach to modeling the evolutionary constraint of protein sequence, introducing the stabilizing selection of protein function into the nearly-neutral theory. In other words, protein function under stabilizing selection generates the evolutionary conservation at the sequence level. With the help of random mutational effects of nucleotides on protein function, I have derived the distribution of selection coefficient among sites, called the S-distribution whose parameters have clear biological interpretations. Moreover, I have studied the inverse relationship between the evolutionary rate and the effective population size, showing that the number of molecular phenotypes of protein function, i.e., independent components in the fitness of the organism, may play a key role for the molecular clock under the nearly-neutral theory. These results are helpful for having a better understanding of the underlying evolutionary mechanism of protein sequences, as well as human disease-related mutations.     

Predicting trait values and measuring selection in complex life histories: reproductive allocation decisions in Soay sheep

Accurate prediction of life history phenomena and characterisation of selection in free-living animal populations are fundamental goals in evolutionary ecology. In density regulated, structured populations, where individual state influences fate, simple and widely used approaches based on individual lifetime measures of fitness are difficult to justify. We combine recently developed structured population modelling tools with ideas from modern evolutionary game theory (adaptive dynamics) to understand selection on allocation of female reproductive effort to singletons or twins in a size-structured population of feral sheep. In marked contrast to the classical selection analyses, our model-based approach predicts that the female allocation strategy is under negligible directional selection. These differences arise because classical selection analysis ignores components of offspring fitness and fails to consider selection over the complete life cycle.     

Thermodynamical interpretation of evolutionary dynamics on a fitness landscape in an evolution reactor,II

In our previous report [Aita, T., Morinaga, S., Hosimi, Y., 2004. Thermodynamical interpretation of evolutionary dynamics on a fitness landscape in an evolution reactor I. Bull. Math. Biol. 66, 1371–1403], an analogy between thermodynamics and adaptive walks on a Mt. Fuji-type fitness landscape in an artificial selection system was presented. Introducing the ‘free fitness’ as the sum of a fitness term and an entropy term and ‘evolutionary force’ as the gradient of free fitness on a fitness coordinate, we demonstrated that the adaptive walk (=evolution) is driven by the evolutionary force in the direction in which free fitness increases. In this report, we examine the effect of various modifications of the original model on the properties of the adaptive walk. The modifications were as follows: first, mutation distance d was distributed obeying binomial distribution; second, the selection process obeyed the natural selection protocol; third, ruggedness was introduced to the landscape according to the NK model; fourth, a noise was included in the fitness measurement. The effect of each modification was described in the same theoretical framework as the original model by introducing ‘effective’ quantities such as the effective mutation distance or the effective screening size.     

Consequences of fluctuating group size for the evolution of cooperation

Studies of cooperation have traditionally focused on discrete games such as the well-known prisoner’s dilemma, in which players choose between two pure strategies: cooperation and defection. Increasingly, however, cooperation is being studied in continuous games that feature a continuum of strategies determining the level of cooperative investment. For the continuous snowdrift game, it has been shown that a gradually evolving monomorphic population may undergo evolutionary branching, resulting in the emergence of a defector strategy that coexists with a cooperator strategy. This phenomenon has been dubbed the ‘tragedy of the commune’. Here we study the effects of fluctuating group size on the tragedy of the commune and derive analytical conditions for evolutionary branching. Our results show that the effects of fluctuating group size on evolutionary dynamics critically depend on the structure of payoff functions. For games with additively separable benefits and costs, fluctuations in group size make evolutionary branching less likely, and sufficiently large fluctuations in group size can always turn an evolutionary branching point into a locally evolutionarily stable strategy. For games with multiplicatively separable benefits and costs, fluctuations in group size can either prevent or induce the tragedy of the commune. For games with general interactions between benefits and costs, we derive a general classification scheme based on second derivatives of the payoff function, to elucidate when fluctuations in group size help or hinder cooperation.     

On the relation between the instability of ESS in discrete dynamics and segregation distortion

There is a simple correspondence between discrete dynamical systems associated with evolutionary game dynamics and general locus multiallele selection models with non-Mendelian segregation. When interpreted properly the payoff matrix has two components, a fitness matrix component and a segregation matrix component. The presence of segregation distortion which corresponds to a non-symmetric payoff matrix, is a source of instability. With non-symmetric payoff an ESS does not usually correspond to a stable equilibrium. It is always externally stable but does not necessarily have an internally stable equilibrium.