An ESS maximum principle for matrix games

Previous work has demonstrated that for games defined by differential or difference equations with a continuum of strategies, there exists a G-function, related to individual fitness, that must take on a maximum with respect to a virtual variable v whenever v is one of the vectors in the coalition of vectors which make up the evolutionarily stable strategy (ESS). This result, called the ESS maximum principle, is quite useful in determining candidates for an ESS. This principle is reformulated here, so that it may be conveniently applied to matrix games. In particular, we define a matrix game to be one in which fitness is expressed in terms of strategy frequencies and a matrix of expected payoffs. It is shown that the G-function in the matrix game setting must again take on a maximum value at all the strategies which make up the ESS coalition vector. The reformulated maximum principle is applicable to both bilinear and nonlinear matrix games. One advantage in employing this principle to solve the traditional bilinear matrix game is that the same G-function is used to find both pure and mixed strategy solutions by simply specifying an appropriate strategy space. Furthermore we show how the theory may be used to solve matrix games which are not in the usual bilinear form. We examine in detail two nonlinear matrix games: the game between relatives and the sex ratio game. In both of these games an ESS solution is determined. These examples not only illustrate the usefulness of this approach to finding solutions to an expanded class of matrix games, but aids in understanding the nature of the ESS as well.     

Evolutionarily stable strategies for a finite population and a variable contest size

This paper presents a generalization of Maynard Smith's concept of an evolutionarily stable strategy (ESS) to cover the cases of a finite population and a variable contest size. Both equilibrium and stability conditions are analysed. The standard Maynard Smith ESS with an infinite population and a contest size of two (pairwise contests) is shown to be a special case of this generalized ESS. An important implication of the generalized ESS is that in finite populations the behaviour of an ESS player is "spiteful", in the sense that an ESS player acts not only to increase his payoff but also to decrease the payoffs of his competitors. The degree of this "spiteful" behaviour is shown to increase with a decrease in the population size, and so is most likely to be observed in small populations. The paper concludes with an extended example: a symmetric two-pure-strategies two-player game for a finite population. It is shown that a mixed strategy ESS is globally stable against invasion by any one type of mutant strategist. The condition for the start of simultaneous invasion by two types of mutant is also given.     

A theory for the evolutionary game

We present an evolutionary game theory. This theory differs in several respects from current theories related to Maynard Smith's pioneering work on evolutionary stable strategies (ESS). Most current work deals with two person matrix games. For these games the strategy set is finite. We consider evolutionary games which are defined over a continuous strategy set and which permit any number of players. Matrix games are included as a bilinear continuous game. However, under our definition, such games will not posses an ESS on the interior of the strategy set. We extend previous work on continuous games by developing an ESS definition which permits the ESS to be composed of a coalition of several strategies. This definition requires that the coalition must not only be stable with respect to perturbations in strategy frequencies which comprise the coalition, but the coalition must also satisfy the requirement that no mutant strategies can invade. Ecological processes are included in the model by explicitly considering population size and density dependent selection.     

Stability in an evolutionary game

A behavior or strategy which is evolutionarily stable must be both optimal and stable. The strategy must be optimal in that it maximizes the expected fitness of all the individuals using it. In addition, the strategy must be resistant to invasion by a mutant. The difference between the Nash solution of game theory and the ESS used in ecology is that the Nash solution only satisfies an optimality criterion and not an evolutionary stability criterion. We extend the ESS definition of Maynard Smith and Price so that it can be applied directly to two-strategy evolutionary games. The concept of a balanced game is introduced, and necessary conditions are derived which are similar to the Nash necessary conditions. The balanced game necessary conditions may be used for direct calculation of ESS candidates. These results are used to examine the optimal flowering time of an annual plant experiencing competition from neighboring plants. The plant competition model is general, and the results may be applied to a wide range of interference competition problems.     

A generalized war of attrition

In the “War of Attrition” model of animal conflict, introduced by Maynard Smith, a reward is obtained by whichever of two opponents displays longer, each individual incurring a cost associated with the length of the contest. This model is generalized to allow more general reward and cost functions, and restrictions on the length of contest permitted. This permits unification of the “War of Attrition” model and the “Graduated Risks” model, and also the extension to models in which contests may end either due to injury, or to retreat.In each case it is demonstrated that either (i) there is no evolutionary stable strategy (ESS) or (ii) there is a unique ESS, which is fully specified. In the case where only a finite number of pure strategies are available, global convergence to the ESS is shown.A variety of interesting conclusions of biological relevance emerge, perhaps the most striking being the occurrence of a dichotomous behavioural pattern in an essentially continuous conflict.     

On the war of attrition and other games among kin

Evolutionarily stable strategies or ESSs of games among kin have been calculated in the literature by both personal-fitness and inclusive-fitness methods. These methods were compared by Hines and Maynard Smith (1979) for games with bilinear payoffs. Although Hines and Maynard Smith regarded the first method as correct, they regarded the second method as useful because the inclusive-fitness conditions for an ESS gave necessary conditions for a personal-fitness ESS in the class of games they considered. In general, however, satisfying the inclusive-fitness conditions is neither necessary nor sufficient for satisfying the personal-fitness conditions, although the two methods may often yield identical ESSs. This result is established by reformulating the classic war-of-attrition model to allow variation in energy reserves, assumed to have a Gamma distribution. For this game, the two methods may disagree for intermediate values of relatedness. By the correct method, if the coefficient of variation in energy reserves is sufficiently high, then the game has a unique ESS in pure strategies at which populations with higher coefficients of variation or relatedness display for shorter times. Unrelated contestants are prepared to expend at least half of their reserves. For populations with lower variation coefficients, the ESS exists only if the cost of displaying per unit time is low compared to the rate at which remaining reserves translate into expected future reproductive success for the victor. The critical variation coefficient, below which the ESS exists regardless of cost, decreases from 0.52 to 0 as the coefficient of relatedness increases from 0 to 1. Although there is no assessment, contests are always won by the animal with greater energy reserves in a population at the ESS.     

Ecological stability,evolutionary stability and the ESS maximum principle

Summary Since the fitness of each individual organism in a biological community may be affected by the strategies of all other individuals in the community, the essential element of a game exists. This game is an evolutionary game where the individual organisms (players) inherit their strategies from continuous play of the game through time. Here, the strategies are assumed to be constants associated with certain adaptive parameters (such as sunlight conversion efficiency for plants or body length in animals) in a set of differential equations which describe the population dynamics of the community. By means of natural selection, these parameters will evolve to a set of strategy values that natural selection, by itself, can no longer modify, i.e. an evolutionarily stable strategy (ESS). For a given class of models, it is possible to predict the outcome of this evolutionary process by determining ESSs using an ESS maximum principle. However, heretofore, the proof of this principle has been based on a limited set of conditions. Herein, we generalize the proof by removing certain restrictions and use instead the concept of an ecological stable equilibrium (ESE). Individuals in a biological community will be at an ESE if fixing the strategies used by the individuals results in stable population densities subject to perturbations in those densities. We present both necessary and sufficient conditions for an ESE to exist and then use the ESE concept to provide a very simple proof of the ESS maximum principle (which is a necessary condition for an ESS). A simple example is used to illustrate the difference between a strategy that maximizes fitness and one that satisfies the ESS maximum principle. In general they are different. We also look for ESEs in Lotka—Volterra competition and use the maximum principle to determine when an ESE will be an ESS. Finally, we examine the applicability of these ideas to matrix games.     

The theory of games and the evolution of animal conflicts

The evolution of behaviour patterns used in animal conflicts is discussed, using models based on the theory of games. The paper extends arguments used by Maynard Smith &; Price (1973) showing that ritualized behaviour can evolve by individual selection. The concept of an evolutionarily stable strategy, or ESS, is defined. Two types of ritualized contests are distinguished, “tournaments” and “displays”; the latter, defined as contests without physical contact in which victory goes to the contestant which continues longer, are analyzed in detail. Three main conclusions are drawn. The degree of persistence should be very variable, either between individuals or for the same individual at different times; a negative exponential distribution of persistence times is predicted. Individuals should display with constant intensity, independent of how much longer they will in fact continue. An initial asymmetry in the conditions of a contest can be used to settle it, even if it is irrelevant to the outcome of a more protracted conflict if one were to take place.     

Does frequency-dependent selection optimize fitness?

In evolutionary biology, the axiom that natural selection tends ideally to maximize inclusive fitness of the individual or some other suitable quantity is often advanced (Cody, 1974; Maynard Smith, 1978; Krebs & McCleery, 1984; Houston et al., 1988). Moreover, the evolutionists generally distinguish two situations (Dawkins, 1980; Maynard Smith, 1982): one in which fitness is independent of the frequency of the phenotypes present in the population (frequency-independent selection), and one in which it does depend on this frequency (frequency-dependent selection). This led some authors such as Parker (1984), and more recently Parker & Maynard Smith (1990), to consider "a 2-speed optimization": frequency-independent selection should lead to a "simple optimum" at the end of the selective process, since all the individuals should have the same strategy and the mean fitness of the population should be maximized; frequency-dependent selection, formulated in terms of the theory of games, should lead to a "competitive optimum" even though the "evolutionary stable strategy" (or "ESS"; Maynard Smith & Price, 1973) characterizing the equilibrium "is not the strategy that maximizes fitness in a population sense" (Parker & Maynard Smith, 1990: 30). Our aim in this short communication is to criticize the concept of "competitive optimum" by Parker & Maynard Smith, as well as the general ability of natural selection to "maximize fitness", even in "phenotypic models" (Lloyd, 1977). These models, devoid of genetic constraints since each strategist is assumed to reproduce its own kind, are especially suitable for examining the ideal effect of natural selection.     

Individual-based simulations of the War of Attrition

The War of Attrition model of John Maynard Smith predicts a single, mixed evolutionarily stable strategy (ESS) for animal contests which are settled by conventional displays with no assessment of the opponent's fighting ability. We test the predictions of the model by simulating the evolution of strategies in a finite population of animals under various assumptions on how possible strategies are coded and mutated. While our simulations for the most part confirm the predictions of the model, we also discovered some significant deviations from the theoretically predicted ESS. Specifically, we found that if inheritance of strategies is somewhat imprecise, then a population can evolve that achieves on average a higher payoff than a population at the theoretically predicted ESS. Moreover, if the ESS is realized as a polymorphism of fixed persistence times, then for small populations, sufficiently stringent statistical tests will reject the hypothesis that these times are distributed as theoretically predicted.     

Evolutionary stability for large populations

We present a revision of Maynard Smith's evolutionary stability criteria for populations which are very large (though technically finite) and of unknown size. We call this the large population ESS, as distinct from Maynard Smith's infinite population ESS and Schaffer's finite population ESS. Building on Schaffer's finite population model, we define the large population ESS as a strategy which cannot be invaded by any finite number of mutants, as long as the population size is sufficiently large. The large population ESS is not equivalent to the infinite population ESS: we give examples of games in which a large population ESS exists but an infinite population ESS does not, and vice versa. Our main contribution is a simple set of two criteria for a large population ESS, which are similar (but not identical) to those originally proposed by Maynard Smith for infinite populations.     

Properties of a mixed ESS candidate in continuous strategy sets

An evolutionarily stable strategy (ESS) is a strategy that if almost all members of the population adopt, then this population cannot be invaded by any mutant strategy. An ESS is not necessarily a possible end point of the evolutionary process. Moreover, there are cases where the population evolves towards a strategy that is not an ESS. This paper studies the properties of a unique mixed ESS candidate in a continuous time animal conflict. A member of a group sized three finds itself at risk and needs the assistance of another group member to be saved. In this conflict, a player's strategy is to choose the probability distribution of the interval between the beginning of the game and the moment it assists the player which is at risk. We first assume that a player is only allowed to choose an exponential distribution, and show that in this case the ESS candidate is an attracting ESS; the population will always evolve towards this strategy, and once it is adopted by most members of the population it cannot be invaded by mutant strategies. Then, we extend the strategy sets and allow a player to choose any continuous distribution. We show that although this ESS candidate may no longer be an ESS, under fairly general conditions the population will tend towards it. This is done by characterizing types of strategies that if established in the population, can be invaded by this ESS candidate, and by presenting possible paths of transition from other types of common strategies to this ESS candidate.     

Arms races in evolution—An ESS to the opponent-independent costs game

The opponent-independent cost game is an animal contest in which the bigger and more heavily-armed opponent wins a disputed resource without significant fighting costs. A strategy is a choice of investment level in armament. Increasing armament is assumed to have fitness costs that are unrelated to contests; i.e. the cost of an individual's investment in arms is independent of the strategy played by an opponent.Previous work with this model showed that no ESS exists if a strategy prescribes an arms level exactly. This is equivalent to the notion that there is no environmental variation in the arms level attained by a given strategy. If environmental variation is introduced, a pure ESS can generally exist. A strategy is assumed to prescribe an exact investment cost, but this is translated into a probability distribution of arms levels attained, rather than an exact arms level. Increasing investment increases the mean of the arms level distribution. The ESS investment level depends both on how environmental factors distribute arms levels, and on the shape of the cost function (i.e. on the way that costs increase with investment); in some instances there is no ESS. Two types of model are investigated; in one fitness is additive (benefit-cost), in the other it is multiplicative (benefit × survivorship). The multiplicative model is likely to apply to the case where contests are between males for access to females. Here the ESS investment level (an ESS degree of risk that a male sustains as a result of armament) increases as fewer individuals guard the available resources. Thus sexual size dimorphism (male/female size) and relative male armament should increase as harem size increases. The ESS investment level will also be highest if most individuals are small and poorly armed, as would often be the case where size increases throughout life.The model can be applied to coevolutionary arms races between two classes of opponent, such as prey and predator, or parent and offspring. Here the ESS is likely to be a pair of ESS arms levels, one for each class of opponent.     

Strategic analysis in evolutionary genetics and the theory of games

This paper is written in memory of John Maynard Smith. In a brief survey it discusses essential aspects of how game theory in biology relates to its counterpart in economics, the major transition in game theory initiated by Maynard Smith, the discrepancies between genetic and phenotypic models in evolutionary biology, and a balanced way of reconciling these models. In addition, the paper discusses modern problems in understanding games at the genetic level using the examples of conflict between endosymbionts and their hosts, and the molecular interactions between parasites and the mammalian immune system.     

Evolutionary matrix games and optimization theory

An evolutionarily stable strategy (ESS) is only required to be capable of resisting invasion by rare mutant strategies. In contrast, an absolute invader strategy (AIS) is a rare mutant strategy that can invade any established strategy. We show that the predictions of the outcome of evolution made by optimization models are compatible with those made by the classical expected payoff comparisons in matrix games. We also show that if a matrix game has an AIS that AIS is unique and is also an ESS. But an ESS need not be an AIS. In pure-strategy submodels, an AIS need not be unique. An AIS of a matrix game has global asymptotic stability property in the game dynamics which involve only pure strategies including the AIS.     

The ESS under spatial variation with applications to sex allocation

I derive a new approximation which uses the backward Kolmogorov equation to describe evolution when individuals have variable numbers of offspring. This approximation is based on an explicit fixed population size assumption and therefore differs from previous models. I show that for individuals to accept an increase in the variance of offspring number, they must be compensated by an increase in mean offspring number. Based on this model and any given set of feasible alleles, an evolutionary stable strategy (ESS) can be found. Four types of ESS are possible and can be discriminated by graphical methods. These ESS values depend on population size, but population size can be reinterpreted as deme size in a structured population. I adapt this theory to the problem of sex allocation under variable returns to male and female function and derive the ESS sex allocation strategy. I show that allocation to the more variable sexual function should be reduced, but that this effect decreases as population size increases and as variability decreases. These results are compared with results from exact matrix models and computer simulations, all of which show strong congruence.     

Simulating natural selection as a culling mechanism on finite populations with the hawk-dove game

The behaviors of individuals and species are often explained in terms of evolutionary stable strategies (ESSs). The analysis of ESSs determines which, if any, combinations of behaviors cannot be invaded by alternative strategies. Two assumptions required to generate an ESS (i.e., an infinite population and payoffs described only on the average) do not hold under natural conditions. Previous experiments indicated that under more realistic conditions of finite populations and stochastic payoffs, populations may evolve in trajectories that are unrelated to an ESS, even in very simple games. The simulations offered here extend earlier research by employing truncation selection with random parental selection in a hawk-dove game. Payoffs are determined in pairwise contests using either the expected outcome, or the result of a random variable. In each case, however, the mean fraction of hawks over many generations and across many independent trials does not conform to the expected ESS. Implications of these results and philosophical underpinnings of ESS theory are offered.     

Strong evolutionary equilibrium and the war of attrition

In developing the concept of an evolutionarily stable strategy, Maynard Smith proposed formal conditions for stability. These conditions have since been shown to be neither necessary nor sufficient for evolutionary stability in finite populations. This paper provides a strong stability condition which is sensitive to the population size. It is then demonstrated that in the war of attrition with uncertain rewards there is a unique “strong evolutionary equilibrium” strategy. As the population becomes large this is shown to approach the solution strategy proposed by Bishop, Cannings and Maynard Smith.The analysis is then extended to wars of attrition between different populations. It is concluded that for such contests there is a whole family of potential strong evolutionary equilibria.     

Fighting for food: a dynamic version of the Hawk-Dove game

Summary The Hawk-Dove game (Maynard Smith, 1982) has been used to analyse conflicts over resources such as food. At the evolutionarily stable strategy (ESS), either a proportionp* of animals always play Hawk, or each animal has a probabilityp* of playing Hawk. We modify the standard Hawk-Dove game to include a state variable,x, that represents the animal's level of energy reserves. A strategy is now a rule for choosing an action as a function ofx and time of day. We consider a non-reproductive period and adopt the criterion of minimizing mortality over this period. We find the ESS, which has the form play Hawk if reserves are belowc* (t) at timet, otherwise play Dove. This ESS is very different from the ESS in the standard Hawk-Dove game. It is a pure ESS that depends on the animal's state and on time. Furthermore, it is characterized by the strong condition that any single mutant that does not adopt the ESS suffers a reduction in fitness. The standard Hawk-Dove game assumes pay-offs that are related to fitness; our approach starts from a definition of fitness and derives the pay-offs in the process of finding the ESS. When the environment becomes worse (e.g. food becomes less reliable or energy expenditure increases) the ESS changes in such a way as to increase the proportion of animals that will play Hawk.