Evolutionary dynamics of continuous strategy games on graphs and social networks under weak selection

Understanding the emergence of cooperation among selfish individuals has been a long-standing puzzle, which has been studied by a variety of game models. Most previous studies presumed that interactions between individuals are discrete, but it seems unrealistic in real systems. Recently, there are increasing interests in studying game models with a continuous strategy space. Existing research work on continuous strategy games mainly focuses on well-mixed populations. Especially, little theoretical work has been conducted on their evolutionary dynamics in a structured population. In the previous work (Zhong et al., BioSystems, 2012), we showed that under strong selection, continuous and discrete strategies have significantly different equilibrium and game dynamics in spatially structured populations. In this paper, we further study evolutionary dynamics of continuous strategy games under weak selection in structured populations. By using the fixation probability based stochastic dynamics, we derive exact conditions of natural selection favoring cooperation for the death–birth updating scheme. We also present a network gain decomposition of the game equilibrium, which might provide a new view of the network reciprocity in a quantitative way. Finally, we make a detailed comparison between games using discrete and continuous strategies. As compared to the former, we find that for the latter (i) the same selection conditions are derived for the general 2 × 2 game; especially, the rule b/c > k in a simplified Prisoner's Dilemma is valid as well; however, (ii) for a coordination game, interestingly, the risk-dominant strategy is disfavored. Numerical simulations have also been conducted to validate our results.     

Strategy selection in structured populations

Evolutionary game theory studies frequency dependent selection. The fitness of a strategy is not constant, but depends on the relative frequencies of strategies in the population. This type of evolutionary dynamics occurs in many settings of ecology, infectious disease dynamics, animal behavior and social interactions of humans. Traditionally evolutionary game dynamics are studied in well-mixed populations, where the interaction between any two individuals is equally likely. There have also been several approaches to study evolutionary games in structured populations. In this paper we present a simple result that holds for a large variety of population structures. We consider the game between two strategies, A and B, described by the payoff matrix . We study a mutation and selection process. For weak selection strategy A is favored over B if and only if σa+b>c+σd. This means the effect of population structure on strategy selection can be described by a single parameter, σ. We present the values of σ for various examples including the well-mixed population, games on graphs, games in phenotype space and games on sets. We give a proof for the existence of such a σ, which holds for all population structures and update rules that have certain (natural) properties. We assume weak selection, but allow any mutation rate. We discuss the relationship between σ and the critical benefit to cost ratio for the evolution of cooperation. The single parameter, σ, allows us to quantify the ability of a population structure to promote the evolution of cooperation or to choose efficient equilibria in coordination games.     

Fixation probabilities in evolutionary game dynamics with a two-strategy game in finite diploid populations

Fixation processes in evolutionary game dynamics in finite diploid populations are investigated. Traditionally, frequency dependent evolutionary dynamics is modeled as deterministic replicator dynamics. This implies that the infinite size of the population is assumed implicitly. In nature, however, population sizes are finite. Recently, stochastic processes in finite populations have been introduced in order to study finite size effects in evolutionary game dynamics. One of the most significant studies on evolutionary dynamics in finite populations was carried out by Nowak et al. which describes “one-third law” [Nowak, et al., 2004. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 646-650]. It states that under weak selection, if the fitness of strategy α is greater than that of strategy β when α has a frequency , strategy α fixates in a β-population with selective advantage. In their study, it is assumed that the inheritance of strategies is asexual, i.e. the population is haploid. In this study, we apply their framework to a diploid population that plays a two-strategy game with two ESSs (a bistable game). The fixation probability of a mutant allele in this diploid population is derived. A “three-tenth law” for a completely recessive mutant allele and a “two-fifth law” for a completely dominant mutant allele are found; other cases are also discussed.     

Mutation-selection equilibrium in games with mixed strategies

We develop a new method for studying stochastic evolutionary game dynamics of mixed strategies. We consider the general situation: there are n pure strategies whose interactions are described by an n×n payoff matrix. Players can use mixed strategies, which are given by the vector (p₁,…,p_n). Each entry specifies the probability to use the corresponding pure strategy. The sum over all entries is one. Therefore, a mixed strategy is a point in the simplex S_n. We study evolutionary dynamics in a well-mixed population of finite size. Individuals reproduce proportional to payoff. We consider the case of weak selection, which means the payoff from the game is only a small contribution to overall fitness. Reproduction can be subject to mutation; a mutant adopts a randomly chosen mixed strategy. We calculate the average abundance of every mixed strategy in the stationary distribution of the mutation-selection process. We find the crucial conditions that specify if a strategy is favored or opposed by selection. One condition holds for low mutation rate, another for high mutation rate. The result for any mutation rate is a linear combination of those two. As a specific example we study the Hawk-Dove game. We prove general statements about the relationship between games with pure and with mixed strategies.     

Gradients for the evolution of bimatrix games

The evolutionary dynamics of bimatrix games is studied for rescaled partnership games and zero sum games. The former case leads to gradient systems. The selection equations for sexual and asexual reproduction of genotypes corresponding to mixed strategies are analysed. As examples, the origin of anisogamy and cyclic chases for predator-prey coevolution are studied.     

The interaction energy of charged filaments in an electrolyte: Results for all filament spacings

Recent studies have explored interactions between evolutionary game dynamics and population structure. Yet most studies so far mainly paid attention to unweighted and static networks. Here we explore evolutionary games played on dynamically weighted networks. Players update their strategies according to the payoffs they obtain. Players also update weights of their adjacent links depending on payoffs they gain through those links; profitable links are reinforced whereas unprofitable ones are weakened. The system is characterized by two time scales, the one for strategy update, β_S, and the other for weight adjustment, β_W. We find that, under a mean-field approximation, the asymptotic behavior of the system is described by the replicator equation with an effective payoff matrix, which is a combination of the original game matrix A and its transpose, A^T. Both analytical and numerical results show that such an adaptive weight adjustment mechanism dramatically promotes evolution of cooperation.     

Mutation-selection equilibrium in games with multiple strategies

In evolutionary games the fitness of individuals is not constant but depends on the relative abundance of the various strategies in the population. Here we study general games among n strategies in populations of large but finite size. We explore stochastic evolutionary dynamics under weak selection, but for any mutation rate. We analyze the frequency dependent Moran process in well-mixed populations, but almost identical results are found for the Wright-Fisher and Pairwise Comparison processes. Surprisingly simple conditions specify whether a strategy is more abundant on average than 1/n, or than another strategy, in the mutation-selection equilibrium. We find one condition that holds for low mutation rate and another condition that holds for high mutation rate. A linear combination of these two conditions holds for any mutation rate. Our results allow a complete characterization of n×n games in the limit of weak selection.     

Global mutations and local mutations have very different effects on evolution, illustrated by mixed strategies of asymmetric binary games

We study the evolutionary effect of rare mutations causing global changes in traits. We consider asymmetric binary games between two players. The first player takes two alternative options with probability x and 1−x; and the second player takes options with probability y and 1−y. Due to natural selection and recurrent mutation, the population evolves to have broad distributions of x and y. We analyze three cases showing qualitatively different dynamics, exemplified by (1) vigilance-intrusion game, (2) asymmetric hawk-dove game and (3) cleaner-client game. We found that the evolutionary outcome is strongly dependent upon the distribution of mutants’ traits, more than the mutation rates. For example in the vigilance-intrusion game, the evolutionary dynamics show a perpetual stable oscillation if mutants are always close to the parent (local-mutation mode), whilst the population converges to a stable equilibrium distribution if mutants can be quite different from the parent (global-mutation mode), even for extremely low mutation rate. When common local mutations and rare global mutations occur simultaneously, the evolutionary outcome is controlled by the latter.     

On Nash Equilibrium and Evolutionarily Stable States That Are Not Characterised by the Folk Theorem

In evolutionary game theory, evolutionarily stable states are characterised by the folk theorem because exact solutions to the replicator equation are difficult to obtain. It is generally assumed that the folk theorem, which is the fundamental theory for non-cooperative games, defines all Nash equilibria in infinitely repeated games. Here, we prove that Nash equilibria that are not characterised by the folk theorem do exist. By adopting specific reactive strategies, a group of players can be better off by coordinating their actions in repeated games. We call it a type-k equilibrium when a group of k players coordinate their actions and they have no incentive to deviate from their strategies simultaneously. The existence and stability of the type-k equilibrium in general games is discussed. This study shows that the sets of Nash equilibria and evolutionarily stable states have greater cardinality than classic game theory has predicted in many repeated games.     

Unpredictability induced by unfocused games in evolutionary game dynamics

Evolutionary game theory is a basis of replicator systems and has applications ranging from animal behavior and human language to ecosystems and other hierarchical network systems. Most studies in evolutionary game dynamics have focused on a single game, but, in many situations, we see that many games are played simultaneously. We construct a replicator equation with plural games by assuming that a reward of a player is a simple summation of the reward of each game. Even if the numbers of the strategies of the games are different, its dynamics can be described in one replicator equation. We here show that when players play several games at the same time, the fate of a single game cannot be determined without knowing the structures of the whole other games. The most absorbing fact is that even if a single game has a ESS (evolutionary stable strategy), the relative frequencies of strategies in the game does not always converge to the ESS point when other games are played simultaneously.     

Emergence of Super Cooperation of Prisoner’s Dilemma Games on Scale-Free Networks

Recently, the authors proposed a quantum prisoner’s dilemma game based on the spatial game of Nowak and May, and showed that the game can be played classically. By using this idea, we proposed three generalized prisoner’s dilemma (GPD, for short) games based on the weak Prisoner’s dilemma game, the full prisoner’s dilemma game and the normalized Prisoner’s dilemma game, written by GPD_W, GPD_F and GPD_N respectively. Our games consist of two players, each of which has three strategies: cooperator (C), defector (D) and super cooperator (denoted by Q), and have a parameter γ to measure the entangled relationship between the two players. We found that our generalised prisoner’s dilemma games have new Nash equilibrium principles, that entanglement is the principle of emergence and convergence (i.e., guaranteed emergence) of super cooperation in evolutions of our generalised prisoner’s dilemma games on scale-free networks, that entanglement provides a threshold for a phase transition of super cooperation in evolutions of our generalised prisoner’s dilemma games on scale-free networks, that the role of heterogeneity of the scale-free networks in cooperations and super cooperations is very limited, and that well-defined structures of scale-free networks allow coexistence of cooperators and super cooperators in the evolutions of the weak version of our generalised prisoner’s dilemma games.     

Coevolution of trustful buyers and cooperative sellers in the trust game

Many online marketplaces enjoy great success. Buyers and sellers in successful markets carry out cooperative transactions even if they do not know each other in advance and a moral hazard exists. An indispensable component that enables cooperation in such social dilemma situations is the reputation system. Under the reputation system, a buyer can avoid transacting with a seller with a bad reputation. A transaction in online marketplaces is better modeled by the trust game than other social dilemma games, including the donation game and the prisoner's dilemma. In addition, most individuals participate mostly as buyers or sellers; each individual does not play the two roles with equal probability. Although the reputation mechanism is known to be able to remove the moral hazard in games with asymmetric roles, competition between different strategies and population dynamics of such a game are not sufficiently understood. On the other hand, existing models of reputation-based cooperation, also known as indirect reciprocity, are based on the symmetric donation game. We analyze the trust game with two fixed roles, where trustees (i.e., sellers) but not investors (i.e., buyers) possess reputation scores. We study the equilibria and the replicator dynamics of the game. We show that the reputation mechanism enables cooperation between unacquainted buyers and sellers under fairly generous conditions, even when such a cooperative equilibrium coexists with an asocial equilibrium in which buyers do not buy and sellers cheat. In addition, we show that not many buyers may care about the seller's reputation under cooperative equilibrium. Buyers' trusting behavior and sellers' reputation-driven cooperative behavior coevolve to alleviate the social dilemma.     

Ecological public goods games: cooperation and bifurcation

The Public Goods Game is one of the most popular models for studying the origin and maintenance of cooperation. In its simplest form, this evolutionary game has two regimes: defection goes to fixation if the multiplication factor r is smaller than the interaction group size N, whereas cooperation goes to fixation if the multiplication factor r is larger than the interaction group size N. Hauert et al. [Hauert, C., Holmes, M., Doebeli, M., 2006a. Evolutionary games and population dynamics: Maintenance of cooperation in public goods games. Proc. R. Soc. Lond. B 273, 2565-2570] have introduced the Ecological Public Goods Game by viewing the payoffs from the evolutionary game as birth rates in a population dynamic model. This results in a feedback between ecological and evolutionary dynamics: if defectors are prevalent, birth rates are low and population densities decline, which leads to smaller interaction groups for the Public Goods game, and hence to dominance of cooperators, with a concomitant increase in birth rates and population densities. This feedback can lead to stable co-existence between cooperators and defectors. Here we provide a detailed analysis of the dynamics of the Ecological Public Goods Game, showing that the model exhibits various types of bifurcations, including supercritical Hopf bifurcations, which result in stable limit cycles, and hence in oscillatory co-existence of cooperators and defectors. These results show that including population dynamics in evolutionary games can have important consequences for the evolutionary dynamics of cooperation.     

How is the equilibrium of continuous strategy game different from that of discrete strategy game?

Cooperation in the prisoner's dilemma (PD) played on various networks has been explained by so-called network reciprocity. Most of the previous studies presumed that players can offer either cooperation (C) or defection (D). This discrete strategy seems unrealistic in the real world, since actual provisions might not be discrete, but rather continuous. This paper studies the differences between continuous and discrete strategies in two aspects under the condition that the payoff function of the former is a linear interpolation of the payoff matrix of the latter. The first part of this paper proves theoretically that for two-player games, continuous and discrete strategies have different equilibria and game dynamics in a well-mixed but finite population. The second part, conducting a series of numerical experiments, reveals that such differences become considerably large in the case of PD games on networks. Furthermore, it shows, using the Wilcoxon sign-rank test, that continuous and discrete strategy games are statistically significantly different in terms of equilibria. Intensive discussion by comparing these two kinds of games elucidates that describing a strategy as a real number blunts D strategy invasion to C clusters on a network in the early stage of evolution. Thus, network reciprocity is enhanced by the continuous strategy.     

Stable Polyglutamine Dimers Can Contain β-Hairpins with Interdigitated Side Chains—But Not α-Helices, β-Nanotubes, β-Pseudohelices,or Steric Zippers

A common thread connecting nine fatal neurodegenerative protein aggregation diseases is an abnormally expanded polyglutamine tract found in the respective proteins. Although the structure of this tract in the large mature aggregates is increasingly well described, its structure in the small early aggregates remains largely unknown. As experimental evidence suggests that the most toxic species along the aggregation pathway are the small early ones, developing strategies to alleviate disease pathology calls for understanding the structure of polyglutamine peptides in the early stages of aggregation. Here, we present a criterion, grounded in available experimental data, that allows for using kinetic stability of dimers to assess whether a given polyglutamine conformer can be on the aggregation path. We then demonstrate that this criterion can be assessed using present-day molecular dynamics simulations. We find that although the α-helical conformer of polyglutamine is very stable, dimers of α-helices lack the kinetic stability necessary to support further oligomerization. Dimers of steric zipper, β-nanotube, and β-pseudohelix conformers are also too short-lived to initiate aggregation. The β-hairpin-containing conformers, instead, invariably form very stable dimers when their side chains are interdigitated. Combining these findings with the implications of recent solid-state NMR data on mature fibrils, we propose a possible pathway for the initial stages of polyglutamine aggregation, in which β-hairpin-containing conformers act as templates for fibril formation.     

Runaway social games,genetic cycles driven by alternative male and female strategies,and the origin of morphs

Analysis of evolutionarily stable strategies (ESS) and decade-long field studies indicate that two color morphs of female side-blotched lizards exhibit density- and frequency-dependent strategies. Orange females are r-strategists: they lay large clutches of small progeny that are favored at low density. Conversely, yellow females are K-strategists: they lay small clutches of large progeny that are favored when carrying capacity is exceeded and the population crashes to low density. Interactions among three male morphs resembles a rock-paper-scissors (RPS) game. Fertilization success of males depends on frequency of neighboring morphs. Orange males usurp territory from blue neighbors and thereby mate with many females. However, orange males are vulnerable to cuckoldry by sneaky yellow males that mimic females. The yellow strategy is thwarted in turn by the mate-guarding strategy of blue. Sinervo and Lively (1996) developed a simple asexual model of the RPS game. Here, we model the dynamics of male and female morphs with one- and two-locus genetic models. Male and female games were considered in isolation and modeled as games that were genetically coupled by the same locus. Parameters for payoff matrices, which describe the force of frequency-dependent selection in ESS games, were estimated from free-ranging animals. Period of cycles in nature was 5 years for males and 2 years for females. Only the one locus model with three alleles (o, b, y) was capable of driving rapid cycles in male and female games. Furthermore, the o allele must be dominant to the y allele in females. Finally, the amplitude of male cycles was only reproduced in genetic models which allowed for irreversible plasticity of by genotypes, which is consistent with hormonally-induced changes that transform some males with yellow to dark blue. We also critique experimental designs that are necessary to detect density- and frequency-dependent selection in nature. Finally, runaway ESS games are discussed in the context of self-reinforcing genetic correlations that build and promote the formation of morphotypic variation.     

A theory for the evolutionary game

We present an evolutionary game theory. This theory differs in several respects from current theories related to Maynard Smith's pioneering work on evolutionary stable strategies (ESS). Most current work deals with two person matrix games. For these games the strategy set is finite. We consider evolutionary games which are defined over a continuous strategy set and which permit any number of players. Matrix games are included as a bilinear continuous game. However, under our definition, such games will not posses an ESS on the interior of the strategy set. We extend previous work on continuous games by developing an ESS definition which permits the ESS to be composed of a coalition of several strategies. This definition requires that the coalition must not only be stable with respect to perturbations in strategy frequencies which comprise the coalition, but the coalition must also satisfy the requirement that no mutant strategies can invade. Ecological processes are included in the model by explicitly considering population size and density dependent selection.     

Game dynamics and evolutionary transitions

An evolutionary model based on the Taylor-Jonker game dynamics is presented. A set of strategies is compatible if there exists a dynamical equilibrium between its members and there is an evolutionary transition to another compatible set if new mutant strategies bring about a passage to another equilibrium. We apply these concepts to supergame strategies, which play repeatedly a given matrix game and at each time step choose their pure strategy according to the preceding moves of the opponent. We investigate the patterns of evolution in zero-sum games, games of partnership, the prisoner's dilemma and the hawkdove game.     

Evolution of Conformity in Social Dilemmas

People often deviate from their individual Nash equilibrium strategy in game experiments based on the prisoner’s dilemma (PD) game and the public goods game (PGG), whereas conditional cooperation, or conformity, is supported by the data from these experiments. In a complicated environment with no obvious “dominant” strategy, conformists who choose the average strategy of the other players in their group could be able to avoid risk by guaranteeing their income will be close to the group average. In this paper, we study the repeated PD game and the repeated m-person PGG, where individuals’ strategies are restricted to the set of conforming strategies. We define a conforming strategy by two parameters, initial action in the game and the influence of the other players’ choices in the previous round. We are particularly interested in the tit-for-tat (TFT) strategy, which is the well-known conforming strategy in theoretical and empirical studies. In both the PD game and the PGG, TFT can prevent the invasion of non-cooperative strategy if the expected number of rounds exceeds a critical value. The stability analysis of adaptive dynamics shows that conformity in general promotes the evolution of cooperation, and that a regime of cooperation can be established in an AllD population through TFT-like strategies. These results provide insight into the emergence of cooperation in social dilemma games.     

Evolutionary conservation of a testes-specific proteasome subunit gene in Drosophila

Proteasomes are large multisubunit particles that act as the proteolytic machinery for the ubiquitin-dependent proteolytic pathway. The core of this complex, the 20S proteasome, is made up of seven α-type and seven β-type subunits, arranged in an (α1–α7)(β1–β7)(β1–β7)(α1–α7) configuration. Previous work had shown that there exist alternative isoforms of the Drosophila melanogaster α4-type subunit, encoded by two distinct genes, α4t1_dm and α4t2_dm, and that these are expressed exclusively in the germline of the testes. We sought to investigate the evolutionary conservation of this phenomenon by screening for orthologs of the α4-type gene family in the distantly related Drosophila species, D. virilis. We isolated the D. virilis orthologs of the somatically expressed gene, α4_dm, and the testes-specific gene, α4t2_dm. We failed to find an ortholog of the other testes-specific gene, α4t1_dm. The α4_dv gene maps to the X chromosome at 12A-C, its product shares 90% amino acid identity with α4_dm, and it is expressed at high levels in both males and females. The other gene, α4t_dv, encodes a protein most similar to the testes-specific α4t2_dm proteasome subunit (59% a.a. identity), and it maps to position 27 on chomosome 2. The expression of the α4t_dv gene is testes-specific, like that of α4t2_dm. The existence of testes-specific α4-type subunits in two widely diverged subgenera of Drosophila suggests that these subunit isoforms have important functional roles in spermatogenesis.