Pavlovian Prisoner's Dilemma—Analytical results, the quasi-regular phase and spatio-temporal patterns

The Prisoner's Dilemma (PD) game is applied in several research fields due to the emergence of cooperation among selfish players. In this work the PD is studied in a one-dimensional lattice, where each cell represents a player, which in turn can interact with the neighbors playing the PD (cooperate or defect). The update of states adopts the Pavlovian Evolutionary Strategy (PES) or Darwinian Evolutionary Strategy (DES). Adopting PES, if a player receives a positive payoff greater than his/her aspiration level, he/she keeps the current state, and switches otherwise. Adopting DES, player compares his/her payoff with payoff of opponents. If it is not the highest, he/she copies the state of fittest player, switching the state if it is different of his/her current state. The critical temptation values obtained analytically are reported, and the cluster patterns that emerge from the interactions among the players are shown. Also we defined analytical functions that calculate the maximum/minimum size of defective/cooperative clusters. Also, the parameter space is explored with exhaustive computational simulations, which confirm the analytical results and reinforce that Pavlovian strategy foments cooperation among players. In steady state, system can reach the cooperative or quasi-regular phases, when adopting the PES, and cooperative, defective or chaotic phases, adopting the DES. The new quasi-regular phase occurs when several players switch their states in each round, but the proportion of cooperators does not show significant variation. Additionally, the present work shows that the lowest temptation level (T=1) may be considered a trivial case only for the particular case where the players interact with only one neighbor, otherwise system presents the same features that for higher temptation values.     

Equilibrium selection in evolutionary games with random matching of players

We discuss stochastic dynamics of populations of individuals playing games. Our models possess two evolutionarily stable strategies: an efficient one, where a population is in a state with the maximal payoff (fitness) and a risk-dominant one, where players are averse to risks. We assume that individuals play with randomly chosen opponents (they do not play against average strategies as in the standard replicator dynamics). We show that the long-run behavior of a population depends on its size and the mutation level.     

The impact of random frequency-dependent mutations on the average population fitness

ABSTRACT: BACKGROUND: In addition to selection, the process of evolution is accompanied by stochastic effects, such as changing environmental conditions, genetic drift and mutations. Commonly it is believed that without genetic drift, advantageous mutations quickly fixate in a halpoid population due to strong selection and lead to a continuous increase of the average fitness. This conclusion is based on the assumption of constant fitness. However, for frequency dependent fitness, where the fitness of an individual depends on the interactions with other individuals in the population, this does not hold. RESULTS: We propose a mathematical model that allows to understand the consequences of random frequency dependent mutations on the dynamics of an infinite large population. The frequencies of different types change according to the replicator equations and the fitness of a mutant is random and frequency dependent. To capture the interactions of different types, we employ a payoff matrix of variable size and thus are able to accommodate an arbitrary number of mutations. We assume that at most one mutant type arises at a time. The payoff entries to describe the mutant type are random variables obeying a probability distribution which is related to the fitness of the parent type. CONCLUSIONS: We show that a random mutant can decrease the average fitness under frequency dependent selection, based on analytical results for two types, and on simulations for n types. Interestingly, in the case of at most two types the probabilities to increase or decrease the average fitness are independent of the concrete probability density function. Instead, they only depend on the probability that the payoff entries of the mutant are larger than the payoff entries of the parent type.     

Evolution of Cooperation in Spatial Traveler's Dilemma Game

Traveler''s dilemma (TD) is one of social dilemmas which has been well studied in the economics community, but it is attracted little attention in the physics community. The TD game is a two-person game. Each player can select an integer value between and () as a pure strategy. If both of them select the same value, the payoff to them will be that value. If the players select different values, say and (), then the payoff to the player who chooses the small value will be and the payoff to the other player will be . We term the player who selects a large value as the cooperator, and the one who chooses a small value as the defector. The reason is that if both of them select large values, it will result in a large total payoff. The Nash equilibrium of the TD game is to choose the smallest value . However, in previous behavioral studies, players in TD game typically select values that are much larger than , and the average selected value exhibits an inverse relationship with . To explain such anomalous behavior, in this paper, we study the evolution of cooperation in spatial traveler''s dilemma game where the players are located on a square lattice and each player plays TD games with his neighbors. Players in our model can adopt their neighbors'' strategies following two standard models of spatial game dynamics. Monte-Carlo simulation is applied to our model, and the results show that the cooperation level of the system, which is proportional to the average value of the strategies, decreases with increasing until is greater than the critical value where cooperation vanishes. Our findings indicate that spatial reciprocity promotes the evolution of cooperation in TD game and the spatial TD game model can interpret the anomalous behavior observed in previous behavioral experiments.     

Leading the Game,Losing the Competition: Identifying Leaders and Followers in a Repeated Game

We explore a new method for identifying leaders and followers, LF, in repeated games by analyzing an experimental, repeated (50 rounds) game where Row player shifts the payoff between small and large values–a type of “investor” and Column player determines who gets the payoff–a type of “manager”. We found that i) the Investor (Row) most often is a leading player and the manager (Column) a follower. The longer the Investor leads the game, the higher is both player’s payoff. Surprisingly however, it is always the Manager that achieves the largest payoff. ii) The game has an efficient cooperative strategy where the players alternate in receiving a high payoff, but the players never identify, or accept, that strategy. iii) Under the assumption that the information used by the players is closely associated with the leader- follower sequence, and that information is available before the player’s decisions are made, the players switched LF- strategy primarily as a function of information on the Investor’s investment and moves and secondly as a function of the Manager’s payoff.     

Transforming the dilemma

How does natural selection lead to cooperation between competing individuals? The Prisoner's Dilemma captures the essence of this problem. Two players can either cooperate or defect. The payoff for mutual cooperation, R, is greater than the payoff for mutual defection, P. But a defector versus a cooperator receives the highest payoff, T, where as the cooperator obtains the lowest payoff, S. Hence, the Prisoner's Dilemma is defined by the payoff ranking T > R > P > S . In a well‐mixed population, defectors always have a higher expected payoff than cooperators, and therefore natural selection favors defectors. The evolution of cooperation requires specific mechanisms. Here we discuss five mechanisms for the evolution of cooperation: direct reciprocity, indirect reciprocity, kin selection, group selection, and network reciprocity (or graph selection). Each mechanism leads to a transformation of the Prisoner's Dilemma payoff matrix. From the transformed matrices, we derive the fundamental conditions for the evolution of cooperation. The transformed matrices can be used in standard frameworks of evolutionary dynamics such as the replicator equation or stochastic processes of game dynamics in finite populations.     

Win-stay, lose-shift strategies for repeated games-memory length, aspiration levels and noise.

Win-stay, lose-shift, the principle to retain a successful action is a simple and general learning rule that can be applied to all types of repeated decision problems. In this paper I consider win-stay, lose-shift strategies with diverse memory sizes and strategies that adapt their aspiration levels, i.e. the payoff level considered as "success". I study their evolution for the Prisoner's Dilemma, as well as in a rapidly changing environment, where a randomly selected game is assigned to the players. For win-stay, lose-shift strategies with memory one the average payoffs are computed and their evolutionary stability is discussed. Using computer simulations I show that the win-stay, lose-shift strategies with longer memory are very successful both for the Prisoner's Dilemma, where cooperation dominates even for high noise levels, and the randomly assigned games, where the players achieve nearly the expected Pareto optimal payoffs. I discuss the impact of noise and show that the memory length of the players increases with the noise level. These results indicate that the win-stay, lose-shift principle is a very successful strategy in repeated games with noise.     

A generalized adaptive dynamics framework can describe the evolutionary Ultimatum Game

Adaptive dynamics describes the evolution of games where the strategies are continuous functions of some parameters. The standard adaptive dynamics framework assumes that the population is homogeneous at any one time. Differential equations point to the direction of the mutant that has maximum payoff against the resident population. The population then moves towards this mutant. The standard adaptive dynamics formulation cannot deal with games in which the payoff is not differentiable. Here we present a generalized framework which can. We assume that the population is not homogeneous but distributed around an average strategy. This approach can describe the long-term dynamics of the Ultimatum Game and also explain the evolution of fairness in a one-parameter Ultimatum Game.     

Threshold Games and Cooperation on Multiplayer Graphs

ObjectiveThe study investigates the effect on cooperation in multiplayer games, when the population from which all individuals are drawn is structured—i.e. when a given individual is only competing with a small subset of the entire population.MethodTo optimize the focus on multiplayer effects, a class of games were chosen for which the payoff depends nonlinearly on the number of cooperators—this ensures that the game cannot be represented as a sum of pair-wise interactions, and increases the likelihood of observing behaviour different from that seen in two-player games. The chosen class of games are named “threshold games”, and are defined by a threshold, M > 0, which describes the minimal number of cooperators in a given match required for all the participants to receive a benefit. The model was studied primarily through numerical simulations of large populations of individuals, each with interaction neighbourhoods described by various classes of networks.ResultsWhen comparing the level of cooperation in a structured population to the mean-field model, we find that most types of structure lead to a decrease in cooperation. This is both interesting and novel, simply due to the generality and breadth of relevance of the model—it is likely that any model with similar payoff structure exhibits related behaviour. More importantly, we find that the details of the behaviour depends to a large extent on the size of the immediate neighbourhoods of the individuals, as dictated by the network structure. In effect, the players behave as if they are part of a much smaller, fully mixed, population, which we suggest an expression for.

Highlights

• Observed behaviour depends on the size of each player’s immediate interaction neighbourhood.
• When the number of players is much larger than the number of required cooperators, average payoff decreases.
• Most network structures lead to a decrease in cooperation compared to the fully mixed case.

     

How Feeling Betrayed Affects Cooperation

For a population of interacting self-interested agents, we study how the average cooperation level is affected by some individuals'' feelings of being betrayed and guilt. We quantify these feelings as adjusted payoffs in asymmetric games, where for different emotions, the payoff matrix takes the structure of that of either a prisoner''s dilemma or a snowdrift game. Then we analyze the evolution of cooperation in a well-mixed population of agents, each of whom is associated with such a payoff matrix. At each time-step, an agent is randomly chosen from the population to update her strategy based on the myopic best-response update rule. According to the simulations, decreasing the feeling of being betrayed in a portion of agents does not necessarily increase the level of cooperation in the population. However, this resistance of the population against low-betrayal-level agents is effective only up to some extend that is explicitly determined by the payoff matrices and the number of agents associated with these matrices. Two other models are also considered where the betrayal factor of an agent fluctuates as a function of the number of cooperators and defectors that she encounters. Unstable behaviors are observed for the level of cooperation in these cases; however, we show that one can tune the parameters in the function to make the whole population become cooperative or defective.     

Replicator dynamics for optional public good games

The public goods game represents a straightforward generalization of the prisoner's dilemma to an arbitrary number of players. Since the dominant strategy is to defect, both classical and evolutionary game theory predict the asocial outcome that no player contributes to the public goods. In contrast to the compulsory public goods game, optional participation provides a natural way to avoid deadlocks in the state of mutual defection. The three resulting strategies--collaboration or defection in the public goods game, as well as not joining at all--are studied by means of a replicator dynamics, which can be completely analysed in spite of the fact that the payoff terms are nonlinear. If cooperation is valuable enough, the dynamics exhibits a rock-scissors-paper type of cycling between the three strategies, leading to sizeable average levels of cooperation in the population. Thus, voluntary participation makes cooperation feasible. But for each strategy, the average payoff value remains equal to the earnings of those not participating in the public goods game.     

One-third rules with equality: Second-order evolutionary stability conditions in finite populations

The one-third law of evolutionary dynamics [Nowak et al. 2004. Emergence of cooperation and evolutionary stability in finite populations. Nature 428, 246-650] describes a robustness criterion for evolution in a finite population: If at an A-frequency of 1/3, the fitness of an A player is greater (smaller) than the fitness of a B player, then a single A mutant that appears in a population of otherwise all B has a fixation probability greater (smaller) than the neutral threshold 1/N, the inverse population size. We examine the case where at an A-frequency of 1/3, the fitness of an A player is exactly equal to the fitness of a B player. We find that in this case the relative magnitude of the cross payoffs matters: If the payoff of A against B is larger (smaller) than the payoff of B against A, then a single A mutant has a fixation probability larger (smaller) than 1/N. If the cross payoffs coincide, we are in the special case of a partnership game, where the deviation cost from an inefficient equilibrium is exactly balanced by the potential gain of switching to the payoff dominant equilibrium. We show that in this case the fixation probability of A is lower than 1/N. Finally, we illustrate our findings by a language game with differentiated costs of signals.     

Interaction characteristics as evolutionary features for the spatial Prisoner’s Dilemma in a population modeled by continuous probabilistic cellular automata and evolutionary algorithm

Evolutionary games usually take into consideration individuals’ strategies as the transformative characteristic which leads to the evolution of the population. Here, besides the strategies, interaction aspects are also considered as evolutionary attributes which can change over time as the replacement dynamic renovates the population choosing locally better individuals to reproduce. The population is modeled by cellular automata, interactions by the Prisoner’s Dilemma game and the replacement process is ruled by two versions of death-birth dynamic. Although the average payoff per game is considered as the fitness for choosing better individuals, the number of games per time step and a maximum radius of interaction with neighbours are also present in the individual’s chromosome which is passed to the next generation. Numerical simulations show that individual interaction properties and cooperation level are linked to the version of death-birth dynamic used and the game payoff. For instance, when the fitness bias is on the death event, individuals have more interactions in a larger radius, and the cooperation level is usually lower than the case where the fitness bias is on the birth event. Also, the individuals’ interaction profiles are heterogeneous, and cooperative individuals form clusters in the lattice to protect themselves.     

Effect of the presence of empty sites on the evolution of cooperation by costly punishment in spatial games

Cooperation and spiteful behavior are still evolutionary puzzles. Costly punishment, for which the game payoff is the same as that of spiteful behavior, is one mechanism for promoting the evolution of cooperation. A spatially structured population facilitates the evolution of either cooperation or spite/punishment if cooperation is linked explicitly or implicitly with spite/punishment; a cooperator cooperates with another cooperator and punishes/spites the other type of player. Different updating rules in the evolutionary game produce different evolutionary outcomes: with one updating rule—the score-dependent viability model, in which a player dies with a probability inversely proportional to the game score and the resulting unoccupied site is colonized by one player chosen randomly—the evolution of spite/punishment is promoted more than with the other updating rule—the score-dependent fertility model, in which, after a player dies randomly, the site is colonized by a player with a higher game score. If the population has empty sites, spiteful players or punishers should have less chance to interact with others and then spite/punish others. Thus the presence of empty sites would affect the evolutionary dynamics of spite/punishment. Here, we investigated whether the presence of empty sites discourages the evolution of spite/punishment in both a lattice-structured population and a completely mixing population where players interact with others randomly, especially when the score-dependent viability model is adopted. In the lattice-structured population adopting this viability model, the presence of empty sites promoted the evolution of cooperation and did not reduce the effect of spite/punishment. In the completely mixing population, the presence of empty sites did not promote evolution of cooperation by punishment. The evolutionary dynamics of the score-dependent viability model with empty sites were close to those of the score-dependent fertility model.     

Direct Reciprocity in Spatial Populations Enhances R-Reciprocity As Well As ST-Reciprocity

As is well-known, spatial reciprocity plays an important role in facilitating the emergence of cooperative traits, and the effect of direct reciprocity is also obvious for explaining the cooperation dynamics. However, how the combination of these two scenarios influences cooperation is still unclear. In the present work, we study the evolution of cooperation in 2×2 games via considering both spatial structured populations and direct reciprocity driven by the strategy with 1-memory length. Our results show that cooperation can be significantly facilitated on the whole parameter plane. For prisoner''s dilemma game, cooperation dominates the system even at strong dilemma, where maximal social payoff is still realized. In this sense, R-reciprocity forms and it is robust to the extremely strong dilemma. Interestingly, when turning to chicken game, we find that ST-reciprocity is also guaranteed, through which social average payoff and cooperation is greatly enhanced. This reciprocity mechanism is supported by mean-field analysis and different interaction topologies. Thus, our study indicates that direct reciprocity in structured populations can be regarded as a more powerful factor for the sustainability of cooperation.     

Mutation in evolutionary games can increase average fitness at equilibrium

We study game dynamical interactions between two strategies, A and B, and analyse whether the average fitness of the population at equilibrium can be increased by adding mutation from A to B. Classifying all two by two games with payoff matrix [(a,b),(c,d)], we show that mutation from A to B enhances the average fitness of the whole population (i) if both a and d are less than (b + c)/2 and (ii) if c is less than b. Furthermore, we study conditions for maximizing the productivity of strategy A, and we analyse the effect of mutations in both directions. Depending on the biological system, a mutation in an evolutionary game can be interpreted as a genetic alteration, a cellular differentiation, a change in gene expression, an accidental or deliberate modification in cultural transmission, or a learning error. In a cultural context, our results indicate that the equilibrium payoff of the population can be increased if players sometimes choose the strategy with lower payoff. In a genetic context, we have shown that for frequency-dependent selection mutation can enhance the average fitness of the population at equilibrium.     

How is the equilibrium of continuous strategy game different from that of discrete strategy game?

Cooperation in the prisoner's dilemma (PD) played on various networks has been explained by so-called network reciprocity. Most of the previous studies presumed that players can offer either cooperation (C) or defection (D). This discrete strategy seems unrealistic in the real world, since actual provisions might not be discrete, but rather continuous. This paper studies the differences between continuous and discrete strategies in two aspects under the condition that the payoff function of the former is a linear interpolation of the payoff matrix of the latter. The first part of this paper proves theoretically that for two-player games, continuous and discrete strategies have different equilibria and game dynamics in a well-mixed but finite population. The second part, conducting a series of numerical experiments, reveals that such differences become considerably large in the case of PD games on networks. Furthermore, it shows, using the Wilcoxon sign-rank test, that continuous and discrete strategy games are statistically significantly different in terms of equilibria. Intensive discussion by comparing these two kinds of games elucidates that describing a strategy as a real number blunts D strategy invasion to C clusters on a network in the early stage of evolution. Thus, network reciprocity is enhanced by the continuous strategy.     

Migration through a Major Andean Ecogeographic Disruption as a Driver of Genetic and Phenotypic Diversity in a Wild Tomato Species

Evolutionary dynamics at the population level play a central role in creating the diversity of life on our planet. In this study, we sought to understand the origins of such population-level variation in mating systems and defensive acylsugar chemistry in Solanum habrochaites—a wild tomato species found in diverse Andean habitats in Ecuador and Peru. Using Restriction-site-Associated-DNA-Sequencing (RAD-seq) of 50 S. habrochaites accessions, we identified eight population clusters generated via isolation and hybridization dynamics of 4–6 ancestral populations. Detailed characterization of mating systems of these clusters revealed emergence of multiple self-compatible (SC) groups from progenitor self-incompatible populations in the northern part of the species range. Emergence of these SC groups was also associated with fixation of deleterious alleles inactivating acylsugar acetylation. The Amotape-Huancabamba Zone—a geographical landmark in the Andes with high endemism and isolated microhabitats—was identified as a major driver of differentiation in the northern species range, whereas large geographical distances contributed to population structure and evolution of a novel SC group in the central and southern parts of the range, where the species was also inferred to have originated. Findings presented here highlight the role of the diverse ecogeography of Peru and Ecuador in generating population differentiation, and enhance our understanding of the microevolutionary processes that create biological diversity.     

Adaptive Dynamics of Extortion and Compliance

Direct reciprocity is a mechanism for the evolution of cooperation. For the iterated prisoner’s dilemma, a new class of strategies has recently been described, the so-called zero-determinant strategies. Using such a strategy, a player can unilaterally enforce a linear relationship between his own payoff and the co-player’s payoff. In particular the player may act in such a way that it becomes optimal for the co-player to cooperate unconditionally. In this way, a player can manipulate and extort his co-player, thereby ensuring that the own payoff never falls below the co-player’s payoff. However, using a compliant strategy instead, a player can also ensure that his own payoff never exceeds the co-player’s payoff. Here, we use adaptive dynamics to study when evolution leads to extortion and when it leads to compliance. We find a remarkable cyclic dynamics: in sufficiently large populations, extortioners play a transient role, helping the population to move from selfish strategies to compliance. Compliant strategies, however, can be subverted by altruists, which in turn give rise to selfish strategies. Whether cooperative strategies are favored in the long run critically depends on the size of the population; we show that cooperation is most abundant in large populations, in which case average payoffs approach the social optimum. Our results are not restricted to the case of the prisoners dilemma, but can be extended to other social dilemmas, such as the snowdrift game. Iterated social dilemmas in large populations do not lead to the evolution of strategies that aim to dominate their co-player. Instead, generosity succeeds.     

Mutation-selection equilibrium in games with mixed strategies

We develop a new method for studying stochastic evolutionary game dynamics of mixed strategies. We consider the general situation: there are n pure strategies whose interactions are described by an n×n payoff matrix. Players can use mixed strategies, which are given by the vector (p₁,…,p_n). Each entry specifies the probability to use the corresponding pure strategy. The sum over all entries is one. Therefore, a mixed strategy is a point in the simplex S_n. We study evolutionary dynamics in a well-mixed population of finite size. Individuals reproduce proportional to payoff. We consider the case of weak selection, which means the payoff from the game is only a small contribution to overall fitness. Reproduction can be subject to mutation; a mutant adopts a randomly chosen mixed strategy. We calculate the average abundance of every mixed strategy in the stationary distribution of the mutation-selection process. We find the crucial conditions that specify if a strategy is favored or opposed by selection. One condition holds for low mutation rate, another for high mutation rate. The result for any mutation rate is a linear combination of those two. As a specific example we study the Hawk-Dove game. We prove general statements about the relationship between games with pure and with mixed strategies.