Perfect reciprocity is the only evolutionarily stable strategy in the continuous iterated prisoner's dilemma

Theoretical studies have shown that cooperation tends to evolve when interacting individuals have positively correlated phenotypes. In the present article, we explore the situation where this correlation results from information exchange between social partners, and behavioral flexibility. We consider the game 'continuous iterated prisoner's dilemma'. The level of cooperation expressed by individuals in this game, together with their ability to respond to one another, both evolve as two aspects of their behavioral strategy. The conditions for a strategy to be evolutionarily stable in this game are degenerate, and earlier works were thus unable to find a single ESS. However, a detailed invasion analysis, together with the study of evolution in finite populations, reveals that natural selection favors strategies whereby individuals respond to their opponent's actions in a perfectly mirrored (i.e., correlated) fashion. As a corollary, the overall payoff of social interactions (i.e., the amount of cooperation) is maximized because couples of correlated partners effectively become the units of selection.     

The evolution of punishment and apology: an iterated prisoner's dilemma model

Recently, behaviors that seem to function as punishment or apology have been reported among non-human primates as well as humans. Such behaviors appear to play an important role in maintaining cooperation between individuals. Therefore, the evolution of these behaviors should be examined from the viewpoint of the evolution of cooperation. The iterated prisoner's dilemma (IPD) game is generally considered to be a standard model for the evolution of cooperation. In the present study, strategies accompanied by punishment-like attacks or apology-like behavior were introduced into the common IPD simulation. Punishment and apology were represented by the P signal and the AS signal given immediately after defection. A strategy with the P and AS signals, named the pPAS strategy, was proved to be an evolutionarily stable strategy under certain conditions. Numerical simulations were carried out according to different assigned values of the costs of punishment and apology. The simulations showed that pPAS could dominate the population (1) when the cost of giving P is relatively small, (2) when the cost of receiving P is relatively large, or (3) when the cost of giving AS is relatively large. The relative cost of giving AS had the clearest effect on the success of pPAS. pPAS can dominate the population even when a dominance asymmetry of the costs between two players was introduced. The present results suggest the possible evolution of social behaviors like punishment or apology as a means of maintaining cooperation. This revised version was published online in November 2006 with corrections to the Cover Date.     

Resolving the iterated prisoner's dilemma: theory and reality

Pairs of unrelated individuals face a prisoner's dilemma if cooperation is the best mutual outcome, but each player does best to defect regardless of his partner's behaviour. Although mutual defection is the only evolutionarily stable strategy in one-shot games, cooperative solutions based on reciprocity can emerge in iterated games. Among the most prominent theoretical solutions are the so-called bookkeeping strategies, such as tit-for-tat, where individuals copy their partner's behaviour in the previous round. However, the lack of empirical data conforming to predicted strategies has prompted the suggestion that the iterated prisoner's dilemma (IPD) is neither a useful nor realistic basis for investigating cooperation. Here, we discuss several recent studies where authors have used the IPD framework to interpret their data. We evaluate the validity of their approach and highlight the diversity of proposed solutions. Strategies based on precise accounting are relatively uncommon, perhaps because the full set of assumptions of the IPD model are rarely satisfied. Instead, animals use a diverse array of strategies that apparently promote cooperation, despite the temptation to cheat. These include both positive and negative reciprocity, as well as long-term mutual investments based on 'friendships'. Although there are various gaps in these studies that remain to be filled, we argue that in most cases, individuals could theoretically benefit from cheating and that cooperation cannot therefore be explained with the concept of positive pseudo-reciprocity. We suggest that by incorporating empirical data into the theoretical framework, we may gain fundamental new insights into the evolution of mutual reciprocal investment in nature.     

Evolutionary escape from the prisoner's dilemma

The classic prisoner's dilemma model of game theory is modified by introducing occasional variations on the options available to players. Mutation and selection of game options reliably change the game matrix, gradually, from a prisoner's dilemma game into a byproduct mutualism one, in which cooperation is stable, and "temptation to defect" is replaced by temptation to cooperate. This result suggests that when there are many different potential ways of interacting, exploring those possibilities may make escape from prisoner's dilemmas a common outcome in the world. A consequence is that persistent prisoner's dilemma structures may be less common than one might otherwise expect.     

Critical dynamics in the evolution of stochastic strategies for the iterated prisoner's dilemma

The observed cooperation on the level of genes, cells, tissues, and individuals has been the object of intense study by evolutionary biologists, mainly because cooperation often flourishes in biological systems in apparent contradiction to the selfish goal of survival inherent in Darwinian evolution. In order to resolve this paradox, evolutionary game theory has focused on the Prisoner's Dilemma (PD), which incorporates the essence of this conflict. Here, we encode strategies for the iterated Prisoner's Dilemma (IPD) in terms of conditional probabilities that represent the response of decision pathways given previous plays. We find that if these stochastic strategies are encoded as genes that undergo Darwinian evolution, the environmental conditions that the strategies are adapting to determine the fixed point of the evolutionary trajectory, which could be either cooperation or defection. A transition between cooperative and defective attractors occurs as a function of different parameters such as mutation rate, replacement rate, and memory, all of which affect a player's ability to predict an opponent's behavior. These results imply that in populations of players that can use previous decisions to plan future ones, cooperation depends critically on whether the players can rely on facing the same strategies that they have adapted to. Defection, on the other hand, is the optimal adaptive response in environments that change so quickly that the information gathered from previous plays cannot usefully be integrated for a response.     

Iterated prisoner's dilemma in an asocial world dominated by loners, not by defectors

Cooperation among genetically unrelated individuals can arise when pairs of individuals interact repeatedly in the Prisoner’s Dilemma. However, the conditions allowing the evolution of reciprocal cooperation become extremely restrictive as the size of the cooperative group increases, because defectors can exploit cooperators more efficiently in larger groups. Here we consider three strategies: Tit for Tat, defector, and loner. Loner beats defector in a non-cooperative world. However, a cooperative strategy Tit for Tat (TFT₀) that stops cooperation after the first iteration when there is at least one defector in the group, can invade a world of loners, even in sizable groups, if both the TFT₀ and the defector strategies arise at the same frequency by mutation.     

Natural selection of memory-one strategies for the iterated prisoner's dilemma

In the iterated Prisoner's Dilemma, mutually cooperative behavior can become established through Darwinian natural selection. In simulated interactions of stochastic memory-one strategies for the Iterated Prisoner's Dilemma, Nowak and Sigmund discovered that cooperative agents using a Pavlov (Win-Stay Lose-Switch) type strategy eventually dominate a random population. This emergence follows more directly from a deterministic dynamical system based on differential reproductive success or natural selection. When restricted to an environment of memory-one agents interacting in iterated Prisoner's Dilemma games with a 1% noise level, the Pavlov agent is the only cooperative strategy and one of very few others that cannot be invaded by a similar strategy. Pavlov agents are trusting but no suckers. They will exploit weakness but repent if punished for cheating.     

The iterated continuous prisoner's dilemma game cannot explain the evolution of interspecific mutualism in unstructured populations

The evolutionionary origin of inter- and intra-specific cooperation among non-related individuals has been a great challenge for biologists for decades. Recently, the continuous prisoner's dilemma game has been introduced to study this problem. In function of previous payoffs, individuals can change their cooperative investment iteratively in this model system. Killingback and Doebeli (Am. Nat. 160 (2002) 421-438) have shown analytically that intra-specific cooperation can emerge in this model system from originally non-cooperating individuals living in a non-structured population. However, it is also known from an earlier numerical work that inter-specific cooperation (mutualism) cannot evolve in a very similar model. The only difference here is that cooperation occurs among individuals of different species. Based on the model framework used by Killingback and Doebeli (2002), this Note proves analytically that mutualism indeed cannot emerge in this model system. Since numerical results have revealed that mutualism can evolve in this model system if individuals interact in a spatially structured manner, our work emphasizes indirectly the role of spatial structure of populations in the origin of mutualism.     

Friendship, cliquishness, and the emergence of cooperation

The evolution of cooperation is a central problem in biology and the social sciences. While theoretical work using the iterated prisoner's dilemma (IPD) has shown that cooperation among non-kin can be sustained among reciprocal strategies (i.e. tit-for-tat), these results are sensitive to errors in strategy execution, cyclical invasions by free riders, and the specific ecology of strategies. Moreover, the IPD assumes that a strategy's probability of playing the PD game with other individuals is independent of the decisions made by others. Here, we remove the assumption of independent pairing by studying a more plausible cooperative dilemma in which players can preferentially interact with a limited set of known partners and also deploy longer-term accounting strategies that can counteract the effects of random errors. We show that cooperative strategies readily emerge and persist in a range of noisy environments, with successful cooperative strategies (henceforth, cliquers) maintaining medium-term memories for partners and low thresholds for acceptable cooperation (i.e. forgiveness). The success of these strategies relies on their cliquishness-a propensity to defect with strangers if they already have an adequate number of partners. Notably, this combination of medium-term accounting, forgiveness, and cliquishness fits with empirical studies of friendship and other long-term relationships among humans.     

Evolving learning rules and emergence of cooperation in spatial prisoner's dilemma

In the evolutionary Prisoner's dilemma (PD) game, agents play with each other and update their strategies in every generation according to some microscopic dynamical rule. In its spatial version, agents do not play with every other but, instead, interact only with their neighbours, thus mimicking the existing of a social or contact network that defines who interacts with whom. In this work, we explore evolutionary, spatial PD systems consisting of two types of agents, each with a certain update (reproduction, learning) rule. We investigate two different scenarios: in the first case, update rules remain fixed for the entire evolution of the system; in the second case, agents update both strategy and update rule in every generation. We show that in a well-mixed population the evolutionary outcome is always full defection. We subsequently focus on two-strategy competition with nearest-neighbour interactions on the contact network and synchronised update of strategies. Our results show that, for an important range of the parameters of the game, the final state of the system is largely different from that arising from the usual setup of a single, fixed dynamical rule. Furthermore, the results are also very different if update rules are fixed or evolve with the strategies. In these respect, we have studied representative update rules, finding that some of them may become extinct while others prevail. We describe the new and rich variety of final outcomes that arise from this co-evolutionary dynamics. We include examples of other neighbourhoods and asynchronous updating that confirm the robustness of our conclusions. Our results pave the way to an evolutionary rationale for modelling social interactions through game theory with a preferred set of update rules.     

Structured population dynamics: continuous size and discontinuous stage structures

A nonlinear stochastic model for the dynamics of a population with either a continuous size structure or a discontinuous stage structure is formulated in the Eulerian formalism. It takes into account dispersion effects due to stochastic variability of the development process of the individuals. The discrete equations of the numerical approximation are derived, and an analysis of the existence and stability of the equilibrium states is performed. An application to a copepod population is illustrated; numerical results of Eulerian and Lagrangian models are compared.      

The volunteer's dilemma and the optimal size of a social group

If one or few individuals are enough to perform an action that produces a collective good and if this action has a cost, living in group can be beneficial because the cost can be shared with other individuals. Without coordination, however, the production of a collective good by the contribution of one or few individuals is inefficient and can be modelled as a volunteer's dilemma. In the volunteer's dilemma the individuals that pay the cost for the production of the collective good benefit from their action if nobody else volunteers, but the cost is wasted if too many individuals volunteer. Increasing group size reduces the need of volunteering for each member of the group; the overall benefit for the group, however, decreases too because the larger the group is, the less likely it is that the collective good is produced. This problem persists even with a high degree of relatedness between group members; an optimal, intermediate group size exists that maximizes the probability to produce the collective good.     

Self-deception as a mediator of cooperation and defection in varying social contexts described in the iterated prisoner''s dilemma

Deception and self-deception may be two related psychological capacities serving to enhance an individual's inclusive fitness. Self-deception has been defined as any psychological act in which one thought or belief is held at the expense of another. This study was designed to test the notion that we may deceive ourselves about our own intentions or the intentions of others in order to facilitate cooperation, if the cooperative behavior has the potential to improve fitness. We may also deceive ourselves in situations where we may need to compete with others, if not competing might represent a loss of fitness. The Self-Deceptio Questionnaire was administered to 80 women and 70 men to determine their levels of self-deception. A series of vignettes, conforming to the Prisoner's Dilemma game format, were administered in questionnaire material to measure participant's predispositions to cooperate or defect in three different contexts (family, mating, and neutral). Men predicted they would be less cooperative than women over all conditions, and participants anticipated cooperating more with kin than with non-kin. Moreover, participants who were high self-deceivers chose more cooperative responses in both family and mating contexts than those with low self-deception scores. Finally, the three-way interaction among sex, context, and level of self-deception suggested that the role of selfdeception in cooperation may vary according to the sex of the individual and the context.     

The continuous prisoner's dilemma and the evolution of cooperation through reciprocal altruism with variable investment

Understanding the evolutionary origin and persistence of cooperative behavior is a fundamental biological problem. The standard "prisoner's dilemma," which is the most widely adopted framework for studying the evolution of cooperation through reciprocal altruism between unrelated individuals, does not allow for varying degrees of cooperation. Here we study the continuous iterated prisoner's dilemma, in which cooperative investments can vary continuously in each round. This game has been previously considered for a class of reactive strategies in which current investments are based on the partner's previous investment. In the standard iterated prisoner's dilemma, such strategies are inferior to strategies that take into account both players' previous moves, as is exemplified by the evolutionary dominance of "Pavlov" over "tit for tat." Consequently, we extend the analysis of the continuous prisoner's dilemma to a class of strategies in which current investments depend on previous payoffs and, hence, on both players' previous investments. We show, both analytically and by simulation, that payoff-based strategies, which embody the intuitively appealing idea that individuals invest more in cooperative interactions when they profit from these interactions, provide a natural explanation for the gradual evolution of cooperation from an initially noncooperative state and for the maintenance of cooperation thereafter.     

Evolutionary dynamics of continuous strategy games on graphs and social networks under weak selection

Understanding the emergence of cooperation among selfish individuals has been a long-standing puzzle, which has been studied by a variety of game models. Most previous studies presumed that interactions between individuals are discrete, but it seems unrealistic in real systems. Recently, there are increasing interests in studying game models with a continuous strategy space. Existing research work on continuous strategy games mainly focuses on well-mixed populations. Especially, little theoretical work has been conducted on their evolutionary dynamics in a structured population. In the previous work (Zhong et al., BioSystems, 2012), we showed that under strong selection, continuous and discrete strategies have significantly different equilibrium and game dynamics in spatially structured populations. In this paper, we further study evolutionary dynamics of continuous strategy games under weak selection in structured populations. By using the fixation probability based stochastic dynamics, we derive exact conditions of natural selection favoring cooperation for the death–birth updating scheme. We also present a network gain decomposition of the game equilibrium, which might provide a new view of the network reciprocity in a quantitative way. Finally, we make a detailed comparison between games using discrete and continuous strategies. As compared to the former, we find that for the latter (i) the same selection conditions are derived for the general 2 × 2 game; especially, the rule b/c > k in a simplified Prisoner's Dilemma is valid as well; however, (ii) for a coordination game, interestingly, the risk-dominant strategy is disfavored. Numerical simulations have also been conducted to validate our results.     

Replicator dynamics of reward & reputation in public goods games

Public goods games have become the mathematical metaphor for game theoretical investigations of cooperative behavior in groups of interacting individuals. Cooperation is a conundrum because cooperators make a sacrifice to benefit others at some cost to themselves. Exploiters or defectors reap the benefits and forgo costs. Despite the fact that groups of cooperators outperform groups of defectors, Darwinian selection or utilitarian principles based on rational choice should favor defectors. In order to overcome this social dilemma, much effort has been expended for investigations pertaining to punishment and sanctioning measures against defectors. Interestingly, the complementary approach to create positive incentives and to reward cooperation has received considerably less attention—despite being heavily advocated in education and social sciences for increasing productivity or preventing conflicts. Here we show that rewards can indeed stimulate cooperation in interaction groups of arbitrary size but, in contrast to punishment, fail to stabilize it. In both cases, however, reputation is essential. The combination of reward and reputation result in complex dynamics dominated by unpredictable oscillations.     

A stochastic model of evolutionary dynamics with deterministic large-population asymptotics

An evolutionary birth-death process is proposed as a model of evolutionary dynamics. Agents residing in a continuous spatial environment X, play a game G, with a continuous strategy set S, against other agents in the environment. The agents’ positions and strategies continuously change in response to other agents and to random effects. Agents spawn asexually at rates that depend on their current fitness, and agents die at rates that depend on their local population density. Agents’ individual evolutionary trajectories in X and S are governed by a system of stochastic ODEs. When the number of agents is large and distributed in a smooth density on (X,S), the collective dynamics of the entire population is governed by a certain (deterministic) PDE, which we call a fitness-diffusion equation.     

The resident's dilemma: a female choice model for the evolution of alternative mating strategies in lekking male ruffs (Philomachus pugnax)

Previous models for the evolution of alternative male matingbehavior have virtually ignored the role of female choice. Wepresent a model in which female choice favors the evolutionand maintenance of alternative mating strategies in male ruffs,Philomachus pugnax. Resident male ruffe establish and defendcourts on leks against other residents, while non-territorialsatellite males move between leks and among courts on a lek.Residents appear to actively recruit satellites to their courts,even though satellites may mate with females once there. Residentbehavior toward satellites and data on female behavior suggestthat residents benefit from a satellite's presence due to somefemale preference for mating on co-occupied courts. However,if all residents accept satellites, none gains any relativeadvantage, yet all pay the costs of having satellites on theircourt. We present a game theoretical model that shows that therelative nature of female choice places residents in an evolutionarydilemma with respect to satellite acceptance. Although all residentswould benefit if satellites could be cooperatively excludedfrom leks, the only evolutionarily stable strategy for individualresidents is to defect and accept satellites. The model alsodemonstrates that this "resident's dilemma" likely exists onlyin a local sense, since the failure of residents to cooperativelyexclude satellites from leks need not result in globally lowerpayoffs, due to frequency-dependent selection on the proportionof satellites in the population. Our analysis suggests thatthe resident-satellite relationship in ruffs, despite its obviouscompetitive elements, is fundamentally a cooperative associationfavored by female choice. Female choice has also been proposedas the primary mechanism selecting for male association to formleks in ruffe. In this context, resident-satellite associationsmay be thought of as transitory "leks within a lek     

Emergence and resilience of cooperation in the spatial prisoner's dilemma via a reward mechanism

We study the problem of the emergence of cooperation in the spatial Prisoner's Dilemma. The pioneering work by Nowak and May [1992. Evolutionary games and spatial chaos. Nature 415, 424-426] showed that large initial populations of cooperators can survive and sustain cooperation in a square lattice with imitate-the-best evolutionary dynamics. We revisit this problem in a cost-benefit formulation suitable for a number of biological applications. We show that if a fixed-amount reward is established for cooperators to share, a single cooperator can invade a population of defectors and form structures that are resilient to re-invasion even if the reward mechanism is turned off. We discuss analytically the case of the invasion by a single cooperator and present agent-based simulations for small initial fractions of cooperators. Large cooperation levels, in the sustainability range, are found. In the conclusions we discuss possible applications of this model as well as its connections with other mechanisms proposed to promote the emergence of cooperation.     

Neighbourhood size,dispersal distance and the complex dynamics of the spatial Ricker model

Amongst the most frequently made assumptions in simple population models are that individuals interact equally with every other individual and that dispersal occurs with equal likelihood to any location. This is especially true for models of a single population (as opposed to a patchy population or metapopulation). For many species of animals and probably for all plant species these assumptions are unlikely to hold true. Here one much-studied population model—the Ricker model—is reformulated such that interactions occur only between individuals located within a certain distance of each other and dispersal distance is finite. Two alternative reformulations are presented. Results demonstrate that both limiting the interaction neighbourhood and reducing dispersal distance tend to stabilise the global population dynamics, although the extent to which this occurs depends upon the reformulation used. Spatial pattern formation is a feature of the simulated population. At lower intrinsic rates of growth (r) these patterns tend to be static, while for higher r, they are dynamic. Both the stabilisation of global dynamics and spatial pattern formation are well-described features of metapopulation models. Here, similar effects are shown to occur on a single contiguous patch of habitat.