Oscillations in the evolution of reciprocity

A game-theoretical analysis of the Iterated Prisoner's Dilemma shows that the evolution of ensembles of stochastic strategies displays a dynamics of high complexity and unpredictability.     

Natural selection of memory-one strategies for the iterated prisoner's dilemma

In the iterated Prisoner's Dilemma, mutually cooperative behavior can become established through Darwinian natural selection. In simulated interactions of stochastic memory-one strategies for the Iterated Prisoner's Dilemma, Nowak and Sigmund discovered that cooperative agents using a Pavlov (Win-Stay Lose-Switch) type strategy eventually dominate a random population. This emergence follows more directly from a deterministic dynamical system based on differential reproductive success or natural selection. When restricted to an environment of memory-one agents interacting in iterated Prisoner's Dilemma games with a 1% noise level, the Pavlov agent is the only cooperative strategy and one of very few others that cannot be invaded by a similar strategy. Pavlov agents are trusting but no suckers. They will exploit weakness but repent if punished for cheating.     

Variable investment, the Continuous Prisoner's Dilemma, and the origin of cooperation.

Cooperation is fundamental to many biological systems. A common metaphor for studying the evolution of cooperation is the Prisoner's Dilemma, a game with two strategies: cooperate or defect. However, cooperation is rare all or nothing, and its evolution probably involves the gradual extension of initially modest degrees of assistance. The inability of the Prisoner's Dilemma to capture this basic aspect limits its use for understanding the evolutionary origins of cooperation. Here we consider a framework for cooperation based on the concept of investment: an act which is costly, but which benefits other individuals, where the cost and benefit depend on the level of investment made. In the resulting Continuous Prisoner's Dilemma the essential problem of cooperation remains: in the absence of any additional structure non-zero levels of investment cannot evolve. However, if investments are considered in a spatially structured context, selfish individuals who make arbitrarily low investments can be invaded by higher-investing mutants. This results in the mean level of investment evolving to significant levels, where it is maintained indefinitely. This approach provides a natural solution to the fundamental problem of how cooperation gradually increases from a non-cooperative state.     

Cooperative boundary populations: the evolution of cooperation on mortality risk gradients

Cooperative or altruistic behavior is known to be vulnerable to destructive exploitation in the absence of spatial segregation and perceptual discrimination on the part of cooperators. In this study, a non-standard, agent-based, spatially explicit model of the evolution of cooperation shows that spatial gradients of increasing individual mortality risk can allow cooperative subpopulations to persist among players randomly matched for one-shot Prisoner's Dilemma. Further, the dynamically stable cooperator population formed on the gradient at the boundary of the survivable non-cooperative range provides ideal conditions for the evolution of discriminating strategies such as tit-for-tat. It is suggested that such gradients may commonly exist at the boundaries of the ranges of existing populations, providing a new basic mechanism for the evolution of cooperation.     

The continuous Prisoner's dilemma: II. Linear reactive strategies with noise.

We present a general model for the Continuous Prisoner's Dilemma and study the effect of errors. We find that cooperative strategies that can resist invasion by defectors are optimistic (make high initial offers), generous (always offer more cooperation than the partner did in the previous round) and uncompromising (offer full cooperation only if the partner does). A necessary condition for the emergence of cooperation in the continuous Prisoner's Dilemma with noise is b (1-p)>c, where b and c denote, respectively, the benefit and cost of cooperation, while p is the error rate. This relation can be reformulated as an error threshold: cooperation can only emerge if the probability of making a mistake is below a critical value. We note, however, that cooperation in the continuous Prisoner's Dilemma with noise does not seem to be evolutionarily stable: while it is possible to find cooperative strategies that resist invasion by defectors, such cooperators are generally invaded by more cooperative strategies which eventually yield to defectors. Thus, the long-term evolution of the continuous Prisoner's Dilemma is either characterized by unending cycles or by stable polymorphisms of cooperators and defectors.     

A two-trial two-strategy conflict

Iterated conflicts allow the possibility of co-operative-like behaviour in games such as the Prisoner's Dilemma. The present paper is an attempt to initiate the study of iterated conflicts when, (a) the number of iterations is fixed and finite and (b) the underlying payoff matrix is general, e.g. a mixed Evolutionary Stable Strategy (ESS) could occur in the non-iterated coflict. These assumptions are in contrast to the Iterated Prisoner's Dilemma. We consider a somewhat special case which none the less produces results of an interesting nature. For those cases where there is no internal ESS in the one trial case the two-trial case is easily resolved. When the former has an internal ESS then the two-trial case yields two ESSs whose supports are a partition of the space of strategies.     

Direct reciprocity with costly punishment: generous tit-for-tat prevails

The standard model for direct reciprocity is the repeated Prisoner's Dilemma, where in each round players choose between cooperation and defection. Here we extend the standard framework to include costly punishment. Now players have a choice between cooperation, defection and costly punishment. We study the set of all reactive strategies, where the behavior depends on what the other player has done in the previous round. We find all cooperative strategies that are Nash equilibria. If the cost of cooperation is greater than the cost of punishment, then the only cooperative Nash equilibrium is generous-tit-for-tat (GTFT), which does not use costly punishment. If the cost of cooperation is less than the cost of punishment, then there are infinitely many cooperative Nash equilibria and the response to defection can include costly punishment. We also perform computer simulations of evolutionary dynamics in populations of finite size. These simulations show that in the context of direct reciprocity, (i) natural selection prefers generous tit-for-tat over strategies that use costly punishment, and (ii) that costly punishment does not promote the evolution of cooperation. We find quantitative agreement between our simulation results and data from experimental observations.     

The paradox of cooperation benefits

It seems obvious that as the benefits of cooperation increase, the share of cooperators in the population should also increase. It is well known that positive assortment between cooperative types, for instance in spatially structured populations, provide better conditions for the evolution of cooperation than complete mixing. This study demonstrates that, assuming positive assortment, under most conditions higher cooperation benefits also increase the share of cooperators. On the other hand, under a specified range of payoff values, when at least two payoff parameters are modified, the reverse is true. The conditions for this paradox are determined for two-person social dilemmas: the Prisoner's Dilemma, the Hawks and Doves game, and the Stag Hunt game, assuming global selection and positive assortment.     

Win-stay, lose-shift strategies for repeated games-memory length, aspiration levels and noise.

Win-stay, lose-shift, the principle to retain a successful action is a simple and general learning rule that can be applied to all types of repeated decision problems. In this paper I consider win-stay, lose-shift strategies with diverse memory sizes and strategies that adapt their aspiration levels, i.e. the payoff level considered as "success". I study their evolution for the Prisoner's Dilemma, as well as in a rapidly changing environment, where a randomly selected game is assigned to the players. For win-stay, lose-shift strategies with memory one the average payoffs are computed and their evolutionary stability is discussed. Using computer simulations I show that the win-stay, lose-shift strategies with longer memory are very successful both for the Prisoner's Dilemma, where cooperation dominates even for high noise levels, and the randomly assigned games, where the players achieve nearly the expected Pareto optimal payoffs. I discuss the impact of noise and show that the memory length of the players increases with the noise level. These results indicate that the win-stay, lose-shift principle is a very successful strategy in repeated games with noise.     

Behavior-dependent contexts for repeated plays of the Prisoner's Dilemma: II. Dynamical aspects of the evolution of cooperation

Iterated Prisoner's Dilemma models are proposed in which, at any trial, the probability of staying in the game depends on the outcome of the previous trial. If a player's choice depends on its own play (cooperate or defect) at the previous trial, it becomes possible for cooperative strategies to increase when rare in a population of egoists. A dynamic analysis is used to demonstrate that stable polymorphisms may result, and may involve more strategies than just Tit-for-Tat and all-Defect. The tendency for clustering among like strategists to enhance their initial increase when rare is also explored dynamically.     

Does mobility decrease cooperation?

We explore the minimal conditions for sustainable cooperation on a spatially distributed population of memoryless, unconditional strategies (cooperators and defectors) in presence of unbiased, non-contingent mobility in the context of the Prisoner's Dilemma game. We find that cooperative behavior is not only possible but may even be enhanced by such an "always-move" rule, when compared with the strongly viscous ("never-move") case. In addition, mobility also increases the capability of cooperation to emerge and invade a population of defectors, what may have a fundamental role in the problem of the onset of cooperation.     

The continuous prisoner's dilemma: I. linear reactive strategies.

We present a general model for the Prisoner's Dilemma in which variable degrees of cooperation are possible, and payoffs are scaled accordingly. We describe a continuous strategy space, and divide this space into strategy families. We derive the payoff function for these families analytically, and study the evolutionary outcome when a wide range of strategies play against each other. Our results show that the initial degree of cooperation offered by a strategy is a decisive factor for evolutionary robustness: the most successful strategies in our model offer full cooperation as an initial move, but thereafter cooperate fully only if their opponent does the same. These strategies gradually raise the stakes when playing a strategy which is initially reticent to cooperate, but differ from the strategies predicted by other continuous models in that they are not only generous, but are also consistently optimistic and uncompromising.     

Asynchronous spatial evolutionary games

Over the past 50 years, much attention has been given to the Prisoner's Dilemma as a metaphor for problems surrounding the evolution and maintenance of cooperative and altruistic behavior. The bulk of this work has dealt with the successfulness and robustness of various strategies. Nowak and May (1992) considered an alternative approach to studying evolutionary games. They assumed that players were distributed across a two-dimensional (2D) lattice, interactions between players occurred locally, rather than at long range as in the well mixed situation. The resulting spatial evolutionary games display dynamics not seen in their well-mixed counterparts. An assumption underlying much of the work on spatial evolutionary games is that the state of all players is updated in unison or in synchrony. Using the framework outlined in Nowak and May (1992), we examine the effect of various asynchronous updating schemes on the dynamics of spatial evolutionary games. There are potential implications for the dynamics of a wide variety of spatially extended systems in biology, physics and chemistry.     

Evolution of host life-history traits in a spatially structured host-parasite system

Most models for the evolution of host defense against parasites assume that host populations are not spatially structured. Yet local interactions and limited dispersal can strongly affect the evolutionary outcome, because they significantly alter epidemiological feedbacks and the spatial genetic structuring of the host and pathogen populations. We provide a general framework to study the evolution of a number of host life-history traits in a spatially structured host population infected by a horizontally transmitted parasite. Our analysis teases apart the selective pressures on hosts and helps disentangle the direct fitness effect of mutations and their indirect effects via the influence of spatial structure on the genetic, demographic, and epidemiological structure of the host population. We then illustrate the evolutionary consequences of spatial structure by focusing on the evolution of two host defense strategies against parasitism: suicide upon infection and reduced transmission. Because they bring no direct fitness benefit, these strategies are counterselected or selectively neutral in a nonspatial setting, but we show that they can be selected for in a spatially structured environment. Our study thus sheds light on the evolution of altruistic defense mechanisms that have been observed in various biological systems.     

SPATIALLY CORRELATED EXTINCTIONS SELECT FOR LESS EMIGRATION BUT LARGER DISPERSAL DISTANCES IN THE SPIDER MITE TETRANYCHUS URTICAE

Dispersal is a central process to almost all species on earth, as it connects spatially structured populations and thereby increases population persistence. Dispersal is subject to (rapid) evolution and local patch extinctions are an important selective force in this context. In contrast to the randomly distributed local extinctions considered in most theoretical studies, habitat fragmentation or other anthropogenic interventions will lead to spatially correlated extinction patterns. Under such conditions natural selection is thought to lead to more long‐distance dispersal, but this theoretical prediction has not yet been verified empirically. We test this prediction in experimental spatially structured populations of the spider mite Tetranychus urticae and supplement these empirical results with insights from an individual‐based evolutionary model. We demonstrate that the spatial correlation of local extinctions changes the entire distribution of dispersal distances (dispersal kernel) and selects for overall less emigration but more long‐distance dispersal.     

Reactive strategies in indirect reciprocity

Evolution of reactive strategy of indirect reciprocity is discussed, where individuals interact with others through the one-shot Prisoner's Dilemma game, changing their partners in every round. We investigate all of the reactive strategies that are stochastic, including deterministic ones as special cases. First we study adaptive dynamics of reactive strategies by assuming monomorphic population. Results are very similar to the corresponding evolutionary dynamics of direct reciprocity. The discriminating strategy, which prescribes cooperation only with those who cooperated in the previous round, cannot be an outcome of the evolution. Next we examine the case where the population includes a diversity of strategies. We find that only the mean 'discriminatoriness' in the population is the parameter that affects the evolutionary dynamics. The discriminating strategy works as a promoter of cooperation there. However, it is again not the end point of the evolution. This is because retaliatory defection, which was prescribed by the discriminating strategy, is regarded as another defection toward the society. These results caution that we have to reconsider the role of retaliatory defection much more carefully.     

A new route to the evolution of cooperation

The Prisoner's Dilemma (PD) constitutes a widely used metaphor to investigate problems related to the evolution of cooperation. Whenever evolution takes place in well-mixed populations engaged in single rounds of the PD, cooperators cannot resist invasion by defectors, a feature, which is somewhat alleviated whenever populations are spatially distributed. In both cases the populations are characterized by a homogeneous pattern of connectivity, in which every individual is equivalent, sharing the same number of neighbours. Recently, compelling evidence has been accumulated on the strong heterogeneous nature of the network of contacts between individuals in populations. Here we describe the networks of contacts in terms of graphs and show that heterogeneity provides a new mechanism for cooperation to survive. Specifically, we show that cooperators are capable of exploring the heterogeneity of the population structure to become evolutionary competitive. As a result, cooperation becomes the dominating trait in scale-free networks of contacts in which the few highly connected individuals are directly inter-connected, in this way contributing to self-sustain cooperation.     

Cooperation driven by mutations in multi-person Prisoner's Dilemma

The n-person Prisoner's Dilemma is a widely used model for populations where individuals interact in groups. The evolutionary stability of populations has been analysed in the literature for the case where mutations in the population may be considered as isolated events. For this case, and assuming simple trigger strategies and many iterations per game, we analyse the rate of convergence to the evolutionarily stable populations. We find that for some values of the payoff parameters of the Prisoner's Dilemma this rate is so low that the assumption, that mutations in the population are infrequent on that time-scale, is unreasonable. Furthermore, the problem is compounded as the group size is increased. In order to address this issue, we derive a deterministic approximation of the evolutionary dynamics with explicit, stochastic mutation processes, valid when the population size is large. We then analyse how the evolutionary dynamics depends on the following factors: mutation rate, group size, the value of the payoff parameters, and the structure of the initial population. In order to carry out the simulations for groups of more than just a few individuals, we derive an efficient way of calculating the fitness values. We find that when the mutation rate per individual and generation is very low, the dynamics is characterized by populations which are evolutionarily stable. As the mutation rate is increased, other fixed points with a higher degree of cooperation become stable. For some values of the payoff parameters, the system is characterized by (apparently) stable limit cycles dominated by cooperative behaviour. The parameter regions corresponding to high degree of cooperation grow in size with the mutation rate, and in number with the group size. For some parameter values, we find more than one stable fixed point, corresponding to different structures of the initial population.     

Optimality under noise: higher memory strategies for the alternating prisoner's dilemma.

The Alternating Prisoner's Dilemma is a variant of the iterated Prisoner's Dilemma in which the players alternate in the roles of actor and recipient. We searched for strategies which are "optimal" in the Alternating Prisoner's Dilemma with noise (a non-zero probability that a player's decision will be transmitted incorrectly). In order to achieve success against a variety of other strategies, a strategy must be "self-cooperating" (able to achieve mutual cooperation with its clone), "C-exploiting" (able to exploit unconditional cooperators), and "D-unexploitable" (able to resist exploitation by defectors). It must also have high evolutionary "dominance", a general measure of evolutionary performance which considers both resistance to invasion and the ability to invade other strategies. A strategy which meets these optimality criteria can evolve cooperation by invading a population of defectors and establishing a stable cooperative society. Most of the strategies commonly discussed in the Alternating Prisoner's Dilemma literature are low-memory strategies such as Tit For Tat, Pavlov, and Firm But Fair, but none of these strategies can simultaneously meet all of the optimality criteria. However, we discovered a class of higher memory "Firm Pavlov" strategies, which not only meet our stringent optimality criteria, but also achieve remarkable success in round-robin tournaments and evolutionary interactions. These higher memory strategies are friendly enough to cooperate with their clone, pragmatic enough to exploit unconditional cooperators, and wary enough to resist exploitation by defectors: they are truly "optimal under noise" in the Alternating Prisoner's Dilemma.     

Kin selection may inhibit the evolution of reciprocation

Kin selection and reciprocal cooperation provide two candidate explanations for the evolution of cooperation. Models of the evolution of cooperation have typically focussed on one or the other mechanism, despite claims that kin selection could pave the way for the evolution of reciprocal cooperation. We describe a computer simulation model that explicitly supports both kin selection and reciprocal cooperation. The model simulates a viscous population of discrete individuals with social interaction taking the form of the Prisoner's Dilemma and selection acting on performance in these interactions. We recount how the analytical and empirical study of this model led to the conclusion that kin selection may actually inhibit the evolution of effective strategies for establishing reciprocal cooperation.