Win-stay-lose-learn promotes cooperation in the spatial prisoner's dilemma game

Holding on to one's strategy is natural and common if the later warrants success and satisfaction. This goes against widespread simulation practices of evolutionary games, where players frequently consider changing their strategy even though their payoffs may be marginally different than those of the other players. Inspired by this observation, we introduce an aspiration-based win-stay-lose-learn strategy updating rule into the spatial prisoner's dilemma game. The rule is simple and intuitive, foreseeing strategy changes only by dissatisfied players, who then attempt to adopt the strategy of one of their nearest neighbors, while the strategies of satisfied players are not subject to change. We find that the proposed win-stay-lose-learn rule promotes the evolution of cooperation, and it does so very robustly and independently of the initial conditions. In fact, we show that even a minute initial fraction of cooperators may be sufficient to eventually secure a highly cooperative final state. In addition to extensive simulation results that support our conclusions, we also present results obtained by means of the pair approximation of the studied game. Our findings continue the success story of related win-stay strategy updating rules, and by doing so reveal new ways of resolving the prisoner's dilemma.     

Physical attractiveness and cooperation in a prisoner's dilemma game

The modulating role of age on the relationship between physical attractiveness and cooperativeness in a prisoner's dilemma game (PDG) was investigated. Previous studies have shown that physical attractiveness is negatively related to cooperative choices among young men but not young women. Following the argument that the negative relationship between physical attractiveness and cooperation is a product of short-term mating strategies among attractive men, we predicted that this relationship is unique to young men and absent among women and older men. We tested this hypothesis with 175 participants (aged 22–69 years). The results showed that physical attractiveness was negatively related to cooperative behavior among young men but not among women or older men. We further observed that the negative relationship between physical attractiveness and cooperation among young men was particularly strong when attractiveness was judged by women.     

Evolution of interactions and cooperation in the spatial prisoner's dilemma game

We study the evolution of cooperation in the spatial prisoner's dilemma game where players are allowed to establish new interactions with others. By employing a simple coevolutionary rule entailing only two crucial parameters, we find that different selection criteria for the new interaction partners as well as their number vitally affect the outcome of the game. The resolution of the social dilemma is most probable if the selection favors more successful players and if their maximally attainable number is restricted. While the preferential selection of the best players promotes cooperation irrespective of game parametrization, the optimal number of new interactions depends somewhat on the temptation to defect. Our findings reveal that the "making of new friends" may be an important activity for the successful evolution of cooperation, but also that partners must be selected carefully and their number limited.     

Moving away from nasty encounters enhances cooperation in ecological prisoner's dilemma game

We study the role of migration in the evolution of cooperation. Individuals spatially located on a square lattice play the prisoner's dilemma game. Dissatisfied players, who have been exploited by defectors, tend to terminate interaction with selfish partners by leaving the current habitats, and explore unknown physical niches available surrounding them. The time scale ratio of game interaction to natural selection governs how many game rounds occur before individuals experience strategy updating. Under local migration and strong selection, simulation results demonstrate that cooperation can be stabilized for a wide range of model parameters, and the slower the natural selection, the more favorable for the emergence of cooperation. Besides, how the selection intensity affects cooperators' evolutionary fate is also investigated. We find that increasing it weakens cooperators' viability at different speeds for different time scale ratios. However, cooperation is greatly improved provided that individuals are offered with enough chance to agglomerate, while cooperation can always establish under weak selection but vanishes under very strong selection whenever individuals have less odds to migrate. Whenever the migration range restriction is removed, the parameter area responsible for the emergence of cooperation is, albeit somewhat compressed, still remarkable, validating the effectiveness of collectively migrating in promoting cooperation.     

Mistakes allow evolutionary stability in the repeated prisoner's dilemma game

The repeated prisoner's dilemma game has been widely used in analyses of the evolution of reciprocal altruism. Recently it was shown that no pure strategy could be evolutionarily stable in the repeated prisoner's dilemma. Here I show that if there is always some probability that individuals will make a mistake, then a pure strategy can be evolutionarily stable provided that it is "strong perfect equilibria" against itself. To be a strong perfect equilibrium against itself, a strategy must be the best response to itself after every possible sequence of behavior. I show that both unconditional defection and a modified version of tit-for-tat have this property.     

Eco-evolutionary feedback and the invasion of cooperation in prisoner's dilemma games

Unveiling the origin and forms of cooperation in nature poses profound challenges in evolutionary ecology. The prisoner's dilemma game is an important metaphor for studying the evolution of cooperation. We here classified potential mechanisms for cooperation evolution into schemes of frequency- and density-dependent selection, and focused on the density-dependent selection in the ecological prisoner's dilemma games. We found that, although assortative encounter is still the necessary condition in ecological games for cooperation evolution, a harsh environment, indicated by a high mortality, can foster the invasion of cooperation. The Hamilton rule provides a fundamental condition for the evolution of cooperation by ensuring an enhanced relatedness between players in low-density populations. Incorporating ecological dynamics into evolutionary games opens up a much wider window for the evolution of cooperation, and exhibits a variety of complex behaviors of dynamics, such as limit and heteroclinic cycles. An alternative evolutionary, or rather succession, sequence was proposed that cooperation first appears in harsh environments, followed by the invasion of defection, which leads to a common catastrophe. The rise of cooperation (and altruism), thus, could be much easier in the density-dependent ecological games than in the classic frequency-dependent evolutionary games.     

Evolving learning rules and emergence of cooperation in spatial prisoner's dilemma

In the evolutionary Prisoner's dilemma (PD) game, agents play with each other and update their strategies in every generation according to some microscopic dynamical rule. In its spatial version, agents do not play with every other but, instead, interact only with their neighbours, thus mimicking the existing of a social or contact network that defines who interacts with whom. In this work, we explore evolutionary, spatial PD systems consisting of two types of agents, each with a certain update (reproduction, learning) rule. We investigate two different scenarios: in the first case, update rules remain fixed for the entire evolution of the system; in the second case, agents update both strategy and update rule in every generation. We show that in a well-mixed population the evolutionary outcome is always full defection. We subsequently focus on two-strategy competition with nearest-neighbour interactions on the contact network and synchronised update of strategies. Our results show that, for an important range of the parameters of the game, the final state of the system is largely different from that arising from the usual setup of a single, fixed dynamical rule. Furthermore, the results are also very different if update rules are fixed or evolve with the strategies. In these respect, we have studied representative update rules, finding that some of them may become extinct while others prevail. We describe the new and rich variety of final outcomes that arise from this co-evolutionary dynamics. We include examples of other neighbourhoods and asynchronous updating that confirm the robustness of our conclusions. Our results pave the way to an evolutionary rationale for modelling social interactions through game theory with a preferred set of update rules.     

Emergence and resilience of cooperation in the spatial prisoner's dilemma via a reward mechanism

We study the problem of the emergence of cooperation in the spatial Prisoner's Dilemma. The pioneering work by Nowak and May [1992. Evolutionary games and spatial chaos. Nature 415, 424-426] showed that large initial populations of cooperators can survive and sustain cooperation in a square lattice with imitate-the-best evolutionary dynamics. We revisit this problem in a cost-benefit formulation suitable for a number of biological applications. We show that if a fixed-amount reward is established for cooperators to share, a single cooperator can invade a population of defectors and form structures that are resilient to re-invasion even if the reward mechanism is turned off. We discuss analytically the case of the invasion by a single cooperator and present agent-based simulations for small initial fractions of cooperators. Large cooperation levels, in the sustainability range, are found. In the conclusions we discuss possible applications of this model as well as its connections with other mechanisms proposed to promote the emergence of cooperation.     

Beyond the prisoner's dilemma: Toward models to discriminate among mechanisms of cooperation in nature

The iterated prisoner's dilemma game, or IPD, has now established itself as the orthodox paradigm for theoretical investigations of the evolution of cooperation; but its scope is restricted to reciprocity, which is only one of three categories of cooperation among unrelated individuals. Even within that category, a cooperative encounter has in general three phases, and the IPD has nothing to say about two of them. To distinguish among mechanisms of cooperation in nature, future theoretical work on the evolution of cooperation must distance itself from economics and develop games as a refinement of ethology's comparative approach.     

Evolutionary escape from the prisoner's dilemma

The classic prisoner's dilemma model of game theory is modified by introducing occasional variations on the options available to players. Mutation and selection of game options reliably change the game matrix, gradually, from a prisoner's dilemma game into a byproduct mutualism one, in which cooperation is stable, and "temptation to defect" is replaced by temptation to cooperate. This result suggests that when there are many different potential ways of interacting, exploring those possibilities may make escape from prisoner's dilemmas a common outcome in the world. A consequence is that persistent prisoner's dilemma structures may be less common than one might otherwise expect.     

The iterated continuous prisoner's dilemma game cannot explain the evolution of interspecific mutualism in unstructured populations

The evolutionionary origin of inter- and intra-specific cooperation among non-related individuals has been a great challenge for biologists for decades. Recently, the continuous prisoner's dilemma game has been introduced to study this problem. In function of previous payoffs, individuals can change their cooperative investment iteratively in this model system. Killingback and Doebeli (Am. Nat. 160 (2002) 421-438) have shown analytically that intra-specific cooperation can emerge in this model system from originally non-cooperating individuals living in a non-structured population. However, it is also known from an earlier numerical work that inter-specific cooperation (mutualism) cannot evolve in a very similar model. The only difference here is that cooperation occurs among individuals of different species. Based on the model framework used by Killingback and Doebeli (2002), this Note proves analytically that mutualism indeed cannot emerge in this model system. Since numerical results have revealed that mutualism can evolve in this model system if individuals interact in a spatially structured manner, our work emphasizes indirectly the role of spatial structure of populations in the origin of mutualism.     

The continuous prisoner's dilemma and the evolution of cooperation through reciprocal altruism with variable investment

Understanding the evolutionary origin and persistence of cooperative behavior is a fundamental biological problem. The standard "prisoner's dilemma," which is the most widely adopted framework for studying the evolution of cooperation through reciprocal altruism between unrelated individuals, does not allow for varying degrees of cooperation. Here we study the continuous iterated prisoner's dilemma, in which cooperative investments can vary continuously in each round. This game has been previously considered for a class of reactive strategies in which current investments are based on the partner's previous investment. In the standard iterated prisoner's dilemma, such strategies are inferior to strategies that take into account both players' previous moves, as is exemplified by the evolutionary dominance of "Pavlov" over "tit for tat." Consequently, we extend the analysis of the continuous prisoner's dilemma to a class of strategies in which current investments depend on previous payoffs and, hence, on both players' previous investments. We show, both analytically and by simulation, that payoff-based strategies, which embody the intuitively appealing idea that individuals invest more in cooperative interactions when they profit from these interactions, provide a natural explanation for the gradual evolution of cooperation from an initially noncooperative state and for the maintenance of cooperation thereafter.     

Simple adaptive strategy wins the prisoner's dilemma

The prisoner's dilemma has become the leading paradigm to explain the evolution of cooperation among selfish individuals. Here, we present an adaptive strategy that implements new mechanisms to process information about past encounters. The history of moves is summarized in an internal state which then determines the subsequent move. This enables the strategy to adjust its decisions to the character of the current opponent and to adapt the most promising strategic behavior. For this reason, we call such strategies Adaptor. Through evolutionary simulations, we demonstrate that the concept of Adaptor leads to strategic patterns that are (a) highly cooperative when playing against kin, (b) stable in a sense that goes far beyond the concept of evolutionary stability, (c) robust to environmental changes, i.e. variations of the parameter values and finally (d) superior in performance to the most prominent strategies in the literature.     

Evolutionary dynamics of the continuous iterated prisoner's dilemma

The iterated prisoner's dilemma (IPD) has been widely used in the biological and social sciences to model dyadic cooperation. While most of this work has focused on the discrete prisoner's dilemma, in which actors choose between cooperation and defection, there has been some analysis of the continuous IPD, in which actors can choose any level of cooperation from zero to one. Here, we analyse a model of the continuous IPD with a limited strategy set, and show that a generous strategy achieves the maximum possible payoff against its own type. While this strategy is stable in a neighborhood of the equilibrium point, the equilibrium point itself is always vulnerable to invasion by uncooperative strategies, and hence subject to eventual destabilization. The presence of noise or errors has no effect on this result. Instead, generosity is favored because of its role in increasing contributions to the most efficient level, rather than in counteracting the corrosiveness of noise. Computer simulation using a single-locus infinite alleles Gaussian mutation model suggest that outcomes ranging from a stable cooperative polymorphism to complete collapse of cooperation are possible depending on the magnitude of the mutational variance. Also, making the cost of helping a convex function of the amount of help provided makes it more difficult for cooperative strategies to invade a non-cooperative equilibrium, and for the cooperative equilibrium to resist destabilization by non-cooperative strategies. Finally, we demonstrate that a much greater degree of assortment is required to destabilize a non-cooperative equilibrium in the continuous IPD than in the discrete IPD. The continuous model outlined here suggests that incremental amounts of cooperation lead to rapid decay of cooperation and thus even a large degree of assortment will not be sufficient to allow cooperation to increase when cooperators are rare. The extreme degree of assortment required to destabilize the non-cooperative equilibrium, as well as the instability of the cooperative equilibrium, may help explain why cooperation in Prisoner's Dilemmas is so rare in nature.     

On the iterated prisoner's dilemma in a finite population

Two standard assumptions in analytical work on the iterated prisoner's dilemma are that the population is infinite, and that opponents—though randomly selected—are fixed for the duration of the game. This paper explores the consequences of relaxing both assumptions. It is shown in particular that if opponents are drawn at random throughout the game, then stable cooperation via reciprocity requires both that the probability of a further interaction be sufficiently high—higher than when opponents are fixed—and that the population not exceed a certain critical size, which depends on the probability of further interaction.     

A microbial modified prisoner's dilemma game: how frequency-dependent selection can lead to random phase variation

Random phase variation (RPV) is a control strategy in which the expression of a cell state or phenotype randomly alternates between discrete 'on' and 'off' states. Though this mode of control is common for bacterial virulence factors like pili and toxins, precise conditions under which RPV confers an advantage have not been well defined. In Part I of this study, we predicted that fluctuating environments select for RPV if transitions between different selective environments cannot be reliably sensed (J. Theor. Biol. (2005)). However, selective forces both inside and outside of human hosts are also likely to be frequency dependent in the sense that the fitnesses of some bacterial states are greatest when rare. Here we show that RPV at slow rates can provide a survival advantage in such a frequency-dependent environment by generating population heterogeneity, essentially mimicking a polymorphism. More surprisingly, RPV at a faster 'optimal' rate can shift the population composition toward an optimal growth rate that exceeds that possible for polymorphic populations, but this optimal strategy is not evolutionarily stable. The population would be most fit if all cells randomly phase varied at the optimal rate, but individual cells have a growth-rate incentive to defect (mutate) to other switching rates or non-phase variable phenotype expression, leading to an overall loss of fitness of the individual and the population. This scenario describes a modified Prisoner's Dilemma game (Evolution and the Theory of Games, Cambridge University Press, Cambridge, New York, 1982, viii, 224pp.; Nature 398 (6726) (1999) 367), with random phase variation at optimal switching rates serving as the cooperation strategy.     

Evolution,altruism, and the prisoner's dilemma

I first argue against Peter Singer's exciting thesis that the Prisoner's Dilemma explains why there could be an evolutionary advantage in making reciprocal exchanges that are ultimately motivated by genuine altruism over making such exchanges on the basis of enlightened long-term self-interest. I then show that an alternative to Singer's thesis — one that is also meant to corroborate the view that natural selection favors genuine altruism, recently defended by Gregory Kavka, fails as well. Finally, I show that even granting Singer's and Kavka's claim about the selective advantage of altruism proper, it is doubtful whether that type of claim can be used in a particular sort of sociobiological argument against psychological egoism.     

Teaching and learning in a probabilistic prisoner's dilemma

The prisoner's dilemma is much studied in social psychology and decision-making because it models many real-world conflicts. In everyday terms, the choice to 'cooperate' (maximize reward for the group) or 'defect' (maximize reward for the individual) is often attributed to altruistic or selfish motives. Alternatively, behavior during a dilemma may be understood as a function of reinforcement and punishment. Human participants played a prisoner's-dilemma-type game (for points exchangeable for money) with a computer that employed either a teaching strategy (a probabilistic version of tit-for-tat), in which the computer reinforced or punished participants' cooperation or defection, or a learning strategy (a probabilistic version of Pavlov), in which the computer's responses were reinforced and punished by participants' cooperation and defection. Participants learned to cooperate against both computer strategies. However, in a second experiment which varied the context of the game, they learned to cooperate only against one or other strategy; participants did not learn to cooperate against tit-for-tat when they believed that they were playing against another person; participants did not learn to cooperate against Pavlov when the computer's cooperation probability was signaled by a spinner. The results are consistent with the notion that people are biased not only to cooperate or defect on individual social choices, but also to employ one or other strategy of interaction in a pattern across social choices.     

Resolving the iterated prisoner's dilemma: theory and reality

Pairs of unrelated individuals face a prisoner's dilemma if cooperation is the best mutual outcome, but each player does best to defect regardless of his partner's behaviour. Although mutual defection is the only evolutionarily stable strategy in one-shot games, cooperative solutions based on reciprocity can emerge in iterated games. Among the most prominent theoretical solutions are the so-called bookkeeping strategies, such as tit-for-tat, where individuals copy their partner's behaviour in the previous round. However, the lack of empirical data conforming to predicted strategies has prompted the suggestion that the iterated prisoner's dilemma (IPD) is neither a useful nor realistic basis for investigating cooperation. Here, we discuss several recent studies where authors have used the IPD framework to interpret their data. We evaluate the validity of their approach and highlight the diversity of proposed solutions. Strategies based on precise accounting are relatively uncommon, perhaps because the full set of assumptions of the IPD model are rarely satisfied. Instead, animals use a diverse array of strategies that apparently promote cooperation, despite the temptation to cheat. These include both positive and negative reciprocity, as well as long-term mutual investments based on 'friendships'. Although there are various gaps in these studies that remain to be filled, we argue that in most cases, individuals could theoretically benefit from cheating and that cooperation cannot therefore be explained with the concept of positive pseudo-reciprocity. We suggest that by incorporating empirical data into the theoretical framework, we may gain fundamental new insights into the evolution of mutual reciprocal investment in nature.