Evolutionary escape from the prisoner's dilemma

The classic prisoner's dilemma model of game theory is modified by introducing occasional variations on the options available to players. Mutation and selection of game options reliably change the game matrix, gradually, from a prisoner's dilemma game into a byproduct mutualism one, in which cooperation is stable, and "temptation to defect" is replaced by temptation to cooperate. This result suggests that when there are many different potential ways of interacting, exploring those possibilities may make escape from prisoner's dilemmas a common outcome in the world. A consequence is that persistent prisoner's dilemma structures may be less common than one might otherwise expect.     

Perfect reciprocity is the only evolutionarily stable strategy in the continuous iterated prisoner's dilemma

Theoretical studies have shown that cooperation tends to evolve when interacting individuals have positively correlated phenotypes. In the present article, we explore the situation where this correlation results from information exchange between social partners, and behavioral flexibility. We consider the game 'continuous iterated prisoner's dilemma'. The level of cooperation expressed by individuals in this game, together with their ability to respond to one another, both evolve as two aspects of their behavioral strategy. The conditions for a strategy to be evolutionarily stable in this game are degenerate, and earlier works were thus unable to find a single ESS. However, a detailed invasion analysis, together with the study of evolution in finite populations, reveals that natural selection favors strategies whereby individuals respond to their opponent's actions in a perfectly mirrored (i.e., correlated) fashion. As a corollary, the overall payoff of social interactions (i.e., the amount of cooperation) is maximized because couples of correlated partners effectively become the units of selection.     

Resolution of the stochastic strategy spatial prisoner's dilemma by means of particle swarm optimization

We study the evolution of cooperation among selfish individuals in the stochastic strategy spatial prisoner's dilemma game. We equip players with the particle swarm optimization technique, and find that it may lead to highly cooperative states even if the temptations to defect are strong. The concept of particle swarm optimization was originally introduced within a simple model of social dynamics that can describe the formation of a swarm, i.e., analogous to a swarm of bees searching for a food source. Essentially, particle swarm optimization foresees changes in the velocity profile of each player, such that the best locations are targeted and eventually occupied. In our case, each player keeps track of the highest payoff attained within a local topological neighborhood and its individual highest payoff. Thus, players make use of their own memory that keeps score of the most profitable strategy in previous actions, as well as use of the knowledge gained by the swarm as a whole, to find the best available strategy for themselves and the society. Following extensive simulations of this setup, we find a significant increase in the level of cooperation for a wide range of parameters, and also a full resolution of the prisoner's dilemma. We also demonstrate extreme efficiency of the optimization algorithm when dealing with environments that strongly favor the proliferation of defection, which in turn suggests that swarming could be an important phenomenon by means of which cooperation can be sustained even under highly unfavorable conditions. We thus present an alternative way of understanding the evolution of cooperative behavior and its ubiquitous presence in nature, and we hope that this study will be inspirational for future efforts aimed in this direction.     

Evolutionary dynamics of the continuous iterated prisoner's dilemma

The iterated prisoner's dilemma (IPD) has been widely used in the biological and social sciences to model dyadic cooperation. While most of this work has focused on the discrete prisoner's dilemma, in which actors choose between cooperation and defection, there has been some analysis of the continuous IPD, in which actors can choose any level of cooperation from zero to one. Here, we analyse a model of the continuous IPD with a limited strategy set, and show that a generous strategy achieves the maximum possible payoff against its own type. While this strategy is stable in a neighborhood of the equilibrium point, the equilibrium point itself is always vulnerable to invasion by uncooperative strategies, and hence subject to eventual destabilization. The presence of noise or errors has no effect on this result. Instead, generosity is favored because of its role in increasing contributions to the most efficient level, rather than in counteracting the corrosiveness of noise. Computer simulation using a single-locus infinite alleles Gaussian mutation model suggest that outcomes ranging from a stable cooperative polymorphism to complete collapse of cooperation are possible depending on the magnitude of the mutational variance. Also, making the cost of helping a convex function of the amount of help provided makes it more difficult for cooperative strategies to invade a non-cooperative equilibrium, and for the cooperative equilibrium to resist destabilization by non-cooperative strategies. Finally, we demonstrate that a much greater degree of assortment is required to destabilize a non-cooperative equilibrium in the continuous IPD than in the discrete IPD. The continuous model outlined here suggests that incremental amounts of cooperation lead to rapid decay of cooperation and thus even a large degree of assortment will not be sufficient to allow cooperation to increase when cooperators are rare. The extreme degree of assortment required to destabilize the non-cooperative equilibrium, as well as the instability of the cooperative equilibrium, may help explain why cooperation in Prisoner's Dilemmas is so rare in nature.     

Evolution,altruism, and the prisoner's dilemma

I first argue against Peter Singer's exciting thesis that the Prisoner's Dilemma explains why there could be an evolutionary advantage in making reciprocal exchanges that are ultimately motivated by genuine altruism over making such exchanges on the basis of enlightened long-term self-interest. I then show that an alternative to Singer's thesis — one that is also meant to corroborate the view that natural selection favors genuine altruism, recently defended by Gregory Kavka, fails as well. Finally, I show that even granting Singer's and Kavka's claim about the selective advantage of altruism proper, it is doubtful whether that type of claim can be used in a particular sort of sociobiological argument against psychological egoism.     

Win-stay-lose-learn promotes cooperation in the spatial prisoner's dilemma game

Holding on to one's strategy is natural and common if the later warrants success and satisfaction. This goes against widespread simulation practices of evolutionary games, where players frequently consider changing their strategy even though their payoffs may be marginally different than those of the other players. Inspired by this observation, we introduce an aspiration-based win-stay-lose-learn strategy updating rule into the spatial prisoner's dilemma game. The rule is simple and intuitive, foreseeing strategy changes only by dissatisfied players, who then attempt to adopt the strategy of one of their nearest neighbors, while the strategies of satisfied players are not subject to change. We find that the proposed win-stay-lose-learn rule promotes the evolution of cooperation, and it does so very robustly and independently of the initial conditions. In fact, we show that even a minute initial fraction of cooperators may be sufficient to eventually secure a highly cooperative final state. In addition to extensive simulation results that support our conclusions, we also present results obtained by means of the pair approximation of the studied game. Our findings continue the success story of related win-stay strategy updating rules, and by doing so reveal new ways of resolving the prisoner's dilemma.     

On the iterated prisoner's dilemma in a finite population

Two standard assumptions in analytical work on the iterated prisoner's dilemma are that the population is infinite, and that opponents—though randomly selected—are fixed for the duration of the game. This paper explores the consequences of relaxing both assumptions. It is shown in particular that if opponents are drawn at random throughout the game, then stable cooperation via reciprocity requires both that the probability of a further interaction be sufficiently high—higher than when opponents are fixed—and that the population not exceed a certain critical size, which depends on the probability of further interaction.     

Heterogeneous aspirations promote cooperation in the prisoner's dilemma game

To be the fittest is central to proliferation in evolutionary games. Individuals thus adopt the strategies of better performing players in the hope of successful reproduction. In structured populations the array of those that are eligible to act as strategy sources is bounded to the immediate neighbors of each individual. But which one of these strategy sources should potentially be copied? Previous research dealt with this question either by selecting the fittest or by selecting one player uniformly at random. Here we introduce a parameter that interpolates between these two extreme options. Setting equal to zero returns the random selection of the opponent, while positive favor the fitter players. In addition, we divide the population into two groups. Players from group select their opponents as dictated by the parameter , while players from group do so randomly irrespective of . We denote the fraction of players contained in groups and by and , respectively. The two parameters and allow us to analyze in detail how aspirations in the context of the prisoner''s dilemma game influence the evolution of cooperation. We find that for sufficiently positive values of there exist a robust intermediate for which cooperation thrives best. The robustness of this observation is tested against different levels of uncertainty in the strategy adoption process and for different interaction networks. We also provide complete phase diagrams depicting the dependence of the impact of and for different values of , and contrast the validity of our conclusions by means of an alternative model where individual aspiration levels are subject to evolution as well. Our study indicates that heterogeneity in aspirations may be key for the sustainability of cooperation in structured populations.     

Resolving the iterated prisoner's dilemma: theory and reality

Pairs of unrelated individuals face a prisoner's dilemma if cooperation is the best mutual outcome, but each player does best to defect regardless of his partner's behaviour. Although mutual defection is the only evolutionarily stable strategy in one-shot games, cooperative solutions based on reciprocity can emerge in iterated games. Among the most prominent theoretical solutions are the so-called bookkeeping strategies, such as tit-for-tat, where individuals copy their partner's behaviour in the previous round. However, the lack of empirical data conforming to predicted strategies has prompted the suggestion that the iterated prisoner's dilemma (IPD) is neither a useful nor realistic basis for investigating cooperation. Here, we discuss several recent studies where authors have used the IPD framework to interpret their data. We evaluate the validity of their approach and highlight the diversity of proposed solutions. Strategies based on precise accounting are relatively uncommon, perhaps because the full set of assumptions of the IPD model are rarely satisfied. Instead, animals use a diverse array of strategies that apparently promote cooperation, despite the temptation to cheat. These include both positive and negative reciprocity, as well as long-term mutual investments based on 'friendships'. Although there are various gaps in these studies that remain to be filled, we argue that in most cases, individuals could theoretically benefit from cheating and that cooperation cannot therefore be explained with the concept of positive pseudo-reciprocity. We suggest that by incorporating empirical data into the theoretical framework, we may gain fundamental new insights into the evolution of mutual reciprocal investment in nature.     

An ESS-analysis for ensembles of prisoner's dilemma strategies

The ESS (Evolutionary Stable Strategy) concept of Maynard Smith can be applied in its weak form to ensembles of competing PD ("Prisoner's Dilemma") strategies memorizing two to three of one's own and one's opponent's moves. The format of our study is: (1) games have very long duration; (2) Taylor-Jonker dynamics applies; (3) Effects of finite population size can be ignored. It is shown that in the case R greater than (T + S)/2 a set of strategies can be singled out which do not lose against any other strategy while co-operating with themselves. Such a set is uninvadable by other PD strategies if it constitutes more than half of the total population.     

Teaching and learning in a probabilistic prisoner's dilemma

The prisoner's dilemma is much studied in social psychology and decision-making because it models many real-world conflicts. In everyday terms, the choice to 'cooperate' (maximize reward for the group) or 'defect' (maximize reward for the individual) is often attributed to altruistic or selfish motives. Alternatively, behavior during a dilemma may be understood as a function of reinforcement and punishment. Human participants played a prisoner's-dilemma-type game (for points exchangeable for money) with a computer that employed either a teaching strategy (a probabilistic version of tit-for-tat), in which the computer reinforced or punished participants' cooperation or defection, or a learning strategy (a probabilistic version of Pavlov), in which the computer's responses were reinforced and punished by participants' cooperation and defection. Participants learned to cooperate against both computer strategies. However, in a second experiment which varied the context of the game, they learned to cooperate only against one or other strategy; participants did not learn to cooperate against tit-for-tat when they believed that they were playing against another person; participants did not learn to cooperate against Pavlov when the computer's cooperation probability was signaled by a spinner. The results are consistent with the notion that people are biased not only to cooperate or defect on individual social choices, but also to employ one or other strategy of interaction in a pattern across social choices.     

Mistakes allow evolutionary stability in the repeated prisoner's dilemma game

The repeated prisoner's dilemma game has been widely used in analyses of the evolution of reciprocal altruism. Recently it was shown that no pure strategy could be evolutionarily stable in the repeated prisoner's dilemma. Here I show that if there is always some probability that individuals will make a mistake, then a pure strategy can be evolutionarily stable provided that it is "strong perfect equilibria" against itself. To be a strong perfect equilibrium against itself, a strategy must be the best response to itself after every possible sequence of behavior. I show that both unconditional defection and a modified version of tit-for-tat have this property.     

Natural selection of memory-one strategies for the iterated prisoner's dilemma

In the iterated Prisoner's Dilemma, mutually cooperative behavior can become established through Darwinian natural selection. In simulated interactions of stochastic memory-one strategies for the Iterated Prisoner's Dilemma, Nowak and Sigmund discovered that cooperative agents using a Pavlov (Win-Stay Lose-Switch) type strategy eventually dominate a random population. This emergence follows more directly from a deterministic dynamical system based on differential reproductive success or natural selection. When restricted to an environment of memory-one agents interacting in iterated Prisoner's Dilemma games with a 1% noise level, the Pavlov agent is the only cooperative strategy and one of very few others that cannot be invaded by a similar strategy. Pavlov agents are trusting but no suckers. They will exploit weakness but repent if punished for cheating.     

Physical attractiveness and cooperation in a prisoner's dilemma game

The modulating role of age on the relationship between physical attractiveness and cooperativeness in a prisoner's dilemma game (PDG) was investigated. Previous studies have shown that physical attractiveness is negatively related to cooperative choices among young men but not young women. Following the argument that the negative relationship between physical attractiveness and cooperation is a product of short-term mating strategies among attractive men, we predicted that this relationship is unique to young men and absent among women and older men. We tested this hypothesis with 175 participants (aged 22–69 years). The results showed that physical attractiveness was negatively related to cooperative behavior among young men but not among women or older men. We further observed that the negative relationship between physical attractiveness and cooperation among young men was particularly strong when attractiveness was judged by women.     

Reciprocal altruism as a felony: Antitrust and the prisoner's dilemma

Biologist study reciprocity. Because U.S. law outlaws cooperative commercial efforts to establish and maintain cartelized prices, for almost a century lawyers and judges also have been interested in the phenomenon of reciprocity. Judges and enforcement officials have set policy based upon intuitive hunches about the likelihood of reciprocity. This law is not consistent. Older court decisions exhibit fear that reciprocity is probable, while more recent doctrine and commentary have disparaged the likelihood of successful cooperation. This antitrust dispute can be understood in game theoretic terms similar to analyses of reciprocity developed by biological theorists. Axelrod's recent research on the repeated prisoner's dilemma game provides an explanation of how cooperative pricing can evolve despite competitive adversity. This research implies both support for the older antitrust fear of collusive reciprocity as well as criticism of recent and supposedly modern skepticism about the older view. But Axelrod's research as it stands is too simplistic for the market context. Before antitrust can use it, this research must be extended to account for three factors: for the existence of more than two equally sized players, for uncertainty in players' communication, and for the manner in which firms learn their managerial strategies.     

Evolution of interactions and cooperation in the spatial prisoner's dilemma game

We study the evolution of cooperation in the spatial prisoner's dilemma game where players are allowed to establish new interactions with others. By employing a simple coevolutionary rule entailing only two crucial parameters, we find that different selection criteria for the new interaction partners as well as their number vitally affect the outcome of the game. The resolution of the social dilemma is most probable if the selection favors more successful players and if their maximally attainable number is restricted. While the preferential selection of the best players promotes cooperation irrespective of game parametrization, the optimal number of new interactions depends somewhat on the temptation to defect. Our findings reveal that the "making of new friends" may be an important activity for the successful evolution of cooperation, but also that partners must be selected carefully and their number limited.     

Eco-evolutionary feedback and the invasion of cooperation in prisoner's dilemma games

Unveiling the origin and forms of cooperation in nature poses profound challenges in evolutionary ecology. The prisoner's dilemma game is an important metaphor for studying the evolution of cooperation. We here classified potential mechanisms for cooperation evolution into schemes of frequency- and density-dependent selection, and focused on the density-dependent selection in the ecological prisoner's dilemma games. We found that, although assortative encounter is still the necessary condition in ecological games for cooperation evolution, a harsh environment, indicated by a high mortality, can foster the invasion of cooperation. The Hamilton rule provides a fundamental condition for the evolution of cooperation by ensuring an enhanced relatedness between players in low-density populations. Incorporating ecological dynamics into evolutionary games opens up a much wider window for the evolution of cooperation, and exhibits a variety of complex behaviors of dynamics, such as limit and heteroclinic cycles. An alternative evolutionary, or rather succession, sequence was proposed that cooperation first appears in harsh environments, followed by the invasion of defection, which leads to a common catastrophe. The rise of cooperation (and altruism), thus, could be much easier in the density-dependent ecological games than in the classic frequency-dependent evolutionary games.     

Critical dynamics in the evolution of stochastic strategies for the iterated prisoner's dilemma

The observed cooperation on the level of genes, cells, tissues, and individuals has been the object of intense study by evolutionary biologists, mainly because cooperation often flourishes in biological systems in apparent contradiction to the selfish goal of survival inherent in Darwinian evolution. In order to resolve this paradox, evolutionary game theory has focused on the Prisoner's Dilemma (PD), which incorporates the essence of this conflict. Here, we encode strategies for the iterated Prisoner's Dilemma (IPD) in terms of conditional probabilities that represent the response of decision pathways given previous plays. We find that if these stochastic strategies are encoded as genes that undergo Darwinian evolution, the environmental conditions that the strategies are adapting to determine the fixed point of the evolutionary trajectory, which could be either cooperation or defection. A transition between cooperative and defective attractors occurs as a function of different parameters such as mutation rate, replacement rate, and memory, all of which affect a player's ability to predict an opponent's behavior. These results imply that in populations of players that can use previous decisions to plan future ones, cooperation depends critically on whether the players can rely on facing the same strategies that they have adapted to. Defection, on the other hand, is the optimal adaptive response in environments that change so quickly that the information gathered from previous plays cannot usefully be integrated for a response.     

Social experiments in the mesoscale: humans playing a spatial prisoner's dilemma

Background

The evolutionary origin of cooperation among unrelated individuals remains a key unsolved issue across several disciplines. Prominent among the several mechanisms proposed to explain how cooperation can emerge is the existence of a population structure that determines the interactions among individuals. Many models have explored analytically and by simulation the effects of such a structure, particularly in the framework of the Prisoner''s Dilemma, but the results of these models largely depend on details such as the type of spatial structure or the evolutionary dynamics. Therefore, experimental work suitably designed to address this question is needed to probe these issues.

Methods and Findings

We have designed an experiment to test the emergence of cooperation when humans play Prisoner''s Dilemma on a network whose size is comparable to that of simulations. We find that the cooperation level declines to an asymptotic state with low but nonzero cooperation. Regarding players'' behavior, we observe that the population is heterogeneous, consisting of a high percentage of defectors, a smaller one of cooperators, and a large group that shares features of the conditional cooperators of public goods games. We propose an agent-based model based on the coexistence of these different strategies that is in good agreement with all the experimental observations.

Conclusions

In our large experimental setup, cooperation was not promoted by the existence of a lattice beyond a residual level (around 20%) typical of public goods experiments. Our findings also indicate that both heterogeneity and a “moody” conditional cooperation strategy, in which the probability of cooperating also depends on the player''s previous action, are required to understand the outcome of the experiment. These results could impact the way game theory on graphs is used to model human interactions in structured groups.