Win-stay-lose-learn promotes cooperation in the spatial prisoner's dilemma game

Holding on to one's strategy is natural and common if the later warrants success and satisfaction. This goes against widespread simulation practices of evolutionary games, where players frequently consider changing their strategy even though their payoffs may be marginally different than those of the other players. Inspired by this observation, we introduce an aspiration-based win-stay-lose-learn strategy updating rule into the spatial prisoner's dilemma game. The rule is simple and intuitive, foreseeing strategy changes only by dissatisfied players, who then attempt to adopt the strategy of one of their nearest neighbors, while the strategies of satisfied players are not subject to change. We find that the proposed win-stay-lose-learn rule promotes the evolution of cooperation, and it does so very robustly and independently of the initial conditions. In fact, we show that even a minute initial fraction of cooperators may be sufficient to eventually secure a highly cooperative final state. In addition to extensive simulation results that support our conclusions, we also present results obtained by means of the pair approximation of the studied game. Our findings continue the success story of related win-stay strategy updating rules, and by doing so reveal new ways of resolving the prisoner's dilemma.     

The continuous prisoner's dilemma and the evolution of cooperation through reciprocal altruism with variable investment

Understanding the evolutionary origin and persistence of cooperative behavior is a fundamental biological problem. The standard "prisoner's dilemma," which is the most widely adopted framework for studying the evolution of cooperation through reciprocal altruism between unrelated individuals, does not allow for varying degrees of cooperation. Here we study the continuous iterated prisoner's dilemma, in which cooperative investments can vary continuously in each round. This game has been previously considered for a class of reactive strategies in which current investments are based on the partner's previous investment. In the standard iterated prisoner's dilemma, such strategies are inferior to strategies that take into account both players' previous moves, as is exemplified by the evolutionary dominance of "Pavlov" over "tit for tat." Consequently, we extend the analysis of the continuous prisoner's dilemma to a class of strategies in which current investments depend on previous payoffs and, hence, on both players' previous investments. We show, both analytically and by simulation, that payoff-based strategies, which embody the intuitively appealing idea that individuals invest more in cooperative interactions when they profit from these interactions, provide a natural explanation for the gradual evolution of cooperation from an initially noncooperative state and for the maintenance of cooperation thereafter.     

Coevolution of Quantum and Classical Strategies on Evolving Random Networks

We study the coevolution of quantum and classical strategies on weighted and directed random networks in the realm of the prisoner’s dilemma game. During the evolution, agents can break and rewire their links with the aim of maximizing payoffs, and they can also adjust the weights to indicate preferences, either positive or negative, towards their neighbors. The network structure itself is thus also subject to evolution. Importantly, the directionality of links does not affect the accumulation of payoffs nor the strategy transfers, but serves only to designate the owner of each particular link and with it the right to adjust the link as needed. We show that quantum strategies outperform classical strategies, and that the critical temptation to defect at which cooperative behavior can be maintained rises, if the network structure is updated frequently. Punishing neighbors by reducing the weights of their links also plays an important role in maintaining cooperation under adverse conditions. We find that the self-organization of the initially random network structure, driven by the evolutionary competition between quantum and classical strategies, leads to the spontaneous emergence of small average path length and a large clustering coefficient.     

Societal background influences social learning in cooperative decision making

Humans owe their ecological success to their great capacities for social learning and cooperation: learning from others helps individuals adjust to their environment and can promote cooperation in groups. Classic and recent studies indicate that the cultural organization of societies shapes the influence of social information on decision making and suggest that collectivist values (prioritizing the group relative to the individual) increase tendencies to conform to the majority. However, it is unknown whether and how societal background impacts social learning in cooperative interactions. Here we show that social learning in cooperative decision making systematically varies across two societies. We experimentally compare people's basic propensities for social learning in samples from a collectivist (China) and an individualist society (United Kingdom; total n?=?540) in a social dilemma and a coordination game. We demonstrate that Chinese participants base their cooperation decisions on information about their peers much more frequently than their British counterparts. Moreover, our results reveal remarkable societal differences in the type of peer information people consider. In contrast to the consensus view, Chinese participants tend to be substantially less majority-oriented than the British. While Chinese participants are inclined to adopt peer behavior that leads to higher payoffs, British participants tend to cooperate only if sufficiently many peers do so too. These results indicate that the basic processes underlying social transmission are not universal; rather, they vary with cultural conditions. As success-based learning is associated with selfish behavior and majority-based learning can help foster cooperation, our study suggests that in different societies social learning can play diverging roles in the emergence and maintenance of cooperation.     

Direct Reciprocity in Spatial Populations Enhances R-Reciprocity As Well As ST-Reciprocity

As is well-known, spatial reciprocity plays an important role in facilitating the emergence of cooperative traits, and the effect of direct reciprocity is also obvious for explaining the cooperation dynamics. However, how the combination of these two scenarios influences cooperation is still unclear. In the present work, we study the evolution of cooperation in 2×2 games via considering both spatial structured populations and direct reciprocity driven by the strategy with 1-memory length. Our results show that cooperation can be significantly facilitated on the whole parameter plane. For prisoner''s dilemma game, cooperation dominates the system even at strong dilemma, where maximal social payoff is still realized. In this sense, R-reciprocity forms and it is robust to the extremely strong dilemma. Interestingly, when turning to chicken game, we find that ST-reciprocity is also guaranteed, through which social average payoff and cooperation is greatly enhanced. This reciprocity mechanism is supported by mean-field analysis and different interaction topologies. Thus, our study indicates that direct reciprocity in structured populations can be regarded as a more powerful factor for the sustainability of cooperation.     

A Model of Human Cooperation in Social Dilemmas

Social dilemmas are situations in which collective interests are at odds with private interests: pollution, depletion of natural resources, and intergroup conflicts, are at their core social dilemmas. Because of their multidisciplinarity and their importance, social dilemmas have been studied by economists, biologists, psychologists, sociologists, and political scientists. These studies typically explain tendency to cooperation by dividing people in proself and prosocial types, or appealing to forms of external control or, in iterated social dilemmas, to long-term strategies. But recent experiments have shown that cooperation is possible even in one-shot social dilemmas without forms of external control and the rate of cooperation typically depends on the payoffs. This makes impossible a predictive division between proself and prosocial people and proves that people have attitude to cooperation by nature. The key innovation of this article is in fact to postulate that humans have attitude to cooperation by nature and consequently they do not act a priori as single agents, as assumed by standard economic models, but they forecast how a social dilemma would evolve if they formed coalitions and then they act according to their most optimistic forecast. Formalizing this idea we propose the first predictive model of human cooperation able to organize a number of different experimental findings that are not explained by the standard model. We show also that the model makes satisfactorily accurate quantitative predictions of population average behavior in one-shot social dilemmas.     

Effects of temporal clumping and payoff accumulation on impulsiveness and cooperation

Animals show impulsiveness when they prefer a smaller more immediate option, even though a larger more delayed option produces a higher intake rate. This impulsive behavior has implications for several behavioral problems including social cooperation. This paper presents two experiments using captive blue jays (Cyanocitta cristata) that consider the effects of payoff accumulation and temporal clumping on impulsiveness and cooperation. Payoff accumulation refers to a situation where the benefits gained from each choice trial accumulate from one trial to the next, and only become available to the animal after it has completed a fixed number of trials. We hypothesized that this would reduce impulsiveness because it removes the advantage of quickly realizing food gains. Clumping refers to situation in which the animal experiences several choice trials in quick succession followed by a long pause before the next clump. We hypothesized that if payoffs accumulated over a clump of trials this would enhance the effect of accumulation. We tested the effects of accumulation and clumping on impulsiveness in a self-control situation. We found a significant interaction between clumping and accumulation. Payoff accumulation reduced impulsiveness, but only when trials were clumped. Post hoc analyses suggest that clumping alone increases impulsiveness. A second experiment applied these results to cooperation. This experiment reveals an interaction between payoff accumulation and trial's position within the clump. Jays were more likely to cooperate on the first trial of a clump, but the likelihood of cooperation dropped after the first trial. However, this drop was larger when payoffs did not accumulate. This observation suggests that the difference between accumulated and un-accumulated treatments that we reported previously may be largely due to differences in how animals behave in the first trial of a clump.     

Human cooperation in social dilemmas: comparing the Snowdrift game with the Prisoner's Dilemma

Explaining the evolution of cooperation among non-relatives is one of the major challenges for evolutionary biology. In this study, we experimentally examined human cooperation in the iterated Snowdrift game (ISD), which has received little attention so far, and compared it with human cooperation in the iterated Prisoner's Dilemma (IPD), which has become the paradigm for the evolution of cooperation. We show that iteration in the ISD leads to consistently higher levels of cooperation than in the IPD. We further demonstrate that the most successful strategies known for the IPD (generous Tit-for-Tat and Pavlov) were also successfully used in the ISD. Interestingly, we found that female players cooperated significantly more often than male players in the IPD but not in the ISD. Moreover, female players in the IPD applied Tit-for-Tat-like or Pavlovian strategies significantly more often than male players, thereby achieving significantly higher pay-offs than male players did. These data demonstrate that the willingness to cooperate does not only depend on the type of the social dilemma, but also on the class of individuals involved. Altogether, our study shows that the ISD can potentially explain high levels of cooperation among non-relatives in humans. In addition, the ISD seems to reflect the social dilemma more realistically than the IPD because individuals obtain immediate direct benefits from the cooperative acts they perform and costs of cooperation are shared between cooperators.     

Emergence of Super Cooperation of Prisoner’s Dilemma Games on Scale-Free Networks

Recently, the authors proposed a quantum prisoner’s dilemma game based on the spatial game of Nowak and May, and showed that the game can be played classically. By using this idea, we proposed three generalized prisoner’s dilemma (GPD, for short) games based on the weak Prisoner’s dilemma game, the full prisoner’s dilemma game and the normalized Prisoner’s dilemma game, written by GPD_W, GPD_F and GPD_N respectively. Our games consist of two players, each of which has three strategies: cooperator (C), defector (D) and super cooperator (denoted by Q), and have a parameter γ to measure the entangled relationship between the two players. We found that our generalised prisoner’s dilemma games have new Nash equilibrium principles, that entanglement is the principle of emergence and convergence (i.e., guaranteed emergence) of super cooperation in evolutions of our generalised prisoner’s dilemma games on scale-free networks, that entanglement provides a threshold for a phase transition of super cooperation in evolutions of our generalised prisoner’s dilemma games on scale-free networks, that the role of heterogeneity of the scale-free networks in cooperations and super cooperations is very limited, and that well-defined structures of scale-free networks allow coexistence of cooperators and super cooperators in the evolutions of the weak version of our generalised prisoner’s dilemma games.     

Adaptive Dynamics of Extortion and Compliance

Direct reciprocity is a mechanism for the evolution of cooperation. For the iterated prisoner’s dilemma, a new class of strategies has recently been described, the so-called zero-determinant strategies. Using such a strategy, a player can unilaterally enforce a linear relationship between his own payoff and the co-player’s payoff. In particular the player may act in such a way that it becomes optimal for the co-player to cooperate unconditionally. In this way, a player can manipulate and extort his co-player, thereby ensuring that the own payoff never falls below the co-player’s payoff. However, using a compliant strategy instead, a player can also ensure that his own payoff never exceeds the co-player’s payoff. Here, we use adaptive dynamics to study when evolution leads to extortion and when it leads to compliance. We find a remarkable cyclic dynamics: in sufficiently large populations, extortioners play a transient role, helping the population to move from selfish strategies to compliance. Compliant strategies, however, can be subverted by altruists, which in turn give rise to selfish strategies. Whether cooperative strategies are favored in the long run critically depends on the size of the population; we show that cooperation is most abundant in large populations, in which case average payoffs approach the social optimum. Our results are not restricted to the case of the prisoners dilemma, but can be extended to other social dilemmas, such as the snowdrift game. Iterated social dilemmas in large populations do not lead to the evolution of strategies that aim to dominate their co-player. Instead, generosity succeeds.     

Evolutionary dynamics of the continuous iterated prisoner's dilemma

The iterated prisoner's dilemma (IPD) has been widely used in the biological and social sciences to model dyadic cooperation. While most of this work has focused on the discrete prisoner's dilemma, in which actors choose between cooperation and defection, there has been some analysis of the continuous IPD, in which actors can choose any level of cooperation from zero to one. Here, we analyse a model of the continuous IPD with a limited strategy set, and show that a generous strategy achieves the maximum possible payoff against its own type. While this strategy is stable in a neighborhood of the equilibrium point, the equilibrium point itself is always vulnerable to invasion by uncooperative strategies, and hence subject to eventual destabilization. The presence of noise or errors has no effect on this result. Instead, generosity is favored because of its role in increasing contributions to the most efficient level, rather than in counteracting the corrosiveness of noise. Computer simulation using a single-locus infinite alleles Gaussian mutation model suggest that outcomes ranging from a stable cooperative polymorphism to complete collapse of cooperation are possible depending on the magnitude of the mutational variance. Also, making the cost of helping a convex function of the amount of help provided makes it more difficult for cooperative strategies to invade a non-cooperative equilibrium, and for the cooperative equilibrium to resist destabilization by non-cooperative strategies. Finally, we demonstrate that a much greater degree of assortment is required to destabilize a non-cooperative equilibrium in the continuous IPD than in the discrete IPD. The continuous model outlined here suggests that incremental amounts of cooperation lead to rapid decay of cooperation and thus even a large degree of assortment will not be sufficient to allow cooperation to increase when cooperators are rare. The extreme degree of assortment required to destabilize the non-cooperative equilibrium, as well as the instability of the cooperative equilibrium, may help explain why cooperation in Prisoner's Dilemmas is so rare in nature.     

The shared reward dilemma

One of the most direct human mechanisms of promoting cooperation is rewarding it. We study the effect of sharing a reward among cooperators in the most stringent form of social dilemma, namely the prisoner's dilemma (PD). Specifically, for a group of players that collect payoffs by playing a pairwise PD game with their partners, we consider an external entity that distributes a fixed reward equally among all cooperators. Thus, individuals confront a new dilemma: on the one hand, they may be inclined to choose the shared reward despite the possibility of being exploited by defectors; on the other hand, if too many players do that, cooperators will obtain a poor reward and defectors will outperform them. By appropriately tuning the amount to be shared a vast variety of scenarios arises, including the traditional ones in the study of cooperation as well as more complex situations where unexpected behavior can occur. We provide a complete classification of the equilibria of the n-player game as well as of its evolutionary dynamics.     

Is population growth conducive to the sustainability of cooperation?

This paper asks whether population growth is conducive to the sustainability of cooperation. A simple model is developed in which farmers who live around a circular lake engage in trade with their adjacent neighbors. The payoffs from this activity are governed by a prisoner's dilemma “rule of engagement.” Every farmer has one son when the population is not growing, or two sons when it is growing. In the former case, the son takes over the farm when his father dies. In the latter case, one son stays on his father's farm, whereas the other son settles around another lake, along with the “other” sons of the other farmers. During his childhood, each son observes the strategies and the payoffs of his father and of the trading partners of his father, and imitates the most successful strategy when starting farming on his own. Then mutant defectors are introduced into an all-cooperator community. The defector strategy may spread. A comparison is drawn between the impact in terms of the sustainability of cooperation of the appearance of the mutants in a population that is not growing, and in one that is growing. It is shown that the ex-ante probability of sustaining the cooperation strategy is higher for a community that is growing than for a stagnant community.     

Invasion and expansion of cooperators in lattice populations: Prisoner's dilemma vs. snowdrift games

The evolution of cooperation is an enduring conundrum in biology and the social sciences. Two social dilemmas, the prisoner's dilemma and the snowdrift game have emerged as the most promising mathematical metaphors to study cooperation. Spatial structure with limited local interactions has long been identified as a potent promoter of cooperation in the prisoner's dilemma but in the spatial snowdrift game, space may actually enhance or inhibit cooperation. Here we investigate and link the microscopic interaction between individuals to the characteristics of the emerging macroscopic patterns generated by the spatial invasion process of cooperators in a world of defectors. In our simulations, individuals are located on a square lattice with Moore neighborhood and update their strategies by probabilistically imitating the strategies of better performing neighbors. Under sufficiently benign conditions, cooperators can survive in both games. After rapid local equilibration, cooperators expand quadratically until global saturation is reached. Under favorable conditions, cooperators expand as a large contiguous cluster in both games with minor differences concerning the shape of embedded defectors. Under less favorable conditions, however, distinct differences arise. In the prisoner's dilemma, cooperators break up into isolated, compact clusters. The compact clustering reduces exploitation and leads to positive assortment, such that cooperators interact more frequently with other cooperators than with defectors. In contrast, in the snowdrift game, cooperators form small, dendritic clusters, which results in negative assortment and cooperators interact more frequently with defectors than with other cooperators. In order to characterize and quantify the emerging spatial patterns, we introduce a measure for the cluster shape and demonstrate that the macroscopic patterns can be used to determine the characteristics of the underlying microscopic interactions.     

An escape from ‘the prisoner's dilemma‘

Conventional escapes from the paradox that noncooperation between two organisms may be rational, even when cooperation would yield a higher reward to each, are based on the mechanism of reciprocity; but an analytical model of foraging among oviposition sites reveals a more immediate rationale, namely, equivalence of selfishness and altruism. The resulting game is unconditionally the prisoner's dilemma if the players have perfect recognition; however, in the absence thereof and for three different parameter regimes, it yields either the prisoner's dilemma, or two evolutionarily stable strategies, or a unique cooperative ESS. Thus unrecognition can favor cooperation; and environments can exist in which cooperation persists, or even invades, without reciprocity. The results suggest that different mechanisms for cooperation may operate at different levels of neural complexity.     

Chimpanzees coordinate in a negotiation game

A crucially important aspect of human cooperation is the ability to negotiate to cooperative outcomes when interests over resources conflict. Although chimpanzees and other social species may negotiate conflicting interests regarding travel direction or activity timing, very little is known about their ability to negotiate conflicting preferences over food. In the current study, we presented pairs of chimpanzees with a choice between two cooperative tasks—one with equal payoffs (e.g., 5-5) and one with unequal payoffs (higher and lower than in the equal option, e.g., 10-1). This created a conflict of interests between partners with failure to work together on the same cooperative task resulting in no payoff for either partner. The chimpanzee pairs cooperated successfully in as many as 78–94% of the trials across experiments. Even though dominant chimpanzees preferred the unequal option (as they would obtain the largest payoff), subordinate chimpanzees were able to get their way (the equal option) in 22–56% of trials across conditions. Various analyses showed that subjects were both strategic and also cognizant of the strategies used by their partners. These results demonstrate that one of our two closest primate relatives, the chimpanzee, can settle conflicts of interest over resources in mutually satisfying ways—even without the social norms of equity, planned strategies of reciprocity, and the complex communication characteristic of human negotiation.     

Avoiding the bullies: The resilience of cooperation among unequals

Can egalitarian norms or conventions survive the presence of dominant individuals who are ensured of victory in conflicts? We investigate the interaction of power asymmetry and partner choice in games of conflict over a contested resource. Previous models of cooperation do not include both power inequality and partner choice. Furthermore, models that do include power inequalities assume a static game where a bully’s advantage does not change. They have therefore not attempted to model complex and realistic properties of social interaction. Here, we introduce three models to study the emergence and resilience of cooperation among unequals when interaction is random, when individuals can choose their partners, and where power asymmetries dynamically depend on accumulated payoffs. We find that the ability to avoid bullies with higher competitive ability afforded by partner choice mostly restores cooperative conventions and that the competitive hierarchy never forms. Partner choice counteracts the hyper dominance of bullies who are isolated in the network and eliminates the need for others to coordinate in a coalition. When competitive ability dynamically depends on cumulative payoffs, complex cycles of coupled network-strategy-rank changes emerge. Effective collaborators gain popularity (and thus power), adopt aggressive behavior, get isolated, and ultimately lose power. Neither the network nor behavior converge to a stable equilibrium. Despite the instability of power dynamics, the cooperative convention in the population remains stable overall and long-term inequality is completely eliminated. The interaction between partner choice and dynamic power asymmetry is crucial for these results: without partner choice, bullies cannot be isolated, and without dynamic power asymmetry, bullies do not lose their power even when isolated. We analytically identify a single critical point that marks a phase transition in all three iterations of our models. This critical point is where the first individual breaks from the convention and cycles start to emerge.     

Chimpanzees and children avoid mutual defection in a social dilemma

Cooperation often comes with the temptation to defect and benefit at the cost of others. This tension between cooperation and defection is best captured in social dilemmas like the Prisoner's Dilemma. Adult humans have specific strategies to maintain cooperation during Prisoner's Dilemma interactions. Yet, little is known about the ontogenetic and phylogenetic origins of human decision-making strategies in conflict scenarios. To shed light on this question, we compared the strategies used by chimpanzees and 5-year old children to overcome a social dilemma. In our task, waiting for the partner to act first produced the best results for the subject. Alternatively, they could mutually cooperate and divide the rewards. Our findings indicate that the two species differed substantially in their strategies to solve the task. Chimpanzees became more strategic across the study period by waiting longer to act in the social dilemma. Children developed a more efficient strategy of taking turns to reciprocate their rewards. Moreover, children used specific types of communication to coordinate with their partners. These results suggest that while both species behaved strategically to overcome a conflict situation, only children engaged in active cooperation to solve a social dilemma.     

The establishment of communication systems depends on the scale of competition

How communication systems emerge and remain stable is an important question in both cognitive science and evolutionary biology. For communication to arise, not only must individuals cooperate by signaling reliable information, but they must also coordinate and perpetuate signals. Most studies on the emergence of communication in humans typically consider scenarios where individuals implicitly share the same interests. Likewise, most studies on human cooperation consider scenarios where shared conventions of signals and meanings cannot be developed de novo. Here, we combined both approaches with an economic experiment where participants could develop a common language, but under different conditions fostering or hindering cooperation. Participants endeavored to acquire a resource through a learning task in a computer-based environment. After this task, participants had the option to transmit a signal (a color) to a fellow group member, who would subsequently play the same learning task. We varied the way participants competed with each other (either global scale or local scale) and the cost of transmitting a signal (either costly or noncostly) and tracked the way in which signals were used as communication among players. Under global competition, players signaled more often and more consistently, scored higher individual payoffs, and established shared associations of signals and meanings. In addition, costly signals were also more likely to be used under global competition; whereas under local competition, fewer signals were sent and no effective communication system was developed. Our results demonstrate that communication involves both a coordination and a cooperative dilemma and show the importance of studying language evolution under different conditions influencing human cooperation.     

How is human cooperation different?

Although cooperation is a widespread phenomenon in nature, human cooperation exceeds that of all other species with regard to the scale and range of cooperative activities. Here we review and discuss differences between humans and non-humans in the strategies employed to maintain cooperation and control free-riders. We distinguish forms of cooperative behaviour based on their influence on the immediate payoffs of actor and recipient. If the actor has immediate costs and only the recipient obtains immediate benefits, we term this investment. If the behaviour has immediate positive effects for both actor and recipient, we call this a self-serving mutually beneficial behaviour or mutual cooperation. We argue that humans, in contrast to all other species, employ a wider range of enforcement mechanisms, which allow higher levels of cooperation to evolve and stabilize among unrelated individuals and in large groups. We also discuss proximate mechanisms underlying cooperative behaviour and focus on our experimental work with humans and our closest primate relatives. Differences in the proximate mechanisms also seem to contribute to explaining humans'' greater ability to cooperate and enforce cooperation.