Friendship, cliquishness, and the emergence of cooperation

The evolution of cooperation is a central problem in biology and the social sciences. While theoretical work using the iterated prisoner's dilemma (IPD) has shown that cooperation among non-kin can be sustained among reciprocal strategies (i.e. tit-for-tat), these results are sensitive to errors in strategy execution, cyclical invasions by free riders, and the specific ecology of strategies. Moreover, the IPD assumes that a strategy's probability of playing the PD game with other individuals is independent of the decisions made by others. Here, we remove the assumption of independent pairing by studying a more plausible cooperative dilemma in which players can preferentially interact with a limited set of known partners and also deploy longer-term accounting strategies that can counteract the effects of random errors. We show that cooperative strategies readily emerge and persist in a range of noisy environments, with successful cooperative strategies (henceforth, cliquers) maintaining medium-term memories for partners and low thresholds for acceptable cooperation (i.e. forgiveness). The success of these strategies relies on their cliquishness-a propensity to defect with strangers if they already have an adequate number of partners. Notably, this combination of medium-term accounting, forgiveness, and cliquishness fits with empirical studies of friendship and other long-term relationships among humans.     

The evolution of punishment and apology: an iterated prisoner's dilemma model

Recently, behaviors that seem to function as punishment or apology have been reported among non-human primates as well as humans. Such behaviors appear to play an important role in maintaining cooperation between individuals. Therefore, the evolution of these behaviors should be examined from the viewpoint of the evolution of cooperation. The iterated prisoner's dilemma (IPD) game is generally considered to be a standard model for the evolution of cooperation. In the present study, strategies accompanied by punishment-like attacks or apology-like behavior were introduced into the common IPD simulation. Punishment and apology were represented by the P signal and the AS signal given immediately after defection. A strategy with the P and AS signals, named the pPAS strategy, was proved to be an evolutionarily stable strategy under certain conditions. Numerical simulations were carried out according to different assigned values of the costs of punishment and apology. The simulations showed that pPAS could dominate the population (1) when the cost of giving P is relatively small, (2) when the cost of receiving P is relatively large, or (3) when the cost of giving AS is relatively large. The relative cost of giving AS had the clearest effect on the success of pPAS. pPAS can dominate the population even when a dominance asymmetry of the costs between two players was introduced. The present results suggest the possible evolution of social behaviors like punishment or apology as a means of maintaining cooperation. This revised version was published online in November 2006 with corrections to the Cover Date.     

Resolving the iterated prisoner's dilemma: theory and reality

Pairs of unrelated individuals face a prisoner's dilemma if cooperation is the best mutual outcome, but each player does best to defect regardless of his partner's behaviour. Although mutual defection is the only evolutionarily stable strategy in one-shot games, cooperative solutions based on reciprocity can emerge in iterated games. Among the most prominent theoretical solutions are the so-called bookkeeping strategies, such as tit-for-tat, where individuals copy their partner's behaviour in the previous round. However, the lack of empirical data conforming to predicted strategies has prompted the suggestion that the iterated prisoner's dilemma (IPD) is neither a useful nor realistic basis for investigating cooperation. Here, we discuss several recent studies where authors have used the IPD framework to interpret their data. We evaluate the validity of their approach and highlight the diversity of proposed solutions. Strategies based on precise accounting are relatively uncommon, perhaps because the full set of assumptions of the IPD model are rarely satisfied. Instead, animals use a diverse array of strategies that apparently promote cooperation, despite the temptation to cheat. These include both positive and negative reciprocity, as well as long-term mutual investments based on 'friendships'. Although there are various gaps in these studies that remain to be filled, we argue that in most cases, individuals could theoretically benefit from cheating and that cooperation cannot therefore be explained with the concept of positive pseudo-reciprocity. We suggest that by incorporating empirical data into the theoretical framework, we may gain fundamental new insights into the evolution of mutual reciprocal investment in nature.     

Beyond the prisoner's dilemma: Toward models to discriminate among mechanisms of cooperation in nature

The iterated prisoner's dilemma game, or IPD, has now established itself as the orthodox paradigm for theoretical investigations of the evolution of cooperation; but its scope is restricted to reciprocity, which is only one of three categories of cooperation among unrelated individuals. Even within that category, a cooperative encounter has in general three phases, and the IPD has nothing to say about two of them. To distinguish among mechanisms of cooperation in nature, future theoretical work on the evolution of cooperation must distance itself from economics and develop games as a refinement of ethology's comparative approach.     

Evolutionary dynamics of the continuous iterated prisoner's dilemma

The iterated prisoner's dilemma (IPD) has been widely used in the biological and social sciences to model dyadic cooperation. While most of this work has focused on the discrete prisoner's dilemma, in which actors choose between cooperation and defection, there has been some analysis of the continuous IPD, in which actors can choose any level of cooperation from zero to one. Here, we analyse a model of the continuous IPD with a limited strategy set, and show that a generous strategy achieves the maximum possible payoff against its own type. While this strategy is stable in a neighborhood of the equilibrium point, the equilibrium point itself is always vulnerable to invasion by uncooperative strategies, and hence subject to eventual destabilization. The presence of noise or errors has no effect on this result. Instead, generosity is favored because of its role in increasing contributions to the most efficient level, rather than in counteracting the corrosiveness of noise. Computer simulation using a single-locus infinite alleles Gaussian mutation model suggest that outcomes ranging from a stable cooperative polymorphism to complete collapse of cooperation are possible depending on the magnitude of the mutational variance. Also, making the cost of helping a convex function of the amount of help provided makes it more difficult for cooperative strategies to invade a non-cooperative equilibrium, and for the cooperative equilibrium to resist destabilization by non-cooperative strategies. Finally, we demonstrate that a much greater degree of assortment is required to destabilize a non-cooperative equilibrium in the continuous IPD than in the discrete IPD. The continuous model outlined here suggests that incremental amounts of cooperation lead to rapid decay of cooperation and thus even a large degree of assortment will not be sufficient to allow cooperation to increase when cooperators are rare. The extreme degree of assortment required to destabilize the non-cooperative equilibrium, as well as the instability of the cooperative equilibrium, may help explain why cooperation in Prisoner's Dilemmas is so rare in nature.     

The importance of mechanisms for the evolution of cooperation

Studies aimed at explaining the evolution of phenotypic traits have often solely focused on fitness considerations, ignoring underlying mechanisms. In recent years, there has been an increasing call for integrating mechanistic perspectives in evolutionary considerations, but it is not clear whether and how mechanisms affect the course and outcome of evolution. To study this, we compare four mechanistic implementations of two well-studied models for the evolution of cooperation, the Iterated Prisoner''s Dilemma (IPD) game and the Iterated Snowdrift (ISD) game. Behavioural strategies are either implemented by a 1 : 1 genotype–phenotype mapping or by a simple neural network. Moreover, we consider two different scenarios for the effect of mutations. The same set of strategies is feasible in all four implementations, but the probability that a given strategy arises owing to mutation is largely dependent on the behavioural and genetic architecture. Our individual-based simulations show that this has major implications for the evolutionary outcome. In the ISD, different evolutionarily stable strategies are predominant in the four implementations, while in the IPD each implementation creates a characteristic dynamical pattern. As a consequence, the evolved average level of cooperation is also strongly dependent on the underlying mechanism. We argue that our findings are of general relevance for the evolution of social behaviour, pleading for the integration of a mechanistic perspective in models of social evolution.     

Critical dynamics in the evolution of stochastic strategies for the iterated prisoner's dilemma

The observed cooperation on the level of genes, cells, tissues, and individuals has been the object of intense study by evolutionary biologists, mainly because cooperation often flourishes in biological systems in apparent contradiction to the selfish goal of survival inherent in Darwinian evolution. In order to resolve this paradox, evolutionary game theory has focused on the Prisoner's Dilemma (PD), which incorporates the essence of this conflict. Here, we encode strategies for the iterated Prisoner's Dilemma (IPD) in terms of conditional probabilities that represent the response of decision pathways given previous plays. We find that if these stochastic strategies are encoded as genes that undergo Darwinian evolution, the environmental conditions that the strategies are adapting to determine the fixed point of the evolutionary trajectory, which could be either cooperation or defection. A transition between cooperative and defective attractors occurs as a function of different parameters such as mutation rate, replacement rate, and memory, all of which affect a player's ability to predict an opponent's behavior. These results imply that in populations of players that can use previous decisions to plan future ones, cooperation depends critically on whether the players can rely on facing the same strategies that they have adapted to. Defection, on the other hand, is the optimal adaptive response in environments that change so quickly that the information gathered from previous plays cannot usefully be integrated for a response.     

The continuous prisoner's dilemma and the evolution of cooperation through reciprocal altruism with variable investment

Understanding the evolutionary origin and persistence of cooperative behavior is a fundamental biological problem. The standard "prisoner's dilemma," which is the most widely adopted framework for studying the evolution of cooperation through reciprocal altruism between unrelated individuals, does not allow for varying degrees of cooperation. Here we study the continuous iterated prisoner's dilemma, in which cooperative investments can vary continuously in each round. This game has been previously considered for a class of reactive strategies in which current investments are based on the partner's previous investment. In the standard iterated prisoner's dilemma, such strategies are inferior to strategies that take into account both players' previous moves, as is exemplified by the evolutionary dominance of "Pavlov" over "tit for tat." Consequently, we extend the analysis of the continuous prisoner's dilemma to a class of strategies in which current investments depend on previous payoffs and, hence, on both players' previous investments. We show, both analytically and by simulation, that payoff-based strategies, which embody the intuitively appealing idea that individuals invest more in cooperative interactions when they profit from these interactions, provide a natural explanation for the gradual evolution of cooperation from an initially noncooperative state and for the maintenance of cooperation thereafter.     

Different reactions to adverse neighborhoods in games of cooperation

In social dilemmas, cooperation among randomly interacting individuals is often difficult to achieve. The situation changes if interactions take place in a network where the network structure jointly evolves with the behavioral strategies of the interacting individuals. In particular, cooperation can be stabilized if individuals tend to cut interaction links when facing adverse neighborhoods. Here we consider two different types of reaction to adverse neighborhoods, and all possible mixtures between these reactions. When faced with a gloomy outlook, players can either choose to cut and rewire some of their links to other individuals, or they can migrate to another location and establish new links in the new local neighborhood. We find that in general local rewiring is more favorable for the evolution of cooperation than emigration from adverse neighborhoods. Rewiring helps to maintain the diversity in the degree distribution of players and favors the spontaneous emergence of cooperative clusters. Both properties are known to favor the evolution of cooperation on networks. Interestingly, a mixture of migration and rewiring is even more favorable for the evolution of cooperation than rewiring on its own. While most models only consider a single type of reaction to adverse neighborhoods, the coexistence of several such reactions may actually be an optimal setting for the evolution of cooperation.     

Correlated pay-offs are key to cooperation

The general belief that cooperation and altruism in social groups result primarily from kin selection has recently been challenged, not least because results from cooperatively breeding insects and vertebrates have shown that groups may be composed mainly of non-relatives. This allows testing predictions of reciprocity theory without the confounding effect of relatedness. Here, we review complementary and alternative evolutionary mechanisms to kin selection theory and provide empirical examples of cooperative behaviour among unrelated individuals in a wide range of taxa. In particular, we focus on the different forms of reciprocity and on their underlying decision rules, asking about evolutionary stability, the conditions selecting for reciprocity and the factors constraining reciprocal cooperation. We find that neither the cognitive requirements of reciprocal cooperation nor the often sequential nature of interactions are insuperable stumbling blocks for the evolution of reciprocity. We argue that simple decision rules such as ‘help anyone if helped by someone’ should get more attention in future research, because empirical studies show that animals apply such rules, and theoretical models find that they can create stable levels of cooperation under a wide range of conditions. Owing to its simplicity, behaviour based on such a heuristic may in fact be ubiquitous. Finally, we argue that the evolution of exchange and trading of service and commodities among social partners needs greater scientific focus.     

Cooperation among Norway Rats: The Importance of Visual Cues for Reciprocal Cooperation,and the Role of Coercion

Some animals reciprocate help, but the underlying proximate mechanisms are largely unclear. Norway rats (Rattus norvegicus) have been shown to cooperate in a variant of the iterated prisoner's dilemma paradigm, yet it is unknown which sensory modalities they use. Visual information is often implicitly assumed to play a major role in social interactions, but primarily nocturnal species such as Norway rats may rely on different cues when deciding to reciprocate received help. We used an instrumental cooperative task to compare the test rats' propensity to reciprocate received help between two experimental conditions, with and without visual information exchange between social partners. Our results show that visual information is not required for reciprocal cooperation among social partners because even when it was lacking, test rats provided food significantly earlier to partners that had helped them to obtain food before than to those that had not done so. The mean decision speed did not differ between the two experimental conditions, with or without visual information. Social partners sometimes showed aggressive behaviour towards focal test individuals. When including this in the analyses to assess the possible role of aggression as a trigger of cooperation, aggression received from cooperators apparently reduced the cooperation propensity, whereas aggression received from defectors increased it. Hence, in addition to reciprocity, coercion seems to provide additional means to generate altruistic help in Norway rats.     

Female cleaner fish cooperate more with unfamiliar males

Joint group membership is of major importance for cooperation in humans, and close ties or familiarity with a partner are also thought to promote cooperation in other animals. Here, we present the opposite pattern: female cleaner fish, Labroides dimidiatus, behave more cooperatively (by feeding more against their preference) when paired with an unfamiliar male rather than with their social partner. We propose that cooperation based on asymmetric punishment causes this reversed pattern. Males are larger than and dominant to female partners and are more aggressive to unfamiliar than to familiar female partners. In response, females behave more cooperatively with unfamiliar male partners. Our data suggest that in asymmetric interactions, weaker players might behave more cooperatively with out-group members than with in-group members to avoid harsher punishment.     

The evolution of reciprocity in sizable groups

Recently, several authors have investigated the evolution of reciprocal altruism using the repeated prisoner's dilemma game. These models suggest that natural selection is likely to favor behavioral strategies leading to reciprocal cooperation when pairs of individuals interact repeatedly in potentially cooperative situations. Using the repeated n-person prisoner's dilemma game, we consider whether reciprocal altruism is also likely to evolve when social interactions involve more individuals. We show that the conditions that allow the evolution of reciprocal cooperation become extremely restrictive as group size increases.     

Second‐order cooperation: Cooperative offspring as a living public good arising from second‐order selection on non‐cooperative individuals

Switching rate between cooperating and non‐cooperating genotypes is a crucial social evolution factor, often neglected by game theory‐inspired theoretical and experimental frameworks. We show that the evolution of alleles increasing the mutation or phenotypic switching rates toward cooperation is in itself a social dilemma. Although cooperative offspring are often unlikely to reproduce, due to high cost of cooperation, they can be seen both as a living public good and a part of the extended parental phenotype. The competition between individuals that generate cooperators and ones that do not is often more relevant than the competition between cooperators and non‐cooperators. The dilemma of second‐order cooperation we describe relates directly to eusociality, but can be also interpreted as a division of labor or a soma‐germline distinction. The results of our simulations shine a new light on what Darwin had already termed a “special difficulty” of evolutionary theory and describe a novel type of cooperation dynamics.     

Whence tit-for-tat?

Summary In theoretical and empirical studies of the evolution of cooperation, the tit-for-tat strategy (i.e. cooperate unless your partner did not cooperate in the previous interaction) is widely considered to be of central importance. Nevertheless, surprisingly little is known about the conditions in which tit-for-tat appears and disappears across generations in a population of interacting individuals. Here, we apply a newly developed classifier-system model (EvA) in addressing this issue when the key features of interactions are caricatured using the iterated prisoner's dilemma game. Our simple representation of behavioural strategies as algorithms composed of two interacting rules allowed us to determine conditions in which tit-for-tat can replace noncooperative strategies and vice versa. Using direct game-theoretic analysis and simulations with the EvA model, we determined that no strategy is evolutionarily stable, but larger population sizes and longer sequences of interactions between individuals can yield transient dominance by tit-for-tat. Genetic drift among behaviourally equivalent strategies is the key mechanism underlying this dominance. Our analysis suggests that tit-for-tat could be important in nature for cognitively simple organisms of limited memory capacity, in strongly kin-selected or group-selected populations, when interaction sequences between individuals are relatively short, in moderate-sized populations of widely interacting individuals and when defectors appear in the population with moderate frequency.     

Evolution of interactions and cooperation in the spatial prisoner's dilemma game

We study the evolution of cooperation in the spatial prisoner's dilemma game where players are allowed to establish new interactions with others. By employing a simple coevolutionary rule entailing only two crucial parameters, we find that different selection criteria for the new interaction partners as well as their number vitally affect the outcome of the game. The resolution of the social dilemma is most probable if the selection favors more successful players and if their maximally attainable number is restricted. While the preferential selection of the best players promotes cooperation irrespective of game parametrization, the optimal number of new interactions depends somewhat on the temptation to defect. Our findings reveal that the "making of new friends" may be an important activity for the successful evolution of cooperation, but also that partners must be selected carefully and their number limited.     

Invasion and expansion of cooperators in lattice populations: Prisoner's dilemma vs. snowdrift games

The evolution of cooperation is an enduring conundrum in biology and the social sciences. Two social dilemmas, the prisoner's dilemma and the snowdrift game have emerged as the most promising mathematical metaphors to study cooperation. Spatial structure with limited local interactions has long been identified as a potent promoter of cooperation in the prisoner's dilemma but in the spatial snowdrift game, space may actually enhance or inhibit cooperation. Here we investigate and link the microscopic interaction between individuals to the characteristics of the emerging macroscopic patterns generated by the spatial invasion process of cooperators in a world of defectors. In our simulations, individuals are located on a square lattice with Moore neighborhood and update their strategies by probabilistically imitating the strategies of better performing neighbors. Under sufficiently benign conditions, cooperators can survive in both games. After rapid local equilibration, cooperators expand quadratically until global saturation is reached. Under favorable conditions, cooperators expand as a large contiguous cluster in both games with minor differences concerning the shape of embedded defectors. Under less favorable conditions, however, distinct differences arise. In the prisoner's dilemma, cooperators break up into isolated, compact clusters. The compact clustering reduces exploitation and leads to positive assortment, such that cooperators interact more frequently with other cooperators than with defectors. In contrast, in the snowdrift game, cooperators form small, dendritic clusters, which results in negative assortment and cooperators interact more frequently with defectors than with other cooperators. In order to characterize and quantify the emerging spatial patterns, we introduce a measure for the cluster shape and demonstrate that the macroscopic patterns can be used to determine the characteristics of the underlying microscopic interactions.     

Correlates of cooperation in a one-shot high-stakes televised prisoners' dilemma

Explaining cooperation between non-relatives is a puzzle for both evolutionary biology and the social sciences. In humans, cooperation is often studied in a laboratory setting using economic games such as the prisoners' dilemma. However, such experiments are sometimes criticized for being played for low stakes and by misrepresentative student samples. Golden balls is a televised game show that uses the prisoners' dilemma, with a diverse range of participants, often playing for very large stakes. We use this non-experimental dataset to investigate the factors that influence cooperation when "playing" for considerably larger stakes than found in economic experiments. The game show has earlier stages that allow for an analysis of lying and voting decisions. We found that contestants were sensitive to the stakes involved, cooperating less when the stakes were larger in both absolute and relative terms. We also found that older contestants were more likely to cooperate, that liars received less cooperative behavior, but only if they told a certain type of lie, and that physical contact was associated with reduced cooperation, whereas laughter and promises were reliable signals or cues of cooperation, but were not necessarily detected.     

Kin selection may inhibit the evolution of reciprocation

Kin selection and reciprocal cooperation provide two candidate explanations for the evolution of cooperation. Models of the evolution of cooperation have typically focussed on one or the other mechanism, despite claims that kin selection could pave the way for the evolution of reciprocal cooperation. We describe a computer simulation model that explicitly supports both kin selection and reciprocal cooperation. The model simulates a viscous population of discrete individuals with social interaction taking the form of the Prisoner's Dilemma and selection acting on performance in these interactions. We recount how the analytical and empirical study of this model led to the conclusion that kin selection may actually inhibit the evolution of effective strategies for establishing reciprocal cooperation.     

Resolution of the stochastic strategy spatial prisoner's dilemma by means of particle swarm optimization

We study the evolution of cooperation among selfish individuals in the stochastic strategy spatial prisoner's dilemma game. We equip players with the particle swarm optimization technique, and find that it may lead to highly cooperative states even if the temptations to defect are strong. The concept of particle swarm optimization was originally introduced within a simple model of social dynamics that can describe the formation of a swarm, i.e., analogous to a swarm of bees searching for a food source. Essentially, particle swarm optimization foresees changes in the velocity profile of each player, such that the best locations are targeted and eventually occupied. In our case, each player keeps track of the highest payoff attained within a local topological neighborhood and its individual highest payoff. Thus, players make use of their own memory that keeps score of the most profitable strategy in previous actions, as well as use of the knowledge gained by the swarm as a whole, to find the best available strategy for themselves and the society. Following extensive simulations of this setup, we find a significant increase in the level of cooperation for a wide range of parameters, and also a full resolution of the prisoner's dilemma. We also demonstrate extreme efficiency of the optimization algorithm when dealing with environments that strongly favor the proliferation of defection, which in turn suggests that swarming could be an important phenomenon by means of which cooperation can be sustained even under highly unfavorable conditions. We thus present an alternative way of understanding the evolution of cooperative behavior and its ubiquitous presence in nature, and we hope that this study will be inspirational for future efforts aimed in this direction.