Transforming the dilemma

How does natural selection lead to cooperation between competing individuals? The Prisoner's Dilemma captures the essence of this problem. Two players can either cooperate or defect. The payoff for mutual cooperation, R, is greater than the payoff for mutual defection, P. But a defector versus a cooperator receives the highest payoff, T, where as the cooperator obtains the lowest payoff, S. Hence, the Prisoner's Dilemma is defined by the payoff ranking T > R > P > S . In a well‐mixed population, defectors always have a higher expected payoff than cooperators, and therefore natural selection favors defectors. The evolution of cooperation requires specific mechanisms. Here we discuss five mechanisms for the evolution of cooperation: direct reciprocity, indirect reciprocity, kin selection, group selection, and network reciprocity (or graph selection). Each mechanism leads to a transformation of the Prisoner's Dilemma payoff matrix. From the transformed matrices, we derive the fundamental conditions for the evolution of cooperation. The transformed matrices can be used in standard frameworks of evolutionary dynamics such as the replicator equation or stochastic processes of game dynamics in finite populations.     

The excuse principle can maintain cooperation through forgivable defection in the Prisoner's Dilemma game

Reciprocal altruism describes a situation in which an organism acts in a manner that temporarily reduces its fitness while increasing another organism''s fitness, but there is an ultimate fitness benefit based on an expectation that the other organism will act in a similar manner at a later time. It creates the obvious dilemma in which there is always a short-term benefit to cheating, therefore cooperating individuals must avoid being exploited by non-cooperating cheaters. This is achieved by following various decision rules, usually variants of the tit-for-tat (TFT) strategy. The strength of TFT, however, is also its weakness—mistakes in implementation or interpretation of moves, or the inability to cooperate, lead to a permanent breakdown in cooperation. We show that pied flycatchers (Ficedula hypoleuca) use a TFT with an embedded ‘excuse principle’ to forgive the neighbours that were perceived as unable to cooperate during mobbing of predators. The excuse principle dramatically increases the stability of TFT-like behavioural strategies within the Prisoner''s Dilemma game.     

The Prisoner's Dilemma and polymorphism in yeast SUC genes

The SUC multigene family of the single-celled yeast Saccharomyces cerevisiae is polymorphic, with genes varying both in number and activity. All of the genes encode invertase, an enzyme that is secreted to digest sucrose outside of the cell. This communal endeavour creates the potential for individual cells to defect (cheat) by stealing the sugar digested by their neighbours without contributing the enzyme themselves. We measured the fitness of a defector, with a deleted suc2 gene, relative to an otherwise isogenic cooperator, with a functional SUC2 gene. We manipulated the level of social interaction within the community by varying the population density and found that the defector is less fit than the cooperator at low levels of sociality but more fit in dense communities. We propose that selection for antisocial cheating causes SUC polymorphism in nature. The infamous Prisoner's Dilemma game shows that social behaviour is generally unstable, and the success of both cooperation and defection can vary continuously in time and space. The variation in SUC genes reflects constant adaptation to an ever-changing biotic environment that is a consequence of the instability of cooperation. It is interesting that social interactions can have a direct effect on molecular evolution, even in an organism as simple as yeast.     

Effects of increasing the number of players and memory size in the iterated Prisoner's Dilemma: a numerical approach

The Prisoner''s Dilemma has become a paradigm for the evolution of altruistic behaviour. Here we present results of numerical simulations of the infinitely iterated stochastic simultaneous Prisoner''s Dilemma considering players with longer memory, encounters of more than two players as well as different pay-off values. This provides us with a better foundation to compare theoretical results to experimental data. We show that the success of the strategy Pavlov, regardless of its simplicity, is far more general by having an outstanding role in the iterated N-player N-memory Prisoner''s Dilemma. Besides, we study influences of increased memory sizes in the iterated two-player Prisoner''s Dilemma, and present comparisons to results of experiments with first-year students.     

Facial expressions as honest signals of cooperative intent in a one-shot anonymous Prisoner's Dilemma game

Previous research has posited that facial expressions of emotion may serve as honest signals of cooperation. Although findings from several empirical studies support this position, prior studies have not used comprehensive and dynamic measures of facial expression as potential predictors of behaviorally defined cooperation. The authors investigated (a) specific positive and negative facial actions displayed among strangers immediately following verbal promises of commitment within an unrestricted acquaintance period and (b) anonymous, behaviorally defined decisions of cooperation or defection in a one-shot, two-person Prisoner's Dilemma game occurring directly following the acquaintance period. The Facial Action Coding System [Ekman P. & Friesen W.V. (1978). Facial Action Coding System. Palo Alto, CA: Consulting Psychology Press] was used to measure affect-related facial actions. It was found that facial actions related to enjoyment were predictive of cooperative decisions within dyads; additionally, facial actions related to contempt were predictive of noncooperative decisions within dyads. Furthermore, and consistent with previous works, participants were able to accurately predict their partner's decisions following the acquaintance period. These results suggest that facial actions may function as honest signals of cooperative intent. These findings also provide a possible explanation for the association between subjective affective experience and facial expression that advances understanding of cooperative behavior.     

Stochastic evolutionary dynamics of direct reciprocity

Evolutionary game theory is the study of frequency-dependent selection. The success of an individual depends on the frequencies of strategies that are used in the population. We propose a new model for studying evolutionary dynamics in games with a continuous strategy space. The population size is finite. All members of the population use the same strategy. A mutant strategy is chosen from some distribution over the strategy space. The fixation probability of the mutant strategy in the resident population is calculated. The new mutant takes over the population with this probability. In this case, the mutant becomes the new resident. Otherwise, the existing resident remains. Then, another mutant is generated. These dynamics lead to a stationary distribution over the entire strategy space. Our new approach generalizes classical adaptive dynamics in three ways: (i) the population size is finite; (ii) mutants can be drawn non-locally and (iii) the dynamics are stochastic. We explore reactive strategies in the repeated Prisoner''s Dilemma. We perform ‘knock-out experiments’ to study how various strategies affect the evolution of cooperation. We find that ‘tit-for-tat’ is a weak catalyst for the emergence of cooperation, while ‘always cooperate’ is a strong catalyst for the emergence of defection. Our analysis leads to a new understanding of the optimal level of forgiveness that is needed for the evolution of cooperation under direct reciprocity.     

Capuchin monkeys (Sapajus [Cebus] apella) play Nash equilibria in dynamic games,but their decisions are likely not influenced by oxytocin

Comparative approaches to experimental economics have shed light on the evolution of social decision‐making across a range of primate species, including humans. Here we replicate our previous work looking at six pairs of capuchin monkeys' (Sapajus [Cebus] apella) responses to scenarios requiring both coordination (Assurance Game) and anti‐coordination (Hawk‐Dove Game). This then provides a foundation for assessing their responses to two additional games, one with a scenario of beneficial cooperation with a temptation to defect (Prisoner's Dilemma) and one with an environment requiring changing strategies within short temporal proximity (Alternating Economic Game). We additionally explored the effects of exogenous oxytocin on decision‐making. Oxytocin did not affect decisions in any of our games. Results from the first two games largely replicated our previous findings. Responses to the Prisoner's Dilemma were more varied than was seen in previous games, with pairs respectively cooperating, defecting, and failing to establish stable strategies. Such variability indicates that this game may be a good assay for individual differences in social decision‐making. Finally, capuchins were able to flexibly switch between their previously established strategies within each of the different games, even when the games were presented within the same session, requiring strategy adjustments within short temporal proximity. These results build on earlier findings showing that capuchins can alter decision‐making strategies as the context demands, which is likely essential for decision‐making in naturally occurring contexts.     

Cooperation and Stability through Periodic Impulses

Basic games, where each individual chooses between two strategies, illustrate several issues that immediately emerge from the standard approach that applies strategic reasoning, based on rational decisions, to predict population behavior where no rationality is assumed. These include how mutual cooperation (which corresponds to the best outcome from the population perspective) can evolve when the only individually rational choice is to defect, illustrated by the Prisoner''s Dilemma (PD) game, and how individuals can randomize between two strategies when neither is individually rational, illustrated by the Battle of the Sexes (BS) game that models male-female conflict over parental investment in offspring. We examine these questions from an evolutionary perspective where the evolutionary dynamics includes an impulsive effect that models sudden changes in collective population behavior. For the PD game, we show analytically that cooperation can either coexist with defection or completely take over the population, depending on the strength of the impulse. By extending these results for the PD game, we also show that males and females each evolve to a single strategy in the BS game when the impulsive effect is strong and that weak impulses stabilize the randomized strategies of this game.     

Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game

Understanding the mechanisms that can lead to the evolution of cooperation through natural selection is a core problem in biology. Among the various attempts at constructing a theory of cooperation, game theory has played a central role. Here, we review models of cooperation that are based on two simple games: the Prisoner's Dilemma, and the Snowdrift game. Both games are two‐person games with two strategies, to cooperate and to defect, and both games are social dilemmas. In social dilemmas, cooperation is prone to exploitation by defectors, and the average payoff in populations at evolutionary equilibrium is lower than it would be in populations consisting of only cooperators. The difference between the games is that cooperation is not maintained in the Prisoner's Dilemma, but persists in the Snowdrift game at an intermediate frequency. As a consequence, insights gained from studying extensions of the two games differ substantially. We review the most salient results obtained from extensions such as iteration, spatial structure, continuously variable cooperative investments, and multi‐person interactions. Bridging the gap between theoretical and empirical research is one of the main challenges for future studies of cooperation, and we conclude by pointing out a number of promising natural systems in which the theory can be tested experimentally.     

Evolution in heterogeneous environments: Effects of migration on habitat specialization

Summary Richard Levins introduced fitness sets as a tool for investigating evolution within heterogeneous environments. Evolutionary game theory permits a synthesis and generalization of this approach by considering the evolutionary response of organisms to any scale of habitat heterogeneity. As scales of heterogeneity increase from fine to coarse, the evolutionary stable strategy (ESS) switches from a single generalist species to several species that become increasingly specialized on distinct habitats. Depending upon the organisms' ecology, the switch from one to two species may occur at high migration rates (relatively fine-grained environment), or may only occur at very low migration rates (coarse-grained environment). At the ESS, the evolutionary context of a species is the entire landscape, while its ecological context may be a single habitat.Evolution towards the ESS can be represented with adaptive landscapes. In the absence of frequency-dependence, shifting from a single strategy ESS to a two strategy ESS poses the problem of evolving across valleys in the adaptive surface to occupy new peaks (hence, Sewell Wright's shifting balance theory). Frequency-dependent processes facilitate evolution across valleys. If a system with a two strategy ESS is constrained to possess a single strategy, the population may actually evolve a strategy that minimizes fitness. Because the population now rests at the bottom of a valley, evolution by natural selection can drive populations to occupy both peaks.     

The importance of mechanisms for the evolution of cooperation

Studies aimed at explaining the evolution of phenotypic traits have often solely focused on fitness considerations, ignoring underlying mechanisms. In recent years, there has been an increasing call for integrating mechanistic perspectives in evolutionary considerations, but it is not clear whether and how mechanisms affect the course and outcome of evolution. To study this, we compare four mechanistic implementations of two well-studied models for the evolution of cooperation, the Iterated Prisoner''s Dilemma (IPD) game and the Iterated Snowdrift (ISD) game. Behavioural strategies are either implemented by a 1 : 1 genotype–phenotype mapping or by a simple neural network. Moreover, we consider two different scenarios for the effect of mutations. The same set of strategies is feasible in all four implementations, but the probability that a given strategy arises owing to mutation is largely dependent on the behavioural and genetic architecture. Our individual-based simulations show that this has major implications for the evolutionary outcome. In the ISD, different evolutionarily stable strategies are predominant in the four implementations, while in the IPD each implementation creates a characteristic dynamical pattern. As a consequence, the evolved average level of cooperation is also strongly dependent on the underlying mechanism. We argue that our findings are of general relevance for the evolution of social behaviour, pleading for the integration of a mechanistic perspective in models of social evolution.     

Natural selection of cooperation and degree hierarchy in heterogeneous populations

One of the current theoretical challenges to the explanatory powers of Evolutionary Theory is the understanding of the observed evolutionary survival of cooperative behavior when selfish actions provide higher fitness (reproductive success). In unstructured populations natural selection drives cooperation to extinction. However, when individuals are allowed to interact only with their neighbors, specified by a graph of social contacts, cooperation-promoting mechanisms (known as lattice reciprocity) offer to cooperation the opportunity of evolutionary survival. Recent numerical works on the evolution of Prisoner's Dilemma in complex network settings have revealed that graph heterogeneity dramatically enhances the lattice reciprocity. Here we show that in highly heterogeneous populations, under the graph analog of replicator dynamics, the fixation of a strategy in the whole population is in general an impossible event, for there is an asymptotic partition of the population in three subsets, two in which fixation of cooperation or defection has been reached and a third one which experiences cycles of invasion by the competing strategies. We show how the dynamical partition correlates with connectivity classes and characterize the temporal fluctuations of the fluctuating set, unveiling the mechanisms stabilizing cooperation in macroscopic scale-free structures.     

The influence of postreliance detection on the deceptive efficacy of dishonest signals of intent: Understanding facial clues to deceit as the outcome of signaling tradeoffs

Evolutionary communication theory posits that signalers and receivers are in a coevolutionary arms race. Receivers attempt to predict the behavior of signalers, and signalers attempt to manipulate the behavior of receivers (often through the use of dishonest signals of intent). This has led to the perception that deceitful signalers prefer perfectly deceptive signals. However, it is often easy for receivers to determine that a signal of intent was dishonest after relying on it to their detriment. Even the best deceivers may then acquire a reputation for being dishonest. For instance, in Prisoner's Dilemma (PD)-like social situations, predictable defectors make better social partners than unpredictable defectors. When opportunities to engage in social interaction depend on one's reputation for predictability, those who are better at concealing their defecting intentions may suffer the most from the reputations they acquire. Deceivers then face a tradeoff between the short-term benefits of successful deception and the long-term costs to their reputations. A mathematical model is developed and it is shown that the tradeoff often favors signalers who produce imperfectly deceptive signals over perfectly honest or perfectly deceptive ones. Implications for understanding human facial expressions and sociopathy are drawn.     

Long-term social bonds promote cooperation in the iterated Prisoner's Dilemma

Reciprocal altruism, one of the most probable explanations for cooperation among non-kin, has been modelled as a Prisoner''s Dilemma. According to this game, cooperation could evolve when individuals, who expect to play again, use conditional strategies like tit-for-tat or Pavlov. There is evidence that humans use such strategies to achieve mutual cooperation, but most controlled experiments with non-human animals have failed to find cooperation. One reason for this could be that subjects fail to cooperate because they behave as if they were to play only once. To assess this hypothesis, we conducted an experiment with monogamous zebra finches (Taeniopygia guttata) that were tested in a two-choice apparatus, with either their social partner or an experimental opponent of the opposite sex. We found that zebra finches maintained high levels of cooperation in an iterated Prisoner''s Dilemma game only when interacting with their social partner. Although other mechanisms may have contributed to the observed difference between the two treatments, our results support the hypothesis that animals do not systematically give in to the short-term temptation of cheating when long-term benefits exist. Thus, our findings contradict the commonly accepted idea that reciprocal altruism will be rare in non-human animals.     

Tit-for-tat or win-stay, lose-shift?

The repeated Prisoner's Dilemma is usually known as a story of tit-for-tat (TFT). This remarkable strategy has won both of Robert Axelrod's tournaments. TFT does whatever the opponent has done in the previous round. It will cooperate if the opponent has cooperated, and it will defect if the opponent has defected. But TFT has two weaknesses: (i) it cannot correct mistakes (erroneous moves) and (ii) a population of TFT players is undermined by random drift when mutant strategies appear which play always-cooperate (ALLC). Another equally simple strategy called 'win-stay, lose-shift' (WSLS) has neither of these two disadvantages. WSLS repeats the previous move if the resulting payoff has met its aspiration level and changes otherwise. Here, we use a novel approach of stochastic evolutionary game dynamics in finite populations to study mutation-selection dynamics in the presence of erroneous moves. We compare four strategies: always-defect (ALLD), ALLC, TFT and WSLS. There are two possible outcomes: if the benefit of cooperation is below a critical value then ALLD is selected; if the benefit of cooperation is above this critical value then WSLS is selected. TFT is never selected in this evolutionary process, but lowers the selection threshold for WSLS.     

COEVOLUTION AS AN EVOLUTIONARY GAME

Coevolution is modeled as a continuous game where the fitness-maximizing strategy of an individual is assumed to be a function of the strategy of other individuals who are also under selection to maximize fitness. An evolutionary stable strategy (ESS) is sought such that no rare alternative strategies can invade the community. The approach can be used to model coevolution because the ESS may be composed of a coalition of more than one strategy. This work, by modeling frequency-dependent selection, extends the approach of Roughgarden (1976) which only considered density-dependent selection. In particular, we show that the coevolutionary model of Rummel and Roughgarden (1985) does contain frequency-dependent selection, and thus, their application of Roughgarden's criterion for evolutionary stability to a model for which it is not applicable leads to the erroneous conclusion that the ecological and evolutionary processes are in conflict. The utility of the game theoretic approach is illustrated by two examples. The first considers an ESS composed of a single strategy, the second an ESS composed of a coalition of two strategies. Evolution occurs on a frequency-dependent adaptive landscape. For this reason, the approach is appropriate for modeling competitive speciation (Rosenzweig, 1978). Also, the game theoretic approach is designed to combine the interplay between the background environment (including the biotic components) and the evolutionary potential of the populations or organisms. The actual application of this theory will require knowledge of both.     

Iterated prisoner's dilemma in an asocial world dominated by loners, not by defectors

Cooperation among genetically unrelated individuals can arise when pairs of individuals interact repeatedly in the Prisoner’s Dilemma. However, the conditions allowing the evolution of reciprocal cooperation become extremely restrictive as the size of the cooperative group increases, because defectors can exploit cooperators more efficiently in larger groups. Here we consider three strategies: Tit for Tat, defector, and loner. Loner beats defector in a non-cooperative world. However, a cooperative strategy Tit for Tat (TFT₀) that stops cooperation after the first iteration when there is at least one defector in the group, can invade a world of loners, even in sizable groups, if both the TFT₀ and the defector strategies arise at the same frequency by mutation.     

The Effects of Extra-Somatic Weapons on the Evolution of Human Cooperation towards Non-Kin

Human cooperation and altruism towards non-kin is a major evolutionary puzzle, as is ‘strong reciprocity’ where no present or future rewards accrue to the co-operator/altruist. Here, we test the hypothesis that the development of extra-somatic weapons could have influenced the evolution of human cooperative behaviour, thus providing a new explanation for these two puzzles. Widespread weapons use could have made disputes within hominin groups far more lethal and also equalized power between individuals. In such a cultural niche non-cooperators might well have become involved in such lethal disputes at a higher frequency than cooperators, thereby increasing the relative fitness of genes associated with cooperative behaviour. We employ two versions of the evolutionary Iterated Prisoner''s Dilemma (IPD) model – one where weapons use is simulated and one where it is not. We then measured the performance of 25 IPD strategies to evaluate the effects of weapons use on them. We found that cooperative strategies performed significantly better, and non-cooperative strategies significantly worse, under simulated weapons use. Importantly, the performance of an ‘Always Cooperate’ IPD strategy, equivalent to that of ‘strong reciprocity’, improved significantly more than that of all other cooperative strategies. We conclude that the development of extra-somatic weapons throws new light on the evolution of human altruistic and cooperative behaviour, and particularly ‘strong reciprocity’. The notion that distinctively human altruism and cooperation could have been an adaptive trait in a past environment that is no longer evident in the modern world provides a novel addition to theory that seeks to account for this major evolutionary puzzle.     

Evolution of cooperation in patchy habitat under patch decay and isolation

The spatial version of Prisoners Dilemma (PD) is studied, which incorporates habitat decay through change in the mortality parameter and habitat isolation through change in the colonization coefficient. We found four kinds of evolutionary results, which are affected profoundly by the elements of the payoff matrix and the ratio of the colonization coefficient to the mortality parameter: population extinction, a pure cooperator population, coexistence of cooperators and defectors, and a pure defector population. First, the parameter region of cooperation (pure cooperator and coexistence region) shrinks with an increase in the cooperative cost, and that of defection extends. The increase in cooperative reward makes the cooperative region extend and the defector region become small. Second, the cooperative reward can compensate for the extinction risk due to habitat destruction and allow a population to survive even if the colonization coefficient is smaller than the mortality parameter. Third, although habitat destruction (including decay and isolation) increase the extinction risk of a population, moderate external power can push the evolution of cooperation ahead of population extinction, and even make a completely cooperative world come into being. Finally, for certain values of elements of the payoff matrix, the population suffering habitat destruction can maintain a stable population size by regulating the frequencies of cooperators and defectors. This implies that the multi-behavior strategy within a population may be a mechanism to defend against the influences of a changing environment.     

Attention bias toward noncooperative people. A dot probe classification study in cheating detection

Evolution-inspired research assumes the existence of brain mechanisms that scan for information that might signal noncooperative behavior. In this study, we demonstrate an automatic attention bias for threatening social interactions involving untrustworthy partners. Using a dot probe classification task, we found that, compared to unknown cooperators, attention was oriented significantly more toward the faces of unknown players who decided not to cooperate during a Prisoner's Dilemma Game. The present results thus suggest that an automatic, preconscious focus of attention underlies our ability to identify noncooperative players in social exchange situations.