Human cooperation in social dilemmas: comparing the Snowdrift game with the Prisoner's Dilemma

Explaining the evolution of cooperation among non-relatives is one of the major challenges for evolutionary biology. In this study, we experimentally examined human cooperation in the iterated Snowdrift game (ISD), which has received little attention so far, and compared it with human cooperation in the iterated Prisoner's Dilemma (IPD), which has become the paradigm for the evolution of cooperation. We show that iteration in the ISD leads to consistently higher levels of cooperation than in the IPD. We further demonstrate that the most successful strategies known for the IPD (generous Tit-for-Tat and Pavlov) were also successfully used in the ISD. Interestingly, we found that female players cooperated significantly more often than male players in the IPD but not in the ISD. Moreover, female players in the IPD applied Tit-for-Tat-like or Pavlovian strategies significantly more often than male players, thereby achieving significantly higher pay-offs than male players did. These data demonstrate that the willingness to cooperate does not only depend on the type of the social dilemma, but also on the class of individuals involved. Altogether, our study shows that the ISD can potentially explain high levels of cooperation among non-relatives in humans. In addition, the ISD seems to reflect the social dilemma more realistically than the IPD because individuals obtain immediate direct benefits from the cooperative acts they perform and costs of cooperation are shared between cooperators.     

Friendship, cliquishness, and the emergence of cooperation

The evolution of cooperation is a central problem in biology and the social sciences. While theoretical work using the iterated prisoner's dilemma (IPD) has shown that cooperation among non-kin can be sustained among reciprocal strategies (i.e. tit-for-tat), these results are sensitive to errors in strategy execution, cyclical invasions by free riders, and the specific ecology of strategies. Moreover, the IPD assumes that a strategy's probability of playing the PD game with other individuals is independent of the decisions made by others. Here, we remove the assumption of independent pairing by studying a more plausible cooperative dilemma in which players can preferentially interact with a limited set of known partners and also deploy longer-term accounting strategies that can counteract the effects of random errors. We show that cooperative strategies readily emerge and persist in a range of noisy environments, with successful cooperative strategies (henceforth, cliquers) maintaining medium-term memories for partners and low thresholds for acceptable cooperation (i.e. forgiveness). The success of these strategies relies on their cliquishness-a propensity to defect with strangers if they already have an adequate number of partners. Notably, this combination of medium-term accounting, forgiveness, and cliquishness fits with empirical studies of friendship and other long-term relationships among humans.     

Resolving the iterated prisoner's dilemma: theory and reality

Pairs of unrelated individuals face a prisoner's dilemma if cooperation is the best mutual outcome, but each player does best to defect regardless of his partner's behaviour. Although mutual defection is the only evolutionarily stable strategy in one-shot games, cooperative solutions based on reciprocity can emerge in iterated games. Among the most prominent theoretical solutions are the so-called bookkeeping strategies, such as tit-for-tat, where individuals copy their partner's behaviour in the previous round. However, the lack of empirical data conforming to predicted strategies has prompted the suggestion that the iterated prisoner's dilemma (IPD) is neither a useful nor realistic basis for investigating cooperation. Here, we discuss several recent studies where authors have used the IPD framework to interpret their data. We evaluate the validity of their approach and highlight the diversity of proposed solutions. Strategies based on precise accounting are relatively uncommon, perhaps because the full set of assumptions of the IPD model are rarely satisfied. Instead, animals use a diverse array of strategies that apparently promote cooperation, despite the temptation to cheat. These include both positive and negative reciprocity, as well as long-term mutual investments based on 'friendships'. Although there are various gaps in these studies that remain to be filled, we argue that in most cases, individuals could theoretically benefit from cheating and that cooperation cannot therefore be explained with the concept of positive pseudo-reciprocity. We suggest that by incorporating empirical data into the theoretical framework, we may gain fundamental new insights into the evolution of mutual reciprocal investment in nature.     

Evolutionary dynamics of the continuous iterated prisoner's dilemma

The iterated prisoner's dilemma (IPD) has been widely used in the biological and social sciences to model dyadic cooperation. While most of this work has focused on the discrete prisoner's dilemma, in which actors choose between cooperation and defection, there has been some analysis of the continuous IPD, in which actors can choose any level of cooperation from zero to one. Here, we analyse a model of the continuous IPD with a limited strategy set, and show that a generous strategy achieves the maximum possible payoff against its own type. While this strategy is stable in a neighborhood of the equilibrium point, the equilibrium point itself is always vulnerable to invasion by uncooperative strategies, and hence subject to eventual destabilization. The presence of noise or errors has no effect on this result. Instead, generosity is favored because of its role in increasing contributions to the most efficient level, rather than in counteracting the corrosiveness of noise. Computer simulation using a single-locus infinite alleles Gaussian mutation model suggest that outcomes ranging from a stable cooperative polymorphism to complete collapse of cooperation are possible depending on the magnitude of the mutational variance. Also, making the cost of helping a convex function of the amount of help provided makes it more difficult for cooperative strategies to invade a non-cooperative equilibrium, and for the cooperative equilibrium to resist destabilization by non-cooperative strategies. Finally, we demonstrate that a much greater degree of assortment is required to destabilize a non-cooperative equilibrium in the continuous IPD than in the discrete IPD. The continuous model outlined here suggests that incremental amounts of cooperation lead to rapid decay of cooperation and thus even a large degree of assortment will not be sufficient to allow cooperation to increase when cooperators are rare. The extreme degree of assortment required to destabilize the non-cooperative equilibrium, as well as the instability of the cooperative equilibrium, may help explain why cooperation in Prisoner's Dilemmas is so rare in nature.     

The evolution of punishment and apology: an iterated prisoner's dilemma model

Recently, behaviors that seem to function as punishment or apology have been reported among non-human primates as well as humans. Such behaviors appear to play an important role in maintaining cooperation between individuals. Therefore, the evolution of these behaviors should be examined from the viewpoint of the evolution of cooperation. The iterated prisoner's dilemma (IPD) game is generally considered to be a standard model for the evolution of cooperation. In the present study, strategies accompanied by punishment-like attacks or apology-like behavior were introduced into the common IPD simulation. Punishment and apology were represented by the P signal and the AS signal given immediately after defection. A strategy with the P and AS signals, named the pPAS strategy, was proved to be an evolutionarily stable strategy under certain conditions. Numerical simulations were carried out according to different assigned values of the costs of punishment and apology. The simulations showed that pPAS could dominate the population (1) when the cost of giving P is relatively small, (2) when the cost of receiving P is relatively large, or (3) when the cost of giving AS is relatively large. The relative cost of giving AS had the clearest effect on the success of pPAS. pPAS can dominate the population even when a dominance asymmetry of the costs between two players was introduced. The present results suggest the possible evolution of social behaviors like punishment or apology as a means of maintaining cooperation. This revised version was published online in November 2006 with corrections to the Cover Date.     

The importance of mechanisms for the evolution of cooperation

Studies aimed at explaining the evolution of phenotypic traits have often solely focused on fitness considerations, ignoring underlying mechanisms. In recent years, there has been an increasing call for integrating mechanistic perspectives in evolutionary considerations, but it is not clear whether and how mechanisms affect the course and outcome of evolution. To study this, we compare four mechanistic implementations of two well-studied models for the evolution of cooperation, the Iterated Prisoner''s Dilemma (IPD) game and the Iterated Snowdrift (ISD) game. Behavioural strategies are either implemented by a 1 : 1 genotype–phenotype mapping or by a simple neural network. Moreover, we consider two different scenarios for the effect of mutations. The same set of strategies is feasible in all four implementations, but the probability that a given strategy arises owing to mutation is largely dependent on the behavioural and genetic architecture. Our individual-based simulations show that this has major implications for the evolutionary outcome. In the ISD, different evolutionarily stable strategies are predominant in the four implementations, while in the IPD each implementation creates a characteristic dynamical pattern. As a consequence, the evolved average level of cooperation is also strongly dependent on the underlying mechanism. We argue that our findings are of general relevance for the evolution of social behaviour, pleading for the integration of a mechanistic perspective in models of social evolution.     

HUMAN COOPERATION BASED ON PUNISHMENT REPUTATION

The threat of punishment usually promotes cooperation. However, punishing itself is costly, rare in nonhuman animals, and humans who punish often finish with low payoffs in economic experiments. The evolution of punishment has therefore been unclear. Recent theoretical developments suggest that punishment has evolved in the context of reputation games. We tested this idea in a simple helping game with observers and with punishment and punishment reputation (experimentally controlling for other possible reputational effects). We show that punishers fully compensate their costs as they receive help more often. The more likely defection is punished within a group, the higher the level of within‐group cooperation. These beneficial effects perish if the punishment reputation is removed. We conclude that reputation is key to the evolution of punishment.     

Self-structuring in spatial evolutionary ecology

Spatial self-structuring has been a focus of recent interest among evolutionary ecologists. We review recent developments in the study of the interplay between spatial self-structuring and evolution. We first discuss the relative merits of the various theoretical approaches to spatial modelling in ecology. Second, we synthesize the main theoretical studies of the evolution of cooperation in spatially structured populations. We show that population viscosity is generally beneficial to cooperation, because cooperators can reap additional benefits from being clustered. A similar mechanism can explain the evolution of honest communication and of reduced virulence in host–parasite interactions. We also discuss some recent innovative empirical results that test these theories. Third, we show the relevance of these results to the general field of evolutionary ecology. An important conclusion is that kin selection is the main process that drives evolution of cooperation in viscous populations. Many results of kin selection theory can be recovered as emergent properties of spatial ecological dynamics. We discuss the implications of these results for the study of multilevel selection and evolutionary transitions. We conclude by sketching some perspectives for future research, with a particular emphasis on the topics of evolutionary branching, criticality, spatial fluctuations and experimental tests of theoretical predictions.     

Second‐order cooperation: Cooperative offspring as a living public good arising from second‐order selection on non‐cooperative individuals

Switching rate between cooperating and non‐cooperating genotypes is a crucial social evolution factor, often neglected by game theory‐inspired theoretical and experimental frameworks. We show that the evolution of alleles increasing the mutation or phenotypic switching rates toward cooperation is in itself a social dilemma. Although cooperative offspring are often unlikely to reproduce, due to high cost of cooperation, they can be seen both as a living public good and a part of the extended parental phenotype. The competition between individuals that generate cooperators and ones that do not is often more relevant than the competition between cooperators and non‐cooperators. The dilemma of second‐order cooperation we describe relates directly to eusociality, but can be also interpreted as a division of labor or a soma‐germline distinction. The results of our simulations shine a new light on what Darwin had already termed a “special difficulty” of evolutionary theory and describe a novel type of cooperation dynamics.     

Critical dynamics in the evolution of stochastic strategies for the iterated prisoner's dilemma

The observed cooperation on the level of genes, cells, tissues, and individuals has been the object of intense study by evolutionary biologists, mainly because cooperation often flourishes in biological systems in apparent contradiction to the selfish goal of survival inherent in Darwinian evolution. In order to resolve this paradox, evolutionary game theory has focused on the Prisoner's Dilemma (PD), which incorporates the essence of this conflict. Here, we encode strategies for the iterated Prisoner's Dilemma (IPD) in terms of conditional probabilities that represent the response of decision pathways given previous plays. We find that if these stochastic strategies are encoded as genes that undergo Darwinian evolution, the environmental conditions that the strategies are adapting to determine the fixed point of the evolutionary trajectory, which could be either cooperation or defection. A transition between cooperative and defective attractors occurs as a function of different parameters such as mutation rate, replacement rate, and memory, all of which affect a player's ability to predict an opponent's behavior. These results imply that in populations of players that can use previous decisions to plan future ones, cooperation depends critically on whether the players can rely on facing the same strategies that they have adapted to. Defection, on the other hand, is the optimal adaptive response in environments that change so quickly that the information gathered from previous plays cannot usefully be integrated for a response.     

The Effects of Extra-Somatic Weapons on the Evolution of Human Cooperation towards Non-Kin

Human cooperation and altruism towards non-kin is a major evolutionary puzzle, as is ‘strong reciprocity’ where no present or future rewards accrue to the co-operator/altruist. Here, we test the hypothesis that the development of extra-somatic weapons could have influenced the evolution of human cooperative behaviour, thus providing a new explanation for these two puzzles. Widespread weapons use could have made disputes within hominin groups far more lethal and also equalized power between individuals. In such a cultural niche non-cooperators might well have become involved in such lethal disputes at a higher frequency than cooperators, thereby increasing the relative fitness of genes associated with cooperative behaviour. We employ two versions of the evolutionary Iterated Prisoner''s Dilemma (IPD) model – one where weapons use is simulated and one where it is not. We then measured the performance of 25 IPD strategies to evaluate the effects of weapons use on them. We found that cooperative strategies performed significantly better, and non-cooperative strategies significantly worse, under simulated weapons use. Importantly, the performance of an ‘Always Cooperate’ IPD strategy, equivalent to that of ‘strong reciprocity’, improved significantly more than that of all other cooperative strategies. We conclude that the development of extra-somatic weapons throws new light on the evolution of human altruistic and cooperative behaviour, and particularly ‘strong reciprocity’. The notion that distinctively human altruism and cooperation could have been an adaptive trait in a past environment that is no longer evident in the modern world provides a novel addition to theory that seeks to account for this major evolutionary puzzle.     

Spiking neural networks with different reinforcement learning (RL) schemes in a multiagent setting

This paper investigates the effectiveness of spiking agents when trained with reinforcement learning (RL) in a challenging multiagent task. In particular, it explores learning through reward-modulated spike-timing dependent plasticity (STDP) and compares it to reinforcement of stochastic synaptic transmission in the general-sum game of the Iterated Prisoner's Dilemma (IPD). More specifically, a computational model is developed where we implement two spiking neural networks as two "selfish" agents learning simultaneously but independently, competing in the IPD game. The purpose of our system (or collective) is to maximise its accumulated reward in the presence of reward-driven competing agents within the collective. This can only be achieved when the agents engage in a behaviour of mutual cooperation during the IPD. Previously, we successfully applied reinforcement of stochastic synaptic transmission to the IPD game. The current study utilises reward-modulated STDP with eligibility trace and results show that the system managed to exhibit the desired behaviour by establishing mutual cooperation between the agents. It is noted that the cooperative outcome was attained after a relatively short learning period which enhanced the accumulation of reward by the system. As in our previous implementation, the successful application of the learning algorithm to the IPD becomes possible only after we extended it with additional global reinforcement signals in order to enhance competition at the neuronal level. Moreover it is also shown that learning is enhanced (as indicated by an increased IPD cooperative outcome) through: (i) strong memory for each agent (regulated by a high eligibility trace time constant) and (ii) firing irregularity produced by equipping the agents' LIF neurons with a partial somatic reset mechanism.     

Persistent high burden of invasive pneumococcal disease in South African HIV-infected adults in the era of an antiretroviral treatment program

Background

Highly active antiretroviral treatment (HAART) programs have been associated with declines in the burden of invasive pneumococcal disease (IPD) in industrialized countries. The aim of this study was to evaluate trends in IPD hospitalizations in HIV-infected adults in Soweto, South Africa, associated with up-scaling of the HAART program from 2003 to 2008.

Methods

Laboratory-confirmed IPD cases were identified from 2003 through 2008 through an existing surveillance program. The period 2003-04 was designated as the early-HAART era, 2005–06 as the intermediate-HAART era and 2007–08 as the established-HAART era. The incidence of IPD was compared between the early-HAART and established-HAART eras in HIV-infected and–uninfected individuals.

Results

A total of 2,567 IPD cases among individuals older than 18 years were reported from 2003 through 2008. Overall incidence of IPD (per 100,000) did not change during the study period in HIV-infected adults (207.4 cases in the early-HAART and 214.0 cases in the established-HAART era; p = 0.55). IPD incidence, actually increased 1.16-fold (95% CI: 1.01; 1.62) in HIV-infected females between the early-and established-HAART eras (212.1 cases and 246.2 cases, respectively; p = 0.03). The incidence of IPD remained unchanged in HIV-uninfected adults across the three time periods.

Conclusion

Despite a stable prevalence of HIV and the increased roll-out of HAART for treatment of AIDS patients in our setting, the burden of IPD has not decreased among HIV-infected adults. The study indicates a need for ongoing monitoring of disease and HAART program effectiveness to reduce opportunistic infections in African adults with HIV/AIDS, as well as the need to consider alternate strategies including pneumococcal conjugate vaccine immunization for the prevention of IPD in HIV-infected adults.     

Cooperation creates selection for tactical deception

Conditional social behaviours such as partner choice and reciprocity are held to be key mechanisms facilitating the evolution of cooperation, particularly in humans. Although how these mechanisms select for cooperation has been explored extensively, their potential to select simultaneously for complex cheating strategies has been largely overlooked. Tactical deception, the misrepresentation of the state of the world to another individual, may allow cheaters to exploit conditional cooperation by tactically misrepresenting their past actions and/or current intentions. Here we first use a simple game-theoretic model to show that the evolution of cooperation can create selection pressures favouring the evolution of tactical deception. This effect is driven by deception weakening cheater detection in conditional cooperators, allowing tactical deceivers to elicit cooperation at lower costs, while simple cheats are recognized and discriminated against. We then provide support for our theoretical predictions using a comparative analysis of deception across primate species. Our results suggest that the evolution of conditional strategies may, in addition to promoting cooperation, select for astute cheating and associated psychological abilities. Ultimately, our ability to convincingly lie to each other may have evolved as a direct result of our cooperative nature.     

No unique effect of intergroup competition on cooperation: non-competitive thresholds are as effective as competitions between groups for increasing human cooperative behavior

Explaining cooperation remains a central topic for evolutionary theorists. Many have argued that group selection provides such an explanation: theoretical models show that intergroup competition could have given rise to cooperation that is costly for the individual. Whether group selection actually did play an important role in the evolution of human cooperation, however, is much debated. Recent experiments have shown that intergroup competitions do increase human cooperation, which has been taken as evidence for group selection as a mechanism for the evolution of cooperation. Here we challenge this standard interpretation. Competitions change the payoff structure by creating a threshold effect whereby the group that contributes more earns an additional prize, which creates some incentive for individuals to cooperate. We present four studies that disentangle competition and thresholds, and strongly suggest that it is thresholds – rather than competitions per se – that increase cooperation. Thus, prior intergroup competition experiments provide no evidence of a unique or special role for intergroup competition in promoting human cooperation, and shed no light on whether group selection shaped human evolution.     

Preferential phenotypic association linked with cooperation in paper wasps

Animals can influence their social environment by preferentially associating with certain conspecifics. Such preferential association has gained increasing theoretical attention, as it may influence social evolution and population dynamics. However, relatively little empirical work has examined the occurrence of preferential association and its effects on cooperative group formation. Here, we test the factors associated with cooperative group formation in Polistes dominulus nest‐founding queen wasps. P. dominulus are a good system to study preferential association, as foundresses can nest alone or in groups and group membership is flexible. We found that both social and environmental factors were associated with partner choice. First, facial patterns were associated with cooperation. Wasps with more similar facial patterns were more likely to cooperate than wasps with less similar facial patterns. This preferential phenotypic association fits the theoretical criteria for the evolution of tag‐based cooperation. Season was also associated with cooperation; wasps on early‐season nests were more likely to cooperate than wasps on late‐season nests. High levels of aggression by nest owners during initial interactions were also correlated with lower probabilities of subsequent cooperation, suggesting that nest owners have some control over group membership. Other factors including body weight, weight similarity and nest productivity were not linked with cooperation. Overall, multiple factors influence cooperation in paper wasps, including facial pattern similarity. The occurrence of preferential phenotypic association in paper wasps is quite interesting and may influence the evolution of cooperation and population divergence in this group.     

Reputation and the evolution of cooperation in sizable groups

The evolution of cooperation in social dilemmas has been of considerable concern in various fields such as sociobiology, economics and sociology. It might be that, in the real world, reputation plays an important role in the evolution of cooperation. Recently, studies that have addressed indirect reciprocity have revealed that cooperation can evolve through reputation, even though pairs of individuals interact only a few times. To our knowledge, most indirect reciprocity models have presumed dyadic interaction; no studies have attempted analysis of the evolution of cooperation in large communities where the effect of reputation is included. We investigate the evolution of cooperation in sizable groups in which the reputation of individuals affects the decision-making process. This paper presents the following: (i) cooperation can evolve in a four-person case, (ii) the evolution of cooperation becomes difficult as group size increases, even if the effect of reputation is included, and (iii) three kinds of final social states exist. In medium-sized communities, cooperative species can coexist in a stable manner with betrayal species.     

Correlated pay-offs are key to cooperation

The general belief that cooperation and altruism in social groups result primarily from kin selection has recently been challenged, not least because results from cooperatively breeding insects and vertebrates have shown that groups may be composed mainly of non-relatives. This allows testing predictions of reciprocity theory without the confounding effect of relatedness. Here, we review complementary and alternative evolutionary mechanisms to kin selection theory and provide empirical examples of cooperative behaviour among unrelated individuals in a wide range of taxa. In particular, we focus on the different forms of reciprocity and on their underlying decision rules, asking about evolutionary stability, the conditions selecting for reciprocity and the factors constraining reciprocal cooperation. We find that neither the cognitive requirements of reciprocal cooperation nor the often sequential nature of interactions are insuperable stumbling blocks for the evolution of reciprocity. We argue that simple decision rules such as ‘help anyone if helped by someone’ should get more attention in future research, because empirical studies show that animals apply such rules, and theoretical models find that they can create stable levels of cooperation under a wide range of conditions. Owing to its simplicity, behaviour based on such a heuristic may in fact be ubiquitous. Finally, we argue that the evolution of exchange and trading of service and commodities among social partners needs greater scientific focus.     

LIMITED DISPERSAL, BUDDING DISPERSAL, AND COOPERATION: AN EXPERIMENTAL STUDY

Numerous theoretical studies have investigated how limited dispersal may provide an explanation for the evolution of cooperation, by leading to interactions between relatives. However, despite considerable theoretical attention, there has been a lack of empirical tests. In this article, we test how patterns of dispersal influence the evolution of cooperation, using iron-scavenging in the bacterium Pseudomonas aeruginosa as our cooperative trait. We found that relatively limited dispersal does not favor cooperation. The reason for this is that although limited dispersal increases the relatedness between interacting individuals, it also leads to increased local competition for resources between relatives. This result supports Taylor's prediction that in the simplest possible scenario, the effects of increased relatedness and local competition exactly cancel out. In contrast, we show that one way for cooperation to be favored is if individuals disperse in groups (budding dispersal), because this maintains high relatedness while reducing local competition between relatives (relatively global competition).     

Pleiotropic constraints promote the evolution of cooperation in cellular groups

The evolution of cooperation in cellular groups is threatened by lineages of cheaters that proliferate at the expense of the group. These cell lineages occur within microbial communities, and multicellular organisms in the form of tumours and cancer. In contrast to an earlier study, here we show how the evolution of pleiotropic genetic architectures—which link the expression of cooperative and private traits—can protect against cheater lineages and allow cooperation to evolve. We develop an age-structured model of cellular groups and show that cooperation breaks down more slowly within groups that tie expression to a private trait than in groups that do not. We then show that this results in group selection for pleiotropy, which strongly promotes cooperation by limiting the emergence of cheater lineages. These results predict that pleiotropy will rapidly evolve, so long as groups persist long enough for cheater lineages to threaten cooperation. Our results hold when pleiotropic links can be undermined by mutations, when pleiotropy is itself costly, and in mixed-genotype groups such as those that occur in microbes. Finally, we consider features of multicellular organisms—a germ line and delayed reproductive maturity—and show that pleiotropy is again predicted to be important for maintaining cooperation. The study of cancer in multicellular organisms provides the best evidence for pleiotropic constraints, where abberant cell proliferation is linked to apoptosis, senescence, and terminal differentiation. Alongside development from a single cell, we propose that the evolution of pleiotropic constraints has been critical for cooperation in many cellular groups.

The evolution of cooperation in cellular groups is threatened by lineages of cheaters that proliferate at the expense of the group. In this study, an age-structured model of cellular groups shows that pleiotropy promotes the evolution of cooperation and may have been important for the origins of multicellularity.