Persistent cooperators in nature

The evolution and maintenance of cooperation fascinated researchers for several decades. Recently, theoretical models and experimental evidence show that costly punishment may facilitate cooperation in human societies. The puzzle how the costly punishment behaviour evolves can be solved under voluntary participation. Could the punishers emerge if participation is compulsory? Is the punishment inevitably a selfish behaviour or an altruistic behaviour? The motivations behind punishment are still an enigma. Based on public goods interactions, we present a model in which just a certain portion of the public good is divided equally among all members. The other portion is distributed to contributors when paying a second cost. The contributors who are willing to pay a second cost are called the persistent cooperators (PC), indicating their desire to retrieve the proportion of the payoff derived from their own contributions with persistent efforts. We show that the persistent cooperators can be costly punishers, which may account for the origin of human costly punishment behaviour under compulsory participation. In this sense our models may show theoretically that the original motivation behind punishment is to retrieve deserved payoff from their own contributions, a selfish incentive. But the persistent cooperators can also flourish or dominate the population in other situations. We list many real examples in which contributors are the persistent cooperators, and they benefit. This indicates a simple norm promoting cooperation: contributing more and gaining more.     

Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game

Understanding the mechanisms that can lead to the evolution of cooperation through natural selection is a core problem in biology. Among the various attempts at constructing a theory of cooperation, game theory has played a central role. Here, we review models of cooperation that are based on two simple games: the Prisoner's Dilemma, and the Snowdrift game. Both games are two‐person games with two strategies, to cooperate and to defect, and both games are social dilemmas. In social dilemmas, cooperation is prone to exploitation by defectors, and the average payoff in populations at evolutionary equilibrium is lower than it would be in populations consisting of only cooperators. The difference between the games is that cooperation is not maintained in the Prisoner's Dilemma, but persists in the Snowdrift game at an intermediate frequency. As a consequence, insights gained from studying extensions of the two games differ substantially. We review the most salient results obtained from extensions such as iteration, spatial structure, continuously variable cooperative investments, and multi‐person interactions. Bridging the gap between theoretical and empirical research is one of the main challenges for future studies of cooperation, and we conclude by pointing out a number of promising natural systems in which the theory can be tested experimentally.     

Cooperators benefit through reputation-based partner choice in economic games

Explaining unconditional cooperation, such as donations to charities or contributions to public goods, continues to present a problem. One possibility is that cooperation can pay through developing a reputation that makes one more likely to be chosen for a profitable cooperative partnership, a process termed competitive altruism (CA) or reputation-based partner choice. Here, we show, to our knowledge, for the first time, that investing in a cooperative reputation can bring net benefits through access to more cooperative partners. Participants played a public goods game (PGG) followed by an opportunity to select a partner for a second cooperative game. We found that those who gave more in the PGG were more often selected as desired partners and received more in the paired cooperative game. Reputational competition was even stronger when it was possible for participants to receive a higher payoff from partner choice. The benefits of being selected by a more cooperative partner outweighed the costs of cooperation in the reputation building phase. CA therefore provides an alternative to indirect reciprocity as an explanation for reputation-building behaviour. Furthermore, while indirect reciprocity depends upon individuals giving preference to those of good standing, CA can explain unconditional cooperation.     

The paradox of cooperation benefits

It seems obvious that as the benefits of cooperation increase, the share of cooperators in the population should also increase. It is well known that positive assortment between cooperative types, for instance in spatially structured populations, provide better conditions for the evolution of cooperation than complete mixing. This study demonstrates that, assuming positive assortment, under most conditions higher cooperation benefits also increase the share of cooperators. On the other hand, under a specified range of payoff values, when at least two payoff parameters are modified, the reverse is true. The conditions for this paradox are determined for two-person social dilemmas: the Prisoner's Dilemma, the Hawks and Doves game, and the Stag Hunt game, assuming global selection and positive assortment.     

Joint Effects of Asymmetric Payoff and Reciprocity Mechanisms on Collective Cooperation in Water Sharing Interactions: A Game Theoretic Perspective

Common-pool resource (CPR) dilemmas distinguish themselves from general public good problems by encompassing both social and physical features. This paper examines how a physical mechanism, namely asymmetric payoff; and a social mechanism, reciprocity; simultaneously affect collective cooperation in theoretical water sharing interactions. We present an iterative N-person game theoretic model to investigate the joint effects of these two mechanisms in a linear fully connected river system under three information assumptions. From a simple evolutionary perspective, this paper quantitatively addresses the conditions for Nash Equilibrium in which collective cooperation might be established. The results suggest that direct reciprocity increases every actor’s motivation to contribute to the collective good of the river system. Meanwhile, various upstream and downstream actors manifest individual disparities as a result of the direct reciprocity and asymmetric payoff mechanisms. More specifically, the downstream actors are less willing to cooperate unless there is a high probability that long-term interactions are ensured; however, a greater level of asymmetries is likely to increase upstream actors’ incentives to cooperate even though the interactions could quickly end. The upstream actors also display weak sensitivity to an increase in the total number of actors, which generally results in a reduction in the other actors’ motivation for cooperation. It is also shown that the indirect reciprocity mechanism relaxes the overall conditions for cooperative Nash Equilibrium.     

Asymmetric interaction will facilitate the evolution of cooperation

Explaining the evolution of cooperation remains one of the greatest problems for both biology and social science. The classical theories of cooperation suggest that cooperation equilibrium or evolutionary stable strategy between partners can be maintained through genetic similarity or reciprocity relatedness. These classical theories are based on an assumption that partners interact symmetrically with equal payoffs in a game of cooperation interaction. However, the payoff between partners is usually not equal and therefore they often interact asymmetrically in real cooperative systems. With the Hawk-Dove model, we find that the probability of cooperation between cooperative partners will depend closely on the payoff ratio. The higher the payoff ratio between recipients and cooperative actors, the greater will be the probability of cooperation interaction between involved partners. The greatest probability of conflict between cooperative partners will occur when the payoff between partners is equal. The results show that this asymmetric relationship is one of the key dynamics of the evolution of cooperation, and that pure cooperation strategy (i.e., Nash equilibrium) does not exist in asymmetrical cooperation systems, which well explains the direct conflict observed in almost all of the well documented cooperation systems. The model developed here shows that the cost-to-benefit ratio of cooperation is also negatively correlated with the probability of cooperation interaction. A smaller cost-to-benefit ratio of cooperation might be created by the limited dispersal ability or exit cost of the partners involved, and it will make the punishment of the non-cooperative individuals by the recipient more credible, and therefore make it more possible to maintain stable cooperation interaction.     

Exit,Punishment and Rewards in Commons Dilemmas: An Experimental Study

Commons dilemmas are interaction situations where a common good is provided or exploited by a group of individuals so that optimal collective outcomes clash with private interests. Although in these situations, social norms and institutions exist that might help individuals to cooperate, little is known about the interaction effects between positive and negative incentives and exit options by individuals. We performed a modified public good game experiment to examine the effect of exit, rewards and punishment, as well as the interplay between exit and rewards and punishment. We found that punishment had a stronger effect than rewards on cooperation if considered by itself, whereas rewards had a stronger effect when combined with voluntary participation. This can be explained in terms of the ‘framing effect’, i.e., as the combination of exit and rewards might induce people to attach higher expected payoffs to cooperative strategies and expect better behaviour from others.     

Adaptive Dynamics of Extortion and Compliance

Direct reciprocity is a mechanism for the evolution of cooperation. For the iterated prisoner’s dilemma, a new class of strategies has recently been described, the so-called zero-determinant strategies. Using such a strategy, a player can unilaterally enforce a linear relationship between his own payoff and the co-player’s payoff. In particular the player may act in such a way that it becomes optimal for the co-player to cooperate unconditionally. In this way, a player can manipulate and extort his co-player, thereby ensuring that the own payoff never falls below the co-player’s payoff. However, using a compliant strategy instead, a player can also ensure that his own payoff never exceeds the co-player’s payoff. Here, we use adaptive dynamics to study when evolution leads to extortion and when it leads to compliance. We find a remarkable cyclic dynamics: in sufficiently large populations, extortioners play a transient role, helping the population to move from selfish strategies to compliance. Compliant strategies, however, can be subverted by altruists, which in turn give rise to selfish strategies. Whether cooperative strategies are favored in the long run critically depends on the size of the population; we show that cooperation is most abundant in large populations, in which case average payoffs approach the social optimum. Our results are not restricted to the case of the prisoners dilemma, but can be extended to other social dilemmas, such as the snowdrift game. Iterated social dilemmas in large populations do not lead to the evolution of strategies that aim to dominate their co-player. Instead, generosity succeeds.     

Punishment as a Means of Competition: Implications for Strong Reciprocity Theory

Strong negative reciprocity, that is, sanctions imposed on norm violators at the punisher’s own expense, has powerful cooperation-enhancing effects in both real-life and experimental game situations. However, it is plausible that punishment may obtain alternative roles depending on social context and the personality characteristics of participants. We examined the occurrence of punishing behavior among 80 subjects in a strongly competitive Public Goods game setting. Despite the punishment condition, the amount of the contributions decreased steadily during the game. The amount of contributions had no significant effect on received and imposed punishments. The results indicate that certain social contexts (in this case, intensive competition) exert modifying effects on the role that punishment takes on. Subjects punished each other in order to achieve a higher rank and a financially better outcome. Punishment primarily functioned as a means of rivalry, instead of as a way of second-order cooperation, as strong reciprocity suggests. These results indicate the need for the possible modification of the social conditions of punishment mechanisms described by the strong reciprocity theory as an evolutionary explanation of human cooperation.     

Variation and the response to variation as a basis for successful cooperation

In applying game theory to problems in biology, differences between individuals are often ignored. In particular, when analysing the evolution of cooperation it is often implicitly assumed that ignoring variation will produce predictions that approximate the solution when differences are included. This need not be true. As we demonstrate, differences are not innocuous noise, but can fundamentally change the nature of a game. Even small amounts of variability can stabilize cooperation by, for example, maintaining the need to deal with cheaters. Differences promote the need to learn about others in an interaction, leading to contingent behaviour that can reduce conflict, and to negotiated outcomes that may or may not be more cooperative than unconditional actions. Once there are mechanisms such as mutation and environmental influences that maintain variation within populations, whether cooperation evolves may depend on the variation in the cooperativeness trait. Variation means that it may be worth taking a chance that a partner is cooperative by being cooperative. When there are markets, so that individuals can break off interactions to seek a better partner, variation promotes choosiness and hence penalizes those uncooperative individuals, who are rejected. Variation promotes the need to monitor the previous behaviour of others, and once this social sensitivity exists, the need to maintain a good reputation can promote cooperation.     

Spatial effects in social dilemmas

Social dilemmas and the evolutionary conundrum of cooperation are traditionally studied through various kinds of game theoretical models such as the prisoner's dilemma, public goods games, snowdrift games or by-product mutualism. All of them exemplify situations which are characterized by different degrees of conflicting interests between the individuals and the community. In groups of interacting individuals, cooperators produce a common good benefitting the entire group at some cost to themselves, whereas defectors attempt to exploit the resource by avoiding the costly contributions. Based on synergistic or discounted accumulation of cooperative benefits a unifying theoretical framework was recently introduced that encompasses all games that have traditionally been studied separately (Hauert, Michor, Nowak, Doebeli, 2005. Synergy and discounting of cooperation in social dilemmas. J. Theor. Biol., in press.). Within this framework we investigate the effects of spatial structure with limited local interactions on the evolutionary fate of cooperators and defectors. The quantitative effects of space turn out to be quite sensitive to the underlying microscopic update mechanisms but, more general, we demonstrate that in prisoner's dilemma type interactions spatial structure benefits cooperation-although the parameter range is quite limited-whereas in snowdrift type interactions spatial structure may be beneficial too, but often turns out to be detrimental to cooperation.     

On the further integration of cooperative breeding and cooperation theory

We present a synopsis about the commentaries to the target article "Integrating cooperative breeding into theoretical concepts of cooperation", in which we attempted to integrate general mechanisms to explain cooperative behaviour among unrelated individuals with classic concepts to explain helping behaviour in cooperative breeders that do not invoke kin-based benefits. Here we (1) summarize the positions of the commentators concerning the main issues we raised in the target article and discuss important criticisms and extensions. (2) We relate our target article to some recent reviews on the evolution of cooperation and, (3) clarify how we use terminology with regard to cooperation and cooperative behaviour. (4) We discuss several aspects that were raised with respect to cooperative interactions including by-product mutualism, generalised reciprocity and multi-level selection and, (5) examine the alternatives to our classification scheme as proposed by some commentaries. (6) Finally, we highlight several aspects that might hinder the application of game theoretical mechanisms of cooperation in cooperatively breeding systems. Although there is broad agreement that cooperative breeding theory should be integrated within the more general concepts of cooperation, there is some debate about how this may be achieved. We conclude that the contributions in this special issue provide a fruitful first step and ample suggestions for future directions with regard to a more unified framework of cooperation in cooperative breeders.     

Effect of the presence of empty sites on the evolution of cooperation by costly punishment in spatial games

Cooperation and spiteful behavior are still evolutionary puzzles. Costly punishment, for which the game payoff is the same as that of spiteful behavior, is one mechanism for promoting the evolution of cooperation. A spatially structured population facilitates the evolution of either cooperation or spite/punishment if cooperation is linked explicitly or implicitly with spite/punishment; a cooperator cooperates with another cooperator and punishes/spites the other type of player. Different updating rules in the evolutionary game produce different evolutionary outcomes: with one updating rule—the score-dependent viability model, in which a player dies with a probability inversely proportional to the game score and the resulting unoccupied site is colonized by one player chosen randomly—the evolution of spite/punishment is promoted more than with the other updating rule—the score-dependent fertility model, in which, after a player dies randomly, the site is colonized by a player with a higher game score. If the population has empty sites, spiteful players or punishers should have less chance to interact with others and then spite/punish others. Thus the presence of empty sites would affect the evolutionary dynamics of spite/punishment. Here, we investigated whether the presence of empty sites discourages the evolution of spite/punishment in both a lattice-structured population and a completely mixing population where players interact with others randomly, especially when the score-dependent viability model is adopted. In the lattice-structured population adopting this viability model, the presence of empty sites promoted the evolution of cooperation and did not reduce the effect of spite/punishment. In the completely mixing population, the presence of empty sites did not promote evolution of cooperation by punishment. The evolutionary dynamics of the score-dependent viability model with empty sites were close to those of the score-dependent fertility model.     

Evolution of Cooperation in Spatial Traveler's Dilemma Game

Traveler''s dilemma (TD) is one of social dilemmas which has been well studied in the economics community, but it is attracted little attention in the physics community. The TD game is a two-person game. Each player can select an integer value between and () as a pure strategy. If both of them select the same value, the payoff to them will be that value. If the players select different values, say and (), then the payoff to the player who chooses the small value will be and the payoff to the other player will be . We term the player who selects a large value as the cooperator, and the one who chooses a small value as the defector. The reason is that if both of them select large values, it will result in a large total payoff. The Nash equilibrium of the TD game is to choose the smallest value . However, in previous behavioral studies, players in TD game typically select values that are much larger than , and the average selected value exhibits an inverse relationship with . To explain such anomalous behavior, in this paper, we study the evolution of cooperation in spatial traveler''s dilemma game where the players are located on a square lattice and each player plays TD games with his neighbors. Players in our model can adopt their neighbors'' strategies following two standard models of spatial game dynamics. Monte-Carlo simulation is applied to our model, and the results show that the cooperation level of the system, which is proportional to the average value of the strategies, decreases with increasing until is greater than the critical value where cooperation vanishes. Our findings indicate that spatial reciprocity promotes the evolution of cooperation in TD game and the spatial TD game model can interpret the anomalous behavior observed in previous behavioral experiments.     

A General Scheme to Predict Partner Control Mechanisms in Pairwise Cooperative Interactions Between Unrelated Individuals

Recent years have seen an explosion in the diversity of partner control mechanisms hypothesised to stabilise cooperative behaviour among unrelated individuals. Game theory suggests numerous strategies, each with specific decision rules that allow cooperators to control a non‐contributing partner. This diversity of hypothetical strategies seems likely to reflect diversity in the types of intraspecific cooperation and interspecific mutualism that exist in nature. It is therefore important to provide a framework that explains similarities and differences between the various hypothetical strategies and that predicts how key parameters that describe the natural history of natural systems favour different control mechanisms. We develop a novel unifying framework for pairwise interactions between unrelated individuals, in which we link specific control mechanisms to specific game structures. The latter are defined by unique combinations of the states of five parameters that describe investment, aspects of the payoff matrix, the number of interactions and partner choice. We find that specific control mechanisms potentially have utility in a limited number of game structures; conversely, each game structure may typically offer a few competing control mechanisms. Our framework offers theoreticians specific problems that await mathematical exploration, while at the same time offering empiricists guidelines for evaluating the game structure and corresponding control mechanisms in their systems.     

How strategy environment and wealth shape altruistic behaviour: cooperation rules affecting wealth distribution in dynamic networks

Societies rely on individual contributions to sustain public goods that benefit the entire community. Several mechanisms, that specify how individuals change their decisions based on past experiences, have been proposed to explain how altruists are not outcompeted by selfish counterparts. A key aspect of such strategy updates involves a comparison of an individual''s latest payoff with that of a random neighbour. In reality, both the economic and social milieu often shapes cooperative behaviour. We propose a new decision heuristic, where the propensity of an individual to cooperate depends on the local strategy environment in which she is embedded as well as her wealth relative to that of her neighbours. Our decision-making model allows cooperation to be sustained and also explains the results of recent experiments on social dilemmas in dynamic networks. Final cooperation levels depend only on the extent to which the strategy environment influences altruistic behaviour but are largely unaffected by network restructuring. However, the extent of wealth inequality in the community is affected by a subtle interplay between the environmental influence on a person''s decision to contribute and the likelihood of reshaping social ties, with wealth-inequality levels rising with increasing likelihood of network restructuring in some situations.     

Reputation-based partner choice is an effective alternative to indirect reciprocity in solving social dilemmas

When group interests clash with individual ones, maintaining cooperation poses a problem. However, cooperation can be facilitated by introducing reputational incentives. Through indirect reciprocity, people who cooperate in a social dilemma are more likely to receive cooperative acts from others. Another mechanism that enhances group cooperation is reputation-based partner choice, or competitive altruism. According to this framework, cooperators benefit via increased access to cooperative partners. Our study compared the effectiveness of indirect reciprocity and competitive altruism in re-establishing cooperation after the typical decline found during repeated public goods games. Twenty groups of four participants first played a series of public goods games, which confirmed the expected decline. Subsequently, public goods games were alternated with either indirect reciprocity games (in which participants had an opportunity to give to another individual from whom they would never receive a direct return) or competitive altruism games (in which they could choose partners for directly reciprocal interactions). We found that public goods game contributions increased when interspersed with competitive altruism games; they were also higher than in public goods games interspersed with indirect reciprocity games. Investing in reputation by increasing contributions to public goods was a profitable strategy in that it increased returns in subsequent competitive altruism and indirect reciprocity games. There was also some evidence that these returns were greater under competitive altruism than indirect reciprocity. Our findings indicate that strategic reputation building through competitive altruism provides an effective alternative to indirect reciprocity as a means for restoring cooperation in social dilemmas.     

The evolution of punishment in n-player public goods games: a volunteer's dilemma

The evolution of punishment to stabilize cooperation in n-player games has been treated as a second-order social dilemma, where contributions to punishment of free-riders are altruistic. Hence it may only evolve under highly restricted conditions. Here, we build on recent insights using the volunteer's dilemma as an alternative payoff matrix for the evolution of cooperation. The key feature of a volunteer's dilemma is that the benefits of cooperation are a nonlinear function of the number of contributors, meaning that cooperation is negatively frequency dependent. We propose that nonlinear returns are also an inherent feature of punishment and that this insight allows for a simple and novel explanation of how punishment evolves in groups.     

Cognitive and Motivational Requirements for the Emergence of Cooperation in a Rat Social Game

Background

Game theory and the Prisoner''s Dilemma (PD) game in particular, which captures the paradox of cooperative interactions that lead to benefits but entail costs to the interacting individuals, have constituted a powerful tool in the study of the mechanisms of reciprocity. However, in non-human animals most tests of reciprocity in PD games have resulted in sustained defection strategies. As a consequence, it has been suggested that under such stringent conditions as the PD game humans alone have evolved the necessary cognitive abilities to engage in reciprocity, namely, numerical discrimination, memory and control of temporal discounting.

Methodology/Principal Findings

We use an iterated PD game to test rats (Rattus norvegicus) for the presence of such cognitive abilities by manipulating the strategy of the opponent, Tit-for-Tat and Pseudo-Random, or the relative size of the temptation to defect. We found that rats shape their behaviour according to the opponent''s strategy and the relative outcome resulting from cooperative or defective moves. Finally, we show that the behaviour of rats is contingent upon their motivational state (hungry versus sated).

Conclusions/Significance

Here we show that rats understand the payoff matrix of the PD game and the strategy of the opponent. Importantly, our findings reveal that rats possess the necessary cognitive capacities for reciprocity-based cooperation to emerge in the context of a prisoner''s dilemma. Finally, the validation of the rat as a model to study reciprocity-based cooperation during the PD game opens new avenues of research in experimental neuroscience.     

Altruistic Punishment and Between-Group Competition

Collective action, or the large-scale cooperation in the pursuit of public goods, has been suggested to have evolved through cultural group selection. Previous research suggests that the costly punishment of group members who do not contribute to public goods plays an important role in the resolution of collective action dilemmas. If large-scale cooperation sustained by the punishment of defectors has evolved through the mechanism of cultural group selection, two implications regarding costly punishment follow: (1) that people are more willing to punish defecting group members in a situation of intergroup competition than in a single-group social dilemma game and (2) that levels of "perverse" punishment of cooperators are not affected by intergroup competition. We find confirmation for these hypotheses. However, we find that the effect of intergroup competition on the punishment of defectors is fully explained by the stronger conditionality of punishment on expected punishment levels in the competition condition.