Replicator dynamics of reward & reputation in public goods games

Public goods games have become the mathematical metaphor for game theoretical investigations of cooperative behavior in groups of interacting individuals. Cooperation is a conundrum because cooperators make a sacrifice to benefit others at some cost to themselves. Exploiters or defectors reap the benefits and forgo costs. Despite the fact that groups of cooperators outperform groups of defectors, Darwinian selection or utilitarian principles based on rational choice should favor defectors. In order to overcome this social dilemma, much effort has been expended for investigations pertaining to punishment and sanctioning measures against defectors. Interestingly, the complementary approach to create positive incentives and to reward cooperation has received considerably less attention—despite being heavily advocated in education and social sciences for increasing productivity or preventing conflicts. Here we show that rewards can indeed stimulate cooperation in interaction groups of arbitrary size but, in contrast to punishment, fail to stabilize it. In both cases, however, reputation is essential. The combination of reward and reputation result in complex dynamics dominated by unpredictable oscillations.     

Altruistic Punishment and Between-Group Competition

Collective action, or the large-scale cooperation in the pursuit of public goods, has been suggested to have evolved through cultural group selection. Previous research suggests that the costly punishment of group members who do not contribute to public goods plays an important role in the resolution of collective action dilemmas. If large-scale cooperation sustained by the punishment of defectors has evolved through the mechanism of cultural group selection, two implications regarding costly punishment follow: (1) that people are more willing to punish defecting group members in a situation of intergroup competition than in a single-group social dilemma game and (2) that levels of "perverse" punishment of cooperators are not affected by intergroup competition. We find confirmation for these hypotheses. However, we find that the effect of intergroup competition on the punishment of defectors is fully explained by the stronger conditionality of punishment on expected punishment levels in the competition condition.     

Group-size diversity in public goods games

Public goods games are models of social dilemmas where cooperators pay a cost for the production of a public good while defectors free ride on the contributions of cooperators. In the traditional framework of evolutionary game theory, the payoffs of cooperators and defectors result from interactions in groups formed by binomial sampling from an infinite population. Despite empirical evidence showing that group-size distributions in nature are highly heterogeneous, most models of social evolution assume that the group size is constant. In this article, I remove this assumption and explore the effects of having random group sizes on the evolutionary dynamics of public goods games. By a straightforward application of Jensen's inequality, I show that the outcome of general nonlinear public goods games depends not only on the average group size but also on the variance of the group-size distribution. This general result is illustrated with two nonlinear public goods games (the public goods game with discounting or synergy and the N-person volunteer's dilemma) and three different group-size distributions (Poisson, geometric, and Waring). The results suggest that failing to acknowledge the natural variation of group sizes can lead to an underestimation of the actual level of cooperation exhibited in evolving populations.     

The durability of public goods changes the dynamics and nature of social dilemmas

An implicit assumption underpins basic models of the evolution of cooperation, mutualism and altruism: The benefits (or pay-offs) of cooperation and defection are defined by the current frequency or distribution of cooperators. In social dilemmas involving durable public goods (group resources that can persist in the environment-ubiquitous from microbes to humans) this assumption is violated. Here, we examine the consequences of relaxing this assumption, allowing pay-offs to depend on both current and past numbers of cooperators. We explicitly trace the dynamic of a public good created by cooperators, and define pay-offs in terms of the current public good. By raising the importance of cooperative history in determining the current fate of cooperators, durable public goods cause novel dynamics (e.g., transient increases in cooperation in Prisoner's Dilemmas, oscillations in Snowdrift Games, or shifts in invasion thresholds in Stag-hunt Games), while changes in durability can transform one game into another, by moving invasion thresholds for cooperation or conditions for coexistence with defectors. This enlarged view challenges our understanding of social cheats. For instance, groups of cooperators can do worse than groups of defectors, if they inherit fewer public goods, while a rise in defectors no longer entails a loss of social benefits, at least not in the present moment (as highlighted by concerns over environmental lags). Wherever durable public goods have yet to reach a steady state (for instance due to external perturbations), the history of cooperation will define the ongoing dynamics of cooperators.     

The Effect of Ostracism and Optional Participation on the Evolution of Cooperation in the Voluntary Public Goods Game

Not only animals, plants and microbes but also humans cooperate in groups. The evolution of cooperation in a group is an evolutionary puzzle, because defectors always obtain a higher benefit than cooperators. When people participate in a group, they evaluate group member’s reputations and then decide whether to participate in it. In some groups, membership is open to all who are willing to participate in the group. In other groups, a candidate is excluded from membership if group members regard the candidate’s reputation as bad. We developed an evolutionary game model and investigated how participation in groups and ostracism influence the evolution of cooperation in groups when group members play the voluntary public goods game, by means of computer simulation. When group membership is open to all candidates and those candidates can decide whether to participate in a group, cooperation cannot be sustainable. However, cooperation is sustainable when a candidate cannot be a member unless all group members admit them to membership. Therefore, it is not participation in a group but rather ostracism, which functions as costless punishment on defectors, that is essential to sustain cooperation in the voluntary public goods game.     

Coexistence of cooperation and defection in public goods games

The production of public goods by the contribution of individual volunteers is a social dilemma because an individual that does not volunteer can benefit from the public good produced by the contributions of others. Therefore it is generally believed that public goods can be produced only in the presence of repeated interactions (which allow reciprocation, reputation effects and punishment) or relatedness (kin selection). Cooperation, however, often occurs in the absence of iterations and relatedness. We show that when the production of a public good is a Volunteer's Dilemma, in which a fixed number of cooperators is necessary to produce the public good, cooperators and defectors persist in a mixed equilibrium, without iterations and without relatedness. This mixed equilibrium is absent in the N-person Prisoner's Dilemma, in which the public good is a linear function of the individual contributions. We also show that the Prisoner's Dilemma and the Volunteer's Dilemma are the two opposite extremes of a general public goods game, and that all intermediate cases can have a mixed equilibrium like the Volunteer's Dilemma. The coexistence of cooperators and defectors, therefore, is a typical outcome of most social dilemmas, which requires neither relatedness nor iterations.     

Spatial effects in social dilemmas

Social dilemmas and the evolutionary conundrum of cooperation are traditionally studied through various kinds of game theoretical models such as the prisoner's dilemma, public goods games, snowdrift games or by-product mutualism. All of them exemplify situations which are characterized by different degrees of conflicting interests between the individuals and the community. In groups of interacting individuals, cooperators produce a common good benefitting the entire group at some cost to themselves, whereas defectors attempt to exploit the resource by avoiding the costly contributions. Based on synergistic or discounted accumulation of cooperative benefits a unifying theoretical framework was recently introduced that encompasses all games that have traditionally been studied separately (Hauert, Michor, Nowak, Doebeli, 2005. Synergy and discounting of cooperation in social dilemmas. J. Theor. Biol., in press.). Within this framework we investigate the effects of spatial structure with limited local interactions on the evolutionary fate of cooperators and defectors. The quantitative effects of space turn out to be quite sensitive to the underlying microscopic update mechanisms but, more general, we demonstrate that in prisoner's dilemma type interactions spatial structure benefits cooperation-although the parameter range is quite limited-whereas in snowdrift type interactions spatial structure may be beneficial too, but often turns out to be detrimental to cooperation.     

The Increased Risk of Joint Venture Promotes Social Cooperation

The joint venture of many members is common both in animal world and human society. In these public enterprizes, highly cooperative groups are more likely to while low cooperative groups are still possible but not probable to succeed. Existent literature mostly focuses on the traditional public goods game, in which cooperators create public wealth unconditionally and benefit all group members unbiasedly. We here institute a model addressing this public goods dilemma with incorporating the public resource foraging failure risk. Risk-averse individuals tend to lead a autarkic life, while risk-preferential ones tend to participate in the risky public goods game. For participants, group''s success relies on its cooperativeness, with increasing contribution leading to increasing success likelihood. We introduce a function with one tunable parameter to describe the risk removal pattern and study in detail three representative classes. Analytical results show that the widely replicated population dynamics of cyclical dominance of loner, cooperator and defector disappear, while most of the time loners act as savors while eventually they also disappear. Depending on the way that group''s success relies on its cooperativeness, either cooperators pervade the entire population or they coexist with defectors. Even in the later case, cooperators still hold salient superiority in number as some defectors also survive by parasitizing. The harder the joint venture succeeds, the higher level of cooperation once cooperators can win the evolutionary race. Our work may enrich the literature concerning the risky public goods games.     

Defectors,not norm violators,are punished by third-parties

Punishment of defectors and cooperators is prevalent when their behaviour deviates from the social norm. Why atypical behaviour is more likely to be punished than typical behaviour remains unclear. One possible proximate explanation is that individuals simply dislike norm violators. However, an alternative possibility exists: individuals may be more likely to punish atypical behaviour, because the cost of punishment generally increases with the number of individuals that are punished. We used a public goods game with third-party punishment to test whether punishment of defectors was reduced when defecting was typical, as predicted if punishment is responsive to norm violation. The cost of punishment was fixed, regardless of the number of players punished, meaning that it was not more costly to punish typical, relative to atypical, behaviour. Under these conditions, atypical behaviour was not punished more often than typical behaviour. In fact, most punishment was targeted at defectors, irrespective of whether defecting was typical or atypical. We suggest that the reduced punishment of defectors when they are common might often be explained in terms of the costs to the punisher, rather than responses to norm violators.     

When Punishment Pays

Explaining cooperation in groups remains a key problem because reciprocity breaks down between more than two. Punishing individuals who contribute little provides a potential answer but changes the dilemma to why pay the costs of punishing which, like cooperation itself, provides a public good. Nevertheless, people are observed to punish others in behavioural economic games, posing a problem for existing theory which highlights the difficulty in explaining the spread and persistence of punishment. Here, I consider the apparent mismatch between theory and evidence and show by means of instructive analysis and simulation how much of the experimental evidence for punishment comes from scenarios in which punishers may expect to obtain a net benefit from punishing free-riders. In repeated games within groups, punishment works by imposing costs on defectors so that it pays them to switch to cooperating. Both punishers and non-punishers then benefit from the resulting increase in cooperation, hence investing in punishment can constitute a social dilemma. However, I show the conditions in which the benefits of increased cooperation are so great that they more than offset the costs of punishing, thereby removing the temptation to free-ride on others'' investments and making punishment explicable in terms of direct self-interest. Crucially, this is because of the leveraging effect imposed in typical studies whereby people can pay a small cost to inflict a heavy loss on a punished individual. In contrast to previous models suggesting punishment is disadvantaged when rare, I show it can invade until it comes into a producer-scrounger equilibrium with non-punishers. I conclude that adding punishment to an iterated public goods game can solve the problem of achieving cooperation by removing the social dilemma.     

The shared reward dilemma

One of the most direct human mechanisms of promoting cooperation is rewarding it. We study the effect of sharing a reward among cooperators in the most stringent form of social dilemma, namely the prisoner's dilemma (PD). Specifically, for a group of players that collect payoffs by playing a pairwise PD game with their partners, we consider an external entity that distributes a fixed reward equally among all cooperators. Thus, individuals confront a new dilemma: on the one hand, they may be inclined to choose the shared reward despite the possibility of being exploited by defectors; on the other hand, if too many players do that, cooperators will obtain a poor reward and defectors will outperform them. By appropriately tuning the amount to be shared a vast variety of scenarios arises, including the traditional ones in the study of cooperation as well as more complex situations where unexpected behavior can occur. We provide a complete classification of the equilibria of the n-player game as well as of its evolutionary dynamics.     

Evolutionary games and population dynamics: maintenance of cooperation in public goods games

The emergence and abundance of cooperation in nature poses a tenacious and challenging puzzle to evolutionary biology. Cooperative behaviour seems to contradict Darwinian evolution because altruistic individuals increase the fitness of other members of the population at a cost to themselves. Thus, in the absence of supporting mechanisms, cooperation should decrease and vanish, as predicted by classical models for cooperation in evolutionary game theory, such as the Prisoner's Dilemma and public goods games. Traditional approaches to studying the problem of cooperation assume constant population sizes and thus neglect the ecology of the interacting individuals. Here, we incorporate ecological dynamics into evolutionary games and reveal a new mechanism for maintaining cooperation. In public goods games, cooperation can gain a foothold if the population density depends on the average population payoff. Decreasing population densities, due to defection leading to small payoffs, results in smaller interaction group sizes in which cooperation can be favoured. This feedback between ecological dynamics and game dynamics can generate stable coexistence of cooperators and defectors in public goods games. However, this mechanism fails for pairwise Prisoner's Dilemma interactions and the population is driven to extinction. Our model represents natural extension of replicator dynamics to populations of varying densities.     

Spatial invasion of cooperation

The evolutionary puzzle of cooperation describes situations where cooperators provide a fitness benefit to other individuals at some cost to themselves. Under Darwinian selection, the evolution of cooperation is a conundrum, whereas non-cooperation (or defection) is not. In the absence of supporting mechanisms, cooperators perform poorly and decrease in abundance. Evolutionary game theory provides a powerful mathematical framework to address the problem of cooperation using the prisoner's dilemma. One well-studied possibility to maintain cooperation is to consider structured populations, where each individual interacts only with a limited subset of the population. This enables cooperators to form clusters such that they are more likely to interact with other cooperators instead of being exploited by defectors. Here we present a detailed analysis of how a few cooperators invade and expand in a world of defectors. If the invasion succeeds, the expansion process takes place in two stages: first, cooperators and defectors quickly establish a local equilibrium and then they uniformly expand in space. The second stage provides good estimates for the global equilibrium frequencies of cooperators and defectors. Under hospitable conditions, cooperators typically form a single, ever growing cluster interspersed with specks of defectors, whereas under more hostile conditions, cooperators form isolated, compact clusters that minimize exploitation by defectors. We provide the first quantitative assessment of the way cooperators arrange in space during invasion and find that the macroscopic properties and the emerging spatial patterns reveal information about the characteristics of the underlying microscopic interactions.     

Defectors Can Create Conditions That Rescue Cooperation

Cooperation based on the production of costly common goods is observed throughout nature. This is puzzling, as cooperation is vulnerable to exploitation by defectors which enjoy a fitness advantage by consuming the common good without contributing fairly. Depletion of the common good can lead to population collapse and the destruction of cooperation. However, population collapse implies small population size, which, in a structured population, is known to favor cooperation. This happens because small population size increases variability in cooperator frequency across different locations. Since individuals in cooperator-dominated locations (which are most likely cooperators) will grow more than those in defector-dominated locations (which are most likely defectors), cooperators can outgrow defectors globally despite defectors outgrowing cooperators in each location. This raises the possibility that defectors can lead to conditions that sometimes rescue cooperation from defector-induced destruction. We demonstrate multiple mechanisms through which this can occur, using an individual-based approach to model stochastic birth, death, migration, and mutation events. First, during defector-induced population collapse, defectors occasionally go extinct before cooperators by chance, which allows cooperators to grow. Second, empty locations, either preexisting or created by defector-induced population extinction, can favor cooperation because they allow cooperator but not defector migrants to grow. These factors lead to the counterintuitive result that the initial presence of defectors sometimes allows better survival of cooperation compared to when defectors are initially absent. Finally, we find that resource limitation, inducible by defectors, can select for mutations adaptive to resource limitation. When these mutations are initially present at low levels or continuously generated at a moderate rate, they can favor cooperation by further reducing local population size. We predict that in a structured population, small population sizes precipitated by defectors provide a “built-in” mechanism for the persistence of cooperation.     

Persistent cooperators in nature

The evolution and maintenance of cooperation fascinated researchers for several decades. Recently, theoretical models and experimental evidence show that costly punishment may facilitate cooperation in human societies. The puzzle how the costly punishment behaviour evolves can be solved under voluntary participation. Could the punishers emerge if participation is compulsory? Is the punishment inevitably a selfish behaviour or an altruistic behaviour? The motivations behind punishment are still an enigma. Based on public goods interactions, we present a model in which just a certain portion of the public good is divided equally among all members. The other portion is distributed to contributors when paying a second cost. The contributors who are willing to pay a second cost are called the persistent cooperators (PC), indicating their desire to retrieve the proportion of the payoff derived from their own contributions with persistent efforts. We show that the persistent cooperators can be costly punishers, which may account for the origin of human costly punishment behaviour under compulsory participation. In this sense our models may show theoretically that the original motivation behind punishment is to retrieve deserved payoff from their own contributions, a selfish incentive. But the persistent cooperators can also flourish or dominate the population in other situations. We list many real examples in which contributors are the persistent cooperators, and they benefit. This indicates a simple norm promoting cooperation: contributing more and gaining more.     

Collective Chasing Behavior between Cooperators and Defectors in the Spatial Prisoner’s Dilemma

Cooperation is one of the essential factors for all biological organisms in major evolutionary transitions. Recent studies have investigated the effect of migration for the evolution of cooperation. However, little is known about whether and how an individuals’ cooperativeness coevolves with mobility. One possibility is that mobility enhances cooperation by enabling cooperators to escape from defectors and form clusters; the other possibility is that mobility inhibits cooperation by helping the defectors to catch and exploit the groups of cooperators. In this study we investigate the coevolutionary dynamics by using the prisoner’s dilemma game model on a lattice structure. The computer simulations demonstrate that natural selection maintains cooperation in the form of evolutionary chasing between the cooperators and defectors. First, cooperative groups grow and collectively move in the same direction. Then, mutant defectors emerge and invade the cooperative groups, after which the defectors exploit the cooperators. Then other cooperative groups emerge due to mutation and the cycle is repeated. Here, it is worth noting that, as a result of natural selection, the mobility evolves towards directional migration, but not to random or completely fixed migration. Furthermore, with directional migration, the rate of global population extinction is lower when compared with other cases without the evolution of mobility (i.e., when mobility is preset to random or fixed). These findings illustrate the coevolutionary dynamics of cooperation and mobility through the directional chasing between cooperators and defectors.     

Synergy and discounting of cooperation in social dilemmas

The emergence and maintenance of cooperation by natural selection is an enduring conundrum in evolutionary biology, which has been studied using a variety of game theoretical models inspired by different biological situations. The most widely studied games are the Prisoner's Dilemma, the Snowdrift game and by-product mutualism for pairwise interactions, as well as Public Goods games in larger groups of interacting individuals. Here, we present a general framework for cooperation in social dilemmas in which all the traditional scenarios can be recovered as special cases. In social dilemmas, cooperators provide a benefit to the group at some cost, while defectors exploit the group by reaping the benefits without bearing the costs of cooperation. Using the concepts of discounting and synergy for describing how benefits accumulate when more than one cooperator is present in a group of interacting individuals, we recover the four basic scenarios of evolutionary dynamics given by (i) dominating defection, (ii) coexistence of defectors and cooperators, (iii) dominating cooperation and (iv) bi-stability, in which cooperators and defectors cannot invade each other. Generically, for groups of three or more interacting individuals further, more complex, dynamics can occur. Our framework provides the first unifying approach to model cooperation in different kinds of social dilemmas.     

Ecological public goods games: cooperation and bifurcation

The Public Goods Game is one of the most popular models for studying the origin and maintenance of cooperation. In its simplest form, this evolutionary game has two regimes: defection goes to fixation if the multiplication factor r is smaller than the interaction group size N, whereas cooperation goes to fixation if the multiplication factor r is larger than the interaction group size N. Hauert et al. [Hauert, C., Holmes, M., Doebeli, M., 2006a. Evolutionary games and population dynamics: Maintenance of cooperation in public goods games. Proc. R. Soc. Lond. B 273, 2565-2570] have introduced the Ecological Public Goods Game by viewing the payoffs from the evolutionary game as birth rates in a population dynamic model. This results in a feedback between ecological and evolutionary dynamics: if defectors are prevalent, birth rates are low and population densities decline, which leads to smaller interaction groups for the Public Goods game, and hence to dominance of cooperators, with a concomitant increase in birth rates and population densities. This feedback can lead to stable co-existence between cooperators and defectors. Here we provide a detailed analysis of the dynamics of the Ecological Public Goods Game, showing that the model exhibits various types of bifurcations, including supercritical Hopf bifurcations, which result in stable limit cycles, and hence in oscillatory co-existence of cooperators and defectors. These results show that including population dynamics in evolutionary games can have important consequences for the evolutionary dynamics of cooperation.     

The joker effect: cooperation driven by destructive agents

Understanding the emergence of cooperation is a central issue in evolutionary game theory. The hardest setup for the attainment of cooperation in a population of individuals is the Public Goods game in which cooperative agents generate a common good at their own expenses, while defectors “free-ride” this good. Eventually this causes the exhaustion of the good, a situation which is bad for everybody. Previous results have shown that introducing reputation, allowing for volunteer participation, punishing defectors, rewarding cooperators or structuring agents, can enhance cooperation. Here we present a model which shows how the introduction of rare, malicious agents - that we term jokers - performing just destructive actions on the other agents induce bursts of cooperation. The appearance of jokers promotes a rock-paper-scissors dynamics, where jokers outbeat defectors and cooperators outperform jokers, which are subsequently invaded by defectors. Thus, paradoxically, the existence of destructive agents acting indiscriminately promotes cooperation.     

Invasion and expansion of cooperators in lattice populations: Prisoner's dilemma vs. snowdrift games

The evolution of cooperation is an enduring conundrum in biology and the social sciences. Two social dilemmas, the prisoner's dilemma and the snowdrift game have emerged as the most promising mathematical metaphors to study cooperation. Spatial structure with limited local interactions has long been identified as a potent promoter of cooperation in the prisoner's dilemma but in the spatial snowdrift game, space may actually enhance or inhibit cooperation. Here we investigate and link the microscopic interaction between individuals to the characteristics of the emerging macroscopic patterns generated by the spatial invasion process of cooperators in a world of defectors. In our simulations, individuals are located on a square lattice with Moore neighborhood and update their strategies by probabilistically imitating the strategies of better performing neighbors. Under sufficiently benign conditions, cooperators can survive in both games. After rapid local equilibration, cooperators expand quadratically until global saturation is reached. Under favorable conditions, cooperators expand as a large contiguous cluster in both games with minor differences concerning the shape of embedded defectors. Under less favorable conditions, however, distinct differences arise. In the prisoner's dilemma, cooperators break up into isolated, compact clusters. The compact clustering reduces exploitation and leads to positive assortment, such that cooperators interact more frequently with other cooperators than with defectors. In contrast, in the snowdrift game, cooperators form small, dendritic clusters, which results in negative assortment and cooperators interact more frequently with defectors than with other cooperators. In order to characterize and quantify the emerging spatial patterns, we introduce a measure for the cluster shape and demonstrate that the macroscopic patterns can be used to determine the characteristics of the underlying microscopic interactions.