The evolution of payoff matrices: providing incentives to cooperate

Most of the work in evolutionary game theory starts with a model of a social situation that gives rise to a particular payoff matrix and analyses how behaviour evolves through natural selection. Here, we invert this approach and ask, given a model of how individuals behave, how the payoff matrix will evolve through natural selection. In particular, we ask whether a prisoner's dilemma game is stable against invasions by mutant genotypes that alter the payoffs. To answer this question, we develop a two-tiered framework with goal-oriented dynamics at the behavioural time scale and a diploid population genetic model at the evolutionary time scale. Our results are two-fold: first, we show that the prisoner's dilemma is subject to invasions by mutants that provide incentives for cooperation to their partners, and that the resulting game is a coordination game similar to the hawk-dove game. Second, we find that for a large class of mutants and symmetric games, a stable genetic polymorphism will exist in the locus determining the payoff matrix, resulting in a complex pattern of behavioural diversity in the population. Our results highlight the importance of considering the evolution of payoff matrices to understand the evolution of animal social systems.     

Spatial effects in social dilemmas

Social dilemmas and the evolutionary conundrum of cooperation are traditionally studied through various kinds of game theoretical models such as the prisoner's dilemma, public goods games, snowdrift games or by-product mutualism. All of them exemplify situations which are characterized by different degrees of conflicting interests between the individuals and the community. In groups of interacting individuals, cooperators produce a common good benefitting the entire group at some cost to themselves, whereas defectors attempt to exploit the resource by avoiding the costly contributions. Based on synergistic or discounted accumulation of cooperative benefits a unifying theoretical framework was recently introduced that encompasses all games that have traditionally been studied separately (Hauert, Michor, Nowak, Doebeli, 2005. Synergy and discounting of cooperation in social dilemmas. J. Theor. Biol., in press.). Within this framework we investigate the effects of spatial structure with limited local interactions on the evolutionary fate of cooperators and defectors. The quantitative effects of space turn out to be quite sensitive to the underlying microscopic update mechanisms but, more general, we demonstrate that in prisoner's dilemma type interactions spatial structure benefits cooperation-although the parameter range is quite limited-whereas in snowdrift type interactions spatial structure may be beneficial too, but often turns out to be detrimental to cooperation.     

Maintenance affects the stability of a two-tiered microbial 'food chain'?

Recent studies have shown that constraints on available resources may play an important role in the evolution of cooperation, especially when individuals do not posses the capacity to recognize other individuals, memory or other developed abilities, as it is the case of most unicellular organisms, algae or even plants. We analyze the evolution of cooperation in the case of a limiting resource, which is necessary for reproduction and survival. We show that, if the strategies determine a prisoner's dilemma, the outcome of the interactions may be modified by the limitation of resources allowing cooperators to invade the entire population. Analytic expressions for the region of cooperation are provided. Furthermore we derive expressions for the connection between fitness, as understood in evolutionary game theory, and resource exchanges, which may be of help to link evolutionary game theoretical results with resource based models.     

No unique effect of intergroup competition on cooperation: non-competitive thresholds are as effective as competitions between groups for increasing human cooperative behavior

Explaining cooperation remains a central topic for evolutionary theorists. Many have argued that group selection provides such an explanation: theoretical models show that intergroup competition could have given rise to cooperation that is costly for the individual. Whether group selection actually did play an important role in the evolution of human cooperation, however, is much debated. Recent experiments have shown that intergroup competitions do increase human cooperation, which has been taken as evidence for group selection as a mechanism for the evolution of cooperation. Here we challenge this standard interpretation. Competitions change the payoff structure by creating a threshold effect whereby the group that contributes more earns an additional prize, which creates some incentive for individuals to cooperate. We present four studies that disentangle competition and thresholds, and strongly suggest that it is thresholds – rather than competitions per se – that increase cooperation. Thus, prior intergroup competition experiments provide no evidence of a unique or special role for intergroup competition in promoting human cooperation, and shed no light on whether group selection shaped human evolution.     

Replicator dynamics of reward & reputation in public goods games

Public goods games have become the mathematical metaphor for game theoretical investigations of cooperative behavior in groups of interacting individuals. Cooperation is a conundrum because cooperators make a sacrifice to benefit others at some cost to themselves. Exploiters or defectors reap the benefits and forgo costs. Despite the fact that groups of cooperators outperform groups of defectors, Darwinian selection or utilitarian principles based on rational choice should favor defectors. In order to overcome this social dilemma, much effort has been expended for investigations pertaining to punishment and sanctioning measures against defectors. Interestingly, the complementary approach to create positive incentives and to reward cooperation has received considerably less attention—despite being heavily advocated in education and social sciences for increasing productivity or preventing conflicts. Here we show that rewards can indeed stimulate cooperation in interaction groups of arbitrary size but, in contrast to punishment, fail to stabilize it. In both cases, however, reputation is essential. The combination of reward and reputation result in complex dynamics dominated by unpredictable oscillations.     

The logic of catastrophe

It has been assumed that public or collective goods, such as resource conservation or pollution reduction, will not be created or maintained since rational actors in the international system will choose to consume these goods while avoiding the cost of preserving them. I argue that on the contrary private and public goods are so interrelated that cooperative coalition behavior over their generation is possible. The problem of the maintenance of such cooperative coalitions, via an internal bargaining process, is considered. It is suggested that local optimizing behavior by a coalition may bring about qualitative change in the system, to the extent that the stability of the coalition is destroyed. Such a catastrophic change could lead to entirely new patterns of cooperation, and hence to qualitatively different development paths for the economic and ecological system.An earlier version of this paper, entitled The Logic of Collective Action and the Law of the Commons, was presented at the XIX International Meeting of the Institute of Management Science, on Management Science and the Quality of Life, Houston, Texas, April, 1972.     

Anti-social punishment can prevent the co-evolution of punishment and cooperation

The evolution of cooperation is one of the great puzzles in evolutionary biology. Punishment has been suggested as one solution to this problem. Here punishment is generally defined as incurring a cost to inflict harm on a wrong-doer. In the presence of punishers, cooperators can gain higher payoffs than non-cooperators. Therefore cooperation may evolve as long as punishment is prevalent in the population. Theoretical models have revealed that spatial structure can favor the co-evolution of punishment and cooperation, by allowing individuals to only play and compete with those in their immediate neighborhood. However, those models have usually assumed that punishment is always targeted at non-cooperators. In light of recent empirical evidence of punishment targeted at cooperators, we relax this assumption and study the effect of so-called ‘anti-social punishment’. We find that evolution can favor anti-social punishment, and that when anti-social punishment is possible costly punishment no longer promotes cooperation. As there is no reason to assume that cooperators cannot be the target of punishment during evolution, our results demonstrate serious restrictions on the ability of costly punishment to allow the evolution of cooperation in spatially structured populations. Our results also help to make sense of the empirical observation that defectors will sometimes pay to punish cooperators.     

Resolving the iterated prisoner's dilemma: theory and reality

Pairs of unrelated individuals face a prisoner's dilemma if cooperation is the best mutual outcome, but each player does best to defect regardless of his partner's behaviour. Although mutual defection is the only evolutionarily stable strategy in one-shot games, cooperative solutions based on reciprocity can emerge in iterated games. Among the most prominent theoretical solutions are the so-called bookkeeping strategies, such as tit-for-tat, where individuals copy their partner's behaviour in the previous round. However, the lack of empirical data conforming to predicted strategies has prompted the suggestion that the iterated prisoner's dilemma (IPD) is neither a useful nor realistic basis for investigating cooperation. Here, we discuss several recent studies where authors have used the IPD framework to interpret their data. We evaluate the validity of their approach and highlight the diversity of proposed solutions. Strategies based on precise accounting are relatively uncommon, perhaps because the full set of assumptions of the IPD model are rarely satisfied. Instead, animals use a diverse array of strategies that apparently promote cooperation, despite the temptation to cheat. These include both positive and negative reciprocity, as well as long-term mutual investments based on 'friendships'. Although there are various gaps in these studies that remain to be filled, we argue that in most cases, individuals could theoretically benefit from cheating and that cooperation cannot therefore be explained with the concept of positive pseudo-reciprocity. We suggest that by incorporating empirical data into the theoretical framework, we may gain fundamental new insights into the evolution of mutual reciprocal investment in nature.     

Spatial invasion of cooperation

The evolutionary puzzle of cooperation describes situations where cooperators provide a fitness benefit to other individuals at some cost to themselves. Under Darwinian selection, the evolution of cooperation is a conundrum, whereas non-cooperation (or defection) is not. In the absence of supporting mechanisms, cooperators perform poorly and decrease in abundance. Evolutionary game theory provides a powerful mathematical framework to address the problem of cooperation using the prisoner's dilemma. One well-studied possibility to maintain cooperation is to consider structured populations, where each individual interacts only with a limited subset of the population. This enables cooperators to form clusters such that they are more likely to interact with other cooperators instead of being exploited by defectors. Here we present a detailed analysis of how a few cooperators invade and expand in a world of defectors. If the invasion succeeds, the expansion process takes place in two stages: first, cooperators and defectors quickly establish a local equilibrium and then they uniformly expand in space. The second stage provides good estimates for the global equilibrium frequencies of cooperators and defectors. Under hospitable conditions, cooperators typically form a single, ever growing cluster interspersed with specks of defectors, whereas under more hostile conditions, cooperators form isolated, compact clusters that minimize exploitation by defectors. We provide the first quantitative assessment of the way cooperators arrange in space during invasion and find that the macroscopic properties and the emerging spatial patterns reveal information about the characteristics of the underlying microscopic interactions.     

Evolution of cooperation under -person snowdrift games

In the animal world, performing a given task which is beneficial to an entire group requires the cooperation of several individuals of that group who often share the workload required to perform the task. The mathematical framework to study the dynamics of collective action is game theory. Here we study the evolutionary dynamics of cooperators and defectors in a population in which groups of individuals engage in N-person, non-excludable public goods games. We explore an N-person generalization of the well-known two-person snowdrift game. We discuss both the case of infinite and finite populations, taking explicitly into consideration the possible existence of a threshold above which collective action is materialized. Whereas in infinite populations, an N-person snowdrift game (NSG) leads to a stable coexistence between cooperators and defectors, the introduction of a threshold leads to the appearance of a new interior fixed point associated with a coordination threshold. The fingerprints of the stable and unstable interior fixed points still affect the evolutionary dynamics in finite populations, despite evolution leading the population inexorably to a monomorphic end-state. However, when the group size and population size become comparable, we find that spite sets in, rendering cooperation unfeasible.     

Probabilistic participation in public goods games

Voluntary participation in public goods games (PGGs) has turned out to be a simple but effective mechanism for promoting cooperation under full anonymity. Voluntary participation allows individuals to adopt a risk-aversion strategy, termed loner. A loner refuses to participate in unpromising public enterprises and instead relies on a small but fixed pay-off. This system leads to a cyclic dominance of three pure strategies, cooperators, defectors and loners, but at the same time, there remain two considerable restrictions: the addition of loners cannot stabilize the dynamics and the time average pay-off for each strategy remains equal to the pay-off of loners. Here, we introduce probabilistic participation in PGGs from the standpoint of diversification of risk, namely simple mixed strategies with loners, and prove the existence of a dynamical regime in which the restrictions ono longer hold. Considering two kinds of mixed strategies associated with participants (cooperators or defectors) and non-participants (loners), we can recover all basic evolutionary dynamics of the two strategies: dominance; coexistence; bistability; and neutrality, as special cases depending on pairs of probabilities. Of special interest is that the expected pay-off of each mixed strategy exceeds the pay-off of loners at some interior equilibrium in the coexistence region.     

Cooperation as a volunteer’s dilemma and the strategy of conflict in public goods games

Conflict and cooperation for the exploitation of public goods are usually modelled as an N‐person prisoner’s dilemma. Many social dilemmas, however, would be described more properly as a volunteer’s dilemma, in which a certain number of individuals are necessary to produce a public good. If volunteering is costly, but so is failure to produce the public good, cheaters can invade and form a stable mixed equilibrium with cooperators. The dilemma is that the benefit for the group decreases with group size because the larger the group is, the less likely it is that someone volunteers. This problem persists even in the presence of a high degree of relatedness between group members. This model provides precise, testable predictions for the stability of cooperation. It also suggests a counterintuitive but practical solution for this kind of social dilemmas: increasing the damage resulting from the failure to produce the public good increases the probability that the public good is actually produced. Adopting a strategy that entails a deliberate risk (brinkmanship), therefore, can lead to a benefit for the society without being detrimental for the individual.