Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game

Understanding the mechanisms that can lead to the evolution of cooperation through natural selection is a core problem in biology. Among the various attempts at constructing a theory of cooperation, game theory has played a central role. Here, we review models of cooperation that are based on two simple games: the Prisoner's Dilemma, and the Snowdrift game. Both games are two‐person games with two strategies, to cooperate and to defect, and both games are social dilemmas. In social dilemmas, cooperation is prone to exploitation by defectors, and the average payoff in populations at evolutionary equilibrium is lower than it would be in populations consisting of only cooperators. The difference between the games is that cooperation is not maintained in the Prisoner's Dilemma, but persists in the Snowdrift game at an intermediate frequency. As a consequence, insights gained from studying extensions of the two games differ substantially. We review the most salient results obtained from extensions such as iteration, spatial structure, continuously variable cooperative investments, and multi‐person interactions. Bridging the gap between theoretical and empirical research is one of the main challenges for future studies of cooperation, and we conclude by pointing out a number of promising natural systems in which the theory can be tested experimentally.     

Evolution of cooperation with shared costs and benefits

The quest to determine how cooperation evolves can be based on evolutionary game theory, in spite of the fact that evolutionarily stable strategies (ESS) for most non-zero-sum games are not cooperative. We analyse the evolution of cooperation for a family of evolutionary games involving shared costs and benefits with a continuum of strategies from non-cooperation to total cooperation. This cost-benefit game allows the cooperator to share in the benefit of a cooperative act, and the recipient to be burdened with a share of the cooperator's cost. The cost-benefit game encompasses the Prisoner's Dilemma, Snowdrift game and Partial Altruism. The models produce ESS solutions of total cooperation, partial cooperation, non-cooperation and coexistence between cooperation and non-cooperation. Cooperation emerges from an interplay between the nonlinearities in the cost and benefit functions. If benefits increase at a decelerating rate and costs increase at an accelerating rate with the degree of cooperation, then the ESS has an intermediate level of cooperation. The game also exhibits non-ESS points such as unstable minima, convergent-stable minima and unstable maxima. The emergence of cooperative behaviour in this game represents enlightened self-interest, whereas non-cooperative solutions illustrate the Tragedy of the Commons. Games having either a stable maximum or a stable minimum have the property that small changes in the incentive structure (model parameter values) or culture (starting frequencies of strategies) result in correspondingly small changes in the degree of cooperation. Conversely, with unstable maxima or unstable minima, small changes in the incentive structure or culture can result in a switch from non-cooperation to total cooperation (and vice versa). These solutions identify when human or animal societies have the potential for cooperation and whether cooperation is robust or fragile.     

Exploring the effects of working for endowments on behaviour in standard economic games

In recent years, significant advances have been made in understanding the adaptive (ultimate) and mechanistic (proximate) explanations for the evolution and maintenance of cooperation. Studies of cooperative behaviour in humans invariably use economic games. These games have provided important insights into the mechanisms that maintain economic and social cooperation in our species. However, they usually rely on the division of monetary tokens which are given to participants by the investigator. The extent to which behaviour in such games may reflect behaviour in the real world of biological markets--where money must be earned and behavioural strategies incur real costs and benefits--is unclear. To provide new data on the potential scale of this problem, we investigated whether people behaved differently in two standard economic games (public goods game and dictator game) when they had to earn their monetary endowments through the completion of dull or physically demanding tasks, as compared with simply being given the endowment. The requirement for endowments to be 'earned' through labour did not affect behaviour in the dictator game. However, the requirement to complete a dull task reduced cooperation in the public goods game among the subset of participants who were not familiar with game theory. There has been some effort to test whether the conclusions drawn from standard, token-based cooperation games adequately reflect cooperative behaviour 'in the wild.' However, given the almost total reliance on such games to study cooperation, more exploration of this issue would be welcome. Our data are not unduly worrying, but they do suggest that further exploration is needed if we are to make general inferences about human behaviour from the results of structured economic games.     

Evolutionary games and population dynamics: maintenance of cooperation in public goods games

The emergence and abundance of cooperation in nature poses a tenacious and challenging puzzle to evolutionary biology. Cooperative behaviour seems to contradict Darwinian evolution because altruistic individuals increase the fitness of other members of the population at a cost to themselves. Thus, in the absence of supporting mechanisms, cooperation should decrease and vanish, as predicted by classical models for cooperation in evolutionary game theory, such as the Prisoner's Dilemma and public goods games. Traditional approaches to studying the problem of cooperation assume constant population sizes and thus neglect the ecology of the interacting individuals. Here, we incorporate ecological dynamics into evolutionary games and reveal a new mechanism for maintaining cooperation. In public goods games, cooperation can gain a foothold if the population density depends on the average population payoff. Decreasing population densities, due to defection leading to small payoffs, results in smaller interaction group sizes in which cooperation can be favoured. This feedback between ecological dynamics and game dynamics can generate stable coexistence of cooperators and defectors in public goods games. However, this mechanism fails for pairwise Prisoner's Dilemma interactions and the population is driven to extinction. Our model represents natural extension of replicator dynamics to populations of varying densities.     

Group-size diversity in public goods games

Public goods games are models of social dilemmas where cooperators pay a cost for the production of a public good while defectors free ride on the contributions of cooperators. In the traditional framework of evolutionary game theory, the payoffs of cooperators and defectors result from interactions in groups formed by binomial sampling from an infinite population. Despite empirical evidence showing that group-size distributions in nature are highly heterogeneous, most models of social evolution assume that the group size is constant. In this article, I remove this assumption and explore the effects of having random group sizes on the evolutionary dynamics of public goods games. By a straightforward application of Jensen's inequality, I show that the outcome of general nonlinear public goods games depends not only on the average group size but also on the variance of the group-size distribution. This general result is illustrated with two nonlinear public goods games (the public goods game with discounting or synergy and the N-person volunteer's dilemma) and three different group-size distributions (Poisson, geometric, and Waring). The results suggest that failing to acknowledge the natural variation of group sizes can lead to an underestimation of the actual level of cooperation exhibited in evolving populations.     

Evolutionary games of cooperation: Insights through integration of theory and data

Cooperation is central to the regulation of many ecological processes and the persistence of ecosystems and their associated functions. However, the evolution of cooperation amongst non-kin appears paradoxical. Games such as the prisoner's dilemma, snowdrift and stag hunt have been borrowed from game theory and used extensively to investigate cooperation. Advances in this area have been numerous and have been provided by both empirical and theoretical studies. We outline some of the common games used and review some of the major findings and recent advancements made in this area. We show a clear link between data and theory, and how this link has been key to our understanding of cooperation.     

Eco-evolutionary feedback and the invasion of cooperation in prisoner's dilemma games

Unveiling the origin and forms of cooperation in nature poses profound challenges in evolutionary ecology. The prisoner's dilemma game is an important metaphor for studying the evolution of cooperation. We here classified potential mechanisms for cooperation evolution into schemes of frequency- and density-dependent selection, and focused on the density-dependent selection in the ecological prisoner's dilemma games. We found that, although assortative encounter is still the necessary condition in ecological games for cooperation evolution, a harsh environment, indicated by a high mortality, can foster the invasion of cooperation. The Hamilton rule provides a fundamental condition for the evolution of cooperation by ensuring an enhanced relatedness between players in low-density populations. Incorporating ecological dynamics into evolutionary games opens up a much wider window for the evolution of cooperation, and exhibits a variety of complex behaviors of dynamics, such as limit and heteroclinic cycles. An alternative evolutionary, or rather succession, sequence was proposed that cooperation first appears in harsh environments, followed by the invasion of defection, which leads to a common catastrophe. The rise of cooperation (and altruism), thus, could be much easier in the density-dependent ecological games than in the classic frequency-dependent evolutionary games.     

Interaction characteristics as evolutionary features for the spatial Prisoner’s Dilemma in a population modeled by continuous probabilistic cellular automata and evolutionary algorithm

Evolutionary games usually take into consideration individuals’ strategies as the transformative characteristic which leads to the evolution of the population. Here, besides the strategies, interaction aspects are also considered as evolutionary attributes which can change over time as the replacement dynamic renovates the population choosing locally better individuals to reproduce. The population is modeled by cellular automata, interactions by the Prisoner’s Dilemma game and the replacement process is ruled by two versions of death-birth dynamic. Although the average payoff per game is considered as the fitness for choosing better individuals, the number of games per time step and a maximum radius of interaction with neighbours are also present in the individual’s chromosome which is passed to the next generation. Numerical simulations show that individual interaction properties and cooperation level are linked to the version of death-birth dynamic used and the game payoff. For instance, when the fitness bias is on the death event, individuals have more interactions in a larger radius, and the cooperation level is usually lower than the case where the fitness bias is on the birth event. Also, the individuals’ interaction profiles are heterogeneous, and cooperative individuals form clusters in the lattice to protect themselves.     

The effect of dispersal and neighbourhood in games of cooperation

The prisoner's dilemma (PD) and the snowdrift (SD) games are paradigmatic tools to investigate the origin of cooperation. Whereas spatial structure (e.g. nonrandom spatial distribution of strategies) present in the spatially explicit models facilitates the emergence of cooperation in the PD game, recent investigations have suggested that spatial structure can be unfavourable for cooperation in the SD game. The frequency of cooperators in a spatially explicit SD game can be lower than it would be in an infinitely large well-mixed population. However, the source of this effect cannot be identified with certainty as spatially explicit games differ from well-mixed games in two aspects: (i) they introduce spatial correlations, (ii) and limited neighbourhood. Here we extend earlier investigations to identify the source of this effect, and thus accordingly we study a spatially explicit version of the PD and SD games with varying degrees of dispersal and neighbourhood size. It was found that dispersal favours selfish individuals in both games. We calculated the frequency of cooperators at strong dispersal limit, which in concordance with the numerical results shows that it is the short range of interactions (i.e. limited neighbourhood) and not spatial correlations that decreases the frequency of cooperators in spatially explicit models of populations. Our results demonstrate that spatial correlations are always beneficial to cooperators in both the PD and SD games. We explain the opposite effect of dispersal and neighbourhood structure, and discuss the relevance of distinguishing the two effects in general.     

An institutional mechanism for assortment in an ecology of games

Recent research has revived Long's "ecology of games" model to analyze how social actors cooperate in the context of multiple political and social games. However, there is still a paucity of theoretical work that considers the mechanisms by which large-scale cooperation can be promoted in a dynamic institutional landscape, in which actors can join new games and leave old ones. This paper develops an agent-based model of an ecology of games where agents participate in multiple public goods games. In addition to contribution decisions, the agents can leave and join different games, and these processes are de-coupled. We show that the payoff for cooperation is greater than for defection when limits to the number of actors per game ("capacity constraints") structure the population in ways that allow cooperators to cluster, independent of any complex individual-level mechanisms such as reputation or punishment. Our model suggests that capacity constraints are one effective mechanism for producing positive assortment and increasing cooperation in an ecology of games. The results suggest an important trade-off between the inclusiveness of policy processes and cooperation: Fully inclusive policy processes reduce the chances of cooperation.     

A dynamic over games drives selfish agents to win–win outcomes

Understanding human institutions, animal cultures and other social systems requires flexible formalisms that describe how their members change them from within. We introduce a framework for modelling how agents change the games they participate in. We contrast this between-game ‘institutional evolution’ with the more familiar within-game ‘behavioural evolution’. We model institutional change by following small numbers of persistent agents as they select and play a changing series of games. Starting from an initial game, a group of agents trace trajectories through game space by navigating to increasingly preferable games until they converge on ‘attractor’ games. Agents use their ‘institutional preferences'' for game features (such as stability, fairness and efficiency) to choose between neighbouring games. We use this framework to pose a pressing question: what kinds of games does institutional evolution select for; what is in the attractors? After computing institutional change trajectories over the two-player space, we find that attractors have disproportionately fair outcomes, even though the agents who produce them are strictly self-interested and indifferent to fairness. This seems to occur because game fairness co-occurs with the self-serving features these agents do actually prefer. We thus present institutional evolution as a mechanism for encouraging the spontaneous emergence of cooperation among small groups of inherently selfish agents, without space, reputation, repetition, or other more familiar mechanisms. Game space trajectories provide a flexible, testable formalism for modelling the interdependencies of behavioural and institutional evolutionary processes, as well as a mechanism for the evolution of cooperation.     

Emergence of Super Cooperation of Prisoner’s Dilemma Games on Scale-Free Networks

Recently, the authors proposed a quantum prisoner’s dilemma game based on the spatial game of Nowak and May, and showed that the game can be played classically. By using this idea, we proposed three generalized prisoner’s dilemma (GPD, for short) games based on the weak Prisoner’s dilemma game, the full prisoner’s dilemma game and the normalized Prisoner’s dilemma game, written by GPD_W, GPD_F and GPD_N respectively. Our games consist of two players, each of which has three strategies: cooperator (C), defector (D) and super cooperator (denoted by Q), and have a parameter γ to measure the entangled relationship between the two players. We found that our generalised prisoner’s dilemma games have new Nash equilibrium principles, that entanglement is the principle of emergence and convergence (i.e., guaranteed emergence) of super cooperation in evolutions of our generalised prisoner’s dilemma games on scale-free networks, that entanglement provides a threshold for a phase transition of super cooperation in evolutions of our generalised prisoner’s dilemma games on scale-free networks, that the role of heterogeneity of the scale-free networks in cooperations and super cooperations is very limited, and that well-defined structures of scale-free networks allow coexistence of cooperators and super cooperators in the evolutions of the weak version of our generalised prisoner’s dilemma games.     

Random mobility and spatial structure often enhance cooperation

The effects of an unconditional move rule in the spatial Prisoner's Dilemma, Snowdrift and Stag Hunt games are studied. Spatial structure by itself is known to modify the outcome of many games when compared with a randomly mixed population, sometimes promoting, sometimes inhibiting cooperation. Here we show that random dilution and mobility may suppress the inhibiting factors of the spatial structure in the Snowdrift game, while enhancing the already larger cooperation found in the Prisoner's dilemma and Stag Hunt games.     

Games network and application to PAs system

In this article, we present a game theory based framework, named games network, for modeling biological interactions. After introducing the theory, we more precisely describe the methodology to model biological interactions. Then we apply it to the plasminogen activator system (PAs) which is a signal transduction pathway involved in cancer cell migration. The games network theory extends game theory by including the locality of interactions. Each game in a games network represents local interactions between biological agents. The PAs system is implicated in cytoskeleton modifications via regulation of actin and microtubules, which in turn favors cell migration. The games network model has enabled us a better understanding of the regulation involved in the PAs system.     

Bounded rationality in volunteering public goods games

It is one of the fundamental problems in biology and social sciences how to maintain high levels of cooperation among selfish individuals. Theorists present an effective mechanism promoting cooperation by allowing for voluntary participation in public goods games. But Nash's theory predicts that no one can do better or worse than loners (players unwilling to join the public goods game) in the long run, and that the frequency of participants is independent of loners’ payoff. In this paper, we introduce a degree of rationality and investigate the model by means of an approximate best response dynamics. Our research shows that the payoffs of the loners have a significant effect in anonymous voluntary public goods games by this introduction and that the dynamics will drive the system to a fixed point, which is different from the Nash equilibrium. In addition, we also qualitatively explain the existing experimental results.     

Consequences of fluctuating group size for the evolution of cooperation

Studies of cooperation have traditionally focused on discrete games such as the well-known prisoner’s dilemma, in which players choose between two pure strategies: cooperation and defection. Increasingly, however, cooperation is being studied in continuous games that feature a continuum of strategies determining the level of cooperative investment. For the continuous snowdrift game, it has been shown that a gradually evolving monomorphic population may undergo evolutionary branching, resulting in the emergence of a defector strategy that coexists with a cooperator strategy. This phenomenon has been dubbed the ‘tragedy of the commune’. Here we study the effects of fluctuating group size on the tragedy of the commune and derive analytical conditions for evolutionary branching. Our results show that the effects of fluctuating group size on evolutionary dynamics critically depend on the structure of payoff functions. For games with additively separable benefits and costs, fluctuations in group size make evolutionary branching less likely, and sufficiently large fluctuations in group size can always turn an evolutionary branching point into a locally evolutionarily stable strategy. For games with multiplicatively separable benefits and costs, fluctuations in group size can either prevent or induce the tragedy of the commune. For games with general interactions between benefits and costs, we derive a general classification scheme based on second derivatives of the payoff function, to elucidate when fluctuations in group size help or hinder cooperation.     

An Application of Evolutionary Game Theory to Social Dilemmas: The Traveler's Dilemma and the Minimum Effort Coordination Game

The Traveler''s Dilemma game and the Minimum Effort Coordination game are two social dilemmas that have attracted considerable attention due to the fact that the predictions of classical game theory are at odds with the results found when the games are studied experimentally. Moreover, a direct application of deterministic evolutionary game theory, as embodied in the replicator dynamics, to these games does not explain the observed behavior. In this work, we formulate natural variants of these two games as smoothed continuous-strategy games. We study the evolutionary dynamics of these continuous-strategy games, both analytically and through agent-based simulations, and show that the behavior predicted theoretically is in accord with that observed experimentally. Thus, these variants of the Traveler''s Dilemma and the Minimum Effort Coordination games provide a simple resolution of the paradoxical behavior associated with the original games.     

What initially brought about communications?

This paper reports an intelligent agent equipped with two-layer finite state machines (FSMs) that can communicate by turning lighting on and off, leading to social cooperation that solves the dilemma situation, modeled by a one-shot 2x2 game. This communication between two gaming agents can be observed in hero- and leader-type dilemma games, where alternating reciprocity, repeating cooperation (C)-defeat (D) after D-C, is the equal pareto optimum instead of a sequence of mutual cooperation that is the equal pareto optimum for a prisoner's dilemma (PD) game.     

Evolutionary dynamics in structured populations

Evolutionary dynamics shape the living world around us. At the centre of every evolutionary process is a population of reproducing individuals. The structure of that population affects evolutionary dynamics. The individuals can be molecules, cells, viruses, multicellular organisms or humans. Whenever the fitness of individuals depends on the relative abundance of phenotypes in the population, we are in the realm of evolutionary game theory. Evolutionary game theory is a general approach that can describe the competition of species in an ecosystem, the interaction between hosts and parasites, between viruses and cells, and also the spread of ideas and behaviours in the human population. In this perspective, we review the recent advances in evolutionary game dynamics with a particular emphasis on stochastic approaches in finite sized and structured populations. We give simple, fundamental laws that determine how natural selection chooses between competing strategies. We study the well-mixed population, evolutionary graph theory, games in phenotype space and evolutionary set theory. We apply these results to the evolution of cooperation. The mechanism that leads to the evolution of cooperation in these settings could be called ‘spatial selection’: cooperators prevail against defectors by clustering in physical or other spaces.     

A simple scaling of the effectiveness of supporting mutual cooperation in donor-recipient games by various reciprocity mechanisms

In donor-recipient games (DRG), one of the sub-classes of Prisoner's Dilemma (PD), it is well-known that a game structure is described by two parameters benefit (b) and cost (c) of cooperation. By means of a series of numerical experiments, we proved that the effectiveness of supporting mutual cooperation in DRG by various reciprocity mechanisms can be expressed in a single game structural parameter, b/c. This also implies that the dilemma strength in various donor-recipient games with various reciprocity mechanisms can be evaluated only by b/c, which is consistent with the previous novel finding by Nowak. It was also discussed whether this kind of parameterization idea can be extended to general games in PD game class.