The good, the bad and the discriminator--errors in direct and indirect reciprocity

This paper presents, in a series of simple diagrams, concise results about the replicator dynamics of direct and indirect reciprocity. We consider repeated interactions between donors and recipients, and analyse the relationship between three basic strategies for the donor: unconditional cooperation, all-out defection, and conditional cooperation. In other words, we investigate the competition of discriminating and indiscriminating altruists with defectors. Discriminators and defectors form a bistable community, and hence a population of discriminators cannot be invaded by defectors. But unconditional altruists can invade a discriminating population and 'soften it up' for a subsequent invasion by defectors. The resulting dynamics exhibits various forms of rock-paper-scissors cycles and depends in subtle ways on noise, in the form of errors in implementation. The probability for another round (in the case of direct reciprocity), and information about the co-player (in the case of indirect reciprocity), add further elements to the ecology of reciprocation.     

Evolution of indirect reciprocity in groups of various sizes and comparison with direct reciprocity

Recently many studies have investigated the evolution of indirect reciprocity through which cooperative action is returned by a third individual, e.g. individual A helped B and then receives help from C. Most studies on indirect reciprocity have presumed that only two individuals take part in a single interaction (group), e.g. A helps B and C helps A. In this paper, we investigate the evolution of indirect reciprocity when more than two individuals take part in a single group, and compare the result with direct reciprocity through which cooperative action is directly returned by the recipient. Our analyses show the following. In the population with discriminating cooperators and unconditional defectors, whether implementation error is included or not, (i) both strategies are evolutionarily stable and the evolution of indirect reciprocity becomes more difficult as group size increases, and (ii) the condition for the evolution of indirect reciprocity under standing reputation criterion where the third individuals distinguish between justified and unjustified defections is more relaxed than that under image scoring reputation criterion in which the third individuals do not distinguish with. Furthermore, in the population that also includes unconditional cooperators, (iii) in the presence of errors in implementation, the discriminating strategy is evolutionarily stable not only under standing but also under image scoring if group size is larger than two. Finally, (iv) in the absence of errors in implementation, the condition for the evolution of direct reciprocity is equivalent to that for the evolution of indirect reciprocity under standing, and, in the presence of errors, the condition for the evolution of direct reciprocity is very close to that for the evolution of indirect reciprocity under image scoring.     

The leading eight: social norms that can maintain cooperation by indirect reciprocity

The theory of indirect reciprocation explains the evolution of cooperation among unrelated individuals, engaging in one-shot interaction. Using reputation, a player acquires information on who are worth cooperating and who are not. In a previous paper, we formalized the reputation dynamics, a rule to assign a binary reputation (good or bad) to each player when his action, his current reputation, and the opponent's reputation are given. We then examined all the possible reputation dynamics, and found that there exist only eight reputation dynamics named "leading eight" that can maintain the ESS with a high level of cooperation, even if errors are included in executing intended cooperation and in reporting the observation to the public. In this paper, we study the nature of these successful social norms. First, we characterize the role of each pivot of the reputation dynamics common to all of the leading eight. We conclude that keys to the success in indirect reciprocity are to be nice (maintenance of cooperation among themselves), retaliatory (detection of defectors, punishment, and justification of punishment), apologetic, and forgiving. Second, we prove the two basic properties of the leading eight, which give a quantitative evaluation of the ESS condition and the level of cooperation maintained at the ESS.     

Evolutionary game dynamics with non-uniform interaction rates

The classical setting of evolutionary game theory, the replicator equation, assumes uniform interaction rates. The rate at which individuals meet and interact is independent of their strategies. Here we extend this framework by allowing the interaction rates to depend on the strategies. This extension leads to non-linear fitness functions. We show that a strict Nash equilibrium remains uninvadable for non-uniform interaction rates, but the conditions for evolutionary stability need to be modified. We analyze all games between two strategies. If the two strategies coexist or exclude each other, then the evolutionary dynamics do not change qualitatively, only the location of the equilibrium point changes. If, however, one strategy dominates the other in the classical setting, then the introduction of non-uniform interaction rates can lead to a pair of interior equilibria. For the Prisoner's Dilemma, non-uniform interaction rates allow the coexistence between cooperators and defectors. For the snowdrift game, non-uniform interaction rates change the equilibrium frequency of cooperators.     

Evolving learning rules and emergence of cooperation in spatial prisoner's dilemma

In the evolutionary Prisoner's dilemma (PD) game, agents play with each other and update their strategies in every generation according to some microscopic dynamical rule. In its spatial version, agents do not play with every other but, instead, interact only with their neighbours, thus mimicking the existing of a social or contact network that defines who interacts with whom. In this work, we explore evolutionary, spatial PD systems consisting of two types of agents, each with a certain update (reproduction, learning) rule. We investigate two different scenarios: in the first case, update rules remain fixed for the entire evolution of the system; in the second case, agents update both strategy and update rule in every generation. We show that in a well-mixed population the evolutionary outcome is always full defection. We subsequently focus on two-strategy competition with nearest-neighbour interactions on the contact network and synchronised update of strategies. Our results show that, for an important range of the parameters of the game, the final state of the system is largely different from that arising from the usual setup of a single, fixed dynamical rule. Furthermore, the results are also very different if update rules are fixed or evolve with the strategies. In these respect, we have studied representative update rules, finding that some of them may become extinct while others prevail. We describe the new and rich variety of final outcomes that arise from this co-evolutionary dynamics. We include examples of other neighbourhoods and asynchronous updating that confirm the robustness of our conclusions. Our results pave the way to an evolutionary rationale for modelling social interactions through game theory with a preferred set of update rules.     

Global mutations and local mutations have very different effects on evolution, illustrated by mixed strategies of asymmetric binary games

We study the evolutionary effect of rare mutations causing global changes in traits. We consider asymmetric binary games between two players. The first player takes two alternative options with probability x and 1−x; and the second player takes options with probability y and 1−y. Due to natural selection and recurrent mutation, the population evolves to have broad distributions of x and y. We analyze three cases showing qualitatively different dynamics, exemplified by (1) vigilance-intrusion game, (2) asymmetric hawk-dove game and (3) cleaner-client game. We found that the evolutionary outcome is strongly dependent upon the distribution of mutants’ traits, more than the mutation rates. For example in the vigilance-intrusion game, the evolutionary dynamics show a perpetual stable oscillation if mutants are always close to the parent (local-mutation mode), whilst the population converges to a stable equilibrium distribution if mutants can be quite different from the parent (global-mutation mode), even for extremely low mutation rate. When common local mutations and rare global mutations occur simultaneously, the evolutionary outcome is controlled by the latter.     

A game theoretical model of deforestation in human-environment relationships

We studied a two-person game regarding deforestation in human-environment relationships. Each landowner manages a single land parcel where the state of land-use is forested, agricultural, or abandoned. The landowner has two strategies available: forest conservation and deforestation. The choice of deforestation provides a high return to the landowner, but it degrades the forest ecosystem services produced on a neighboring land parcel managed by a different landowner. Given spatial interactions between the two landowners, each landowner decides which strategy to choose by comparing the expected discounted utility of each strategy. Expected discounted utility is determined by taking into account the current and future utilities to be received, according to the state transition on the two land parcels. The state transition is described by a Markov chain that incorporates a landowner's choice about whether to deforest and the dynamics of agricultural abandonment and forest regeneration. By considering a stationary distribution of the Markov chain for land-use transitions, we derive explicit conditions for Nash equilibrium. We found that a slow regeneration of forests favors mutual cooperation (forest conservation). As the forest regenerates faster, mutual cooperation transforms to double Nash equilibria (mutual cooperation and mutual defection), and finally mutual defection (deforestation) leads to a unique Nash equilibrium. Two different types of social dilemma emerge in our deforestation game. The stag-hunt dilemma is most likely to occur under an unsustainable resource supply, where forest regenerates extremely slowly but agricultural abandonment happens quite rapidly. In contrast, the prisoner's dilemma is likely under a persistent or circulating supply of resources, where forest regenerates rapidly and agricultural abandonment occurs slowly or rapidly. These results show how humans and the environment mutually shape the dilemma structure in forest management, implying that solutions to dilemmas depend on environmental properties.     

The effect of dispersal and neighbourhood in games of cooperation

The prisoner's dilemma (PD) and the snowdrift (SD) games are paradigmatic tools to investigate the origin of cooperation. Whereas spatial structure (e.g. nonrandom spatial distribution of strategies) present in the spatially explicit models facilitates the emergence of cooperation in the PD game, recent investigations have suggested that spatial structure can be unfavourable for cooperation in the SD game. The frequency of cooperators in a spatially explicit SD game can be lower than it would be in an infinitely large well-mixed population. However, the source of this effect cannot be identified with certainty as spatially explicit games differ from well-mixed games in two aspects: (i) they introduce spatial correlations, (ii) and limited neighbourhood. Here we extend earlier investigations to identify the source of this effect, and thus accordingly we study a spatially explicit version of the PD and SD games with varying degrees of dispersal and neighbourhood size. It was found that dispersal favours selfish individuals in both games. We calculated the frequency of cooperators at strong dispersal limit, which in concordance with the numerical results shows that it is the short range of interactions (i.e. limited neighbourhood) and not spatial correlations that decreases the frequency of cooperators in spatially explicit models of populations. Our results demonstrate that spatial correlations are always beneficial to cooperators in both the PD and SD games. We explain the opposite effect of dispersal and neighbourhood structure, and discuss the relevance of distinguishing the two effects in general.     

Anti-social punishment can prevent the co-evolution of punishment and cooperation

The evolution of cooperation is one of the great puzzles in evolutionary biology. Punishment has been suggested as one solution to this problem. Here punishment is generally defined as incurring a cost to inflict harm on a wrong-doer. In the presence of punishers, cooperators can gain higher payoffs than non-cooperators. Therefore cooperation may evolve as long as punishment is prevalent in the population. Theoretical models have revealed that spatial structure can favor the co-evolution of punishment and cooperation, by allowing individuals to only play and compete with those in their immediate neighborhood. However, those models have usually assumed that punishment is always targeted at non-cooperators. In light of recent empirical evidence of punishment targeted at cooperators, we relax this assumption and study the effect of so-called ‘anti-social punishment’. We find that evolution can favor anti-social punishment, and that when anti-social punishment is possible costly punishment no longer promotes cooperation. As there is no reason to assume that cooperators cannot be the target of punishment during evolution, our results demonstrate serious restrictions on the ability of costly punishment to allow the evolution of cooperation in spatially structured populations. Our results also help to make sense of the empirical observation that defectors will sometimes pay to punish cooperators.     

Spatial invasion of cooperation

The evolutionary puzzle of cooperation describes situations where cooperators provide a fitness benefit to other individuals at some cost to themselves. Under Darwinian selection, the evolution of cooperation is a conundrum, whereas non-cooperation (or defection) is not. In the absence of supporting mechanisms, cooperators perform poorly and decrease in abundance. Evolutionary game theory provides a powerful mathematical framework to address the problem of cooperation using the prisoner's dilemma. One well-studied possibility to maintain cooperation is to consider structured populations, where each individual interacts only with a limited subset of the population. This enables cooperators to form clusters such that they are more likely to interact with other cooperators instead of being exploited by defectors. Here we present a detailed analysis of how a few cooperators invade and expand in a world of defectors. If the invasion succeeds, the expansion process takes place in two stages: first, cooperators and defectors quickly establish a local equilibrium and then they uniformly expand in space. The second stage provides good estimates for the global equilibrium frequencies of cooperators and defectors. Under hospitable conditions, cooperators typically form a single, ever growing cluster interspersed with specks of defectors, whereas under more hostile conditions, cooperators form isolated, compact clusters that minimize exploitation by defectors. We provide the first quantitative assessment of the way cooperators arrange in space during invasion and find that the macroscopic properties and the emerging spatial patterns reveal information about the characteristics of the underlying microscopic interactions.     

Three-person game facilitates indirect reciprocity under image scoring

Reputation building plays an important role in the evolution of reciprocal altruism when the same individuals do not interact repeatedly because, by referring to reputation, a reciprocator can know which partners are cooperative and can reciprocate with a cooperator. This reciprocity based on reputation is called indirect reciprocity. Previous studies of indirect reciprocity have focused only on two-person games in which only two individuals participate in a single interaction, and have claimed that indirectly reciprocal cooperation cannot be established under image scoring reputation criterion where the reputation of an individual who has cooperated (defected) becomes good (bad). In this study, we specifically examine three-person games, and reveal that indirectly reciprocal cooperation can be formed and maintained stably, even under image scoring, by a nucleus shield mechanism. In the nucleus shield, reciprocators are a shield that keeps out unconditional defectors, whereas unconditional cooperators are the backbone of cooperation that retains a good reputation among the population.     

Cross entropy minimization in uninvadable states of complex populations

Selection is often. viewed as a process that maximizes the average fitness of a population. However, there are often constraints even on the phenotypic level which may prevent fitness optimization. Consequently, in evolutionary game theory, models of frequency dependent selection are investigated, which focus on equilibrium states that are characterized by stability (or uninvadability) rather than by optimality. The aim of this article is to show that nevertheless there is a biologically meaningful quantity, namely cross (fitness) entropy, which is optimized during the course of evolution: a dynamical model adapted to evolutionary games is presented which has the property that relative entropy decreases monotonically, if the state of a (complex) population is close to an uninvadable state. This result may be interpreted as if evolution has an order stabilizing effect.     

The logic of reprobation: assessment and action rules for indirect reciprocation

Ever since image-based models for indirect reciprocity were introduced, the relative merits of scoring vs. standing have been discussed to find out how important it is to differentiate between justified and non-justified defections. This is analogous to the question whether punishment can sustain cooperation even when it is costly. We show that an answer to this question can depend on details of the model, for instance concerning the probability distribution of the number of interactions experienced per player. We use extensive individual-based simulations to compare scoring, standing and other forms of assessing defections, and show that several forms of indirect reciprocation can robustly sustain cooperation. By most standards, standing is better than scoring, but nevertheless scoring is able to sustain cooperation in the presence of errors. The model presented here is based on three specifications: each player has a personal list of images of all co-players, a specific way of judging an observed situation, and a specific strategy to decide whether to cooperate or not.     

Two wrongs don't make a right: the initial viability of different assessment rules in the evolution of indirect reciprocity

Indirect reciprocity models are meant to correspond to simple moral systems, in which individuals assess the interactions of third parties in order to condition their cooperative behavior. Despite the staggering number of possible assessment rules in even the simplest of these models, previous research suggests that only a handful are evolutionarily stable against invasion by free riders. These successful assessment rules fall into two categories, one which positively judges miscreants when they refuse to help other miscreants, the other which does not. Previous research has not, however, demonstrated that all of these rules can invade an asocial population—a requirement for a complete theory of social evolution. Here, I present a general analytical model of indirect reciprocity and show that the class of assessment rules which positively judges a refusal to help scofflaws cannot invade a population of defectors, whereas the other class can. When rare, assessment rules which positively judge a refusal to help bad people produce a poor correlation between reputation and behavior. It is this correlation that generates the assortment crucial in sustaining cooperation through indirect reciprocity. Only assessment rules that require good deeds to achieve a good reputation guarantee a strong correlation between behavior and reputation.     

The robustness of altruism as an evolutionary strategy

Kin selection, reciprocity and group selection are widely regarded as evolutionary mechanisms capable of sustaining altruism among humans andother cooperative species. Our research indicates, however, that these mechanisms are only particular examples of a broader set of evolutionary possibilities.In this paper we present the results of a series of simple replicator simulations, run on variations of the 2–player prisoner's dilemma, designed to illustrate the wide range of scenarios under which altruism proves to be robust under evolutionary pressures. The set of mechanisms we explore is divided into four categories:correlation, group selection, imitation, and punishment. We argue that correlation is the core phenomenon at work in all four categories.     

Evolution of cooperation: cooperation defeats defection in the cornfield model

"Cooperation" defines any behavior that enhances the fitness of a group (e.g. a community or species), but which, by its nature, can be exploited by selfish individuals, meaning, firstly, that selfish individuals derive an advantage from exploitation which is greater than the average advantage that accrues to unselfish individuals. Secondly, exploitation has no intrinsic fitness value except in the presence of the "cooperative behavior". The mathematics is described by the simple Prisoner's Dilemma Game (PDG). It has previously been shown that koinophilia (the avoidance of sexual mates displaying unusual or atypical phenotypic features, such as mutations) stabilizes any inherited strategy in the simple or iterated PDG, meaning that it cannot be displaced by rare forms of alternative behavior which arise through mutation or occasional migration. In the present model equal numbers of cooperators and defectors (in the simple PDG) were randomly spread in a two-dimensional "cornfield" with uniformly distributed resources. Every individual was koinophilic, and interacted (sexually and in the PDG tournaments) only with individuals from within its immediate neighborhood. This model therefore tested whether cooperation can outcompete defection or selfishness in a straight, initially equally matched, evolutionary battle. The results show that in the absence of koinophilia cooperation was rapidly driven to extinction. With koinophilia there was a very rapid loss of cooperators in the first few generations, but thereafter cooperation slowly spread, ultimately eliminating defection completely. This result was critically dependent on sampling effects of neighborhoods. Small samples (resulting from low population densities or small neighborhood sizes) increase the probability that a chance neighborhood comes to consist predominantly of cooperators. A sexual preference for the most common phenotype in the neighborhood then makes that phenotype more common still. Once this occurs cooperation's spread becomes almost inevitable.     

Evolutionary stability of first-order-information indirect reciprocity in sizable groups

Indirect reciprocity is considered as a key mechanism for explaining the evolution of cooperation in situations where the same individuals interact only a few times. Under indirect reciprocity, an individual who helps others gets returns indirectly from others who know her good reputation. Recently, many studies have discussed the effect of reputation criteria based only on the former actions of the others (first-order information) and of those based also on the former reputation of opponents of the others (second-order information) on the evolution of indirect reciprocity. In this study, we investigate the evolutionary stability of the indirectly reciprocal strategy (discriminating strategy: DIS), which cooperates only with opponents who have good reputations, in -person games where more than two individuals take part in a single group (interaction). We show that in n-person games, DIS is an evolutionarily stable strategy (ESS) even under the image-scoring reputation criterion, which is based only on first-order information and where cooperations (defections) are judged to be good (bad). This result is in contrast to that of 2-person games where DIS is not an ESS under reputation criteria based only on first-order information.     

The evolution of payoff matrices: providing incentives to cooperate

Most of the work in evolutionary game theory starts with a model of a social situation that gives rise to a particular payoff matrix and analyses how behaviour evolves through natural selection. Here, we invert this approach and ask, given a model of how individuals behave, how the payoff matrix will evolve through natural selection. In particular, we ask whether a prisoner's dilemma game is stable against invasions by mutant genotypes that alter the payoffs. To answer this question, we develop a two-tiered framework with goal-oriented dynamics at the behavioural time scale and a diploid population genetic model at the evolutionary time scale. Our results are two-fold: first, we show that the prisoner's dilemma is subject to invasions by mutants that provide incentives for cooperation to their partners, and that the resulting game is a coordination game similar to the hawk-dove game. Second, we find that for a large class of mutants and symmetric games, a stable genetic polymorphism will exist in the locus determining the payoff matrix, resulting in a complex pattern of behavioural diversity in the population. Our results highlight the importance of considering the evolution of payoff matrices to understand the evolution of animal social systems.     

Invasion of cooperators in lattice populations: Linear and non-linear public good games

A generalized version of the N-person volunteer's dilemma (NVD) Game has been suggested recently for illustrating the problem of N-person social dilemmas. Using standard replicator dynamics it can be shown that coexistence of cooperators and defectors is typical in this model. However, the question of how a rare mutant cooperator could invade a population of defectors is still open.     

The continuous public goods game and the evolution of cooperative sex ratios

Some animals, such as Melittobia wasps and surface-living mites, have extremely female-biased sex ratios that cannot be explained by the existing local mate competition (LMC) theories. The restricted production of sons may entail cooperation among mothers, enabling the production of more daughters and avoiding severe competition among sons for insemination access. These unusual examples are characterized by the long-term cohabitation of egg-layers (foundresses) on resource patches and possible contact with oviposited eggs. By applying the logic of mutual policing, we develop a novel game theoretical model for the evolution of cooperation in sex-ratio traits. This is the first attempt to model the evolution of sex ratios based on iterated games. We assumed that foundresses have two abilities to enable mutual policing: they can monitor the sex ratio in the resource patch, and they can punish defectors that produce an overabundance of males. Numerical analysis and evolutionary simulations demonstrate that cooperative low sex ratios can evolve when the number of foundresses per patch is sufficiently small. Our model predicts a slight, but steady increase in oviposition sex ratios with an increase in the number of foundresses, which mimics the phenomenon observed in several animals with extremely female-biased sex ratios. We also discuss the relationship between our model and other models of sex-ratio evolution.