Moving away from nasty encounters enhances cooperation in ecological prisoner's dilemma game

We study the role of migration in the evolution of cooperation. Individuals spatially located on a square lattice play the prisoner's dilemma game. Dissatisfied players, who have been exploited by defectors, tend to terminate interaction with selfish partners by leaving the current habitats, and explore unknown physical niches available surrounding them. The time scale ratio of game interaction to natural selection governs how many game rounds occur before individuals experience strategy updating. Under local migration and strong selection, simulation results demonstrate that cooperation can be stabilized for a wide range of model parameters, and the slower the natural selection, the more favorable for the emergence of cooperation. Besides, how the selection intensity affects cooperators' evolutionary fate is also investigated. We find that increasing it weakens cooperators' viability at different speeds for different time scale ratios. However, cooperation is greatly improved provided that individuals are offered with enough chance to agglomerate, while cooperation can always establish under weak selection but vanishes under very strong selection whenever individuals have less odds to migrate. Whenever the migration range restriction is removed, the parameter area responsible for the emergence of cooperation is, albeit somewhat compressed, still remarkable, validating the effectiveness of collectively migrating in promoting cooperation.     

A new route to the evolution of cooperation

The Prisoner's Dilemma (PD) constitutes a widely used metaphor to investigate problems related to the evolution of cooperation. Whenever evolution takes place in well-mixed populations engaged in single rounds of the PD, cooperators cannot resist invasion by defectors, a feature, which is somewhat alleviated whenever populations are spatially distributed. In both cases the populations are characterized by a homogeneous pattern of connectivity, in which every individual is equivalent, sharing the same number of neighbours. Recently, compelling evidence has been accumulated on the strong heterogeneous nature of the network of contacts between individuals in populations. Here we describe the networks of contacts in terms of graphs and show that heterogeneity provides a new mechanism for cooperation to survive. Specifically, we show that cooperators are capable of exploring the heterogeneity of the population structure to become evolutionary competitive. As a result, cooperation becomes the dominating trait in scale-free networks of contacts in which the few highly connected individuals are directly inter-connected, in this way contributing to self-sustain cooperation.     

Inferring reputation promotes the evolution of cooperation in spatial social dilemma games

In realistic world individuals with high reputation are more likely to influence the collective behaviors. Due to the cost and error of information dissemination, however, it is unreasonable to assign each individual with a complete cognitive power, which means that not everyone can accurately realize others' reputation situation. Here we introduce the mechanism of inferring reputation into the selection of potential strategy sources to explore the evolution of cooperation. Before the game each player is assigned with a randomly distributed parameter p denoting his ability to infer the reputation of others. The parameter p of each individual is kept constant during the game. The value of p indicates that the neighbor possessing highest reputation is chosen with the probability p and randomly choosing an opponent is left with the probability 1-p. We find that this novel mechanism can be seen as an universally applicable promoter of cooperation, which works on various interaction networks and in different types of evolutionary game. Of particular interest is the fact that, in the early stages of evolutionary process, cooperators with high reputation who are easily regarded as the potential strategy donors can quickly lead to the formation of extremely robust clusters of cooperators that are impervious to defector attacks. These clusters eventually help cooperators reach their undisputed dominance, which transcends what can be warranted by the spatial reciprocity alone. Moreover, we provide complete phase diagrams to depict the impact of uncertainty in strategy adoptions and conclude that the effective interaction topology structure may be altered under such a mechanism. When the estimation of reputation is extended, we also show that the moderate value of evaluation factor enables cooperation to thrive best. We thus present a viable method of understanding the ubiquitous cooperative behaviors in nature and hope that it will inspire further studies to resolve social dilemmas.     

Global analyses of evolutionary dynamics and exhaustive search for social norms that maintain cooperation by reputation

Reputation formation is a key to understanding indirect reciprocity. In particular, the way to assign reputation to each individual, namely a norm that describes who is good and who is bad, greatly affects the possibility of sustained cooperation in the population. Previously, we have exhaustively studied reputation dynamics that are able to maintain a high level of cooperation at the ESS. However, this analysis examined the stability of monomorphic population and did not investigate polymorphic population where several strategies coexist. Here, we study the evolutionary dynamics of multiple behavioral strategies by replicator dynamics. We exhaustively study all 16 possible norms under which the reputation of a player in the next round is determined by the action of the self and the reputation of the opponent. For each norm, we explore evolutionary dynamics of three strategies: unconditional cooperators, unconditional defectors, and conditional cooperators. We find that only three norms, simple-standing, Kandori, and shunning, can make conditional cooperation evolutionarily stable, hence, realize sustained cooperation. The other 13 norms, including scoring, ultimately lead to the invasion by defectors. Also, we study the model in which private reputation errors exist to a small extent. In this case, we find the stable coexistence of unconditional and conditional cooperators under the three norms.     

High strength-of-ties and low mobility enable the evolution of third-party punishment

As punishment can be essential to cooperation and norm maintenance but costly to the punisher, many evolutionary game-theoretic studies have explored how direct punishment can evolve in populations. Compared to direct punishment, in which an agent acts to punish another for an interaction in which both parties were involved, the evolution of third-party punishment (3PP) is even more puzzling, because the punishing agent itself was not involved in the original interaction. Despite significant empirical studies of 3PP, little is known about the conditions under which it can evolve. We find that punishment reputation is not, by itself, sufficient for the evolution of 3PP. Drawing on research streams in sociology and psychology, we implement a structured population model and show that high strength-of-ties and low mobility are critical for the evolution of responsible 3PP. Only in such settings of high social-structural constraint are punishers able to induce self-interested agents toward cooperation, making responsible 3PP ultimately beneficial to individuals as well as the collective. Our results illuminate the conditions under which 3PP is evolutionarily adaptive in populations. Responsible 3PP can evolve and induce cooperation in cases where other mechanisms alone fail to do so.     

Three-person game facilitates indirect reciprocity under image scoring

Reputation building plays an important role in the evolution of reciprocal altruism when the same individuals do not interact repeatedly because, by referring to reputation, a reciprocator can know which partners are cooperative and can reciprocate with a cooperator. This reciprocity based on reputation is called indirect reciprocity. Previous studies of indirect reciprocity have focused only on two-person games in which only two individuals participate in a single interaction, and have claimed that indirectly reciprocal cooperation cannot be established under image scoring reputation criterion where the reputation of an individual who has cooperated (defected) becomes good (bad). In this study, we specifically examine three-person games, and reveal that indirectly reciprocal cooperation can be formed and maintained stably, even under image scoring, by a nucleus shield mechanism. In the nucleus shield, reciprocators are a shield that keeps out unconditional defectors, whereas unconditional cooperators are the backbone of cooperation that retains a good reputation among the population.     

Direct reciprocity on graphs

Direct reciprocity is a mechanism for the evolution of cooperation based on the idea of repeated encounters between the same two individuals. Here we examine direct reciprocity in structured populations, where individuals occupy the vertices of a graph. The edges denote who interacts with whom. The graph represents spatial structure or a social network. For birth-death or pairwise comparison updating, we find that evolutionary stability of direct reciprocity is more restrictive on a graph than in a well-mixed population, but the condition for reciprocators to be advantageous is less restrictive on a graph. For death-birth and imitation updating, in contrast, both conditions are easier to fulfill on a graph. Moreover, for all four update mechanisms, reciprocators can dominate defectors on a graph, which is never possible in a well-mixed population. We also study the effect of an error rate, which increases with the number of links per individual; interacting with more people simultaneously enhances the probability of making mistakes. We provide analytic derivations for all results.     

Spatial invasion of cooperation

The evolutionary puzzle of cooperation describes situations where cooperators provide a fitness benefit to other individuals at some cost to themselves. Under Darwinian selection, the evolution of cooperation is a conundrum, whereas non-cooperation (or defection) is not. In the absence of supporting mechanisms, cooperators perform poorly and decrease in abundance. Evolutionary game theory provides a powerful mathematical framework to address the problem of cooperation using the prisoner's dilemma. One well-studied possibility to maintain cooperation is to consider structured populations, where each individual interacts only with a limited subset of the population. This enables cooperators to form clusters such that they are more likely to interact with other cooperators instead of being exploited by defectors. Here we present a detailed analysis of how a few cooperators invade and expand in a world of defectors. If the invasion succeeds, the expansion process takes place in two stages: first, cooperators and defectors quickly establish a local equilibrium and then they uniformly expand in space. The second stage provides good estimates for the global equilibrium frequencies of cooperators and defectors. Under hospitable conditions, cooperators typically form a single, ever growing cluster interspersed with specks of defectors, whereas under more hostile conditions, cooperators form isolated, compact clusters that minimize exploitation by defectors. We provide the first quantitative assessment of the way cooperators arrange in space during invasion and find that the macroscopic properties and the emerging spatial patterns reveal information about the characteristics of the underlying microscopic interactions.     

Evolutionary dynamics in structured populations

Evolutionary dynamics shape the living world around us. At the centre of every evolutionary process is a population of reproducing individuals. The structure of that population affects evolutionary dynamics. The individuals can be molecules, cells, viruses, multicellular organisms or humans. Whenever the fitness of individuals depends on the relative abundance of phenotypes in the population, we are in the realm of evolutionary game theory. Evolutionary game theory is a general approach that can describe the competition of species in an ecosystem, the interaction between hosts and parasites, between viruses and cells, and also the spread of ideas and behaviours in the human population. In this perspective, we review the recent advances in evolutionary game dynamics with a particular emphasis on stochastic approaches in finite sized and structured populations. We give simple, fundamental laws that determine how natural selection chooses between competing strategies. We study the well-mixed population, evolutionary graph theory, games in phenotype space and evolutionary set theory. We apply these results to the evolution of cooperation. The mechanism that leads to the evolution of cooperation in these settings could be called ‘spatial selection’: cooperators prevail against defectors by clustering in physical or other spaces.     

Evolutionary causes and consequences of consistent individual variation in cooperative behaviour

Behaviour is typically regarded as among the most flexible of animal phenotypic traits. In particular, expression of cooperative behaviour is often assumed to be conditional upon the behaviours of others. This flexibility is a key component of many hypothesized mechanisms favouring the evolution of cooperative behaviour. However, evidence shows that cooperative behaviours are often less flexible than expected and that, in many species, individuals show consistent differences in the amount and type of cooperative and non-cooperative behaviours displayed. This phenomenon is known as ‘animal personality’ or a ‘behavioural syndrome’. Animal personality is evolutionarily relevant, as it typically shows heritable variation and can entail fitness consequences, and hence, is subject to evolutionary change. Here, we review the empirical evidence for individual variation in cooperative behaviour across taxa, we examine the evolutionary processes that have been invoked to explain the existence of individual variation in cooperative behaviour and we discuss the consequences of consistent individual differences on the evolutionary stability of cooperation. We highlight that consistent individual variation in cooperativeness can both stabilize or disrupt cooperation in populations. We conclude that recognizing the existence of consistent individual differences in cooperativeness is essential for an understanding of the evolution and prevalence of cooperation.     

Cooperation under indirect reciprocity and imitative trust

Indirect reciprocity, a key concept in behavioral experiments and evolutionary game theory, provides a mechanism that allows reciprocal altruism to emerge in a population of self-regarding individuals even when repeated interactions between pairs of actors are unlikely. Recent empirical evidence show that humans typically follow complex assessment strategies involving both reciprocity and social imitation when making cooperative decisions. However, currently, we have no systematic understanding of how imitation, a mechanism that may also generate negative effects via a process of cumulative advantage, affects cooperation when repeated interactions are unlikely or information about a recipient's reputation is unavailable. Here we extend existing evolutionary models, which use an image score for reputation to track how individuals cooperate by contributing resources, by introducing a new imitative-trust score, which tracks whether actors have been the recipients of cooperation in the past. We show that imitative trust can co-exist with indirect reciprocity mechanisms up to a threshold and then cooperation reverses -revealing the elusive nature of cooperation. Moreover, we find that when information about a recipient's reputation is limited, trusting the action of third parties towards her (i.e. imitating) does favor a higher collective cooperation compared to random-trusting and share-alike mechanisms. We believe these results shed new light on the factors favoring social imitation as an adaptive mechanism in populations of cooperating social actors.     

Stochastic sampling of interaction partners versus deterministic payoff assignment

Evolutionary game dynamics describes how successful strategies spread in a population. In well-mixed populations, the usual assumption, e.g. underlying the replicator dynamics, is that individuals obtain a payoff from interactions with a representative sample of the population. This determines their fitness. Here, we analyze a situation in which payoffs are obtained through a single interaction, so that individuals of the same type can have different payoffs. We show analytically that for weak selection, this scenario is identical to the usual approach in which an individual interacts with the whole population. For strong selection, however, differences arise that are reflected in the fixation probabilities and lead to deviating evolutionary dynamics.     

EVOLUTION OF HELPING AND HARMING IN HETEROGENEOUS GROUPS

Social groups are often composed of individuals who differ in many respects. Theoretical studies on the evolution of helping and harming behaviors have largely focused upon genetic differences between individuals. However, nongenetic variation between group members is widespread in natural populations, and may mediate differences in individuals’ social behavior. Here, we develop a framework to study how variation in individual quality mediates the evolution of unconditional and conditional social traits. We investigate the scope for the evolution of social traits that are conditional on the quality of the actor and/or recipients. We find that asymmetries in individual quality can lead to the evolution of plastic traits with different individuals expressing helping and harming traits within the same group. In this context, population viscosity can mediate the evolution of social traits, and local competition can promote both helping and harming behaviors. Furthermore, asymmetries in individual quality can lead to the evolution of competition‐like traits between clonal individuals. Overall, we highlight the importance of asymmetries in individual quality, including differences in reproductive value and the ability to engage in successful social interactions, in mediating the evolution of helping and harming behaviors.     

Reinforcement Learning Explains Conditional Cooperation and Its Moody Cousin

Direct reciprocity, or repeated interaction, is a main mechanism to sustain cooperation under social dilemmas involving two individuals. For larger groups and networks, which are probably more relevant to understanding and engineering our society, experiments employing repeated multiplayer social dilemma games have suggested that humans often show conditional cooperation behavior and its moody variant. Mechanisms underlying these behaviors largely remain unclear. Here we provide a proximate account for this behavior by showing that individuals adopting a type of reinforcement learning, called aspiration learning, phenomenologically behave as conditional cooperator. By definition, individuals are satisfied if and only if the obtained payoff is larger than a fixed aspiration level. They reinforce actions that have resulted in satisfactory outcomes and anti-reinforce those yielding unsatisfactory outcomes. The results obtained in the present study are general in that they explain extant experimental results obtained for both so-called moody and non-moody conditional cooperation, prisoner’s dilemma and public goods games, and well-mixed groups and networks. Different from the previous theory, individuals are assumed to have no access to information about what other individuals are doing such that they cannot explicitly use conditional cooperation rules. In this sense, myopic aspiration learning in which the unconditional propensity of cooperation is modulated in every discrete time step explains conditional behavior of humans. Aspiration learners showing (moody) conditional cooperation obeyed a noisy GRIM-like strategy. This is different from the Pavlov, a reinforcement learning strategy promoting mutual cooperation in two-player situations.     

The cancellation effect at the group level

Group selection models combine selection pressure at the individual level with selection pressure at the group level. Cooperation can be costly for individuals, but beneficial for the group, and therefore, if individuals are sufficiently much assorted, and cooperators find themselves in groups with disproportionately many other cooperators, cooperation can evolve. The existing literature on group selection generally assumes that competition between groups takes place in a well-mixed population of groups, where any group competes with any other group equally intensely. Competition between groups however might very well occur locally; groups may compete more intensely with nearby than with far-away groups. We show that if competition between groups is indeed local, then the evolution of cooperation can be hindered significantly by the fact that groups with many cooperators will mostly compete against neighboring groups that are also highly cooperative, and therefore harder to outcompete. The existing empirical method for determining how conducive a group structured population is to the evolution of cooperation also implicitly assumes global between-group competition, and therefore gives (possibly very) biased estimates.     

Cooperation can promote rescue or lead to evolutionary suicide during environmental change

The adaptation of populations to changing conditions may be affected by interactions between individuals. For example, when cooperative interactions increase fecundity, they may allow populations to maintain high densities and thus keep track of moving environmental optima. Simultaneously, changes in population density alter the marginal benefits of cooperative investments, creating a feedback loop between population dynamics and the evolution of cooperation. Here we model how the evolution of cooperation interacts with adaptation to changing environments. We hypothesize that environmental change lowers population size and thus promotes the evolution of cooperation, and that this, in turn, helps the population keep up with the moving optimum. However, we find that the evolution of cooperation can have qualitatively different effects, depending on which fitness component is reduced by the costs of cooperation. If the costs decrease fecundity, cooperation indeed speeds adaptation by increasing population density; if, in contrast, the costs decrease viability, cooperation may instead slow adaptation by lowering the effective population size, leading to evolutionary suicide. Thus, cooperation can either promote or—counterintuitively—hinder adaptation to a changing environment. Finally, we show that our model can also be generalized to other social interactions by discussing the evolution of competition during environmental change.     

Adaptive evolution of cooperation through Darwinian dynamics in Public Goods games

The linear or threshold Public Goods game (PGG) is extensively accepted as a paradigmatic model to approach the evolution of cooperation in social dilemmas. Here we explore the significant effect of nonlinearity of the structures of public goods on the evolution of cooperation within the well-mixed population by adopting Darwinian dynamics, which simultaneously consider the evolution of populations and strategies on a continuous adaptive landscape, and extend the concept of evolutionarily stable strategy (ESS) as a coalition of strategies that is both convergent-stable and resistant to invasion. Results show (i) that in the linear PGG contributing nothing is an ESS, which contradicts experimental data, (ii) that in the threshold PGG contributing the threshold value is a fragile ESS, which cannot resist the invasion of contributing nothing, and (iii) that there exists a robust ESS of contributing more than half in the sigmoid PGG if the return rate is relatively high. This work reveals the significant effect of the nonlinearity of the structures of public goods on the evolution of cooperation, and suggests that, compared with the linear or threshold PGG, the sigmoid PGG might be a more proper model for the evolution of cooperation within the well-mixed population.     

Interaction characteristics as evolutionary features for the spatial Prisoner’s Dilemma in a population modeled by continuous probabilistic cellular automata and evolutionary algorithm

Evolutionary games usually take into consideration individuals’ strategies as the transformative characteristic which leads to the evolution of the population. Here, besides the strategies, interaction aspects are also considered as evolutionary attributes which can change over time as the replacement dynamic renovates the population choosing locally better individuals to reproduce. The population is modeled by cellular automata, interactions by the Prisoner’s Dilemma game and the replacement process is ruled by two versions of death-birth dynamic. Although the average payoff per game is considered as the fitness for choosing better individuals, the number of games per time step and a maximum radius of interaction with neighbours are also present in the individual’s chromosome which is passed to the next generation. Numerical simulations show that individual interaction properties and cooperation level are linked to the version of death-birth dynamic used and the game payoff. For instance, when the fitness bias is on the death event, individuals have more interactions in a larger radius, and the cooperation level is usually lower than the case where the fitness bias is on the birth event. Also, the individuals’ interaction profiles are heterogeneous, and cooperative individuals form clusters in the lattice to protect themselves.     

Social Norms of Cooperation in Small-Scale Societies

Indirect reciprocity, besides providing a convenient framework to address the evolution of moral systems, offers a simple and plausible explanation for the prevalence of cooperation among unrelated individuals. By helping someone, an individual may increase her/his reputation, which may change the pre-disposition of others to help her/him in the future. This, however, depends on what is reckoned as a good or a bad action, i.e., on the adopted social norm responsible for raising or damaging a reputation. In particular, it remains an open question which social norms are able to foster cooperation in small-scale societies, while enduring the wide plethora of stochastic affects inherent to finite populations. Here we address this problem by studying the stochastic dynamics of cooperation under distinct social norms, showing that the leading norms capable of promoting cooperation depend on the community size. However, only a single norm systematically leads to the highest cooperative standards in small communities. That simple norm dictates that only whoever cooperates with good individuals, and defects against bad ones, deserves a good reputation, a pattern that proves robust to errors, mutations and variations in the intensity of selection.     

The option to leave: Conditional dissociation in the evolution of cooperation

Conditional dissociation, i.e. the option to leave an interacting partner in response to his behaviour, is a mechanism that has been shown to promote cooperation in several settings, but the fundamental features that make conditional dissociation work in this way are not yet fully understood. This paper identifies some of the key conditions that make conditional dissociation lead to high levels of cooperation, explains how this mechanism can support the evolutionary coexistence of cooperative and non-cooperative behaviour typically observed in nature, and provides an analytical formula to estimate the expected degree of cooperation thus achieved. Our model involves a population of individuals who are paired to play an iterated prisoner’s dilemma. All individuals share the same capacity to react to the action previously chosen by their partner and, without any other a priori constraint or exclusion, they may use any behavioural rule that is compatible with this capacity. The dynamic evolution of the population eventually enters either a non-cooperative or a partially cooperative regime, depending mainly on the expected lifetime of individuals. Whenever the partially cooperative regime materializes, the cornerstone of its long-run stability is the coexistence of defectors and “Out-for-Tat” strategists, the latter being those who start cooperating and respond to defection by merely leaving. We find, therefore, that conditional dissociation is the essential disciplinary device supporting cooperation, whilst other conditional strategies (such as Tit-for-Tat) remain present only in small population shares. These conclusions are obtained both by extensive numerical simulations and through analytical mean-field methods that approximate the stochastic simulation dynamics and deliver accurate predictions for general parameter configurations.