Chaos, oscillation and the evolution of indirect reciprocity in n-person games

Evolution of cooperation among genetically unrelated individuals has been of considerable concern in various fields such as biology, economics, and psychology. The evolution of cooperation is often explained by reciprocity. Under reciprocity, cooperation can prevail in a society because a donor of cooperation receives reciprocation from the recipient of the cooperation, called direct reciprocity, or from someone else in the community, called indirect reciprocity. Nowak and Sigmund [1993. Chaos and the evolution of cooperation. Proc. Natl. Acad. Sci. USA 90, 5091-5094] have demonstrated that directly reciprocal cooperation in two-person prisoner's dilemma games with mutation of strategies can be maintained dynamically as periodic or chaotic oscillation. Furthermore, Eriksson and Lindgren [2005. Cooperation driven by mutations in multi-person Prisoner's Dilemma. J. Theor. Biol. 232, 399-409] have reported that directly reciprocal cooperation in n-person prisoner's dilemma games (n>2) can be maintained as periodic oscillation. Is dynamic cooperation observed only in direct reciprocity? Results of this study show that indirectly reciprocal cooperation in n-person prisoner's dilemma games can be maintained dynamically as periodic or chaotic oscillation. This is, to our knowledge, the first demonstration of chaos in indirect reciprocity. Furthermore, the results show that oscillatory dynamics are observed in common in the evolution of reciprocal cooperation whether for direct or indirect.     

The importance of subjectivity in perceptual errors on the emergence of indirect reciprocity

Indirect reciprocity is one mechanism that allows for unilateral resource giving among n-persons. Using analytical methods and computer simulations, previous studies have examined a number of strategies that make indirect reciprocity possible. In particular, previous investigations have concentrated on whether differentiating between justified and unjustified not-giving is important. However, whether or not a given strategy is ESS depends on the type of perceptual errors that are assumed. When errors are objective, regarding those who do not give to "bad" as "good" is critical. When perceptual errors are subjective, however, regarding those who give to "bad" as "bad" is critical. Since we believe that there is no guarantee that perceptual errors are shared among all individuals in a society, we argue that the latter moral principle may play a more important role in human interactions.     

Two wrongs don't make a right: the initial viability of different assessment rules in the evolution of indirect reciprocity

Indirect reciprocity models are meant to correspond to simple moral systems, in which individuals assess the interactions of third parties in order to condition their cooperative behavior. Despite the staggering number of possible assessment rules in even the simplest of these models, previous research suggests that only a handful are evolutionarily stable against invasion by free riders. These successful assessment rules fall into two categories, one which positively judges miscreants when they refuse to help other miscreants, the other which does not. Previous research has not, however, demonstrated that all of these rules can invade an asocial population—a requirement for a complete theory of social evolution. Here, I present a general analytical model of indirect reciprocity and show that the class of assessment rules which positively judges a refusal to help scofflaws cannot invade a population of defectors, whereas the other class can. When rare, assessment rules which positively judge a refusal to help bad people produce a poor correlation between reputation and behavior. It is this correlation that generates the assortment crucial in sustaining cooperation through indirect reciprocity. Only assessment rules that require good deeds to achieve a good reputation guarantee a strong correlation between behavior and reputation.     

Reactive strategies in indirect reciprocity

Evolution of reactive strategy of indirect reciprocity is discussed, where individuals interact with others through the one-shot Prisoner's Dilemma game, changing their partners in every round. We investigate all of the reactive strategies that are stochastic, including deterministic ones as special cases. First we study adaptive dynamics of reactive strategies by assuming monomorphic population. Results are very similar to the corresponding evolutionary dynamics of direct reciprocity. The discriminating strategy, which prescribes cooperation only with those who cooperated in the previous round, cannot be an outcome of the evolution. Next we examine the case where the population includes a diversity of strategies. We find that only the mean 'discriminatoriness' in the population is the parameter that affects the evolutionary dynamics. The discriminating strategy works as a promoter of cooperation there. However, it is again not the end point of the evolution. This is because retaliatory defection, which was prescribed by the discriminating strategy, is regarded as another defection toward the society. These results caution that we have to reconsider the role of retaliatory defection much more carefully.     

A study of indirect reciprocity involving a reputation system or a simple tag system in a one-shot, multi-player game

The possibility of the evolution of cooperation backed by indirect reciprocity (IDR) in a one-shot, multi-player game is investigated focusing on two mechanisms. First of all, the reputation system with image score (RS with IS), as proposed by Nowak and Sigmund [Nowak, M.A., Sigmund, K., 1998. Evolution of indirect reciprocity by image scoring. Nature 393, 573–577], is investigated in various game structures. A numerical experiment demonstrates that the RS with IS is a robust mechanism for the support of IDR in various dilemma games, but whose effectiveness decreases with an increase in the number of players in a game. It is fair to say that the RS is an information mapping function to relate between player's cooperative fraction on his action and IS. As the second mechanism, a simple tag system which could be applicable to animals having no cognitive complexity is considered to support IDR. Computer simulations of the tag system's strategy for invading a population initially consisting of AllD, AllC, and Random strategies suggest several novel facts. The Tag strategy with plausible crossover and mutation probabilities can only invade to settle down if the game structure is not Trivial and contains a moderate dilemma. The Tag strategy's evolutionary competition takes place mainly in the presence of the AllD strategy. During the competition, the Tag strategy frequently metamorphoses to shake off the AllD strategy, but stops after winning in order to avoid a shrinking payoff due to fragmentation.     

Evolution of indirect reciprocity in groups of various sizes and comparison with direct reciprocity

Recently many studies have investigated the evolution of indirect reciprocity through which cooperative action is returned by a third individual, e.g. individual A helped B and then receives help from C. Most studies on indirect reciprocity have presumed that only two individuals take part in a single interaction (group), e.g. A helps B and C helps A. In this paper, we investigate the evolution of indirect reciprocity when more than two individuals take part in a single group, and compare the result with direct reciprocity through which cooperative action is directly returned by the recipient. Our analyses show the following. In the population with discriminating cooperators and unconditional defectors, whether implementation error is included or not, (i) both strategies are evolutionarily stable and the evolution of indirect reciprocity becomes more difficult as group size increases, and (ii) the condition for the evolution of indirect reciprocity under standing reputation criterion where the third individuals distinguish between justified and unjustified defections is more relaxed than that under image scoring reputation criterion in which the third individuals do not distinguish with. Furthermore, in the population that also includes unconditional cooperators, (iii) in the presence of errors in implementation, the discriminating strategy is evolutionarily stable not only under standing but also under image scoring if group size is larger than two. Finally, (iv) in the absence of errors in implementation, the condition for the evolution of direct reciprocity is equivalent to that for the evolution of indirect reciprocity under standing, and, in the presence of errors, the condition for the evolution of direct reciprocity is very close to that for the evolution of indirect reciprocity under image scoring.     

The good, the bad and the discriminator--errors in direct and indirect reciprocity

This paper presents, in a series of simple diagrams, concise results about the replicator dynamics of direct and indirect reciprocity. We consider repeated interactions between donors and recipients, and analyse the relationship between three basic strategies for the donor: unconditional cooperation, all-out defection, and conditional cooperation. In other words, we investigate the competition of discriminating and indiscriminating altruists with defectors. Discriminators and defectors form a bistable community, and hence a population of discriminators cannot be invaded by defectors. But unconditional altruists can invade a discriminating population and 'soften it up' for a subsequent invasion by defectors. The resulting dynamics exhibits various forms of rock-paper-scissors cycles and depends in subtle ways on noise, in the form of errors in implementation. The probability for another round (in the case of direct reciprocity), and information about the co-player (in the case of indirect reciprocity), add further elements to the ecology of reciprocation.     

The leading eight: social norms that can maintain cooperation by indirect reciprocity

The theory of indirect reciprocation explains the evolution of cooperation among unrelated individuals, engaging in one-shot interaction. Using reputation, a player acquires information on who are worth cooperating and who are not. In a previous paper, we formalized the reputation dynamics, a rule to assign a binary reputation (good or bad) to each player when his action, his current reputation, and the opponent's reputation are given. We then examined all the possible reputation dynamics, and found that there exist only eight reputation dynamics named "leading eight" that can maintain the ESS with a high level of cooperation, even if errors are included in executing intended cooperation and in reporting the observation to the public. In this paper, we study the nature of these successful social norms. First, we characterize the role of each pivot of the reputation dynamics common to all of the leading eight. We conclude that keys to the success in indirect reciprocity are to be nice (maintenance of cooperation among themselves), retaliatory (detection of defectors, punishment, and justification of punishment), apologetic, and forgiving. Second, we prove the two basic properties of the leading eight, which give a quantitative evaluation of the ESS condition and the level of cooperation maintained at the ESS.     

Upstream reciprocity and the evolution of gratitude

If someone is nice to you, you feel good and may be inclined to be nice to somebody else. This every day experience is borne out by experimental games: the recipients of an act of kindness are more likely to help in turn, even if the person who benefits from their generosity is somebody else. This behaviour, which has been called ‘upstream reciprocity’, appears to be a misdirected act of gratitude: you help somebody because somebody else has helped you. Does this make any sense from an evolutionary or a game theoretic perspective? In this paper, we show that upstream reciprocity alone does not lead to the evolution of cooperation, but it can evolve and increase the level of cooperation if it is linked to either direct or spatial reciprocity. We calculate the random walks of altruistic acts that are induced by upstream reciprocity. Our analysis shows that gratitude and other positive emotions, which increase the willingness to help others, can evolve in the competitive world of natural selection.     

The logic of reprobation: assessment and action rules for indirect reciprocation

Ever since image-based models for indirect reciprocity were introduced, the relative merits of scoring vs. standing have been discussed to find out how important it is to differentiate between justified and non-justified defections. This is analogous to the question whether punishment can sustain cooperation even when it is costly. We show that an answer to this question can depend on details of the model, for instance concerning the probability distribution of the number of interactions experienced per player. We use extensive individual-based simulations to compare scoring, standing and other forms of assessing defections, and show that several forms of indirect reciprocation can robustly sustain cooperation. By most standards, standing is better than scoring, but nevertheless scoring is able to sustain cooperation in the presence of errors. The model presented here is based on three specifications: each player has a personal list of images of all co-players, a specific way of judging an observed situation, and a specific strategy to decide whether to cooperate or not.     

Evolution of indirect reciprocity by social information: the role of trust and reputation in evolution of altruism

The complexity of human's cooperative behavior cannot be fully explained by theories of kin selection and group selection. If reciprocal altruism is to provide an explanation for altruistic behavior, it would have to depart from direct reciprocity, which requires dyads of individuals to interact repeatedly. For indirect reciprocity to rationalize cooperation among genetically unrelated or even culturally dissimilar individuals, information about the reputation of individuals must be assessed and propagated in a population. Here, we propose a new framework for the evolution of indirect reciprocity by social information: information selectively retrieved from and propagated through dynamically evolving networks of friends and acquaintances. We show that for indirect reciprocity to be evolutionarily stable, the differential probability of trusting and helping a reputable individual over a disreputable individual, at a point in time, must exceed the cost-to-benefit ratio of the altruistic act. In other words, the benefit received by the trustworthy must out-weigh the cost of helping the untrustworthy.     

The evolution of strong reciprocity: cooperation in heterogeneous populations

How do human groups maintain a high level of cooperation despite a low level of genetic relatedness among group members? We suggest that many humans have a predisposition to punish those who violate group-beneficial norms, even when this imposes a fitness cost on the punisher. Such altruistic punishment is widely observed to sustain high levels of cooperation in behavioral experiments and in natural settings. We offer a model of cooperation and punishment that we call STRONG RECIPROCITY: where members of a group benefit from mutual adherence to a social norm, strong reciprocators obey the norm and punish its violators, even though as a result they receive lower payoffs than other group members, such as selfish agents who violate the norm and do not punish, and pure cooperators who adhere to the norm but free-ride by never punishing. Our agent-based simulations show that, under assumptions approximating likely human environments over the 100000 years prior to the domestication of animals and plants, the proliferation of strong reciprocators when initially rare is highly likely, and that substantial frequencies of all three behavioral types can be sustained in a population. As a result, high levels of cooperation are sustained. Our results do not require that group members be related or that group extinctions occur.     

Global analyses of evolutionary dynamics and exhaustive search for social norms that maintain cooperation by reputation

Reputation formation is a key to understanding indirect reciprocity. In particular, the way to assign reputation to each individual, namely a norm that describes who is good and who is bad, greatly affects the possibility of sustained cooperation in the population. Previously, we have exhaustively studied reputation dynamics that are able to maintain a high level of cooperation at the ESS. However, this analysis examined the stability of monomorphic population and did not investigate polymorphic population where several strategies coexist. Here, we study the evolutionary dynamics of multiple behavioral strategies by replicator dynamics. We exhaustively study all 16 possible norms under which the reputation of a player in the next round is determined by the action of the self and the reputation of the opponent. For each norm, we explore evolutionary dynamics of three strategies: unconditional cooperators, unconditional defectors, and conditional cooperators. We find that only three norms, simple-standing, Kandori, and shunning, can make conditional cooperation evolutionarily stable, hence, realize sustained cooperation. The other 13 norms, including scoring, ultimately lead to the invasion by defectors. Also, we study the model in which private reputation errors exist to a small extent. In this case, we find the stable coexistence of unconditional and conditional cooperators under the three norms.     

Effects of information and group structure on evolution of altruism: analysis of two-score model by covariance and contextual analyses

An altruistic individual has to gamble on cooperation to a stranger because it does not know whether the stranger is trustworthy before direct interaction. Nowak and Sigmund (Nature 393 (1998a) 573; J. Theor. Biol. 194 (1998b) 561) presented a new theoretical framework of indirect reciprocal altruism by image scoring game where all individuals are informed about a partner's behavior from its image score without direct interaction. Interestingly, in a simplified version of the image scoring game, the evolutionarily stability condition for altruism became a similar form of Hamilton's rule, i.e. inequality that the probability of getting correct information is more than the ratio of cost to benefit. Since the Hamilton's rule was derived by evolutionarily stable analysis, the evolutionary meaning of the probability of getting correct information has not been clearly examined in terms of kin and group selection. In this study, we applied covariance analysis to the two-score model for deriving the Hamilton's rule. We confirmed that the probability of getting correct information was proportional to the bias of altruistic interactions caused by using information about a partner's image score. The Hamilton's rule was dependent on the number of game bouts even though the information reduced the risk of cooperation to selfish one at the first encounter. In addition, we incorporated group structure to the two-score model to examine whether the probability of getting correct information affect selection for altruism by group selection. We calculated a Hamilton's rule of group selection by contextual analysis. Group selection is very effective when either the probability of getting correct information or that of future interaction, or both are low. The two Hamilton's rules derived by covariance and contextual analyses demonstrated the effects of information and group structure on the evolution of altruism. We inferred that information about a partner's behavior and group structure can produce flexible pathways for the evolution of altruism.     

Participation costs can suppress the evolution of upstream reciprocity

Indirect reciprocity, one of the many mechanisms proposed to explain the evolution of cooperation, is the idea that altruistic actions can be rewarded by third parties. Upstream or generalized reciprocity is one type of indirect reciprocity in which individuals help someone if they have been helped by somebody else in the past. Although empirically found to be at work in humans, the evolution of upstream reciprocity is difficult to explain from a theoretical point of view. A recent model of upstream reciprocity, first proposed by Nowak and Roch (2007) and further analyzed by Iwagami and Masuda (2010), shows that while upstream reciprocity alone does not lead to the evolution of cooperation, it can act in tandem with mechanisms such as network reciprocity and increase the total level of cooperativity in the population. We argue, however, that Nowak and Roch's model systematically leads to non-uniform interaction rates, where more cooperative individuals take part in more games than less cooperative ones. As a result, the critical benefit-to-cost ratios derived under this model in previous studies are not invariant with respect to the addition of participation costs. We show that accounting for these costs can hinder and even suppress the evolution of upstream reciprocity, both for populations with non-random encounters and graph-structured populations.     

Game theory and human evolution: a critique of some recent interpretations of experimental games

Economists and psychologists have been testing Nash equilibrium predictions of game theory models of human behavior. In many instances, humans do not conform to the predictions. These results are of great interest to biologists because they also raise questions about well-known ESS models of cooperation. Cooperation in certain one-shot, anonymous interactions, and a willingness to punish others at a net cost to oneself are some of the most intriguing deviations from standard theory. One proposed explanation for these results that is receiving increasing attention invokes the cultural group selection of 'other regarding' social norms. We critically review this explanation. We conclude that experimental results reveal limits in two implicit models of cognitive structure commonly employed by economists and evolutionary biologists.     

Cooperating in the face of uncertainty: a consistent framework for understanding the evolution of cooperation

The evolution of cooperative behaviour, whereby individuals enhance the fitness of others at an apparent cost to themselves, represents one of the greatest paradoxes of evolution. Individuals that engage in such cooperative behaviour can, however, be favoured by natural selection if cooperative actions confer higher fitness than alternative actions. To understand the evolution of cooperative behaviour, the direct and indirect genetic benefits that individuals accrue in the present and future must be summed - this can be accomplished without any reference to the colorful vocabulary typically associated with studies of cooperation. When benefits are accrued indirectly through relatives or directly in the future individuals must be able to assess and enhance their probability of accruing those benefits and behave accordingly. We suggest that, in the same way that studies of kin recognition systems improved our understanding of how individuals assess and enhance their probability of accruing indirect benefits, studies of various forms of inheritance and reciprocation recognition systems will improve our understanding of how individuals assess and enhance their probability of accruing future benefits. Recognizing the parallel between studies of indirect fitness and future fitness, at multiple levels of analysis, will move us toward a simpler and more consistent framework for understanding the evolution of cooperative behaviour.     

Evolutionary stability of first-order-information indirect reciprocity in sizable groups

Indirect reciprocity is considered as a key mechanism for explaining the evolution of cooperation in situations where the same individuals interact only a few times. Under indirect reciprocity, an individual who helps others gets returns indirectly from others who know her good reputation. Recently, many studies have discussed the effect of reputation criteria based only on the former actions of the others (first-order information) and of those based also on the former reputation of opponents of the others (second-order information) on the evolution of indirect reciprocity. In this study, we investigate the evolutionary stability of the indirectly reciprocal strategy (discriminating strategy: DIS), which cooperates only with opponents who have good reputations, in -person games where more than two individuals take part in a single group (interaction). We show that in n-person games, DIS is an evolutionarily stable strategy (ESS) even under the image-scoring reputation criterion, which is based only on first-order information and where cooperations (defections) are judged to be good (bad). This result is in contrast to that of 2-person games where DIS is not an ESS under reputation criteria based only on first-order information.     

Three-person game facilitates indirect reciprocity under image scoring

Reputation building plays an important role in the evolution of reciprocal altruism when the same individuals do not interact repeatedly because, by referring to reputation, a reciprocator can know which partners are cooperative and can reciprocate with a cooperator. This reciprocity based on reputation is called indirect reciprocity. Previous studies of indirect reciprocity have focused only on two-person games in which only two individuals participate in a single interaction, and have claimed that indirectly reciprocal cooperation cannot be established under image scoring reputation criterion where the reputation of an individual who has cooperated (defected) becomes good (bad). In this study, we specifically examine three-person games, and reveal that indirectly reciprocal cooperation can be formed and maintained stably, even under image scoring, by a nucleus shield mechanism. In the nucleus shield, reciprocators are a shield that keeps out unconditional defectors, whereas unconditional cooperators are the backbone of cooperation that retains a good reputation among the population.