Evolutionary stability of first-order-information indirect reciprocity in sizable groups

Indirect reciprocity is considered as a key mechanism for explaining the evolution of cooperation in situations where the same individuals interact only a few times. Under indirect reciprocity, an individual who helps others gets returns indirectly from others who know her good reputation. Recently, many studies have discussed the effect of reputation criteria based only on the former actions of the others (first-order information) and of those based also on the former reputation of opponents of the others (second-order information) on the evolution of indirect reciprocity. In this study, we investigate the evolutionary stability of the indirectly reciprocal strategy (discriminating strategy: DIS), which cooperates only with opponents who have good reputations, in -person games where more than two individuals take part in a single group (interaction). We show that in n-person games, DIS is an evolutionarily stable strategy (ESS) even under the image-scoring reputation criterion, which is based only on first-order information and where cooperations (defections) are judged to be good (bad). This result is in contrast to that of 2-person games where DIS is not an ESS under reputation criteria based only on first-order information.     

Three-person game facilitates indirect reciprocity under image scoring

Reputation building plays an important role in the evolution of reciprocal altruism when the same individuals do not interact repeatedly because, by referring to reputation, a reciprocator can know which partners are cooperative and can reciprocate with a cooperator. This reciprocity based on reputation is called indirect reciprocity. Previous studies of indirect reciprocity have focused only on two-person games in which only two individuals participate in a single interaction, and have claimed that indirectly reciprocal cooperation cannot be established under image scoring reputation criterion where the reputation of an individual who has cooperated (defected) becomes good (bad). In this study, we specifically examine three-person games, and reveal that indirectly reciprocal cooperation can be formed and maintained stably, even under image scoring, by a nucleus shield mechanism. In the nucleus shield, reciprocators are a shield that keeps out unconditional defectors, whereas unconditional cooperators are the backbone of cooperation that retains a good reputation among the population.     

Global analyses of evolutionary dynamics and exhaustive search for social norms that maintain cooperation by reputation

Reputation formation is a key to understanding indirect reciprocity. In particular, the way to assign reputation to each individual, namely a norm that describes who is good and who is bad, greatly affects the possibility of sustained cooperation in the population. Previously, we have exhaustively studied reputation dynamics that are able to maintain a high level of cooperation at the ESS. However, this analysis examined the stability of monomorphic population and did not investigate polymorphic population where several strategies coexist. Here, we study the evolutionary dynamics of multiple behavioral strategies by replicator dynamics. We exhaustively study all 16 possible norms under which the reputation of a player in the next round is determined by the action of the self and the reputation of the opponent. For each norm, we explore evolutionary dynamics of three strategies: unconditional cooperators, unconditional defectors, and conditional cooperators. We find that only three norms, simple-standing, Kandori, and shunning, can make conditional cooperation evolutionarily stable, hence, realize sustained cooperation. The other 13 norms, including scoring, ultimately lead to the invasion by defectors. Also, we study the model in which private reputation errors exist to a small extent. In this case, we find the stable coexistence of unconditional and conditional cooperators under the three norms.     

The good, the bad and the discriminator--errors in direct and indirect reciprocity

This paper presents, in a series of simple diagrams, concise results about the replicator dynamics of direct and indirect reciprocity. We consider repeated interactions between donors and recipients, and analyse the relationship between three basic strategies for the donor: unconditional cooperation, all-out defection, and conditional cooperation. In other words, we investigate the competition of discriminating and indiscriminating altruists with defectors. Discriminators and defectors form a bistable community, and hence a population of discriminators cannot be invaded by defectors. But unconditional altruists can invade a discriminating population and 'soften it up' for a subsequent invasion by defectors. The resulting dynamics exhibits various forms of rock-paper-scissors cycles and depends in subtle ways on noise, in the form of errors in implementation. The probability for another round (in the case of direct reciprocity), and information about the co-player (in the case of indirect reciprocity), add further elements to the ecology of reciprocation.     

The competition of assessment rules for indirect reciprocity

Indirect reciprocity is one of the basic mechanisms to sustain mutual cooperation. Beneficial acts are returned, not by the recipient, but by third parties. Indirect reciprocity is based on reputation and status: it pays to provide help because this makes one more likely to receive help in turn. The mechanism depends on knowing the past behavior of other players, and assessing that behavior. There are many different systems of assessing other individuals, which can be interpreted as rudimentary moral systems (i.e. views on what is ‘good’ or ‘bad’). In this paper, we describe the competition of some of the leading assessment rules called SUGDEN and KANDORI by analytic methods. We show that the sterner rule KANDORI has a slight advantage in the sense that KANDORI-players have more chance to earn higher payoff than SUGDEN-players in the presence of unconditional altruists. On the other hand, we see that the unconditional altruists are eliminated in the long run and that stable polymorphisms of KANDORI and SUGDEN can subsist, but that a moral consensus is realized even in those polymorphic states: all players’ images are the same in each observer's eyes.     

Two wrongs don't make a right: the initial viability of different assessment rules in the evolution of indirect reciprocity

Indirect reciprocity models are meant to correspond to simple moral systems, in which individuals assess the interactions of third parties in order to condition their cooperative behavior. Despite the staggering number of possible assessment rules in even the simplest of these models, previous research suggests that only a handful are evolutionarily stable against invasion by free riders. These successful assessment rules fall into two categories, one which positively judges miscreants when they refuse to help other miscreants, the other which does not. Previous research has not, however, demonstrated that all of these rules can invade an asocial population—a requirement for a complete theory of social evolution. Here, I present a general analytical model of indirect reciprocity and show that the class of assessment rules which positively judges a refusal to help scofflaws cannot invade a population of defectors, whereas the other class can. When rare, assessment rules which positively judge a refusal to help bad people produce a poor correlation between reputation and behavior. It is this correlation that generates the assortment crucial in sustaining cooperation through indirect reciprocity. Only assessment rules that require good deeds to achieve a good reputation guarantee a strong correlation between behavior and reputation.     

Evolution of cooperation through indirect reciprocity

How can cooperation through indirect reciprocity evolve and what would it be like? This problem has previously been studied by simulating evolution in a small group of interacting individuals, assuming no gene flow between groups. In these simulations, certain ''image scoring'' strategies were found to be the most successful. However, analytical arguments show that it would not be in an individual''s interest to use these strategies. Starting with this puzzle, we investigate indirect reciprocity in simulations based on an island model. This has an advantage in that the role of genetic drift can be examined. Our results show that the image scoring strategies depend on very strong drift or a very small cost of giving help. As soon as these factors are absent, selection eliminates image scoring. We also consider other possibilities for the evolution of indirect reciprocity. In particular, we find that the strategy of aiming for ''good standing'' has superior properties. It can be an evolutionarily stable strategy and, even if not, it usually beats image scoring. Furthermore, by introducing quality variation among individuals into the model, we show that the standing strategy can be quality revealing, adding a new dimension to indirect reciprocity. Finally, we discuss general problems with currently popular modelling styles.     

Evolution of indirect reciprocity by social information: the role of trust and reputation in evolution of altruism

The complexity of human's cooperative behavior cannot be fully explained by theories of kin selection and group selection. If reciprocal altruism is to provide an explanation for altruistic behavior, it would have to depart from direct reciprocity, which requires dyads of individuals to interact repeatedly. For indirect reciprocity to rationalize cooperation among genetically unrelated or even culturally dissimilar individuals, information about the reputation of individuals must be assessed and propagated in a population. Here, we propose a new framework for the evolution of indirect reciprocity by social information: information selectively retrieved from and propagated through dynamically evolving networks of friends and acquaintances. We show that for indirect reciprocity to be evolutionarily stable, the differential probability of trusting and helping a reputable individual over a disreputable individual, at a point in time, must exceed the cost-to-benefit ratio of the altruistic act. In other words, the benefit received by the trustworthy must out-weigh the cost of helping the untrustworthy.     

Indirect reciprocity under incomplete observation

Indirect reciprocity, in which individuals help others with a good reputation but not those with a bad reputation, is a mechanism for cooperation in social dilemma situations when individuals do not repeatedly interact with the same partners. In a relatively large society where indirect reciprocity is relevant, individuals may not know each other's reputation even indirectly. Previous studies investigated the situations where individuals playing the game have to determine the action possibly without knowing others' reputations. Nevertheless, the possibility that observers of the game, who generate the reputation of the interacting players, assign reputations without complete information about them has been neglected. Because an individual acts as an interacting player and as an observer on different occasions if indirect reciprocity is endogenously sustained in a society, the incompleteness of information may affect either role. We examine the game of indirect reciprocity when the reputations of players are not necessarily known to observers and to interacting players. We find that the trustful discriminator, which cooperates with good and unknown players and defects against bad players, realizes cooperative societies under seven social norms. Among the seven social norms, three of the four suspicious norms under which cooperation (defection) to unknown players leads to a good (bad) reputation enable cooperation down to a relatively small observation probability. In contrast, the three trustful norms under which both cooperation and defection to unknown players lead to a good reputation are relatively efficient.     

The long-term benefits of human generosity in indirect reciprocity

Among the theories that have been proposed to explain the evolution of altruism are direct reciprocity and indirect reciprocity. The idea of the latter is that helping someone or refusing to do so has an impact on one's reputation within a group. This reputation is constantly assessed and reassessed by others and is taken into account by them in future social interactions. Generosity in indirect reciprocity can evolve if and only if it eventually leads to a net benefit in the long term. Here, we show that this key assumption is met. We let 114 students play for money in an indirect and a subsequent direct reciprocity game. We found that although being generous, i.e., giving something of value to others, had the obvious short-term costs, it paid in the long run because it builds up a reputation that is rewarded by third parties (who thereby themselves increase their reputation). A reputation of being generous also provided an advantage in the subsequent direct reciprocity game, probably because it builds up trust that can lead to more stable cooperation.     

Stern-judging: A simple, successful norm which promotes cooperation under indirect reciprocity

We study the evolution of cooperation under indirect reciprocity, believed to constitute the biological basis of morality. We employ an evolutionary game theoretical model of multilevel selection, and show that natural selection and mutation lead to the emergence of a robust and simple social norm, which we call stern-judging. Under stern-judging, helping a good individual or refusing help to a bad individual leads to a good reputation, whereas refusing help to a good individual or helping a bad one leads to a bad reputation. Similarly for tit-for-tat and win-stay-lose-shift, the simplest ubiquitous strategies in direct reciprocity, the lack of ambiguity of stern-judging, where implacable punishment is compensated by prompt forgiving, supports the idea that simplicity is often associated with evolutionary success.     

An experimental study of strong reciprocity in bacteria

Strong reciprocity, whereby cooperators punish non-cooperators, may help to explain the evolutionary success of cooperative behaviours. However, theory suggests that selection for strong reciprocity can depend upon tight genetic linkage between cooperation and punishment, to avoid the strategy being outcompeted by non-punishing cooperators. We tested this hypothesis using experimental populations of the bacterium Pseudomonas aeruginosa, which cooperate by producing iron-scavenging siderophores and, in this context, punish non-cooperators with toxins. Consistent with theory, we show that cooperative punishers can indeed invade cheats, but only when the traits are tightly linked. These results emphasize that punishment is only likely to be favoured when the punishment itself leads to a direct or indirect fitness benefit to the actor.     

Reputation-based partner choice is an effective alternative to indirect reciprocity in solving social dilemmas

When group interests clash with individual ones, maintaining cooperation poses a problem. However, cooperation can be facilitated by introducing reputational incentives. Through indirect reciprocity, people who cooperate in a social dilemma are more likely to receive cooperative acts from others. Another mechanism that enhances group cooperation is reputation-based partner choice, or competitive altruism. According to this framework, cooperators benefit via increased access to cooperative partners. Our study compared the effectiveness of indirect reciprocity and competitive altruism in re-establishing cooperation after the typical decline found during repeated public goods games. Twenty groups of four participants first played a series of public goods games, which confirmed the expected decline. Subsequently, public goods games were alternated with either indirect reciprocity games (in which participants had an opportunity to give to another individual from whom they would never receive a direct return) or competitive altruism games (in which they could choose partners for directly reciprocal interactions). We found that public goods game contributions increased when interspersed with competitive altruism games; they were also higher than in public goods games interspersed with indirect reciprocity games. Investing in reputation by increasing contributions to public goods was a profitable strategy in that it increased returns in subsequent competitive altruism and indirect reciprocity games. There was also some evidence that these returns were greater under competitive altruism than indirect reciprocity. Our findings indicate that strategic reputation building through competitive altruism provides an effective alternative to indirect reciprocity as a means for restoring cooperation in social dilemmas.     

A tale of two defectors: the importance of standing for evolution of indirect reciprocity

Indirect reciprocity occurs when the cooperative behavior between two individuals is contingent on their previous behavior toward others. Previous theoretical analysis indicates that indirect reciprocity can evolve if individuals use an image-scoring strategy. In this paper, we show that, when errors are added, indirect reciprocity cannot be based on an image-scoring strategy. However, if individuals use a standing strategy, then cooperation through indirect reciprocity is evolutionarily stable. These two strategies differ with respect to the information to which they attend. While image-scoring strategies only need attend to the actions of others, standing strategies also require information about intent. We speculate that this difference may shed light on the evolvability of indirect reciprocity. Additionally, we show that systems of indirect reciprocity are highly sensitive to the availability of information. Finally, we present a model which shows that if indirect reciprocity were to evolve, selection should also favor trusting behavior in relations between strangers.     

From reciprocity to unconditional altruism through signalling benefits

Cooperation among genetically unrelated individuals is commonly explained by the potential for future reciprocity or by the risk of being punished by group members. However, unconditional altruism is more difficult to explain. We demonstrate that unconditional altruism can evolve as a costly signal of individual quality (i.e. a handicap) as a consequence of reciprocal altruism. This is because the emergent correlation between altruism and individual quality in reciprocity games can facilitate the use of altruism as a quality indicator in a much wider context, outside the reciprocity game, thus affecting its further evolution through signalling benefits. Our model, based on multitype evolutionary game theory shows that, when the additive signalling benefit of donating help exceeds the cost for only some individuals (of high-quality state) but not for others (of low-quality state), the population possesses an evolutionarily stable strategy (ESS) profile wherein high-quality individuals cooperate unconditionally while low-quality individuals defect or play tit-for-tat (TfT). Hence, as predicted by Zahavi's handicap model, signalling benefits of altruistic acts can establish a stable generosity by high-quality individuals that no longer depends on the probability of future reciprocation or punishment.     

Cooperation under indirect reciprocity and imitative trust

Indirect reciprocity, a key concept in behavioral experiments and evolutionary game theory, provides a mechanism that allows reciprocal altruism to emerge in a population of self-regarding individuals even when repeated interactions between pairs of actors are unlikely. Recent empirical evidence show that humans typically follow complex assessment strategies involving both reciprocity and social imitation when making cooperative decisions. However, currently, we have no systematic understanding of how imitation, a mechanism that may also generate negative effects via a process of cumulative advantage, affects cooperation when repeated interactions are unlikely or information about a recipient's reputation is unavailable. Here we extend existing evolutionary models, which use an image score for reputation to track how individuals cooperate by contributing resources, by introducing a new imitative-trust score, which tracks whether actors have been the recipients of cooperation in the past. We show that imitative trust can co-exist with indirect reciprocity mechanisms up to a threshold and then cooperation reverses -revealing the elusive nature of cooperation. Moreover, we find that when information about a recipient's reputation is limited, trusting the action of third parties towards her (i.e. imitating) does favor a higher collective cooperation compared to random-trusting and share-alike mechanisms. We believe these results shed new light on the factors favoring social imitation as an adaptive mechanism in populations of cooperating social actors.     

Reactive strategies in indirect reciprocity

Evolution of reactive strategy of indirect reciprocity is discussed, where individuals interact with others through the one-shot Prisoner's Dilemma game, changing their partners in every round. We investigate all of the reactive strategies that are stochastic, including deterministic ones as special cases. First we study adaptive dynamics of reactive strategies by assuming monomorphic population. Results are very similar to the corresponding evolutionary dynamics of direct reciprocity. The discriminating strategy, which prescribes cooperation only with those who cooperated in the previous round, cannot be an outcome of the evolution. Next we examine the case where the population includes a diversity of strategies. We find that only the mean 'discriminatoriness' in the population is the parameter that affects the evolutionary dynamics. The discriminating strategy works as a promoter of cooperation there. However, it is again not the end point of the evolution. This is because retaliatory defection, which was prescribed by the discriminating strategy, is regarded as another defection toward the society. These results caution that we have to reconsider the role of retaliatory defection much more carefully.     

Indirect reciprocity among imperfect individuals

Reciprocal cooperation occurs when the overall benefits of receiving help exceed the costs of donating help (Q. Rev. Biol. 46 (197) 35). That is, individuals in good condition--for whom the pertinent costs are relatively small; donate help in order to secure reciprocity in their hour of need--when the benefits of receiving a donation are large. Consequently, reciprocity occurs among individuals who occasionally need help. In particular, such individuals will be unable to help others, no matter how deserving, when in need of help themselves--involuntary defection. This paper deals with the effects of involuntary defection in the context of a specific model of indirect reciprocity (i.e. reciprocal altruism that is directed toward all the cooperative members of the community) due to Nowak and Sigmund (J. Theor. Biol.194 (1998b) 561: Sections 2-4). In that model, the authors formulate the decision rules for conditional cooperation in the context of indirect reciprocity, and demonstrate that these decision rules can account for a long-term persistence of cooperation. Here we show that addition of involuntary defection to the decision rules formulated by Nowak and Sigmund results in indirect reciprocity that is evolutionary stable under appropriate conditions. Moreover, for a wide range of parameter values, evolutionary stability of cooperation requires a mixture of conditional- and unconditional-altruist behaviors. To recollect, unconditional altruist strategy can be viewed as conditional altruist strategy sans the ability to decide when the help-soliciting individual should be refused help. That is, given involuntary defection, stability of cooperation requires an occasional forgiveness, if only by default, of a failure to donate help. Thus, we see that evolutionary stable indirect reciprocity does not require perfection in either the ability to assess the merits of the help-soliciting individuals, or the ability to donate help when it is merited. On the contrary, we are forced to conclude that reciprocity, at least in the current case, is stable only among imperfect individuals.     

Evolution of direct and indirect reciprocity

Indirect reciprocity (IR) occurs when individuals help those who help others. It is important as a potential explanation for why people might develop cooperative reputations. However, previous models of IR are based on the assumption that individuals never meet again. Yet humans and other animals often interact repeatedly within groups, thereby violating the fundamental basis of these models. Whenever re-meeting can occur, discriminating reciprocators can decide whether to help those who helped others (IR) or those who helped them (direct reciprocity, DR). Here I used simulation models to investigate the conditions in which we can expect the different forms of reciprocity to predominate. I show that IR through image scoring becomes unstable with respect to DR by experience scoring as the probability of re-meeting increases. However, using the standing strategy, which takes into account the context of observed defections, IR can be stable with respect to DR even when individuals interact with few partners many times. The findings are important in showing that IR cannot explain a concern for reputation in typical societies unless reputations provide as reliable a guide to cooperative behaviour as does experience.     

Participation costs can suppress the evolution of upstream reciprocity

Indirect reciprocity, one of the many mechanisms proposed to explain the evolution of cooperation, is the idea that altruistic actions can be rewarded by third parties. Upstream or generalized reciprocity is one type of indirect reciprocity in which individuals help someone if they have been helped by somebody else in the past. Although empirically found to be at work in humans, the evolution of upstream reciprocity is difficult to explain from a theoretical point of view. A recent model of upstream reciprocity, first proposed by Nowak and Roch (2007) and further analyzed by Iwagami and Masuda (2010), shows that while upstream reciprocity alone does not lead to the evolution of cooperation, it can act in tandem with mechanisms such as network reciprocity and increase the total level of cooperativity in the population. We argue, however, that Nowak and Roch's model systematically leads to non-uniform interaction rates, where more cooperative individuals take part in more games than less cooperative ones. As a result, the critical benefit-to-cost ratios derived under this model in previous studies are not invariant with respect to the addition of participation costs. We show that accounting for these costs can hinder and even suppress the evolution of upstream reciprocity, both for populations with non-random encounters and graph-structured populations.