Indirect reciprocity among imperfect individuals

Reciprocal cooperation occurs when the overall benefits of receiving help exceed the costs of donating help (Q. Rev. Biol. 46 (197) 35). That is, individuals in good condition--for whom the pertinent costs are relatively small; donate help in order to secure reciprocity in their hour of need--when the benefits of receiving a donation are large. Consequently, reciprocity occurs among individuals who occasionally need help. In particular, such individuals will be unable to help others, no matter how deserving, when in need of help themselves--involuntary defection. This paper deals with the effects of involuntary defection in the context of a specific model of indirect reciprocity (i.e. reciprocal altruism that is directed toward all the cooperative members of the community) due to Nowak and Sigmund (J. Theor. Biol.194 (1998b) 561: Sections 2-4). In that model, the authors formulate the decision rules for conditional cooperation in the context of indirect reciprocity, and demonstrate that these decision rules can account for a long-term persistence of cooperation. Here we show that addition of involuntary defection to the decision rules formulated by Nowak and Sigmund results in indirect reciprocity that is evolutionary stable under appropriate conditions. Moreover, for a wide range of parameter values, evolutionary stability of cooperation requires a mixture of conditional- and unconditional-altruist behaviors. To recollect, unconditional altruist strategy can be viewed as conditional altruist strategy sans the ability to decide when the help-soliciting individual should be refused help. That is, given involuntary defection, stability of cooperation requires an occasional forgiveness, if only by default, of a failure to donate help. Thus, we see that evolutionary stable indirect reciprocity does not require perfection in either the ability to assess the merits of the help-soliciting individuals, or the ability to donate help when it is merited. On the contrary, we are forced to conclude that reciprocity, at least in the current case, is stable only among imperfect individuals.     

Partner Choice Drives the Evolution of Cooperation via Indirect Reciprocity

Indirect reciprocity potentially provides an important means for generating cooperation based on helping those who help others. However, the use of ‘image scores’ to summarize individuals’ past behaviour presents a dilemma: individuals withholding help from those of low image score harm their own reputation, yet giving to defectors erodes cooperation. Explaining how indirect reciprocity could evolve has therefore remained problematic. In all previous treatments of indirect reciprocity, individuals are assigned potential recipients and decide whether to cooperate or defect based on their reputation. A second way of achieving discrimination is through partner choice, which should enable individuals to avoid defectors. Here, I develop a model in which individuals choose to donate to anyone within their group, or to none. Whereas image scoring with random pairing produces cycles of cooperation and defection, with partner choice there is almost maximal cooperation. In contrast to image scoring with random pairing, partner choice results in almost perfect contingency, producing the correlation between giving and receiving required for cooperation. In this way, partner choice facilitates much higher and more stable levels of cooperation through image scoring than previously reported and provides a simple mechanism through which systems of helping those who help others can work.     

Clustering in large networks does not promote upstream reciprocity

Upstream reciprocity (also called generalized reciprocity) is a putative mechanism for cooperation in social dilemma situations with which players help others when they are helped by somebody else. It is a type of indirect reciprocity. Although upstream reciprocity is often observed in experiments, most theories suggest that it is operative only when players form short cycles such as triangles, implying a small population size, or when it is combined with other mechanisms that promote cooperation on their own. An expectation is that real social networks, which are known to be full of triangles and other short cycles, may accommodate upstream reciprocity. In this study, I extend the upstream reciprocity game proposed for a directed cycle by Boyd and Richerson to the case of general networks. The model is not evolutionary and concerns the conditions under which the unanimity of cooperative players is a Nash equilibrium. I show that an abundance of triangles or other short cycles in a network does little to promote upstream reciprocity. Cooperation is less likely for a larger population size even if triangles are abundant in the network. In addition, in contrast to the results for evolutionary social dilemma games on networks, scale-free networks lead to less cooperation than networks with a homogeneous degree distribution.     

ASSORTMENT AND THE EVOLUTION OF GENERALIZED RECIPROCITY

Reciprocity is often invoked to explain cooperation. Reciprocity is cognitively demanding, and both direct and indirect reciprocity require that individuals store information about the propensity of their partners to cooperate. By contrast, generalized reciprocity, wherein individuals help on the condition that they received help previously, only relies on whether an individual received help in a previous encounter. Such anonymous information makes generalized reciprocity hard to evolve in a well‐mixed population, as the strategy will lose out to pure defectors. Here we analyze a model for the evolution of generalized reciprocity, incorporating assortment of encounters, to investigate the conditions under which it will evolve. We show that, in a well‐mixed population, generalized reciprocity cannot evolve. However, incorporating assortment of encounters can favor the evolution of generalized reciprocity in which indiscriminate cooperation and defection are both unstable. We show that generalized reciprocity can evolve under both the prisoner's dilemma and the snowdrift game.     

Upstream reciprocity and the evolution of gratitude

If someone is nice to you, you feel good and may be inclined to be nice to somebody else. This every day experience is borne out by experimental games: the recipients of an act of kindness are more likely to help in turn, even if the person who benefits from their generosity is somebody else. This behaviour, which has been called ‘upstream reciprocity’, appears to be a misdirected act of gratitude: you help somebody because somebody else has helped you. Does this make any sense from an evolutionary or a game theoretic perspective? In this paper, we show that upstream reciprocity alone does not lead to the evolution of cooperation, but it can evolve and increase the level of cooperation if it is linked to either direct or spatial reciprocity. We calculate the random walks of altruistic acts that are induced by upstream reciprocity. Our analysis shows that gratitude and other positive emotions, which increase the willingness to help others, can evolve in the competitive world of natural selection.     

Cooperation among non-relatives evolves by state-dependent generalized reciprocity

For decades, attempts to understand cooperation between non-kin have generated substantial theoretical and empirical interest in the evolutionary mechanisms of reciprocal altruism. There is growing evidence that the cognitive limitations of animals can hinder direct and indirect reciprocity because the necessary mental capacity is costly. Here, we show that cooperation can evolve by generalized reciprocity (help anyone, if helped by someone) even in large groups, if individuals base their decision to cooperate on a state variable updated by the outcome of the last interaction with an anonymous partner. We demonstrate that this alternative mechanism emerges through small evolutionary steps under a wide range of conditions. Since this state-based generalized reciprocity works without advanced cognitive abilities it may help to understand the evolution of complex social behaviour in a wide range of organisms.     

Generalized reciprocity in rats

The evolution of cooperation among nonrelatives has been explained by direct, indirect, and strong reciprocity. Animals should base the decision to help others on expected future help, which they may judge from past behavior of their partner. Although many examples of cooperative behavior exist in nature where reciprocity may be involved, experimental evidence for strategies predicted by direct reciprocity models remains controversial; and indirect and strong reciprocity have been found only in humans so far. Here we show experimentally that cooperative behavior of female rats is influenced by prior receipt of help, irrespective of the identity of the partner. Rats that were trained in an instrumental cooperative task (pulling a stick in order to produce food for a partner) pulled more often for an unknown partner after they were helped than if they had not received help before. This alternative mechanism, called generalized reciprocity, requires no specific knowledge about the partner and may promote the evolution of cooperation among unfamiliar nonrelatives.     

Mechanistic constraints and the unlikely evolution of reciprocal cooperation

Social evolution theory faces a puzzle: a gap between theoretical and empirical results on reciprocity. On the one hand, models show that reciprocity should evolve easily in a wide range of circumstances. On the other hand, empirically, few clear instances of reciprocity (even in a broad sense) have been found in nonhuman animals. In this paper, I aim to suggest and evaluate a novel reason concurring to solve this puzzle. I propose that it is difficult for reciprocity to evolve because it raises an evolutionary problem of bootstrapping: it requires that two complementary functions: (i) the ability to cooperate and (ii) the ability to respond conditionally to the cooperation of others, arise together and reach a significant frequency, whereas neither of them can be favoured in the absence of the other. I develop analytical models and simulations showing that, for this reason, the evolutionary emergence of reciprocal cooperation is highly unlikely. I then discuss the consequences of this result for our understanding of cooperation.     

The evolution of generalized reciprocity on social interaction networks

Generalized reciprocity (help anyone, if helped by someone) is a minimal strategy capable of supporting cooperation between unrelated individuals. Its simplicity makes it an attractive model to explain the evolution of reciprocal altruism in animals that lack the information or cognitive skills needed for other types of reciprocity. Yet, generalized reciprocity is anonymous and thus defenseless against exploitation by defectors. Recognizing that animals hardly ever interact randomly, we investigate whether social network structure can mitigate this vulnerability. Our results show that heterogeneous interaction patterns strongly support the evolution of generalized reciprocity. The future probability of being rewarded for an altruistic act is inversely proportional to the average connectivity of the social network when cooperators are rare. Accordingly, sparse networks are conducive to the invasion of reciprocal altruism. Moreover, the evolutionary stability of cooperation is enhanced by a modular network structure. Communities of reciprocal altruists are protected against exploitation, because modularity increases the mean access time, that is, the average number of steps that it takes for a random walk on the network to reach a defector. Sparseness and community structure are characteristic properties of vertebrate social interaction patterns, as illustrated by network data from natural populations ranging from fish to primates.     

Revisiting “The Evolution of Reciprocity in Sizable Groups”: Continuous reciprocity in the repeated n-person prisoner's dilemma

For many years in evolutionary science, the consensus view has been that while reciprocal altruism can evolve in dyadic interactions, it is unlikely to evolve in sizable groups. This view had been based on studies which have assumed cooperation to be discrete rather than continuous (i.e., individuals can either fully cooperate or else fully defect, but they cannot continuously vary their level of cooperation). In real world cooperation, however, cooperation is often continuous. In this paper, we re-examine the evolution of reciprocity in sizable groups by presenting a model of the n-person prisoner's dilemma that assumes continuous rather than discrete cooperation. This model shows that continuous reciprocity has a dramatically wider basin of attraction than discrete reciprocity, and that this basin's size increases with efficiency of cooperation (marginal per capita return). Further, we find that assortative interaction interacts synergistically with continuous reciprocity to a much greater extent than it does with discrete reciprocity. These results suggest that previous models may have underestimated reciprocity's adaptiveness in groups. However, we also find that the invasion of continuous reciprocators into a population of unconditional defectors becomes realistic only within a narrow parameter space in which the efficiency of cooperation is close to its maximum bound. Therefore our model suggests that continuous reciprocity can evolve in large groups more easily than discrete reciprocity only under unusual circumstances.     

Direct Reciprocity in Spatial Populations Enhances R-Reciprocity As Well As ST-Reciprocity

As is well-known, spatial reciprocity plays an important role in facilitating the emergence of cooperative traits, and the effect of direct reciprocity is also obvious for explaining the cooperation dynamics. However, how the combination of these two scenarios influences cooperation is still unclear. In the present work, we study the evolution of cooperation in 2×2 games via considering both spatial structured populations and direct reciprocity driven by the strategy with 1-memory length. Our results show that cooperation can be significantly facilitated on the whole parameter plane. For prisoner''s dilemma game, cooperation dominates the system even at strong dilemma, where maximal social payoff is still realized. In this sense, R-reciprocity forms and it is robust to the extremely strong dilemma. Interestingly, when turning to chicken game, we find that ST-reciprocity is also guaranteed, through which social average payoff and cooperation is greatly enhanced. This reciprocity mechanism is supported by mean-field analysis and different interaction topologies. Thus, our study indicates that direct reciprocity in structured populations can be regarded as a more powerful factor for the sustainability of cooperation.     

Participation costs can suppress the evolution of upstream reciprocity

Indirect reciprocity, one of the many mechanisms proposed to explain the evolution of cooperation, is the idea that altruistic actions can be rewarded by third parties. Upstream or generalized reciprocity is one type of indirect reciprocity in which individuals help someone if they have been helped by somebody else in the past. Although empirically found to be at work in humans, the evolution of upstream reciprocity is difficult to explain from a theoretical point of view. A recent model of upstream reciprocity, first proposed by Nowak and Roch (2007) and further analyzed by Iwagami and Masuda (2010), shows that while upstream reciprocity alone does not lead to the evolution of cooperation, it can act in tandem with mechanisms such as network reciprocity and increase the total level of cooperativity in the population. We argue, however, that Nowak and Roch's model systematically leads to non-uniform interaction rates, where more cooperative individuals take part in more games than less cooperative ones. As a result, the critical benefit-to-cost ratios derived under this model in previous studies are not invariant with respect to the addition of participation costs. We show that accounting for these costs can hinder and even suppress the evolution of upstream reciprocity, both for populations with non-random encounters and graph-structured populations.     

Heterogeneity stabilizes reciprocal altruism interactions

In considering the phenomena of reciprocal altruism few would dispute that there are differences in individual quality-in particular, that for some individuals, at least on occasion, the cost of doing favors will exceed the potential of future benefits. That is, at any given time, a typical population is heterogeneous with respect to the affordability of reciprocal altruism. However, methodological limitations of the traditional analytical framework-Single Type (symmetric) Evolutionary Game Theory-have restricted previous analytical efforts to addressing populations idealized in terms of their averages. Here we use the methods of Multitype Evolutionary Game Theory to analyse the role of individual differences in direct reciprocity interactions. Multitype analysis shows that non-idealized populations possess an ESS profile wherein individuals who cannot afford reciprocity (low-quality) defect, while individuals who derive net benefits from reciprocity (high-quality) cooperate. Furthermore, this cooperation is implemented via unmodified tit-for-tat (TfT) strategy. Hence, our results may help resolve a long-standing problem concerning the evolutionary stability of TfT in direct reciprocal altruism. Finally, this difference between idealized and real populations is not restricted to direct reciprocal cooperation. Previously (Lotem et al., 1999) we have demonstrated evolutionarily stable indirect reciprocal cooperation among high-quality individuals in heterogeneous populations.     

Indirect reciprocity under incomplete observation

Indirect reciprocity, in which individuals help others with a good reputation but not those with a bad reputation, is a mechanism for cooperation in social dilemma situations when individuals do not repeatedly interact with the same partners. In a relatively large society where indirect reciprocity is relevant, individuals may not know each other's reputation even indirectly. Previous studies investigated the situations where individuals playing the game have to determine the action possibly without knowing others' reputations. Nevertheless, the possibility that observers of the game, who generate the reputation of the interacting players, assign reputations without complete information about them has been neglected. Because an individual acts as an interacting player and as an observer on different occasions if indirect reciprocity is endogenously sustained in a society, the incompleteness of information may affect either role. We examine the game of indirect reciprocity when the reputations of players are not necessarily known to observers and to interacting players. We find that the trustful discriminator, which cooperates with good and unknown players and defects against bad players, realizes cooperative societies under seven social norms. Among the seven social norms, three of the four suspicious norms under which cooperation (defection) to unknown players leads to a good (bad) reputation enable cooperation down to a relatively small observation probability. In contrast, the three trustful norms under which both cooperation and defection to unknown players lead to a good reputation are relatively efficient.     

How should we define goodness?--reputation dynamics in indirect reciprocity

Theory of indirect reciprocity is important in explaining cooperation between humans. Since a partner of a social interaction often changes, an individual should assess his partner by using social information such as reputation and make decisions whether to help him or not. To those who have 'good' social reputation does a player give aid as reciprocation, whereas he has to refuse to help those who have 'bad' reputation. Otherwise benefits of altruism is easily exploited by them. Little has been known, however, about the definition of 'goodness' in reputation. What kind of actions are and should be regarded as good and what kind of actions bad? And what sort of goodness enables sustaining exchange of altruism? We herein challenge this question with an evolutionary perspective. We generalize social reputation as 'Honor-score' (H-score) and examine the conditions under which individuals in a group stably maintain cooperative relationships based on indirect reciprocity. We examine the condition for evolutionarily stable strategies (ESSs) over 4096 possible cases exhaustively. Mathematical analysis reveals that only eight cases called 'leading eight' are crucial to the evolution of indirect reciprocity. Each in the leading eight shares two common characteristics: (i) cooperation with good persons is regarded as good while defection against them is regarded as bad, and (ii) defection against bad persons should be regarded as a good behavior because it works as sanction. Our results give one solution to the definition of goodness from an evolutionary viewpoint. In addition, we believe that the formalism of reputation dynamics gives general insights into the way social information is generated, handled, and transmitted in animal societies.     

Dynamic indirect reciprocity; When is indirect reciprocity bounded by group membership?

Indirect reciprocity is a strong driver of reputation-based cooperation and previous studies have offered ample evidence as to when and how it guides cooperation towards others. However, the current empirical literature suffers from mixed evidence about the realm of indirect reciprocity; some studies showed that people assume that in-group members, but not out-group members belong to the same system of indirect reciprocity and they, thus, display reputation-based cooperation only towards in-group members. In other words, indirect reciprocity is bounded by group membership. On the other hand, other studies found that people display reputation-based cooperation towards out-group members as well, suggesting that indirect reciprocity is unbounded. Thus, there remains an important question to address. When is indirect reciprocity bounded by group membership? We carefully examined previous studies supporting bounded and unbounded indirect reciprocity perspectives and argue that the realm of indirect reciprocity is conditional to the presence of cues of reputational consequences (i.e., consequences of immediate cooperation). Based on our review, we further propose a new perspective, the dynamic indirect reciprocity perspective, which integrates bounded and unbounded indirect reciprocity perspectives. We propose a highly powered preregistered study to test hypotheses derived from the proposed perspective.     

Indirect reciprocity in asymmetric interactions: when apparent altruism facilitates profitable exploitation

Intraspecific cooperation and interspecific mutualism often feature an asymmetry in the scope for exploitation. We investigate the evolution of indirect reciprocity in an asymmetric game, loosely modelled on interactions between cleaner fishes and clients, in which 'actors' can choose to help or to exploit a 'recipient' that approaches them, while recipients can only choose whether or not to approach an actor (based on the observation of its behaviour towards others). We show that when actors vary in state over time, in a manner that influences the potential gains from exploitation, an equilibrium is possible at which recipients avoid actors whom they have observed exploiting others in the past, and actors help when the potential gains from exploitation are low but choose to exploit when the potential gains are high. In this context, helping is favoured not because it elicits reciprocal altruism ('help so that you may be helped'), but because it facilitates profitable exploitation ('help so that you may gain the opportunity to harm'). The cost of helping one recipient is thereby recouped through exploitation of another. Indirect reciprocity is thus possible even in asymmetric interactions in which one party cannot directly 'punish' exploitation or 'reward' helping by the other.     

The competition of assessment rules for indirect reciprocity

Indirect reciprocity is one of the basic mechanisms to sustain mutual cooperation. Beneficial acts are returned, not by the recipient, but by third parties. Indirect reciprocity is based on reputation and status: it pays to provide help because this makes one more likely to receive help in turn. The mechanism depends on knowing the past behavior of other players, and assessing that behavior. There are many different systems of assessing other individuals, which can be interpreted as rudimentary moral systems (i.e. views on what is ‘good’ or ‘bad’). In this paper, we describe the competition of some of the leading assessment rules called SUGDEN and KANDORI by analytic methods. We show that the sterner rule KANDORI has a slight advantage in the sense that KANDORI-players have more chance to earn higher payoff than SUGDEN-players in the presence of unconditional altruists. On the other hand, we see that the unconditional altruists are eliminated in the long run and that stable polymorphisms of KANDORI and SUGDEN can subsist, but that a moral consensus is realized even in those polymorphic states: all players’ images are the same in each observer's eyes.     

Strong reciprocity, human cooperation, and the enforcement of social norms

This paper provides strong evidence challenging the self-interest assumption that dominates the behavioral sciences and much evolutionary thinking. The evidence indicates that many people have a tendency to voluntarily cooperate, if treated fairly, and to punish noncooperators. We call this behavioral propensity “strong reciprocity” and show empirically that it can lead to almost universal cooperation in circumstances in which purely self-interested behavior would cause a complete breakdown of cooperation. In addition, we show that people are willing to punish those who behaved unfairly towards a third person or who defected in a Prisoner’s Dilemma game with a third person. This suggests that strong reciprocity is a powerful device for the enforcement of social norms involving, for example, food sharing or collective action. Strong reciprocity cannot be rationalized as an adaptive trait by the leading evolutionary theories of human cooperation (in other words, kin selection, reciprocal altruism, indirect reciprocity, and costly signaling theory). However, multilevel selection theories of cultural evolution are consistent with strong reciprocity.     

Upstream reciprocity in heterogeneous networks

Many mechanisms for the emergence and maintenance of altruistic behavior in social dilemma situations have been proposed. Indirect reciprocity is one such mechanism, where other-regarding actions of a player are eventually rewarded by other players with whom the original player has not interacted. The upstream reciprocity (also called generalized indirect reciprocity) is a type of indirect reciprocity and represents the concept that those helped by somebody will help other unspecified players. In spite of the evidence for the enhancement of helping behavior by upstream reciprocity in rats and humans, theoretical support for this mechanism is not strong. In the present study, we numerically investigate upstream reciprocity in heterogeneous contact networks, in which the players generally have different number of neighbors. We show that heterogeneous networks considerably enhance cooperation in a game of upstream reciprocity. In heterogeneous networks, the most generous strategy, by which a player helps a neighbor on being helped and in addition initiates helping behavior, first occupies hubs in a network and then disseminates to other players. The scenario to achieve enhanced altruism resembles that seen in the case of the Prisoner's Dilemma game in heterogeneous networks.