Evolution of cooperation through indirect reciprocity

How can cooperation through indirect reciprocity evolve and what would it be like? This problem has previously been studied by simulating evolution in a small group of interacting individuals, assuming no gene flow between groups. In these simulations, certain ''image scoring'' strategies were found to be the most successful. However, analytical arguments show that it would not be in an individual''s interest to use these strategies. Starting with this puzzle, we investigate indirect reciprocity in simulations based on an island model. This has an advantage in that the role of genetic drift can be examined. Our results show that the image scoring strategies depend on very strong drift or a very small cost of giving help. As soon as these factors are absent, selection eliminates image scoring. We also consider other possibilities for the evolution of indirect reciprocity. In particular, we find that the strategy of aiming for ''good standing'' has superior properties. It can be an evolutionarily stable strategy and, even if not, it usually beats image scoring. Furthermore, by introducing quality variation among individuals into the model, we show that the standing strategy can be quality revealing, adding a new dimension to indirect reciprocity. Finally, we discuss general problems with currently popular modelling styles.     

Direct reciprocity under uncertainty does not explain one-shot cooperation,but demonstrates the benefits of a norm psychology

Humans in many societies cooperate in economic experiments at much higher levels than would be expected if their goal was maximizing economic returns, even when their interactions are anonymous and one-shot. This is a puzzle because paying a cost to benefit another in one-shot interactions gives no direct or indirect benefits to the cooperator. This paper explores the logic of two competing evolutionary hypotheses to explain this behavior. The ``norm psychology'' hypothesis holds that a player's choice of strategy reflects socially-learned cultural norms. Its premise is that over the course of human evolutionary history, cultural norms varied considerably across human societies, and through a process of gene-culture co-evolution, humans evolved mechanisms to learn and adopt the norms that are successful in their particular society. The ``mismatch'' hypothesis holds that pro-social preferences evolved genetically in our hunter–gatherer past where one-shot anonymous interactions were rare and these preferences are misapplied in modern laboratory settings. I compare these hypotheses by adopting a well-known model of the mismatch hypothesis and show that selection for one-shot cooperation in the model is an artifact of agents being confined to only two strategies: Tit-for-Tat and Always Defect. Allowing for repentant and forgiving strategies reverses selection away from one-shot cooperation under all environmental parameters. Direct reciprocity does not necessarily lead to cooperation, but instead generates many different normative equilibria depending on a group's idiosyncratic evolutionary history. Therefore, an agent whose behavior is evoked solely from non-cultural environmental cues will be disadvantaged relative to an agent who learns the locally successful norms. Cooperation in one-shot laboratory experiments is thus more easily explained as the result of a psychology evolved for learning social norms than as a genetic mismatch.     

The leading eight: social norms that can maintain cooperation by indirect reciprocity

The theory of indirect reciprocation explains the evolution of cooperation among unrelated individuals, engaging in one-shot interaction. Using reputation, a player acquires information on who are worth cooperating and who are not. In a previous paper, we formalized the reputation dynamics, a rule to assign a binary reputation (good or bad) to each player when his action, his current reputation, and the opponent's reputation are given. We then examined all the possible reputation dynamics, and found that there exist only eight reputation dynamics named "leading eight" that can maintain the ESS with a high level of cooperation, even if errors are included in executing intended cooperation and in reporting the observation to the public. In this paper, we study the nature of these successful social norms. First, we characterize the role of each pivot of the reputation dynamics common to all of the leading eight. We conclude that keys to the success in indirect reciprocity are to be nice (maintenance of cooperation among themselves), retaliatory (detection of defectors, punishment, and justification of punishment), apologetic, and forgiving. Second, we prove the two basic properties of the leading eight, which give a quantitative evaluation of the ESS condition and the level of cooperation maintained at the ESS.     

Cooperation under indirect reciprocity and imitative trust

Indirect reciprocity, a key concept in behavioral experiments and evolutionary game theory, provides a mechanism that allows reciprocal altruism to emerge in a population of self-regarding individuals even when repeated interactions between pairs of actors are unlikely. Recent empirical evidence show that humans typically follow complex assessment strategies involving both reciprocity and social imitation when making cooperative decisions. However, currently, we have no systematic understanding of how imitation, a mechanism that may also generate negative effects via a process of cumulative advantage, affects cooperation when repeated interactions are unlikely or information about a recipient's reputation is unavailable. Here we extend existing evolutionary models, which use an image score for reputation to track how individuals cooperate by contributing resources, by introducing a new imitative-trust score, which tracks whether actors have been the recipients of cooperation in the past. We show that imitative trust can co-exist with indirect reciprocity mechanisms up to a threshold and then cooperation reverses -revealing the elusive nature of cooperation. Moreover, we find that when information about a recipient's reputation is limited, trusting the action of third parties towards her (i.e. imitating) does favor a higher collective cooperation compared to random-trusting and share-alike mechanisms. We believe these results shed new light on the factors favoring social imitation as an adaptive mechanism in populations of cooperating social actors.     

Contingent movement and cooperation evolve under generalized reciprocity

How cooperation and altruism among non-relatives can persist in the face of cheating remains a key puzzle in evolutionary biology. Although mechanisms such as direct and indirect reciprocity and limited movement have been put forward to explain such cooperation, they cannot explain cooperation among unfamiliar, highly mobile individuals. Here we show that cooperation may be evolutionarily stable if decisions taken to cooperate and to change group membership are both dependent on anonymous social experience (generalized reciprocity). We find that a win-stay, lose-shift rule (where shifting is either moving away from the group or changing tactics within the group after receiving defection) evolves in evolutionary simulations when group leaving is moderately costly (i.e. the current payoff to being alone is low, but still higher than that in a mutually defecting group, and new groups are rarely encountered). This leads to the establishment of widespread cooperation in the population. If the costs of group leaving are reduced, a similar group-leaving rule evolves in association with cooperation in pairs and exploitation of larger anonymous groups. We emphasize that mechanisms of assortment within populations are often behavioural decisions and should not be considered independently of the evolution of cooperation.     

Three-person game facilitates indirect reciprocity under image scoring

Reputation building plays an important role in the evolution of reciprocal altruism when the same individuals do not interact repeatedly because, by referring to reputation, a reciprocator can know which partners are cooperative and can reciprocate with a cooperator. This reciprocity based on reputation is called indirect reciprocity. Previous studies of indirect reciprocity have focused only on two-person games in which only two individuals participate in a single interaction, and have claimed that indirectly reciprocal cooperation cannot be established under image scoring reputation criterion where the reputation of an individual who has cooperated (defected) becomes good (bad). In this study, we specifically examine three-person games, and reveal that indirectly reciprocal cooperation can be formed and maintained stably, even under image scoring, by a nucleus shield mechanism. In the nucleus shield, reciprocators are a shield that keeps out unconditional defectors, whereas unconditional cooperators are the backbone of cooperation that retains a good reputation among the population.     

A study of indirect reciprocity involving a reputation system or a simple tag system in a one-shot, multi-player game

The possibility of the evolution of cooperation backed by indirect reciprocity (IDR) in a one-shot, multi-player game is investigated focusing on two mechanisms. First of all, the reputation system with image score (RS with IS), as proposed by Nowak and Sigmund [Nowak, M.A., Sigmund, K., 1998. Evolution of indirect reciprocity by image scoring. Nature 393, 573–577], is investigated in various game structures. A numerical experiment demonstrates that the RS with IS is a robust mechanism for the support of IDR in various dilemma games, but whose effectiveness decreases with an increase in the number of players in a game. It is fair to say that the RS is an information mapping function to relate between player's cooperative fraction on his action and IS. As the second mechanism, a simple tag system which could be applicable to animals having no cognitive complexity is considered to support IDR. Computer simulations of the tag system's strategy for invading a population initially consisting of AllD, AllC, and Random strategies suggest several novel facts. The Tag strategy with plausible crossover and mutation probabilities can only invade to settle down if the game structure is not Trivial and contains a moderate dilemma. The Tag strategy's evolutionary competition takes place mainly in the presence of the AllD strategy. During the competition, the Tag strategy frequently metamorphoses to shake off the AllD strategy, but stops after winning in order to avoid a shrinking payoff due to fragmentation.     

A simple scaling of the effectiveness of supporting mutual cooperation in donor-recipient games by various reciprocity mechanisms

In donor-recipient games (DRG), one of the sub-classes of Prisoner's Dilemma (PD), it is well-known that a game structure is described by two parameters benefit (b) and cost (c) of cooperation. By means of a series of numerical experiments, we proved that the effectiveness of supporting mutual cooperation in DRG by various reciprocity mechanisms can be expressed in a single game structural parameter, b/c. This also implies that the dilemma strength in various donor-recipient games with various reciprocity mechanisms can be evaluated only by b/c, which is consistent with the previous novel finding by Nowak. It was also discussed whether this kind of parameterization idea can be extended to general games in PD game class.     

Social semantics: altruism, cooperation, mutualism, strong reciprocity and group selection

From an evolutionary perspective, social behaviours are those which have fitness consequences for both the individual that performs the behaviour, and another individual. Over the last 43 years, a huge theoretical and empirical literature has developed on this topic. However, progress is often hindered by poor communication between scientists, with different people using the same term to mean different things, or different terms to mean the same thing. This can obscure what is biologically important, and what is not. The potential for such semantic confusion is greatest with interdisciplinary research. Our aim here is to address issues of semantic confusion that have arisen with research on the problem of cooperation. In particular, we: (i) discuss confusion over the terms kin selection, mutualism, mutual benefit, cooperation, altruism, reciprocal altruism, weak altruism, altruistic punishment, strong reciprocity, group selection and direct fitness; (ii) emphasize the need to distinguish between proximate (mechanism) and ultimate (survival value) explanations of behaviours. We draw examples from all areas, but especially recent work on humans and microbes.     

The evolution of strong reciprocity: cooperation in heterogeneous populations

How do human groups maintain a high level of cooperation despite a low level of genetic relatedness among group members? We suggest that many humans have a predisposition to punish those who violate group-beneficial norms, even when this imposes a fitness cost on the punisher. Such altruistic punishment is widely observed to sustain high levels of cooperation in behavioral experiments and in natural settings. We offer a model of cooperation and punishment that we call STRONG RECIPROCITY: where members of a group benefit from mutual adherence to a social norm, strong reciprocators obey the norm and punish its violators, even though as a result they receive lower payoffs than other group members, such as selfish agents who violate the norm and do not punish, and pure cooperators who adhere to the norm but free-ride by never punishing. Our agent-based simulations show that, under assumptions approximating likely human environments over the 100000 years prior to the domestication of animals and plants, the proliferation of strong reciprocators when initially rare is highly likely, and that substantial frequencies of all three behavioral types can be sustained in a population. As a result, high levels of cooperation are sustained. Our results do not require that group members be related or that group extinctions occur.     

Ostracism and indirect reciprocity: The reproductive significance of humor

Humor is hypothesized to be a social activity that alters the status of the humorist positively and that of the object or victim negatively. Of the two traditionally distinguished classes of humor, “ostracizing” humor singles out a victim, with others present or absent either incidental affiliates of the humorist (and one another) or unaffected. “Affiliative” humor, on the other hand, is focused on creating or maintaining group cohesiveness, with the identity of the victim more or less incidental.     

Strong reciprocity, human cooperation, and the enforcement of social norms

This paper provides strong evidence challenging the self-interest assumption that dominates the behavioral sciences and much evolutionary thinking. The evidence indicates that many people have a tendency to voluntarily cooperate, if treated fairly, and to punish noncooperators. We call this behavioral propensity “strong reciprocity” and show empirically that it can lead to almost universal cooperation in circumstances in which purely self-interested behavior would cause a complete breakdown of cooperation. In addition, we show that people are willing to punish those who behaved unfairly towards a third person or who defected in a Prisoner’s Dilemma game with a third person. This suggests that strong reciprocity is a powerful device for the enforcement of social norms involving, for example, food sharing or collective action. Strong reciprocity cannot be rationalized as an adaptive trait by the leading evolutionary theories of human cooperation (in other words, kin selection, reciprocal altruism, indirect reciprocity, and costly signaling theory). However, multilevel selection theories of cultural evolution are consistent with strong reciprocity.     

A tale of two defectors: the importance of standing for evolution of indirect reciprocity

Indirect reciprocity occurs when the cooperative behavior between two individuals is contingent on their previous behavior toward others. Previous theoretical analysis indicates that indirect reciprocity can evolve if individuals use an image-scoring strategy. In this paper, we show that, when errors are added, indirect reciprocity cannot be based on an image-scoring strategy. However, if individuals use a standing strategy, then cooperation through indirect reciprocity is evolutionarily stable. These two strategies differ with respect to the information to which they attend. While image-scoring strategies only need attend to the actions of others, standing strategies also require information about intent. We speculate that this difference may shed light on the evolvability of indirect reciprocity. Additionally, we show that systems of indirect reciprocity are highly sensitive to the availability of information. Finally, we present a model which shows that if indirect reciprocity were to evolve, selection should also favor trusting behavior in relations between strangers.     

Chaos, oscillation and the evolution of indirect reciprocity in n-person games

Evolution of cooperation among genetically unrelated individuals has been of considerable concern in various fields such as biology, economics, and psychology. The evolution of cooperation is often explained by reciprocity. Under reciprocity, cooperation can prevail in a society because a donor of cooperation receives reciprocation from the recipient of the cooperation, called direct reciprocity, or from someone else in the community, called indirect reciprocity. Nowak and Sigmund [1993. Chaos and the evolution of cooperation. Proc. Natl. Acad. Sci. USA 90, 5091-5094] have demonstrated that directly reciprocal cooperation in two-person prisoner's dilemma games with mutation of strategies can be maintained dynamically as periodic or chaotic oscillation. Furthermore, Eriksson and Lindgren [2005. Cooperation driven by mutations in multi-person Prisoner's Dilemma. J. Theor. Biol. 232, 399-409] have reported that directly reciprocal cooperation in n-person prisoner's dilemma games (n>2) can be maintained as periodic oscillation. Is dynamic cooperation observed only in direct reciprocity? Results of this study show that indirectly reciprocal cooperation in n-person prisoner's dilemma games can be maintained dynamically as periodic or chaotic oscillation. This is, to our knowledge, the first demonstration of chaos in indirect reciprocity. Furthermore, the results show that oscillatory dynamics are observed in common in the evolution of reciprocal cooperation whether for direct or indirect.     

The good, the bad and the discriminator--errors in direct and indirect reciprocity

This paper presents, in a series of simple diagrams, concise results about the replicator dynamics of direct and indirect reciprocity. We consider repeated interactions between donors and recipients, and analyse the relationship between three basic strategies for the donor: unconditional cooperation, all-out defection, and conditional cooperation. In other words, we investigate the competition of discriminating and indiscriminating altruists with defectors. Discriminators and defectors form a bistable community, and hence a population of discriminators cannot be invaded by defectors. But unconditional altruists can invade a discriminating population and 'soften it up' for a subsequent invasion by defectors. The resulting dynamics exhibits various forms of rock-paper-scissors cycles and depends in subtle ways on noise, in the form of errors in implementation. The probability for another round (in the case of direct reciprocity), and information about the co-player (in the case of indirect reciprocity), add further elements to the ecology of reciprocation.     

The return of reciprocity: a psychological approach to the evolution of cooperation

Recent developments in evolutionary game theory argue the superiority of punishment over reciprocity as accounts of large-scale human cooperation. I introduce a distinction between a behavioral and a psychological perspective on reciprocity and punishment to question this view. I examine a narrow and a wide version of a psychological mechanism for reciprocity and conclude that a narrow version is clearly distinguishable from punishment, but inadequate for humans; whereas a wide version is applicable to humans but indistinguishable from punishment. The mechanism for reciprocity in humans emerges as a meta-norm that governs both retaliation and punishment. I make predictions open to empirical investigation to confirm or disconfirm this view.

Punishing free riders: direct and indirect promotion of cooperation

Human cooperation in a large group of genetically unrelated people is an evolutionary puzzle. Despite its costly nature, cooperative behavior is commonly found in all human societies—a fact that has interested researchers from a wide range of disciplines, including biology, economics, and psychology, to name a few. Many behavioral experiments have demonstrated that cooperation within a group can be sustained when free riders are punished. We argue that punishment has both a direct effect and an indirect effect on promoting cooperation. The direct effect of punishment alters the consequences of cooperation and defection in such a way as to make a rational person prefer cooperation. The indirect effect of punishment promotes cooperation among conditional cooperators by providing the condition necessary for their cooperation (i.e., the expectation that other members will also cooperate). Here we present data from two one-shot n-person prisoner's dilemma games, demonstrating that the indirect effect of punishment complements the direct effect to increase cooperation in the game. Furthermore, we show that direct and indirect effects are robust across two forms of punishment technology: either when punishment is voluntarily provided by game players themselves or when it is exogenously provided by the experimenter.