Cognitive and Motivational Requirements for the Emergence of Cooperation in a Rat Social Game

Background

Game theory and the Prisoner''s Dilemma (PD) game in particular, which captures the paradox of cooperative interactions that lead to benefits but entail costs to the interacting individuals, have constituted a powerful tool in the study of the mechanisms of reciprocity. However, in non-human animals most tests of reciprocity in PD games have resulted in sustained defection strategies. As a consequence, it has been suggested that under such stringent conditions as the PD game humans alone have evolved the necessary cognitive abilities to engage in reciprocity, namely, numerical discrimination, memory and control of temporal discounting.

Methodology/Principal Findings

We use an iterated PD game to test rats (Rattus norvegicus) for the presence of such cognitive abilities by manipulating the strategy of the opponent, Tit-for-Tat and Pseudo-Random, or the relative size of the temptation to defect. We found that rats shape their behaviour according to the opponent''s strategy and the relative outcome resulting from cooperative or defective moves. Finally, we show that the behaviour of rats is contingent upon their motivational state (hungry versus sated).

Conclusions/Significance

Here we show that rats understand the payoff matrix of the PD game and the strategy of the opponent. Importantly, our findings reveal that rats possess the necessary cognitive capacities for reciprocity-based cooperation to emerge in the context of a prisoner''s dilemma. Finally, the validation of the rat as a model to study reciprocity-based cooperation during the PD game opens new avenues of research in experimental neuroscience.     

Personal experience and reputation interact in human decisions to help reciprocally

There is ample evidence that human cooperative behaviour towards other individuals is often conditioned on information about previous interactions. This information derives both from personal experience (direct reciprocity) and from experience of others (i.e. reputation; indirect reciprocity). Direct and indirect reciprocity have been studied separately, but humans often have access to both types of information. Here, we experimentally investigate information use in a repeated helping game. When acting as donor, subjects can condition their decisions to help recipients with both types of information at a small cost to access such information. We find that information from direct interactions weighs more heavily in decisions to help, and participants tend to react less forgivingly to negative personal experience than to negative reputation. Moreover, effects of personal experience and reputation interact in decisions to help. If a recipient''s reputation is positive, the personal experience of the donor has a weak effect on the decision to help, and vice versa. Yet if the two types of information indicate conflicting signatures of helpfulness, most decisions to help follow personal experience. To understand the roles of direct and indirect reciprocity in human cooperation, they should be studied in concert, not in isolation.     

Direct Reciprocity in Spatial Populations Enhances R-Reciprocity As Well As ST-Reciprocity

As is well-known, spatial reciprocity plays an important role in facilitating the emergence of cooperative traits, and the effect of direct reciprocity is also obvious for explaining the cooperation dynamics. However, how the combination of these two scenarios influences cooperation is still unclear. In the present work, we study the evolution of cooperation in 2×2 games via considering both spatial structured populations and direct reciprocity driven by the strategy with 1-memory length. Our results show that cooperation can be significantly facilitated on the whole parameter plane. For prisoner''s dilemma game, cooperation dominates the system even at strong dilemma, where maximal social payoff is still realized. In this sense, R-reciprocity forms and it is robust to the extremely strong dilemma. Interestingly, when turning to chicken game, we find that ST-reciprocity is also guaranteed, through which social average payoff and cooperation is greatly enhanced. This reciprocity mechanism is supported by mean-field analysis and different interaction topologies. Thus, our study indicates that direct reciprocity in structured populations can be regarded as a more powerful factor for the sustainability of cooperation.     

Reciprocity and communication of partner quality

In a cooperative exchange, the size of a partner''s contribution is likely to depend both on the partner''s ability to supply help and on the partner''s need for help in return. Referring to such needs and abilities as aspects of partner quality, it follows that variation in the amount of help offered in a relationship could transmit information about partner quality. A plausible behaviour might then be to vary the investment in a partner according to available information about partner quality and to invest little in a partner who offers little in return. Thus, regulation of a relationship through communication of partner quality would tend to follow the principle of reciprocity. In an analysis of an iterated game where players have private information about their needs and abilities, I verify this possibility by describing an evolutionarily stable state space strategy, referred to as ''state-dependent reciprocity'', entailing communication of partner quality. Although the evolution of cooperation has been studied in great detail, there has been no previous analysis of communication of needs and abilities in a relationship. It may well be that such communication is of major importance for the evolution of cooperative behaviour in nature.     

A friend in need is a friend indeed: Need-based sharing,rather than cooperative assortment,predicts experimental resource transfers among Agta hunter-gatherers

Despite much theorizing, the evolutionary reasons why humans cooperate extensively with unrelated individuals are still largely unknown. While reciprocity explains many instances of non-kin cooperation, much remains to be understood. A recent suite of models based upon ‘cooperative assortativity’ suggest that non-kin cooperation can evolve if individuals preferentially assort with certain cooperative phenotypes, such as helping those who help others. Here, we test these assortative hypotheses among the Agta, a population of Filipino hunter-gatherers, using an experimental resource allocation game in which individuals divide resources between themselves and camp-mates. Individuals preferentially shared with less cooperative individuals, arguing against cooperative assortativity as a mechanism sustaining resource transfers in this population. Rather, sharing was often based on the recipient's level of need, in addition to kin-based transfers and reciprocal sharing. Contrary to several recent theoretical accounts, in this real-world setting we find no evidence for cooperative assortativity influencing patterns of cooperation. These results may reflect the demands of living in a foraging ecology characterized by high resource stochasticity, necessitating need-based sharing as a system of long-term reciprocity to mitigate repeated subsistence shortfalls.     

Population structure,spite, and the iterated Prisoner's Dilemma

Evolutionary stability (sensu Maynard Smith: Evolution and the Theory of Games, Cambridge: Cambridge University Press, 1982) of TIT FOR TAT (TFT) under the social ecology of the iterated Prisoner's Dilemma is a function of the number of pure TFT groups (dyads) in the population, relative to the social position of a focal invading defector. Defecting against TFT always raises the defector's relative intragroup fitness; when Axelrod's (Am. Polit. Sci. Rev. 75:306–318, 1981; The Evolution of Cooperation. New York: Basic Books, 1984) Evolutionary stable strategy (ESS) conditions are met, defection also lowers the absolute fitness of the defector. Here the retaliatory (punishing) character of TFT converts defection into spite, permitting pure TFT groups to sufficiently outproduce the defector for the latter's evolutionary suppression. Increasing the relative impact of spiteful defection on a population lowers the range of evolutionary stability for TFT. When individuals participate in multiple dyads, those participating in the greatest number of dyads are most likely to provide a vehicle for the successful invasion of defection. Within social networks, ESS conditions for TFT are thus individual specific. This logic is generalized to the context of an interated n-person Prisoner's Dilemma, providing a cooperative solution conceptually identical with TFT in the two-person game.     

Cooperation beyond the dyad: on simple models and a complex society

Players in Axelrod and Hamilton''s model of cooperation were not only in a Prisoner''s Dilemma, but by definition, they were also trapped in a dyad. But animals are rarely so restricted and even the option to interact with third parties allows individuals to escape from the Prisoner''s Dilemma into a much more interesting and varied world of cooperation, from the apparently rare ‘parcelling’ to the widespread phenomenon of market effects. Our understanding of by-product mutualism, pseudo-reciprocity and the snowdrift game is also enriched by thinking ‘beyond the dyad’. The concepts of by-product mutualism and pseudo-reciprocity force us to think again about our basic definitions of cooperative behaviour (behaviour by a single individual) and cooperation (the outcome of an interaction between two or more individuals). Reciprocity is surprisingly rare outside of humans, even among large-brained ‘intelligent’ birds and mammals. Are humans unique in having extensive cooperative interactions among non-kin and an integrated cognitive system for mediating reciprocity? Perhaps, but our best chance for finding a similar phenomenon may be in delphinids, which also live in large societies with extensive cooperative interactions among non-relatives. A system of nested male alliances in bottlenose dolphins illustrates the potential and difficulties of finding a complex system of cooperation close to our own.     

Free-Riding Behavior in Vaccination Decisions: An Experimental Study

Individual decision-making regarding vaccination may be affected by the vaccination choices of others. As vaccination produces externalities reducing transmission of a disease, it can provide an incentive for individuals to be free-riders who benefit from the vaccination of others while avoiding the cost of vaccination. This study examined an individual''s decision about vaccination in a group setting for a hypothetical disease that is called “influenza” using a computerized experimental game. In the game, interactions with others are allowed. We found that higher observed vaccination rate within the group during the previous round of the game decreased the likelihood of an individual''s vaccination acceptance, indicating the existence of free-riding behavior. The free-riding behavior was observed regardless of parameter conditions on the characteristics of the influenza and vaccine. We also found that other predictors of vaccination uptake included an individual''s own influenza exposure in previous rounds increasing the likelihood of vaccination acceptance, consistent with existing empirical studies. Influenza prevalence among other group members during the previous round did not have a statistically significant effect on vaccination acceptance in the current round once vaccination rate in the previous round was controlled for.     

Cooperators benefit through reputation-based partner choice in economic games

Explaining unconditional cooperation, such as donations to charities or contributions to public goods, continues to present a problem. One possibility is that cooperation can pay through developing a reputation that makes one more likely to be chosen for a profitable cooperative partnership, a process termed competitive altruism (CA) or reputation-based partner choice. Here, we show, to our knowledge, for the first time, that investing in a cooperative reputation can bring net benefits through access to more cooperative partners. Participants played a public goods game (PGG) followed by an opportunity to select a partner for a second cooperative game. We found that those who gave more in the PGG were more often selected as desired partners and received more in the paired cooperative game. Reputational competition was even stronger when it was possible for participants to receive a higher payoff from partner choice. The benefits of being selected by a more cooperative partner outweighed the costs of cooperation in the reputation building phase. CA therefore provides an alternative to indirect reciprocity as an explanation for reputation-building behaviour. Furthermore, while indirect reciprocity depends upon individuals giving preference to those of good standing, CA can explain unconditional cooperation.     

Evolutionary game theory meets social science: Is there a unifying rule for human cooperation?

Evolutionary game theory has shown that human cooperation thrives in different types of social interactions with a PD structure. Models treat the cooperative strategies within the different frameworks as discrete entities and sometimes even as contenders. Whereas strong reciprocity was acclaimed as superior to classic reciprocity for its ability to defeat defectors in public goods games, recent experiments and simulations show that costly punishment fails to promote cooperation in the IR and DR games, where classic reciprocity succeeds. My aim is to show that cooperative strategies across frameworks are capable of a unified treatment, for they are governed by a common underlying rule or norm. An analysis of the reputation and action rules that govern some representative cooperative strategies both in models and in economic experiments confirms that the different frameworks share a conditional action rule and several reputation rules. The common conditional rule contains an option between costly punishment and withholding benefits that provides alternative enforcement methods against defectors. Depending on the framework, individuals can switch to the appropriate strategy and method of enforcement. The stability of human cooperation looks more promising if one mechanism controls successful strategies across frameworks.     

Feel good,do good? Disentangling reciprocity from unconditional prosociality

Direct and generalised reciprocity can establish evolutionarily stable levels of cooperation among unrelated individuals, with animals reciprocating help based on whether they have been helped by a social partner before. It has been argued that the actual cooperative act by a social partner may be of minor importance for seemingly reciprocal cooperation and that a mere positive experience might suffice to enhance helpful behaviour towards a conspecific (‘feel good, do good’). However, this effect could easily be exploited by defectors free‐riding on an individual's enhanced propensity to cooperate after an unspecific positive experience, without investing in reciprocity themselves. Here, we use female Norway rats (Rattus norvegicus) to test if a positive experience that was not provided by a helping partner increases the propensity to subsequently help a social partner. We manipulated the experience of test subjects by providing them with treats, either in the presence or absence of a conspecific. Thereafter, we assessed whether they produce treats and if so, how many, for an unfamiliar social partner compared to a situation in which they had not received treats before. As the treats the test subject received had not been provided by a social partner even if the partner was present, we predicted that the rats should not be more cooperative after they had received treats than if they had not. Indeed, the helping behaviour of rats was apparently not influenced by prior experience made either in a social or non‐social context. Rats have been shown previously to perform both direct and generalised reciprocity in the same variant of the iterated prisoner's dilemma game. Our results suggest that this behaviour cannot be explained by an unspecific positive experience. The decision to help a social partner seems to be contingent on previously receiving help from a social partner (reciprocity), not on any positive experience (unconditional prosociality).     

ASSORTMENT AND THE EVOLUTION OF GENERALIZED RECIPROCITY

Reciprocity is often invoked to explain cooperation. Reciprocity is cognitively demanding, and both direct and indirect reciprocity require that individuals store information about the propensity of their partners to cooperate. By contrast, generalized reciprocity, wherein individuals help on the condition that they received help previously, only relies on whether an individual received help in a previous encounter. Such anonymous information makes generalized reciprocity hard to evolve in a well‐mixed population, as the strategy will lose out to pure defectors. Here we analyze a model for the evolution of generalized reciprocity, incorporating assortment of encounters, to investigate the conditions under which it will evolve. We show that, in a well‐mixed population, generalized reciprocity cannot evolve. However, incorporating assortment of encounters can favor the evolution of generalized reciprocity in which indiscriminate cooperation and defection are both unstable. We show that generalized reciprocity can evolve under both the prisoner's dilemma and the snowdrift game.     

Coevolution of trustful buyers and cooperative sellers in the trust game

Many online marketplaces enjoy great success. Buyers and sellers in successful markets carry out cooperative transactions even if they do not know each other in advance and a moral hazard exists. An indispensable component that enables cooperation in such social dilemma situations is the reputation system. Under the reputation system, a buyer can avoid transacting with a seller with a bad reputation. A transaction in online marketplaces is better modeled by the trust game than other social dilemma games, including the donation game and the prisoner's dilemma. In addition, most individuals participate mostly as buyers or sellers; each individual does not play the two roles with equal probability. Although the reputation mechanism is known to be able to remove the moral hazard in games with asymmetric roles, competition between different strategies and population dynamics of such a game are not sufficiently understood. On the other hand, existing models of reputation-based cooperation, also known as indirect reciprocity, are based on the symmetric donation game. We analyze the trust game with two fixed roles, where trustees (i.e., sellers) but not investors (i.e., buyers) possess reputation scores. We study the equilibria and the replicator dynamics of the game. We show that the reputation mechanism enables cooperation between unacquainted buyers and sellers under fairly generous conditions, even when such a cooperative equilibrium coexists with an asocial equilibrium in which buyers do not buy and sellers cheat. In addition, we show that not many buyers may care about the seller's reputation under cooperative equilibrium. Buyers' trusting behavior and sellers' reputation-driven cooperative behavior coevolve to alleviate the social dilemma.     

Humans reciprocate by discriminating against group peers

Motivated by cycles of intergroup revenge in real-world conflicts, we experimentally test the hypothesis that humans practice group-based reciprocity: if someone harms or helps them, they harm or help other members of that person's group. Subjects played a trust game, then allocated money between other people. Senders whose partners returned more in the trust game gave more to that partner's group members. The effect was about half as large as the effect of direct reciprocity. Receivers' allocations to group members were not affected by their partners' play in the trust game, suggesting that group reciprocity was only triggered by strong norm violations. We discuss the role of group reciprocity in conflict among early humans.     

Partner switching can favour cooperation in a biological market

Intraspecific cooperation and interspecific mutualisms can be promoted by mechanisms that reduce the frequency with which cooperative organisms are exploited by unhelpful partners. One such mechanism consists of changing partners after interacting with an uncooperative individual. I used McNamara et al.'s (Nature, 451, 2008, 189) partner switching model as a framework to examine whether this mechanism can select for increased cooperative investment by house sparrows (Passer domesticus) collaborating to rear offspring; previous research on this species has shown that substantial cooperative investments by both pair members are required to achieve high pay‐offs from collaborating. I found that the poorer the outcome of a breeding attempt relative to the number of eggs the female invested, the greater the likelihood of partner switching. The incidence of partner switching changed seasonally, with peak switching coinciding with an increase in the number of alternative partners available to females. After females switched partners, their breeding outcomes rose to match those of females that remained with the same partner; this was not the case for males that switched partners. Consistent with the model's prediction, males in stable partnerships achieved over 25% higher than average reproductive success, which was attributable to both persistently good breeding outcomes and their older partners' high fecundity. These results provide empirical support for the hypothesis that partner switching favours increased cooperative investment levels, and they demonstrate that variation in the relative value of by‐product benefits can enhance that process.     

Fear or greed? Oxytocin regulates inter-individual conflict by enhancing fear in men

People may choose non-cooperation in social dilemmas either out of fear (if others choose to defect) or out of greed (when others choose to cooperate). Previous studies have shown that exogenous oxytocin motivates a “tend and defend” pattern in inter-group conflict in which oxytocin stimulates in-group cooperation and out-group defense. Using a double-blind placebo-controlled design combined with a modified Prisoner's dilemma game (PDG), we examined the effect of oxytocin on social motivations in inter-individual conflict in men. Results showed that compared with the placebo group, oxytocin-exposed participants were less cooperative in general. Specifically, oxytocin amplified the effect of fear on defection but did not influence the effect of greed. Another non-social control study confirmed participants' decisions were sensitive to social factors. Our findings suggest that even when social group conflict is removed, oxytocin promotes distrust of strangers in “me and you” inter-individual conflict by elevating social fear in men.     

Altruism may arise from individual selection

The fact that humans cooperate with non-kin in large groups, or with people they will never meet again, is a long-standing evolutionary puzzle. Altruism, the capacity to perform costly acts that confer benefits on others, is at the core of cooperative behavior. Behavioral experiments show that humans have a predisposition to cooperate with others and to punish non-cooperators at personal cost (so-called strong reciprocity) which, according to standard evolutionary game theory arguments, cannot arise from selection acting on individuals. This has led to the suggestion of group and cultural selection as the only mechanisms that can explain the evolutionary origin of human altruism. We introduce an agent-based model inspired on the Ultimatum Game, that allows us to go beyond the limitations of standard evolutionary game theory and show that individual selection can indeed give rise to strong reciprocity. Our results are consistent with the existence of neural correlates of fairness and in good agreement with observations on humans and monkeys.     

Indirect reciprocity under incomplete observation

Indirect reciprocity, in which individuals help others with a good reputation but not those with a bad reputation, is a mechanism for cooperation in social dilemma situations when individuals do not repeatedly interact with the same partners. In a relatively large society where indirect reciprocity is relevant, individuals may not know each other's reputation even indirectly. Previous studies investigated the situations where individuals playing the game have to determine the action possibly without knowing others' reputations. Nevertheless, the possibility that observers of the game, who generate the reputation of the interacting players, assign reputations without complete information about them has been neglected. Because an individual acts as an interacting player and as an observer on different occasions if indirect reciprocity is endogenously sustained in a society, the incompleteness of information may affect either role. We examine the game of indirect reciprocity when the reputations of players are not necessarily known to observers and to interacting players. We find that the trustful discriminator, which cooperates with good and unknown players and defects against bad players, realizes cooperative societies under seven social norms. Among the seven social norms, three of the four suspicious norms under which cooperation (defection) to unknown players leads to a good (bad) reputation enable cooperation down to a relatively small observation probability. In contrast, the three trustful norms under which both cooperation and defection to unknown players lead to a good reputation are relatively efficient.     

Evolution of generous cooperative norms by cultural group selection

Evolution of cooperative norms is studied in a population where individual- and group-level selection are both in operation. Individuals play indirect reciprocity game within their group. Individuals are well informed about the previous actions and reputations, and follow second-order norms. Individuals are norm-followers, and imitate their successful group mates. In contrast to previous models where norms classify actions deterministically, we assume that norms determine only the probabilities of actions, and mutants can differ in these probabilities. The central question is how a selective cooperative norm can emerge in a population where initially only non-cooperative norms were present. It is shown that evolution leads to a cooperative state if generous cooperative strategies are dominant, although the “always defecting” and the “always cooperating”-like strategies remain stably present. The characteristics of these generous cooperative strategies and the presence of always defecting and always cooperating strategies are in concordance with experimental observations.     

Transforming the dilemma

How does natural selection lead to cooperation between competing individuals? The Prisoner's Dilemma captures the essence of this problem. Two players can either cooperate or defect. The payoff for mutual cooperation, R, is greater than the payoff for mutual defection, P. But a defector versus a cooperator receives the highest payoff, T, where as the cooperator obtains the lowest payoff, S. Hence, the Prisoner's Dilemma is defined by the payoff ranking T > R > P > S . In a well‐mixed population, defectors always have a higher expected payoff than cooperators, and therefore natural selection favors defectors. The evolution of cooperation requires specific mechanisms. Here we discuss five mechanisms for the evolution of cooperation: direct reciprocity, indirect reciprocity, kin selection, group selection, and network reciprocity (or graph selection). Each mechanism leads to a transformation of the Prisoner's Dilemma payoff matrix. From the transformed matrices, we derive the fundamental conditions for the evolution of cooperation. The transformed matrices can be used in standard frameworks of evolutionary dynamics such as the replicator equation or stochastic processes of game dynamics in finite populations.