Coveting thy neighbors fitness as a means to resolve social dilemmas

In spatial evolutionary games the fitness of each individual is traditionally determined by the payoffs it obtains upon playing the game with its neighbors. Since defection yields the highest individual benefits, the outlook for cooperators is gloomy. While network reciprocity promotes collaborative efforts, chances of averting the impending social decline are slim if the temptation to defect is strong. It is, therefore, of interest to identify viable mechanisms that provide additional support for the evolution of cooperation. Inspired by the fact that the environment may be just as important as inheritance for individual development, we introduce a simple switch that allows a player to either keep its original payoff or use the average payoff of all its neighbors. Depending on which payoff is higher, the influence of either option can be tuned by means of a single parameter. We show that, in general, taking into account the environment promotes cooperation. Yet coveting the fitness of one's neighbors too strongly is not optimal. In fact, cooperation thrives best only if the influence of payoffs obtained in the traditional way is equal to that of the average payoff of the neighborhood. We present results for the prisoner's dilemma and the snowdrift game, for different levels of uncertainty governing the strategy adoption process, and for different neighborhood sizes. Our approach outlines a viable route to increased levels of cooperative behavior in structured populations, but one that requires a thoughtful implementation.     

Evolution of Cooperation in Spatial Traveler's Dilemma Game

Traveler''s dilemma (TD) is one of social dilemmas which has been well studied in the economics community, but it is attracted little attention in the physics community. The TD game is a two-person game. Each player can select an integer value between and () as a pure strategy. If both of them select the same value, the payoff to them will be that value. If the players select different values, say and (), then the payoff to the player who chooses the small value will be and the payoff to the other player will be . We term the player who selects a large value as the cooperator, and the one who chooses a small value as the defector. The reason is that if both of them select large values, it will result in a large total payoff. The Nash equilibrium of the TD game is to choose the smallest value . However, in previous behavioral studies, players in TD game typically select values that are much larger than , and the average selected value exhibits an inverse relationship with . To explain such anomalous behavior, in this paper, we study the evolution of cooperation in spatial traveler''s dilemma game where the players are located on a square lattice and each player plays TD games with his neighbors. Players in our model can adopt their neighbors'' strategies following two standard models of spatial game dynamics. Monte-Carlo simulation is applied to our model, and the results show that the cooperation level of the system, which is proportional to the average value of the strategies, decreases with increasing until is greater than the critical value where cooperation vanishes. Our findings indicate that spatial reciprocity promotes the evolution of cooperation in TD game and the spatial TD game model can interpret the anomalous behavior observed in previous behavioral experiments.     

Adaptive Dynamics of Extortion and Compliance

Direct reciprocity is a mechanism for the evolution of cooperation. For the iterated prisoner’s dilemma, a new class of strategies has recently been described, the so-called zero-determinant strategies. Using such a strategy, a player can unilaterally enforce a linear relationship between his own payoff and the co-player’s payoff. In particular the player may act in such a way that it becomes optimal for the co-player to cooperate unconditionally. In this way, a player can manipulate and extort his co-player, thereby ensuring that the own payoff never falls below the co-player’s payoff. However, using a compliant strategy instead, a player can also ensure that his own payoff never exceeds the co-player’s payoff. Here, we use adaptive dynamics to study when evolution leads to extortion and when it leads to compliance. We find a remarkable cyclic dynamics: in sufficiently large populations, extortioners play a transient role, helping the population to move from selfish strategies to compliance. Compliant strategies, however, can be subverted by altruists, which in turn give rise to selfish strategies. Whether cooperative strategies are favored in the long run critically depends on the size of the population; we show that cooperation is most abundant in large populations, in which case average payoffs approach the social optimum. Our results are not restricted to the case of the prisoners dilemma, but can be extended to other social dilemmas, such as the snowdrift game. Iterated social dilemmas in large populations do not lead to the evolution of strategies that aim to dominate their co-player. Instead, generosity succeeds.     

Transforming the dilemma

How does natural selection lead to cooperation between competing individuals? The Prisoner's Dilemma captures the essence of this problem. Two players can either cooperate or defect. The payoff for mutual cooperation, R, is greater than the payoff for mutual defection, P. But a defector versus a cooperator receives the highest payoff, T, where as the cooperator obtains the lowest payoff, S. Hence, the Prisoner's Dilemma is defined by the payoff ranking T > R > P > S . In a well‐mixed population, defectors always have a higher expected payoff than cooperators, and therefore natural selection favors defectors. The evolution of cooperation requires specific mechanisms. Here we discuss five mechanisms for the evolution of cooperation: direct reciprocity, indirect reciprocity, kin selection, group selection, and network reciprocity (or graph selection). Each mechanism leads to a transformation of the Prisoner's Dilemma payoff matrix. From the transformed matrices, we derive the fundamental conditions for the evolution of cooperation. The transformed matrices can be used in standard frameworks of evolutionary dynamics such as the replicator equation or stochastic processes of game dynamics in finite populations.     

A Theoretical Analysis of Temporal Difference Learning in the Iterated Prisoner’s Dilemma Game

Direct reciprocity is a chief mechanism of mutual cooperation in social dilemma. Agents cooperate if future interactions with the same opponents are highly likely. Direct reciprocity has been explored mostly by evolutionary game theory based on natural selection. Our daily experience tells, however, that real social agents including humans learn to cooperate based on experience. In this paper, we analyze a reinforcement learning model called temporal difference learning and study its performance in the iterated Prisoner’s Dilemma game. Temporal difference learning is unique among a variety of learning models in that it inherently aims at increasing future payoffs, not immediate ones. It also has a neural basis. We analytically and numerically show that learners with only two internal states properly learn to cooperate with retaliatory players and to defect against unconditional cooperators and defectors. Four-state learners are more capable of achieving a high payoff against various opponents. Moreover, we numerically show that four-state learners can learn to establish mutual cooperation for sufficiently small learning rates.     

A study on emergence of alternating reciprocity in a 2 x 2 game with 2-length memory strategy

It is recognized that bilateral cooperation (C), a reward-state in other words, emergently comes up in a 2 × 2 prisoner's dilemma game, if you assume a strategy set with a memory concept. Also observed is a mixture state of cooperation (C) and defect (D), saint- and temptation-state in other words, to obtain a higher payoff than R (R reciprocity) in a hero or leader game that is a chicken-type dilemma game; this phenomenon is called alternating reciprocity (AR) or ST reciprocity. Observing a holistic 2 × 2 game world including trivial games and various dilemma games, where 2-length memory and infinite interactions are assumed, the paper reports on the specific mechanism of AR. It is observed there are three different phases relating to AR, which can be explained by the stress of the dilemma.     

Evolutionary dynamics of continuous strategy games on graphs and social networks under weak selection

Understanding the emergence of cooperation among selfish individuals has been a long-standing puzzle, which has been studied by a variety of game models. Most previous studies presumed that interactions between individuals are discrete, but it seems unrealistic in real systems. Recently, there are increasing interests in studying game models with a continuous strategy space. Existing research work on continuous strategy games mainly focuses on well-mixed populations. Especially, little theoretical work has been conducted on their evolutionary dynamics in a structured population. In the previous work (Zhong et al., BioSystems, 2012), we showed that under strong selection, continuous and discrete strategies have significantly different equilibrium and game dynamics in spatially structured populations. In this paper, we further study evolutionary dynamics of continuous strategy games under weak selection in structured populations. By using the fixation probability based stochastic dynamics, we derive exact conditions of natural selection favoring cooperation for the death–birth updating scheme. We also present a network gain decomposition of the game equilibrium, which might provide a new view of the network reciprocity in a quantitative way. Finally, we make a detailed comparison between games using discrete and continuous strategies. As compared to the former, we find that for the latter (i) the same selection conditions are derived for the general 2 × 2 game; especially, the rule b/c > k in a simplified Prisoner's Dilemma is valid as well; however, (ii) for a coordination game, interestingly, the risk-dominant strategy is disfavored. Numerical simulations have also been conducted to validate our results.     

A simple scaling of the effectiveness of supporting mutual cooperation in donor-recipient games by various reciprocity mechanisms

In donor-recipient games (DRG), one of the sub-classes of Prisoner's Dilemma (PD), it is well-known that a game structure is described by two parameters benefit (b) and cost (c) of cooperation. By means of a series of numerical experiments, we proved that the effectiveness of supporting mutual cooperation in DRG by various reciprocity mechanisms can be expressed in a single game structural parameter, b/c. This also implies that the dilemma strength in various donor-recipient games with various reciprocity mechanisms can be evaluated only by b/c, which is consistent with the previous novel finding by Nowak. It was also discussed whether this kind of parameterization idea can be extended to general games in PD game class.     

Repeated games and direct reciprocity under active linking

Direct reciprocity relies on repeated encounters between the same two individuals. Here we examine the evolution of cooperation under direct reciprocity in dynamically structured populations. Individuals occupy the vertices of a graph, undergoing repeated interactions with their partners via the edges of the graph. Unlike the traditional approach to evolutionary game theory, where individuals meet at random and have no control over the frequency or duration of interactions, we consider a model in which individuals differ in the rate at which they seek new interactions. Moreover, once a link between two individuals has formed, the productivity of this link is evaluated. Links can be broken off at different rates. Whenever the active dynamics of links is sufficiently fast, population structure leads to a simple transformation of the payoff matrix, effectively changing the game under consideration, and hence paving the way for reciprocators to dominate defectors. We derive analytical conditions for evolutionary stability.     

Cognitive and Motivational Requirements for the Emergence of Cooperation in a Rat Social Game

Background

Game theory and the Prisoner''s Dilemma (PD) game in particular, which captures the paradox of cooperative interactions that lead to benefits but entail costs to the interacting individuals, have constituted a powerful tool in the study of the mechanisms of reciprocity. However, in non-human animals most tests of reciprocity in PD games have resulted in sustained defection strategies. As a consequence, it has been suggested that under such stringent conditions as the PD game humans alone have evolved the necessary cognitive abilities to engage in reciprocity, namely, numerical discrimination, memory and control of temporal discounting.

Methodology/Principal Findings

We use an iterated PD game to test rats (Rattus norvegicus) for the presence of such cognitive abilities by manipulating the strategy of the opponent, Tit-for-Tat and Pseudo-Random, or the relative size of the temptation to defect. We found that rats shape their behaviour according to the opponent''s strategy and the relative outcome resulting from cooperative or defective moves. Finally, we show that the behaviour of rats is contingent upon their motivational state (hungry versus sated).

Conclusions/Significance

Here we show that rats understand the payoff matrix of the PD game and the strategy of the opponent. Importantly, our findings reveal that rats possess the necessary cognitive capacities for reciprocity-based cooperation to emerge in the context of a prisoner''s dilemma. Finally, the validation of the rat as a model to study reciprocity-based cooperation during the PD game opens new avenues of research in experimental neuroscience.     

Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game

Understanding the mechanisms that can lead to the evolution of cooperation through natural selection is a core problem in biology. Among the various attempts at constructing a theory of cooperation, game theory has played a central role. Here, we review models of cooperation that are based on two simple games: the Prisoner's Dilemma, and the Snowdrift game. Both games are two‐person games with two strategies, to cooperate and to defect, and both games are social dilemmas. In social dilemmas, cooperation is prone to exploitation by defectors, and the average payoff in populations at evolutionary equilibrium is lower than it would be in populations consisting of only cooperators. The difference between the games is that cooperation is not maintained in the Prisoner's Dilemma, but persists in the Snowdrift game at an intermediate frequency. As a consequence, insights gained from studying extensions of the two games differ substantially. We review the most salient results obtained from extensions such as iteration, spatial structure, continuously variable cooperative investments, and multi‐person interactions. Bridging the gap between theoretical and empirical research is one of the main challenges for future studies of cooperation, and we conclude by pointing out a number of promising natural systems in which the theory can be tested experimentally.     

Joint Effects of Asymmetric Payoff and Reciprocity Mechanisms on Collective Cooperation in Water Sharing Interactions: A Game Theoretic Perspective

Common-pool resource (CPR) dilemmas distinguish themselves from general public good problems by encompassing both social and physical features. This paper examines how a physical mechanism, namely asymmetric payoff; and a social mechanism, reciprocity; simultaneously affect collective cooperation in theoretical water sharing interactions. We present an iterative N-person game theoretic model to investigate the joint effects of these two mechanisms in a linear fully connected river system under three information assumptions. From a simple evolutionary perspective, this paper quantitatively addresses the conditions for Nash Equilibrium in which collective cooperation might be established. The results suggest that direct reciprocity increases every actor’s motivation to contribute to the collective good of the river system. Meanwhile, various upstream and downstream actors manifest individual disparities as a result of the direct reciprocity and asymmetric payoff mechanisms. More specifically, the downstream actors are less willing to cooperate unless there is a high probability that long-term interactions are ensured; however, a greater level of asymmetries is likely to increase upstream actors’ incentives to cooperate even though the interactions could quickly end. The upstream actors also display weak sensitivity to an increase in the total number of actors, which generally results in a reduction in the other actors’ motivation for cooperation. It is also shown that the indirect reciprocity mechanism relaxes the overall conditions for cooperative Nash Equilibrium.     

Direct Sum Matrix Game with Prisoner's Dilemma and Snowdrift Game

A direct sum form is proposed for constructing a composite game from two games, prisoner''s dilemma and snowdrift game. This kind of direct sum form game is called a multiple roles game. The replicator dynamics of the multiple roles game with will-mixed populations is explored. The dynamical behaviors on square lattice are investigated by numerical simulation. It is found that the dynamical behaviors of population on square lattice depend on the mixing proportion of the two simple games. Mixing SD activities to pure PD population inhibits the proportion of cooperators in PD, and mixing PD activities to pure SD population stimulates the proportion of cooperators in SD. Besides spatial reciprocity, our results show that there are roles reciprocities between different types of individuals.     

The paradox of cooperation benefits

It seems obvious that as the benefits of cooperation increase, the share of cooperators in the population should also increase. It is well known that positive assortment between cooperative types, for instance in spatially structured populations, provide better conditions for the evolution of cooperation than complete mixing. This study demonstrates that, assuming positive assortment, under most conditions higher cooperation benefits also increase the share of cooperators. On the other hand, under a specified range of payoff values, when at least two payoff parameters are modified, the reverse is true. The conditions for this paradox are determined for two-person social dilemmas: the Prisoner's Dilemma, the Hawks and Doves game, and the Stag Hunt game, assuming global selection and positive assortment.     

Evolutionary games on networks and payoff invariance under replicator dynamics

The commonly used accumulated payoff scheme is not invariant with respect to shifts of payoff values when applied locally in degree-inhomogeneous population structures. We propose a suitably modified payoff scheme and we show both formally and by numerical simulation, that it leaves the replicator dynamics invariant with respect to affine transformations of the game payoff matrix. We then show empirically that, using the modified payoff scheme, an interesting amount of cooperation can be reached in three paradigmatic non-cooperative two-person games in populations that are structured according to graphs that have a marked degree inhomogeneity, similar to actual graphs found in society. The three games are the Prisoner’s Dilemma, the Hawks-Doves and the Stag-Hunt. This confirms previous important observations that, under certain conditions, cooperation may emerge in such network-structured populations, even though standard replicator dynamics for mixing populations prescribes equilibria in which cooperation is totally absent in the Prisoner’s Dilemma, and it is less widespread in the other two games.     

ASSORTMENT AND THE EVOLUTION OF GENERALIZED RECIPROCITY

Reciprocity is often invoked to explain cooperation. Reciprocity is cognitively demanding, and both direct and indirect reciprocity require that individuals store information about the propensity of their partners to cooperate. By contrast, generalized reciprocity, wherein individuals help on the condition that they received help previously, only relies on whether an individual received help in a previous encounter. Such anonymous information makes generalized reciprocity hard to evolve in a well‐mixed population, as the strategy will lose out to pure defectors. Here we analyze a model for the evolution of generalized reciprocity, incorporating assortment of encounters, to investigate the conditions under which it will evolve. We show that, in a well‐mixed population, generalized reciprocity cannot evolve. However, incorporating assortment of encounters can favor the evolution of generalized reciprocity in which indiscriminate cooperation and defection are both unstable. We show that generalized reciprocity can evolve under both the prisoner's dilemma and the snowdrift game.     

Targeted Cooperative Actions Shape Social Networks

Individual acts of cooperation give rise to dynamic social networks. Traditionally, models for cooperation in structured populations are based on a separation of individual strategies and of population structure. Individuals adopt a strategy—typically cooperation or defection, which determines their behaviour toward their neighbours as defined by an interaction network. Here, we report a behavioural experiment that amalgamates strategies and structure to empirically investigate the dynamics of social networks. The action of paying a cost c to provide a benefit b is represented as a directed link point from the donor to the recipient. Participants can add and/or remove links to up to two recipients in each round. First, we show that dense networks emerge, where individuals are characterized by fairness: they receive to the same extent they provide. More specifically, we investigate how participants use information about the generosity and payoff of others to update their links. It turns out that aversion to payoff inequity was the most consistent update rule: adding links to individuals that are worse off and removing links to individuals that are better off. We then investigate the effect of direct reciprocation, showing that the possibility of direct reciprocation does not increase cooperation as compared to the treatment where participants are totally unaware of who is providing benefits to them.     

Strategy Selection in Evolutionary Game Dynamics on Group Interaction Networks

Evolutionary game theory provides an appropriate tool for investigating the competition and diffusion of behavioral traits in biological or social populations. A core challenge in evolutionary game theory is the strategy selection problem: Given two strategies, which one is favored by the population? Recent studies suggest that the answer depends not only on the payoff functions of strategies but also on the interaction structure of the population. Group interactions are one of the fundamental interactive modes within populations. This work aims to investigate the strategy selection problem in evolutionary game dynamics on group interaction networks. In detail, the strategy selection conditions are obtained for some typical networks with group interactions. Furthermore, the obtained conditions are applied to investigate selection between cooperation and defection in populations. The conditions for evolution of cooperation are derived for both the public goods game and volunteer’s dilemma game. Numerical experiments validate the above analytical results.     

What initially brought about communications?

This paper reports an intelligent agent equipped with two-layer finite state machines (FSMs) that can communicate by turning lighting on and off, leading to social cooperation that solves the dilemma situation, modeled by a one-shot 2x2 game. This communication between two gaming agents can be observed in hero- and leader-type dilemma games, where alternating reciprocity, repeating cooperation (C)-defeat (D) after D-C, is the equal pareto optimum instead of a sequence of mutual cooperation that is the equal pareto optimum for a prisoner's dilemma (PD) game.     

Direct reciprocity on graphs

Direct reciprocity is a mechanism for the evolution of cooperation based on the idea of repeated encounters between the same two individuals. Here we examine direct reciprocity in structured populations, where individuals occupy the vertices of a graph. The edges denote who interacts with whom. The graph represents spatial structure or a social network. For birth-death or pairwise comparison updating, we find that evolutionary stability of direct reciprocity is more restrictive on a graph than in a well-mixed population, but the condition for reciprocators to be advantageous is less restrictive on a graph. For death-birth and imitation updating, in contrast, both conditions are easier to fulfill on a graph. Moreover, for all four update mechanisms, reciprocators can dominate defectors on a graph, which is never possible in a well-mixed population. We also study the effect of an error rate, which increases with the number of links per individual; interacting with more people simultaneously enhances the probability of making mistakes. We provide analytic derivations for all results.