A neural basis for social cooperation

Cooperation based on reciprocal altruism has evolved in only a small number of species, yet it constitutes the core behavioral principle of human social life. The iterated Prisoner's Dilemma Game has been used to model this form of cooperation. We used fMRI to scan 36 women as they played an iterated Prisoner's Dilemma Game with another woman to investigate the neurobiological basis of cooperative social behavior. Mutual cooperation was associated with consistent activation in brain areas that have been linked with reward processing: nucleus accumbens, the caudate nucleus, ventromedial frontal/orbitofrontal cortex, and rostral anterior cingulate cortex. We propose that activation of this neural network positively reinforces reciprocal altruism, thereby motivating subjects to resist the temptation to selfishly accept but not reciprocate favors.     

Evolution,altruism, and the prisoner's dilemma

I first argue against Peter Singer's exciting thesis that the Prisoner's Dilemma explains why there could be an evolutionary advantage in making reciprocal exchanges that are ultimately motivated by genuine altruism over making such exchanges on the basis of enlightened long-term self-interest. I then show that an alternative to Singer's thesis — one that is also meant to corroborate the view that natural selection favors genuine altruism, recently defended by Gregory Kavka, fails as well. Finally, I show that even granting Singer's and Kavka's claim about the selective advantage of altruism proper, it is doubtful whether that type of claim can be used in a particular sort of sociobiological argument against psychological egoism.     

A Prisoner's Dilemma model of the evolution of paternal care

The heavy energetic demands of gestation, lactation and rearing of offspring mean that studies of paternal care in primates usually focus on female reproductive effort. Here it is shown that both male and female reproductive effort must be considered in order to understand how paternal care evolved. This is done using the Prisoner's Dilemma, best known as a model of reciprocal altruism. It is found that the relative cost of reproduction for males and females is crucially important in determining co-operative and competitive strategies. In particular, when male reproductive costs are less than female reproductive costs, males co-operate with females even when females do not reciprocate. This surprising behaviour, termed non-reciprocal altruism, is comparable with male investment in a female and her offspring.     

Natural selection of memory-one strategies for the iterated prisoner's dilemma

In the iterated Prisoner's Dilemma, mutually cooperative behavior can become established through Darwinian natural selection. In simulated interactions of stochastic memory-one strategies for the Iterated Prisoner's Dilemma, Nowak and Sigmund discovered that cooperative agents using a Pavlov (Win-Stay Lose-Switch) type strategy eventually dominate a random population. This emergence follows more directly from a deterministic dynamical system based on differential reproductive success or natural selection. When restricted to an environment of memory-one agents interacting in iterated Prisoner's Dilemma games with a 1% noise level, the Pavlov agent is the only cooperative strategy and one of very few others that cannot be invaded by a similar strategy. Pavlov agents are trusting but no suckers. They will exploit weakness but repent if punished for cheating.     

The continuous Prisoner's dilemma: II. Linear reactive strategies with noise.

We present a general model for the Continuous Prisoner's Dilemma and study the effect of errors. We find that cooperative strategies that can resist invasion by defectors are optimistic (make high initial offers), generous (always offer more cooperation than the partner did in the previous round) and uncompromising (offer full cooperation only if the partner does). A necessary condition for the emergence of cooperation in the continuous Prisoner's Dilemma with noise is b (1-p)>c, where b and c denote, respectively, the benefit and cost of cooperation, while p is the error rate. This relation can be reformulated as an error threshold: cooperation can only emerge if the probability of making a mistake is below a critical value. We note, however, that cooperation in the continuous Prisoner's Dilemma with noise does not seem to be evolutionarily stable: while it is possible to find cooperative strategies that resist invasion by defectors, such cooperators are generally invaded by more cooperative strategies which eventually yield to defectors. Thus, the long-term evolution of the continuous Prisoner's Dilemma is either characterized by unending cycles or by stable polymorphisms of cooperators and defectors.     

The Hermaphrodite's Dilemma

Given sexual conflict, mating encounters between simultaneous hermaphrodites will conform to a new, conditional, non-zero sum game of strategy, the Hermaphrodite's Dilemma; special cases of which include Prisoner's Dilemma and Game of Chicken. The model predicts that hermaphrodite mating systems will be based on reciprocation with cheating in a preferred sexual role. This model suggests that study of hermaphrodite mating systems will provide direct evidence for the existence of sexual conflict and suggests that sexual conflict may act to stabilize hermaphroditism through such mating systems.     

Variable investment, the Continuous Prisoner's Dilemma, and the origin of cooperation.

Cooperation is fundamental to many biological systems. A common metaphor for studying the evolution of cooperation is the Prisoner's Dilemma, a game with two strategies: cooperate or defect. However, cooperation is rare all or nothing, and its evolution probably involves the gradual extension of initially modest degrees of assistance. The inability of the Prisoner's Dilemma to capture this basic aspect limits its use for understanding the evolutionary origins of cooperation. Here we consider a framework for cooperation based on the concept of investment: an act which is costly, but which benefits other individuals, where the cost and benefit depend on the level of investment made. In the resulting Continuous Prisoner's Dilemma the essential problem of cooperation remains: in the absence of any additional structure non-zero levels of investment cannot evolve. However, if investments are considered in a spatially structured context, selfish individuals who make arbitrarily low investments can be invaded by higher-investing mutants. This results in the mean level of investment evolving to significant levels, where it is maintained indefinitely. This approach provides a natural solution to the fundamental problem of how cooperation gradually increases from a non-cooperative state.     

Evolution of Cooperation in Spatially Structured Populations

Using a spatial lattice model of the Iterated Prisoner's Dilemma we studied the evolution of cooperation within the strategy space of all stochastic strategies with a memory of one round. Comparing the spatial model with a randomly mixed model showed that (1) there is more cooperative behaviour in a spatially structured population, (2) PAVLOV and generous variants of it are very successful strategies in the spatial context and (3) in spatially structured populations evolution is much less chaotic than in unstructured populations. In spatially structured populations, generous variants of PAVLOV are found to be very successful strategies in playing the Iterated Prisoner's Dilemma. The main weakness of PAVLOV is that it is exploitable by defective strategies. In a spatial context this disadvantage is much less important than the good error correction of PAVLOV, and especially of generous PAVLOV, because in a spatially structured population successful strategies always build clusters.     

A mechanism of dynamical interactions for two-person social dilemmas

We propose a new mechanism of interactions between game-theoretical agents in which the weights of the connections between interacting individuals are dynamical, payoff-dependent variables. Their evolution depends on the difference between the payoff of the agents from a given type of encounter and their average payoff. The mechanism is studied in the frame of two models: agents distributed on a random graph, and a mean field model. Symmetric and asymmetric connections between the agents are introduced. Long time behavior of both systems is discussed for the Prisoner's Dilemma and the Snow Drift games.     

Human cooperation in social dilemmas: comparing the Snowdrift game with the Prisoner's Dilemma

Explaining the evolution of cooperation among non-relatives is one of the major challenges for evolutionary biology. In this study, we experimentally examined human cooperation in the iterated Snowdrift game (ISD), which has received little attention so far, and compared it with human cooperation in the iterated Prisoner's Dilemma (IPD), which has become the paradigm for the evolution of cooperation. We show that iteration in the ISD leads to consistently higher levels of cooperation than in the IPD. We further demonstrate that the most successful strategies known for the IPD (generous Tit-for-Tat and Pavlov) were also successfully used in the ISD. Interestingly, we found that female players cooperated significantly more often than male players in the IPD but not in the ISD. Moreover, female players in the IPD applied Tit-for-Tat-like or Pavlovian strategies significantly more often than male players, thereby achieving significantly higher pay-offs than male players did. These data demonstrate that the willingness to cooperate does not only depend on the type of the social dilemma, but also on the class of individuals involved. Altogether, our study shows that the ISD can potentially explain high levels of cooperation among non-relatives in humans. In addition, the ISD seems to reflect the social dilemma more realistically than the IPD because individuals obtain immediate direct benefits from the cooperative acts they perform and costs of cooperation are shared between cooperators.     

Optimality under noise: higher memory strategies for the alternating prisoner's dilemma.

The Alternating Prisoner's Dilemma is a variant of the iterated Prisoner's Dilemma in which the players alternate in the roles of actor and recipient. We searched for strategies which are "optimal" in the Alternating Prisoner's Dilemma with noise (a non-zero probability that a player's decision will be transmitted incorrectly). In order to achieve success against a variety of other strategies, a strategy must be "self-cooperating" (able to achieve mutual cooperation with its clone), "C-exploiting" (able to exploit unconditional cooperators), and "D-unexploitable" (able to resist exploitation by defectors). It must also have high evolutionary "dominance", a general measure of evolutionary performance which considers both resistance to invasion and the ability to invade other strategies. A strategy which meets these optimality criteria can evolve cooperation by invading a population of defectors and establishing a stable cooperative society. Most of the strategies commonly discussed in the Alternating Prisoner's Dilemma literature are low-memory strategies such as Tit For Tat, Pavlov, and Firm But Fair, but none of these strategies can simultaneously meet all of the optimality criteria. However, we discovered a class of higher memory "Firm Pavlov" strategies, which not only meet our stringent optimality criteria, but also achieve remarkable success in round-robin tournaments and evolutionary interactions. These higher memory strategies are friendly enough to cooperate with their clone, pragmatic enough to exploit unconditional cooperators, and wary enough to resist exploitation by defectors: they are truly "optimal under noise" in the Alternating Prisoner's Dilemma.     

Network fluctuations hinder cooperation in evolutionary games

In this paper we study the influence of random network fluctuations on the behavior of evolutionary games on Barabási-Albert networks. This network class has been shown to promote cooperation on social dilemmas such as the Prisoner's Dilemma and the Snowdrift games when the population network is fixed. Here we introduce exogenous random fluctuations of the network links through several noise models, and we investigate the evolutionary dynamics comparing them with the known static network case. The results we obtain show that even a moderate amount of random noise on the network links causes a significant loss of cooperation, to the point that cooperation vanishes altogether in the Prisoner's Dilemma when the noise rate is the same as the agents' strategy revision rate. The results appear to be robust since they are essentially the same whatever the type of the exogenous noise. Besides, it turns out that random network noise is more important than strategy noise in suppressing cooperation. Thus, even in the more favorable situation of accumulated payoff in which links have no cost, the mere presence of random external network fluctuations act as a powerful limitation to the attainment of high levels of cooperation.     

Evolutionary games and population dynamics: maintenance of cooperation in public goods games

The emergence and abundance of cooperation in nature poses a tenacious and challenging puzzle to evolutionary biology. Cooperative behaviour seems to contradict Darwinian evolution because altruistic individuals increase the fitness of other members of the population at a cost to themselves. Thus, in the absence of supporting mechanisms, cooperation should decrease and vanish, as predicted by classical models for cooperation in evolutionary game theory, such as the Prisoner's Dilemma and public goods games. Traditional approaches to studying the problem of cooperation assume constant population sizes and thus neglect the ecology of the interacting individuals. Here, we incorporate ecological dynamics into evolutionary games and reveal a new mechanism for maintaining cooperation. In public goods games, cooperation can gain a foothold if the population density depends on the average population payoff. Decreasing population densities, due to defection leading to small payoffs, results in smaller interaction group sizes in which cooperation can be favoured. This feedback between ecological dynamics and game dynamics can generate stable coexistence of cooperators and defectors in public goods games. However, this mechanism fails for pairwise Prisoner's Dilemma interactions and the population is driven to extinction. Our model represents natural extension of replicator dynamics to populations of varying densities.     

Coexistence of cooperation and defection in public goods games

The production of public goods by the contribution of individual volunteers is a social dilemma because an individual that does not volunteer can benefit from the public good produced by the contributions of others. Therefore it is generally believed that public goods can be produced only in the presence of repeated interactions (which allow reciprocation, reputation effects and punishment) or relatedness (kin selection). Cooperation, however, often occurs in the absence of iterations and relatedness. We show that when the production of a public good is a Volunteer's Dilemma, in which a fixed number of cooperators is necessary to produce the public good, cooperators and defectors persist in a mixed equilibrium, without iterations and without relatedness. This mixed equilibrium is absent in the N-person Prisoner's Dilemma, in which the public good is a linear function of the individual contributions. We also show that the Prisoner's Dilemma and the Volunteer's Dilemma are the two opposite extremes of a general public goods game, and that all intermediate cases can have a mixed equilibrium like the Volunteer's Dilemma. The coexistence of cooperators and defectors, therefore, is a typical outcome of most social dilemmas, which requires neither relatedness nor iterations.     

A two-trial two-strategy conflict

Iterated conflicts allow the possibility of co-operative-like behaviour in games such as the Prisoner's Dilemma. The present paper is an attempt to initiate the study of iterated conflicts when, (a) the number of iterations is fixed and finite and (b) the underlying payoff matrix is general, e.g. a mixed Evolutionary Stable Strategy (ESS) could occur in the non-iterated coflict. These assumptions are in contrast to the Iterated Prisoner's Dilemma. We consider a somewhat special case which none the less produces results of an interesting nature. For those cases where there is no internal ESS in the one trial case the two-trial case is easily resolved. When the former has an internal ESS then the two-trial case yields two ESSs whose supports are a partition of the space of strategies.     

Behavior-dependent contexts for repeated plays of the Prisoner's Dilemma: II. Dynamical aspects of the evolution of cooperation

Iterated Prisoner's Dilemma models are proposed in which, at any trial, the probability of staying in the game depends on the outcome of the previous trial. If a player's choice depends on its own play (cooperate or defect) at the previous trial, it becomes possible for cooperative strategies to increase when rare in a population of egoists. A dynamic analysis is used to demonstrate that stable polymorphisms may result, and may involve more strategies than just Tit-for-Tat and all-Defect. The tendency for clustering among like strategists to enhance their initial increase when rare is also explored dynamically.     

Random mobility and spatial structure often enhance cooperation

The effects of an unconditional move rule in the spatial Prisoner's Dilemma, Snowdrift and Stag Hunt games are studied. Spatial structure by itself is known to modify the outcome of many games when compared with a randomly mixed population, sometimes promoting, sometimes inhibiting cooperation. Here we show that random dilution and mobility may suppress the inhibiting factors of the spatial structure in the Snowdrift game, while enhancing the already larger cooperation found in the Prisoner's dilemma and Stag Hunt games.     

Altruism,tit for tat and ‘outlaw’ genes

Summary In a prior study we combined game theory and inclusive fitness models to examine whether the guarded altruism that can evolve among non-relatives (tit for tat, TFT) might also evolve among close relatives, supplanting unconditional altruism. In most cases, TFT replaced unconditional altruism in family-structured models. Even when TFT is selected at a single locus, however, by withholding altruism from non-reciprocating relatives it may qualify as an outlaw from the standpoint of modifier genes at other loci. Here we examine this possibility with a series of haploid, two-locus models in which a modifier gene transforms TFT into unconditional altruism. The modifier allele spreads to fixation whenever Hamilton's Rule is satisfied, resulting in an unconditional altruist replacing the TFT strategy. As such, TFT may be regarded as an outlaw vulnerable to suppression by alleles at other loci.     

Oscillations in the evolution of reciprocity

A game-theoretical analysis of the Iterated Prisoner's Dilemma shows that the evolution of ensembles of stochastic strategies displays a dynamics of high complexity and unpredictability.     

Evolution of coordinated alternating reciprocity in repeated dyadic games

A genetic algorithm incorporating mutation and crossing-over was used to investigate the evolution of social behaviour in repeated Prisoner's Dilemma, Chicken (Hawk-Dove), Battle of the Sexes, and Leader games. The results show that the strategic structure of an interaction has a crucial determining effect on the type of social behaviour that evolves. In particular, simulations using repeated Prisoner's Dilemma and Chicken (Hawk-Dove) games lead to the emergence of genes coding for symmetric reciprocity and the evolution of mutual cooperation, whereas simulations using repeated Battle of the Sexes and Leader games lead to near-fixation of genes coding for asymmetric strategic choices and the evolution of coordinated alternating reciprocity. A mechanism is suggested whereby, in games with asymmetric equilibrium points, coordinated alternating reciprocity might evolve without insight or communication between players.