Variable investment, the Continuous Prisoner's Dilemma, and the origin of cooperation.

Cooperation is fundamental to many biological systems. A common metaphor for studying the evolution of cooperation is the Prisoner's Dilemma, a game with two strategies: cooperate or defect. However, cooperation is rare all or nothing, and its evolution probably involves the gradual extension of initially modest degrees of assistance. The inability of the Prisoner's Dilemma to capture this basic aspect limits its use for understanding the evolutionary origins of cooperation. Here we consider a framework for cooperation based on the concept of investment: an act which is costly, but which benefits other individuals, where the cost and benefit depend on the level of investment made. In the resulting Continuous Prisoner's Dilemma the essential problem of cooperation remains: in the absence of any additional structure non-zero levels of investment cannot evolve. However, if investments are considered in a spatially structured context, selfish individuals who make arbitrarily low investments can be invaded by higher-investing mutants. This results in the mean level of investment evolving to significant levels, where it is maintained indefinitely. This approach provides a natural solution to the fundamental problem of how cooperation gradually increases from a non-cooperative state.     

Why cooperation is not running away

A growing number of experimental and theoretical studies show the importance of partner choice as a mechanism to promote the evolution of cooperation, especially in humans. In this paper, we focus on the question of the precise quantitative level of cooperation that should evolve under this mechanism. When individuals compete to be chosen by others, their level of investment in cooperation evolves towards higher values, a process called competitive altruism, or runaway cooperation. Using a classic adaptive dynamics model, we first show that when the cost of changing partner is low, this runaway process can lead to a profitless escalation of cooperation. In the extreme, when partner choice is entirely frictionless, cooperation even increases up to a level where its cost entirely cancels out its benefit. That is, at evolutionary equilibrium, individuals gain the same payoff than if they had not cooperated at all. Second, importing models from matching theory in economics we, however, show that when individuals can plastically modulate their choosiness in function of their own cooperation level, partner choice stops being a runaway competition to outbid others and becomes a competition to form the most optimal partnerships. In this case, when the cost of changing partner tends towards zero, partner choice leads to the evolution of the socially optimum level of cooperation. This last result could explain the observation that human cooperation seems to be often constrained by considerations of social efficiency.     

The evolution of cooperation by negotiation in a noisy world

Cooperative interactions among individuals are ubiquitous despite the possibility of exploitation by selfish free riders. One mechanism that may promote cooperation is ‘negotiation’: individuals altering their behaviour in response to the behaviour of others. Negotiating individuals decide their actions through a recursive process of reciprocal observation, thereby reducing the possibility of free riding. Evolutionary games with response rules have shown that infinitely many forms of the rule can be evolutionarily stable simultaneously, unless there is variation in individual quality. This potentially restricts the conditions under which negotiation could maintain cooperation. Organisms interact with one another in a noisy world in which cooperative effort and the assessment of effort may be subject to error. Here, we show that such noise can make the number of evolutionarily stable rules finite, even without quality variation, and so noise could help maintain cooperative behaviour. We show that the curvature of the benefit function is the key factor determining whether individuals invest more or less as their partner's investment increases, investing less when the benefit to investment has diminishing returns. If the benefits of low investment are very small then behavioural flexibility tends to promote cooperation, because negotiation enables cooperators to reach large benefits. Under some conditions, this leads to a repeating cycle in which cooperative behaviour rises and falls over time, which may explain between‐population differences in cooperative behaviour. In other conditions, negotiation leads to extremely high levels of cooperative behaviour, suggesting that behavioural flexibility could facilitate the evolution of eusociality in the absence of high relatedness.     

Adaptive and bounded investment returns promote cooperation in spatial public goods games

The public goods game is one of the most famous models for studying the evolution of cooperation in sizable groups. The multiplication factor in this game can characterize the investment return from the public good, which may be variable depending on the interactive environment in realistic situations. Instead of using the same universal value, here we consider that the multiplication factor in each group is updated based on the differences between the local and global interactive environments in the spatial public goods game, but meanwhile limited to within a certain range. We find that the adaptive and bounded investment returns can significantly promote cooperation. In particular, full cooperation can be achieved for high feedback strength when appropriate limitation is set for the investment return. Also, we show that the fraction of cooperators in the whole population can become larger if the lower and upper limits of the multiplication factor are increased. Furthermore, in comparison to the traditionally spatial public goods game where the multiplication factor in each group is identical and fixed, we find that cooperation can be better promoted if the multiplication factor is constrained to adjust between one and the group size in our model. Our results highlight the importance of the locally adaptive and bounded investment returns for the emergence and dominance of cooperative behavior in structured populations.     

Dynamic social behaviour in a bacterium: Pseudomonas aeruginosa partially compensates for siderophore loss to cheats

Cooperation underlies diverse phenomena including the origins of multicellular life, human behaviour in economic markets and the mechanisms by which pathogenic bacteria cause disease. Experiments with microorganisms have advanced our understanding of how, when and why cooperation evolves, but the extent to which microbial cooperation can recapitulate aspects of animal behaviour is debated. For instance, understanding the evolution of behavioural response rules (how should one individual respond to another's decision to cooperate or defect?) is a key part of social evolution theory, but the possible existence of such rules in social microbes has not been explored. In one specific context (biparental care in animals), cooperation is maintained if individuals respond to a partner's defection by increasing their own investment into cooperation, but not so much that this fully compensates for the defector's lack of investment. This is termed ‘partial compensation’. Here, I show that partial compensation for the presence of noncooperating ‘cheats’ is also observed in a microbial social behaviour: the cooperative production of iron‐scavenging siderophores by the bacterium Pseudomonas aeruginosa. A period of evolution in the presence of cheats maintains this response, whereas evolution in the absence of cheats leads to a loss of compensatory behaviour. These results demonstrate (i) the remarkable flexibility of bacterial social behaviour, (ii) the potential generality of partial compensation as a social response rule and (iii) the need for mathematical models to explore the evolution of response rules in multi‐player social interactions.     

Evolution of cooperation by generalized reciprocity

The evolution of cooperation by direct reciprocity requires that individuals recognize their present partner and remember the outcome of their last encounter with that specific partner. Direct reciprocity thus requires advanced cognitive abilities. Here, we demonstrate that if individuals repeatedly interact within small groups with different partners in a two person Prisoner's Dilemma, cooperation can emerge and also be maintained in the absence of such cognitive capabilities. It is sufficient for an individual to base their decision of whether or not to cooperate on the outcome of their last encounter--even if it was with a different partner.     

Reputation-based conditional interaction supports cooperation in well-mixed prisoner's dilemmas

In the well-mixed prisoner's dilemma game, individuals are typically assumed to have no choice about whether to interact with other individuals in the population. In this paper, we instead consider reputation-based conditional interaction and its consequences for the evolution of cooperation. Each individual has a tolerance range, and only interacts with other individuals whose reputation lies within its tolerance range in a chosen sample of the population. Reputation contains information about the number of interaction partners an individual has just cooperated with. We find that the introduction of conditional interaction promotes cooperation in well-mixed populations, and there exist moderate tolerance ranges for which this effect is maximized. For a given tolerance range, there is a critical cost-to-benefit ratio below which cooperation can be promoted. Interestingly, we find that if cooperation evolves, different cooperators' interaction clusters are typically maintained in the population, each around a different reputation level. We further investigate some properties of these cooperators' clusters. Moreover, we examine the effects of the sample number on the evolution of cooperation. Our results highlight the importance of the detailed consideration of modes of interaction for the evolution of cooperation in well-mixed populations.     

Different reactions to adverse neighborhoods in games of cooperation

In social dilemmas, cooperation among randomly interacting individuals is often difficult to achieve. The situation changes if interactions take place in a network where the network structure jointly evolves with the behavioral strategies of the interacting individuals. In particular, cooperation can be stabilized if individuals tend to cut interaction links when facing adverse neighborhoods. Here we consider two different types of reaction to adverse neighborhoods, and all possible mixtures between these reactions. When faced with a gloomy outlook, players can either choose to cut and rewire some of their links to other individuals, or they can migrate to another location and establish new links in the new local neighborhood. We find that in general local rewiring is more favorable for the evolution of cooperation than emigration from adverse neighborhoods. Rewiring helps to maintain the diversity in the degree distribution of players and favors the spontaneous emergence of cooperative clusters. Both properties are known to favor the evolution of cooperation on networks. Interestingly, a mixture of migration and rewiring is even more favorable for the evolution of cooperation than rewiring on its own. While most models only consider a single type of reaction to adverse neighborhoods, the coexistence of several such reactions may actually be an optimal setting for the evolution of cooperation.     

Charitable giving as a signal of trustworthiness: Disentangling the signaling benefits of altruistic acts

It has been shown that psychological predispositions to benefit others can motivate human cooperation and the evolution of such social preferences can be explained with kin or multi-level selection models. It has also been shown that cooperation can evolve as a costly signal of an unobservable quality that makes a person more attractive with regard to other types of social interactions. Here we show that if a proportion of individuals with social preferences is maintained in the population through kin or multi-level selection, cooperative acts that are truly altruistic can be a costly signal of social preferences and make altruistic individuals more trustworthy interaction partners in social exchange. In a computerized laboratory experiment, we test whether altruistic behavior in the form of charitable giving is indeed correlated with trustworthiness and whether a charitable donation increases the observing agents' trust in the donor. Our results support these hypotheses and show that, apart from trust, responses to altruistic acts can have a rewarding or outcome-equalizing purpose. Our findings corroborate that the signaling benefits of altruistic acts that accrue in social exchange can ease the conditions for the evolution of social preferences.     

Cooperation can promote rescue or lead to evolutionary suicide during environmental change

The adaptation of populations to changing conditions may be affected by interactions between individuals. For example, when cooperative interactions increase fecundity, they may allow populations to maintain high densities and thus keep track of moving environmental optima. Simultaneously, changes in population density alter the marginal benefits of cooperative investments, creating a feedback loop between population dynamics and the evolution of cooperation. Here we model how the evolution of cooperation interacts with adaptation to changing environments. We hypothesize that environmental change lowers population size and thus promotes the evolution of cooperation, and that this, in turn, helps the population keep up with the moving optimum. However, we find that the evolution of cooperation can have qualitatively different effects, depending on which fitness component is reduced by the costs of cooperation. If the costs decrease fecundity, cooperation indeed speeds adaptation by increasing population density; if, in contrast, the costs decrease viability, cooperation may instead slow adaptation by lowering the effective population size, leading to evolutionary suicide. Thus, cooperation can either promote or—counterintuitively—hinder adaptation to a changing environment. Finally, we show that our model can also be generalized to other social interactions by discussing the evolution of competition during environmental change.     

LIMITED DISPERSAL, BUDDING DISPERSAL, AND COOPERATION: AN EXPERIMENTAL STUDY

Numerous theoretical studies have investigated how limited dispersal may provide an explanation for the evolution of cooperation, by leading to interactions between relatives. However, despite considerable theoretical attention, there has been a lack of empirical tests. In this article, we test how patterns of dispersal influence the evolution of cooperation, using iron-scavenging in the bacterium Pseudomonas aeruginosa as our cooperative trait. We found that relatively limited dispersal does not favor cooperation. The reason for this is that although limited dispersal increases the relatedness between interacting individuals, it also leads to increased local competition for resources between relatives. This result supports Taylor's prediction that in the simplest possible scenario, the effects of increased relatedness and local competition exactly cancel out. In contrast, we show that one way for cooperation to be favored is if individuals disperse in groups (budding dispersal), because this maintains high relatedness while reducing local competition between relatives (relatively global competition).     

The continuous prisoner's dilemma and the evolution of cooperation through reciprocal altruism with variable investment

Understanding the evolutionary origin and persistence of cooperative behavior is a fundamental biological problem. The standard "prisoner's dilemma," which is the most widely adopted framework for studying the evolution of cooperation through reciprocal altruism between unrelated individuals, does not allow for varying degrees of cooperation. Here we study the continuous iterated prisoner's dilemma, in which cooperative investments can vary continuously in each round. This game has been previously considered for a class of reactive strategies in which current investments are based on the partner's previous investment. In the standard iterated prisoner's dilemma, such strategies are inferior to strategies that take into account both players' previous moves, as is exemplified by the evolutionary dominance of "Pavlov" over "tit for tat." Consequently, we extend the analysis of the continuous prisoner's dilemma to a class of strategies in which current investments depend on previous payoffs and, hence, on both players' previous investments. We show, both analytically and by simulation, that payoff-based strategies, which embody the intuitively appealing idea that individuals invest more in cooperative interactions when they profit from these interactions, provide a natural explanation for the gradual evolution of cooperation from an initially noncooperative state and for the maintenance of cooperation thereafter.     

Density-dependent cooperation as a mechanism for persistence and coexistence

To overcome stress, such as resource limitation, an organism often needs to successfully mediate competition with other members of its own species. This may favor the evolution of defective traits that are harmful to the species population as a whole, and that may lead to its dilution or even to its extinction (the tragedy of the commons). Here, we show that this phenomenon can be circumvented by cooperation plasticity, in which an individual decides, based on environmental conditions, whether to cooperate or to defect. Specifically, we analyze the evolution of density-dependent cooperation. In our model, the population is spatially subdivided, periodically remixed, and comprises several species. We find that evolution pushes individuals to be more cooperative when their own species is at lower densities, and we show that not only could this cooperation prevent the tragedy of the commons, but it could also facilitate coexistence between many species that compete for the same resource.     

The effect of cheats on siderophore diversity in Pseudomonas aeruginosa

Cooperation can be maintained if cooperative behaviours are preferentially directed towards other cooperative individuals. Tag‐based cooperation (greenbeards) – where cooperation benefits individuals with the same tag as the actor – is one way to achieve this. Tag‐based cooperation can be exploited by individuals who maintain the specific tag but do not cooperate, and selection to escape this exploitation can result in the evolution of tag diversity. We tested key predictions crucial for the evolution of cheat‐mediated tag diversity using the production of iron‐scavenging pyoverdine by the opportunistic pathogen, Pseduomonas aeruginosa as a model system. Using two strains that produce different pyoverdine types and their respective cheats, we show that cheats outcompete their homologous pyoverdine producer, but are outcompeted by the heterologous producer in well‐mixed environments. As a consequence, co‐inoculating two types of pyoverdine producer and one type of pyoverdine cheat resulted in the pyoverdine type whose cheat was not present having a large fitness advantage. Theory suggests that in such interactions, cheats can maintain tag diversity in spatially structured environments, but that tag‐based cooperation will be lost in well‐mixed populations, regardless of tag diversity. We saw that when all pyoverdine producers and cheats were co‐inoculated in well‐mixed environments, both types of pyoverdine producers were outcompeted, whereas spatial structure (agar plates and compost microcosms), rather than maintaining diversity, resulted in the domination of one pyoverdine producer. These results suggest cheats may play a more limited role in the evolution of pyoverdine diversity than predicted.     

Evolutionary games and population dynamics: maintenance of cooperation in public goods games

The emergence and abundance of cooperation in nature poses a tenacious and challenging puzzle to evolutionary biology. Cooperative behaviour seems to contradict Darwinian evolution because altruistic individuals increase the fitness of other members of the population at a cost to themselves. Thus, in the absence of supporting mechanisms, cooperation should decrease and vanish, as predicted by classical models for cooperation in evolutionary game theory, such as the Prisoner's Dilemma and public goods games. Traditional approaches to studying the problem of cooperation assume constant population sizes and thus neglect the ecology of the interacting individuals. Here, we incorporate ecological dynamics into evolutionary games and reveal a new mechanism for maintaining cooperation. In public goods games, cooperation can gain a foothold if the population density depends on the average population payoff. Decreasing population densities, due to defection leading to small payoffs, results in smaller interaction group sizes in which cooperation can be favoured. This feedback between ecological dynamics and game dynamics can generate stable coexistence of cooperators and defectors in public goods games. However, this mechanism fails for pairwise Prisoner's Dilemma interactions and the population is driven to extinction. Our model represents natural extension of replicator dynamics to populations of varying densities.     

Sex-biased dispersal of adults mediates the evolution of altruism among juveniles

Population viscosity has been proposed as an important mechanism for the evolution of cooperation. The idea is that if individuals do not disperse far during the course of their lives, they will tend to interact with their genealogical relatives, which may give kin-selected benefits for cooperation. However, in the simplest model of population structure, the evolution of cooperation is unaffected by the rate of dispersal, owing to dispersal also mediating competition between social partners. This surprising result has generated much research interest in recent years. Here I show that dispersal does matter if there is a sex difference in dispersal rate, even when the expression of cooperation is not conditional upon the actor's dispersal status or sex. In particular, I show that cooperation among juveniles is relatively favoured when there is a small sex bias in adult dispersal in favour of the sex with the greatest variance in reproductive success, and is relatively disfavoured when this sex bias is large or in the opposite direction. This is because dispersal by individuals of each sex can have different consequences for the genetic structure of the population.     

The cancellation effect at the group level

Group selection models combine selection pressure at the individual level with selection pressure at the group level. Cooperation can be costly for individuals, but beneficial for the group, and therefore, if individuals are sufficiently much assorted, and cooperators find themselves in groups with disproportionately many other cooperators, cooperation can evolve. The existing literature on group selection generally assumes that competition between groups takes place in a well-mixed population of groups, where any group competes with any other group equally intensely. Competition between groups however might very well occur locally; groups may compete more intensely with nearby than with far-away groups. We show that if competition between groups is indeed local, then the evolution of cooperation can be hindered significantly by the fact that groups with many cooperators will mostly compete against neighboring groups that are also highly cooperative, and therefore harder to outcompete. The existing empirical method for determining how conducive a group structured population is to the evolution of cooperation also implicitly assumes global between-group competition, and therefore gives (possibly very) biased estimates.     

Partner choice creates fairness in humans

Many studies demonstrate that partner choice has played an important role in the evolution of human cooperation, but little work has tested its impact on the evolution of human fairness. In experiments involving divisions of money, people become either over-generous or over-selfish when they are in competition to be chosen as cooperative partners. Hence, it is difficult to see how partner choice could result in the evolution of fair, equal divisions. Here, we show that this puzzle can be solved if we consider the outside options on which partner choice operates. We conduct a behavioural experiment, run agent-based simulations and analyse a game-theoretic model to understand how outside options affect partner choice and fairness. All support the conclusion that partner choice leads to fairness only when individuals have equal outside options. We discuss how this condition has been met in our evolutionary history, and the implications of these findings for our understanding of other aspects of fairness less specific than preferences for equal divisions of resources.     

Resolving the iterated prisoner's dilemma: theory and reality

Pairs of unrelated individuals face a prisoner's dilemma if cooperation is the best mutual outcome, but each player does best to defect regardless of his partner's behaviour. Although mutual defection is the only evolutionarily stable strategy in one-shot games, cooperative solutions based on reciprocity can emerge in iterated games. Among the most prominent theoretical solutions are the so-called bookkeeping strategies, such as tit-for-tat, where individuals copy their partner's behaviour in the previous round. However, the lack of empirical data conforming to predicted strategies has prompted the suggestion that the iterated prisoner's dilemma (IPD) is neither a useful nor realistic basis for investigating cooperation. Here, we discuss several recent studies where authors have used the IPD framework to interpret their data. We evaluate the validity of their approach and highlight the diversity of proposed solutions. Strategies based on precise accounting are relatively uncommon, perhaps because the full set of assumptions of the IPD model are rarely satisfied. Instead, animals use a diverse array of strategies that apparently promote cooperation, despite the temptation to cheat. These include both positive and negative reciprocity, as well as long-term mutual investments based on 'friendships'. Although there are various gaps in these studies that remain to be filled, we argue that in most cases, individuals could theoretically benefit from cheating and that cooperation cannot therefore be explained with the concept of positive pseudo-reciprocity. We suggest that by incorporating empirical data into the theoretical framework, we may gain fundamental new insights into the evolution of mutual reciprocal investment in nature.     

Incentives and opportunism: from the carrot to the stick

Cooperation in public good games is greatly promoted by positive and negative incentives. In this paper, we use evolutionary game dynamics to study the evolution of opportunism (the readiness to be swayed by incentives) and the evolution of trust (the propensity to cooperate in the absence of information on the co-players). If both positive and negative incentives are available, evolution leads to a population where defectors are punished and players cooperate, except when they can get away with defection. Rewarding behaviour does not become fixed, but can play an essential role in catalysing the emergence of cooperation, especially if the information level is low.