Evolution of generous cooperative norms by cultural group selection

Evolution of cooperative norms is studied in a population where individual- and group-level selection are both in operation. Individuals play indirect reciprocity game within their group. Individuals are well informed about the previous actions and reputations, and follow second-order norms. Individuals are norm-followers, and imitate their successful group mates. In contrast to previous models where norms classify actions deterministically, we assume that norms determine only the probabilities of actions, and mutants can differ in these probabilities. The central question is how a selective cooperative norm can emerge in a population where initially only non-cooperative norms were present. It is shown that evolution leads to a cooperative state if generous cooperative strategies are dominant, although the “always defecting” and the “always cooperating”-like strategies remain stably present. The characteristics of these generous cooperative strategies and the presence of always defecting and always cooperating strategies are in concordance with experimental observations.     

Indirect reciprocity under incomplete observation

Indirect reciprocity, in which individuals help others with a good reputation but not those with a bad reputation, is a mechanism for cooperation in social dilemma situations when individuals do not repeatedly interact with the same partners. In a relatively large society where indirect reciprocity is relevant, individuals may not know each other's reputation even indirectly. Previous studies investigated the situations where individuals playing the game have to determine the action possibly without knowing others' reputations. Nevertheless, the possibility that observers of the game, who generate the reputation of the interacting players, assign reputations without complete information about them has been neglected. Because an individual acts as an interacting player and as an observer on different occasions if indirect reciprocity is endogenously sustained in a society, the incompleteness of information may affect either role. We examine the game of indirect reciprocity when the reputations of players are not necessarily known to observers and to interacting players. We find that the trustful discriminator, which cooperates with good and unknown players and defects against bad players, realizes cooperative societies under seven social norms. Among the seven social norms, three of the four suspicious norms under which cooperation (defection) to unknown players leads to a good (bad) reputation enable cooperation down to a relatively small observation probability. In contrast, the three trustful norms under which both cooperation and defection to unknown players lead to a good reputation are relatively efficient.     

Cooperating in the face of uncertainty: a consistent framework for understanding the evolution of cooperation

The evolution of cooperative behaviour, whereby individuals enhance the fitness of others at an apparent cost to themselves, represents one of the greatest paradoxes of evolution. Individuals that engage in such cooperative behaviour can, however, be favoured by natural selection if cooperative actions confer higher fitness than alternative actions. To understand the evolution of cooperative behaviour, the direct and indirect genetic benefits that individuals accrue in the present and future must be summed - this can be accomplished without any reference to the colorful vocabulary typically associated with studies of cooperation. When benefits are accrued indirectly through relatives or directly in the future individuals must be able to assess and enhance their probability of accruing those benefits and behave accordingly. We suggest that, in the same way that studies of kin recognition systems improved our understanding of how individuals assess and enhance their probability of accruing indirect benefits, studies of various forms of inheritance and reciprocation recognition systems will improve our understanding of how individuals assess and enhance their probability of accruing future benefits. Recognizing the parallel between studies of indirect fitness and future fitness, at multiple levels of analysis, will move us toward a simpler and more consistent framework for understanding the evolution of cooperative behaviour.     

Evolution of Cooperation in Spatially Structured Populations

Using a spatial lattice model of the Iterated Prisoner's Dilemma we studied the evolution of cooperation within the strategy space of all stochastic strategies with a memory of one round. Comparing the spatial model with a randomly mixed model showed that (1) there is more cooperative behaviour in a spatially structured population, (2) PAVLOV and generous variants of it are very successful strategies in the spatial context and (3) in spatially structured populations evolution is much less chaotic than in unstructured populations. In spatially structured populations, generous variants of PAVLOV are found to be very successful strategies in playing the Iterated Prisoner's Dilemma. The main weakness of PAVLOV is that it is exploitable by defective strategies. In a spatial context this disadvantage is much less important than the good error correction of PAVLOV, and especially of generous PAVLOV, because in a spatially structured population successful strategies always build clusters.     

Social learning in cooperative dilemmas

Helping is a cornerstone of social organization and commonplace in human societies. A major challenge for the evolutionary sciences is to explain how cooperation is maintained in large populations with high levels of migration, conditions under which cooperators can be exploited by selfish individuals. Cultural group selection models posit that such large-scale cooperation evolves via selection acting on populations among which behavioural variation is maintained by the cultural transmission of cooperative norms. These models assume that individuals acquire cooperative strategies via social learning. This assumption remains empirically untested. Here, I test this by investigating whether individuals employ conformist or payoff-biased learning in public goods games conducted in 14 villages of a forager–horticulturist society, the Pahari Korwa of India. Individuals did not show a clear tendency to conform or to be payoff-biased and are highly variable in their use of social learning. This variation is partly explained by both individual and village characteristics. The tendency to conform decreases and to be payoff-biased increases as the value of the modal contribution increases. These findings suggest that the use of social learning in cooperative dilemmas is contingent on individuals'' circumstances and environments, and question the existence of stably transmitted cultural norms of cooperation.     

Direct reciprocity with costly punishment: generous tit-for-tat prevails

The standard model for direct reciprocity is the repeated Prisoner's Dilemma, where in each round players choose between cooperation and defection. Here we extend the standard framework to include costly punishment. Now players have a choice between cooperation, defection and costly punishment. We study the set of all reactive strategies, where the behavior depends on what the other player has done in the previous round. We find all cooperative strategies that are Nash equilibria. If the cost of cooperation is greater than the cost of punishment, then the only cooperative Nash equilibrium is generous-tit-for-tat (GTFT), which does not use costly punishment. If the cost of cooperation is less than the cost of punishment, then there are infinitely many cooperative Nash equilibria and the response to defection can include costly punishment. We also perform computer simulations of evolutionary dynamics in populations of finite size. These simulations show that in the context of direct reciprocity, (i) natural selection prefers generous tit-for-tat over strategies that use costly punishment, and (ii) that costly punishment does not promote the evolution of cooperation. We find quantitative agreement between our simulation results and data from experimental observations.     

On learning dynamics underlying the evolution of learning rules

In order to understand the development of non-genetically encoded actions during an animal’s lifespan, it is necessary to analyze the dynamics and evolution of learning rules producing behavior. Owing to the intrinsic stochastic and frequency-dependent nature of learning dynamics, these rules are often studied in evolutionary biology via agent-based computer simulations. In this paper, we show that stochastic approximation theory can help to qualitatively understand learning dynamics and formulate analytical models for the evolution of learning rules. We consider a population of individuals repeatedly interacting during their lifespan, and where the stage game faced by the individuals fluctuates according to an environmental stochastic process. Individuals adjust their behavioral actions according to learning rules belonging to the class of experience-weighted attraction learning mechanisms, which includes standard reinforcement and Bayesian learning as special cases. We use stochastic approximation theory in order to derive differential equations governing action play probabilities, which turn out to have qualitative features of mutator-selection equations. We then perform agent-based simulations to find the conditions where the deterministic approximation is closest to the original stochastic learning process for standard 2-action 2-player fluctuating games, where interaction between learning rules and preference reversal may occur. Finally, we analyze a simplified model for the evolution of learning in a producer–scrounger game, which shows that the exploration rate can interact in a non-intuitive way with other features of co-evolving learning rules. Overall, our analyses illustrate the usefulness of applying stochastic approximation theory in the study of animal learning.     

Analysing the Unequal Effects of Positive and Negative Information on the Behaviour of Users of a Taiwanese On-Line Bulletin Board

The impact of social influence causes people to adopt the behaviour of others when interacting with other individuals. The effects of social influence can be direct or indirect. Direct social influence is the result of an individual directly influencing the opinion of another, while indirect social influence is a process taking place when an individual’s opinion and behaviour is affected by the availability of information about others’ actions. Such indirect effect may exhibit a more significant impact in the on-line community because the internet records not only positive but also negative information, for example on-line written text comments. This study focuses on indirect social influence and examines the effect of preceding information on subsequent users’ opinions by fitting statistical models to data collected from an on-line bulletin board. Specifically, the different impacts of information on approval and disapproval comments on subsequent opinions were investigated. Although in an anonymous situation where social influence is assumed to be at minimum, our results demonstrate the tendency of on-line users to adopt both positive and negative information to conform to the neighbouring trend when expressing opinions. Moreover, our results suggest unequal effects of the local approval and disapproval comments in affecting the likelihood of expressing opinions. The impact of neighbouring disapproval densities was stronger than that of neighbouring approval densities on inducing subsequent disapproval relative to approval comments. However, our results suggest no effects of global social influence on subsequent opinion expression.     

Tag-based indirect reciprocity by incomplete social information

Evolution of altruistic behaviour in interacting individuals is accounted for by, for example, kin selection, direct reciprocity, spatially limited interaction and indirect reciprocity. Real social agents, particularly humans, often take actions based on similarity between themselves and others. Although tag-based indirect reciprocity in which altruism occurs exclusively among similar flocks is a natural expectation, its mechanism has not really been established. We propose a model of tag-based indirect reciprocity by assuming that each player may note strategies of others. We show that tag-based altruism can evolve to eradicate other strategies, including unconditional defectors for various initial strategy configurations and parameter sets. A prerequisite for altruism is that the strategy is sometimes, but not always, visible to others. Without visibility of strategies, policing does not take place and defection is optimal. With perfect visibility, what a player does is always witnessed by others and cooperation is optimal. In the intermediate regime, discriminators based on tag proximity, rather than mixture of generous players and defectors, are most likely to evolve. In this situation, altruism is realized based on homophily in which players are exclusively good to similar others.     

A tale of two defectors: the importance of standing for evolution of indirect reciprocity

Indirect reciprocity occurs when the cooperative behavior between two individuals is contingent on their previous behavior toward others. Previous theoretical analysis indicates that indirect reciprocity can evolve if individuals use an image-scoring strategy. In this paper, we show that, when errors are added, indirect reciprocity cannot be based on an image-scoring strategy. However, if individuals use a standing strategy, then cooperation through indirect reciprocity is evolutionarily stable. These two strategies differ with respect to the information to which they attend. While image-scoring strategies only need attend to the actions of others, standing strategies also require information about intent. We speculate that this difference may shed light on the evolvability of indirect reciprocity. Additionally, we show that systems of indirect reciprocity are highly sensitive to the availability of information. Finally, we present a model which shows that if indirect reciprocity were to evolve, selection should also favor trusting behavior in relations between strangers.     

Generalized reciprocity in rats

The evolution of cooperation among nonrelatives has been explained by direct, indirect, and strong reciprocity. Animals should base the decision to help others on expected future help, which they may judge from past behavior of their partner. Although many examples of cooperative behavior exist in nature where reciprocity may be involved, experimental evidence for strategies predicted by direct reciprocity models remains controversial; and indirect and strong reciprocity have been found only in humans so far. Here we show experimentally that cooperative behavior of female rats is influenced by prior receipt of help, irrespective of the identity of the partner. Rats that were trained in an instrumental cooperative task (pulling a stick in order to produce food for a partner) pulled more often for an unknown partner after they were helped than if they had not received help before. This alternative mechanism, called generalized reciprocity, requires no specific knowledge about the partner and may promote the evolution of cooperation among unfamiliar nonrelatives.     

Personal experience and reputation interact in human decisions to help reciprocally

There is ample evidence that human cooperative behaviour towards other individuals is often conditioned on information about previous interactions. This information derives both from personal experience (direct reciprocity) and from experience of others (i.e. reputation; indirect reciprocity). Direct and indirect reciprocity have been studied separately, but humans often have access to both types of information. Here, we experimentally investigate information use in a repeated helping game. When acting as donor, subjects can condition their decisions to help recipients with both types of information at a small cost to access such information. We find that information from direct interactions weighs more heavily in decisions to help, and participants tend to react less forgivingly to negative personal experience than to negative reputation. Moreover, effects of personal experience and reputation interact in decisions to help. If a recipient''s reputation is positive, the personal experience of the donor has a weak effect on the decision to help, and vice versa. Yet if the two types of information indicate conflicting signatures of helpfulness, most decisions to help follow personal experience. To understand the roles of direct and indirect reciprocity in human cooperation, they should be studied in concert, not in isolation.     

Deception Undermines the Stability of Cooperation in Games of Indirect Reciprocity

Indirect reciprocity is often claimed as one of the key mechanisms of human cooperation. It works only if there is a reputational score keeping and each individual can inform with high probability which other individuals were good or bad in the previous round. Gossip is often proposed as a mechanism that can maintain such coherence of reputations in the face of errors of transmission. Random errors, however, are not the only source of uncertainty in such situations. The possibility of deceptive communication, where the signallers aim to misinform the receiver cannot be excluded. While there is plenty of evidence for deceptive communication in humans the possibility of deception is not yet incorporated into models of indirect reciprocity. Here we show that when deceptive strategies are allowed in the population it will cause the collapse of the coherence of reputations and thus in turn it results the collapse of cooperation. This collapse is independent of the norms and the cost and benefit values. It is due to the fact that there is no selection for honest communication in the framework of indirect reciprocity. It follows that indirect reciprocity can be only proposed plausibly as a mechanism of human cooperation if additional mechanisms are specified in the model that maintains honesty.     

Dog’s Discrimination of Human Selfish and Generous Attitudes: The Role of Individual Recognition,Experience, and Experimenters’ Gender

Discrimination of and memory for others’ generous and selfish behaviors could be adaptive abilities in social animals. Dogs have seemingly expressed such skills in both direct and indirect interactions with humans. However, recent studies suggest that their capacity may rely on cues other than people’s individual characteristics, such as the place where the person stands. Thus, the conditions under which dogs recognize individual humans when solving cooperative tasks still remains unclear. With the aim of contributing to this problem, we made dogs interact with two human experimenters, one generous (pointed towards the food, gave ostensive cues, and allowed the dog to eat it) and the other selfish (pointed towards the food, but ate it before the dog could have it). Then subjects could choose between them (studies 1-3). In study 1, dogs took several training trials to learn the discrimination between the generous and the selfish experimenters when both were of the same gender. In study 2, the discrimination was learned faster when the experimenters were of different gender as evidenced both by dogs’ latencies to approach the bowl in training trials as well as by their choices in preference tests. Nevertheless, dogs did not get confused by gender when the experimenters were changed in between the training and the choice phase in study 3. We conclude that dogs spontaneously used human gender as a cue to discriminate between more and less cooperative experimenters. They also relied on some other personal feature which let them avoid being confused by gender when demonstrators were changed. We discuss these results in terms of dogs’ ability to recognize individuals and the potential advantage of this skill for their lives in human environments.     

From P-Values to Objective Probabilities in Assessing Medical Treatments

The assessment of the effectiveness of a treatment in a clinical trial, depends on calculating p-values. However, p-values are only indirect and partial indicators of a genuine effect. Particularly in situations where publication bias is very likely, assessment using a p-value of 0.05 may not be sufficiently cautious. In other situations it seems reasonable to believe that assessment based on p-values may be unduly conservative. Assessments could be improved by using prior information. This implies using a Bayesian approach to take account of prior probability. However, the use of prior information in the form of expert opinion can allow bias. A method is given here that applies to assessments already included or likely to be included in the Cochrane Collaboration, excluding those reviews concerning new drugs. This method uses prior information and a Bayesian approach, but the prior information comes not from expert opinion but simply from the distribution of effectiveness apparent in a random sample of summary statistics in the Cochrane Collaboration. The method takes certain types of summary statistics and their confidence intervals and with the help of a graph, translates this into probabilities that the treatments being trialled are effective.     

Facial expressions as honest signals of cooperative intent in a one-shot anonymous Prisoner's Dilemma game

Previous research has posited that facial expressions of emotion may serve as honest signals of cooperation. Although findings from several empirical studies support this position, prior studies have not used comprehensive and dynamic measures of facial expression as potential predictors of behaviorally defined cooperation. The authors investigated (a) specific positive and negative facial actions displayed among strangers immediately following verbal promises of commitment within an unrestricted acquaintance period and (b) anonymous, behaviorally defined decisions of cooperation or defection in a one-shot, two-person Prisoner's Dilemma game occurring directly following the acquaintance period. The Facial Action Coding System [Ekman P. & Friesen W.V. (1978). Facial Action Coding System. Palo Alto, CA: Consulting Psychology Press] was used to measure affect-related facial actions. It was found that facial actions related to enjoyment were predictive of cooperative decisions within dyads; additionally, facial actions related to contempt were predictive of noncooperative decisions within dyads. Furthermore, and consistent with previous works, participants were able to accurately predict their partner's decisions following the acquaintance period. These results suggest that facial actions may function as honest signals of cooperative intent. These findings also provide a possible explanation for the association between subjective affective experience and facial expression that advances understanding of cooperative behavior.     

A hidden Markov model to identify and adjust for selection bias: an example involving mixed migration strategies

An important assumption in observational studies is that sampled individuals are representative of some larger study population. Yet, this assumption is often unrealistic. Notable examples include online public‐opinion polls, publication biases associated with statistically significant results, and in ecology, telemetry studies with significant habitat‐induced probabilities of missed locations. This problem can be overcome by modeling selection probabilities simultaneously with other predictor–response relationships or by weighting observations by inverse selection probabilities. We illustrate the problem and a solution when modeling mixed migration strategies of northern white‐tailed deer (Odocoileus virginianus). Captures occur on winter yards where deer migrate in response to changing environmental conditions. Yet, not all deer migrate in all years, and captures during mild years are more likely to target deer that migrate every year (i.e., obligate migrators). Characterizing deer as conditional or obligate migrators is also challenging unless deer are observed for many years and under a variety of winter conditions. We developed a hidden Markov model where the probability of capture depends on each individual's migration strategy (conditional versus obligate migrator), a partially latent variable that depends on winter severity in the year of capture. In a 15‐year study, involving 168 white‐tailed deer, the estimated probability of migrating for conditional migrators increased nonlinearly with an index of winter severity. We estimated a higher proportion of obligates in the study cohort than in the population, except during a span of 3 years surrounding back‐to‐back severe winters. These results support the hypothesis that selection biases occur as a result of capturing deer on winter yards, with the magnitude of bias depending on the severity of winter weather. Hidden Markov models offer an attractive framework for addressing selection biases due to their ability to incorporate latent variables and model direct and indirect links between state variables and capture probabilities.     

Strategies for cellular decision‐making

Stochasticity pervades life at the cellular level. Cells receive stochastic signals, perform detection and transduction with stochastic biochemistry, and grow and die in stochastic environments. Here we review progress in going from the molecular details to the information‐processing strategies cells use in their decision‐making. Such strategies are fundamentally influenced by stochasticity. We argue that the cellular decision‐making can only be probabilistic and occurs at three levels. First, cells must infer from noisy signals the probable current and anticipated future state of their environment. Second, they must weigh the costs and benefits of each potential response, given that future. Third, cells must decide in the presence of other, potentially competitive, decision‐makers. In this context, we discuss cooperative responses where some individuals can appear to sacrifice for the common good. We believe that decision‐making strategies will be conserved, with comparatively few strategies being implemented by different biochemical mechanisms in many organisms. Determining the strategy of a decision‐making network provides a potentially powerful coarse‐graining that links systems and evolutionary biology to understand biological design.     

Coevolution of trustful buyers and cooperative sellers in the trust game

Many online marketplaces enjoy great success. Buyers and sellers in successful markets carry out cooperative transactions even if they do not know each other in advance and a moral hazard exists. An indispensable component that enables cooperation in such social dilemma situations is the reputation system. Under the reputation system, a buyer can avoid transacting with a seller with a bad reputation. A transaction in online marketplaces is better modeled by the trust game than other social dilemma games, including the donation game and the prisoner's dilemma. In addition, most individuals participate mostly as buyers or sellers; each individual does not play the two roles with equal probability. Although the reputation mechanism is known to be able to remove the moral hazard in games with asymmetric roles, competition between different strategies and population dynamics of such a game are not sufficiently understood. On the other hand, existing models of reputation-based cooperation, also known as indirect reciprocity, are based on the symmetric donation game. We analyze the trust game with two fixed roles, where trustees (i.e., sellers) but not investors (i.e., buyers) possess reputation scores. We study the equilibria and the replicator dynamics of the game. We show that the reputation mechanism enables cooperation between unacquainted buyers and sellers under fairly generous conditions, even when such a cooperative equilibrium coexists with an asocial equilibrium in which buyers do not buy and sellers cheat. In addition, we show that not many buyers may care about the seller's reputation under cooperative equilibrium. Buyers' trusting behavior and sellers' reputation-driven cooperative behavior coevolve to alleviate the social dilemma.     

The continuous Prisoner's dilemma: II. Linear reactive strategies with noise.

We present a general model for the Continuous Prisoner's Dilemma and study the effect of errors. We find that cooperative strategies that can resist invasion by defectors are optimistic (make high initial offers), generous (always offer more cooperation than the partner did in the previous round) and uncompromising (offer full cooperation only if the partner does). A necessary condition for the emergence of cooperation in the continuous Prisoner's Dilemma with noise is b (1-p)>c, where b and c denote, respectively, the benefit and cost of cooperation, while p is the error rate. This relation can be reformulated as an error threshold: cooperation can only emerge if the probability of making a mistake is below a critical value. We note, however, that cooperation in the continuous Prisoner's Dilemma with noise does not seem to be evolutionarily stable: while it is possible to find cooperative strategies that resist invasion by defectors, such cooperators are generally invaded by more cooperative strategies which eventually yield to defectors. Thus, the long-term evolution of the continuous Prisoner's Dilemma is either characterized by unending cycles or by stable polymorphisms of cooperators and defectors.