Strategy Selection in Evolutionary Game Dynamics on Group Interaction Networks

Evolutionary game theory provides an appropriate tool for investigating the competition and diffusion of behavioral traits in biological or social populations. A core challenge in evolutionary game theory is the strategy selection problem: Given two strategies, which one is favored by the population? Recent studies suggest that the answer depends not only on the payoff functions of strategies but also on the interaction structure of the population. Group interactions are one of the fundamental interactive modes within populations. This work aims to investigate the strategy selection problem in evolutionary game dynamics on group interaction networks. In detail, the strategy selection conditions are obtained for some typical networks with group interactions. Furthermore, the obtained conditions are applied to investigate selection between cooperation and defection in populations. The conditions for evolution of cooperation are derived for both the public goods game and volunteer’s dilemma game. Numerical experiments validate the above analytical results.     

COEVOLUTION AS AN EVOLUTIONARY GAME

Coevolution is modeled as a continuous game where the fitness-maximizing strategy of an individual is assumed to be a function of the strategy of other individuals who are also under selection to maximize fitness. An evolutionary stable strategy (ESS) is sought such that no rare alternative strategies can invade the community. The approach can be used to model coevolution because the ESS may be composed of a coalition of more than one strategy. This work, by modeling frequency-dependent selection, extends the approach of Roughgarden (1976) which only considered density-dependent selection. In particular, we show that the coevolutionary model of Rummel and Roughgarden (1985) does contain frequency-dependent selection, and thus, their application of Roughgarden's criterion for evolutionary stability to a model for which it is not applicable leads to the erroneous conclusion that the ecological and evolutionary processes are in conflict. The utility of the game theoretic approach is illustrated by two examples. The first considers an ESS composed of a single strategy, the second an ESS composed of a coalition of two strategies. Evolution occurs on a frequency-dependent adaptive landscape. For this reason, the approach is appropriate for modeling competitive speciation (Rosenzweig, 1978). Also, the game theoretic approach is designed to combine the interplay between the background environment (including the biotic components) and the evolutionary potential of the populations or organisms. The actual application of this theory will require knowledge of both.     

Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game

Understanding the mechanisms that can lead to the evolution of cooperation through natural selection is a core problem in biology. Among the various attempts at constructing a theory of cooperation, game theory has played a central role. Here, we review models of cooperation that are based on two simple games: the Prisoner's Dilemma, and the Snowdrift game. Both games are two‐person games with two strategies, to cooperate and to defect, and both games are social dilemmas. In social dilemmas, cooperation is prone to exploitation by defectors, and the average payoff in populations at evolutionary equilibrium is lower than it would be in populations consisting of only cooperators. The difference between the games is that cooperation is not maintained in the Prisoner's Dilemma, but persists in the Snowdrift game at an intermediate frequency. As a consequence, insights gained from studying extensions of the two games differ substantially. We review the most salient results obtained from extensions such as iteration, spatial structure, continuously variable cooperative investments, and multi‐person interactions. Bridging the gap between theoretical and empirical research is one of the main challenges for future studies of cooperation, and we conclude by pointing out a number of promising natural systems in which the theory can be tested experimentally.     

The sociobiology of sex: inclusive fitness consequences of inter-sexual interactions

The diversity of social interactions between sexual partners has long captivated biologists, and its evolution has been interpreted largely in terms of 'direct fitness' pay-offs to partners and their descendants. Inter-sexual interactions also have 'indirect effects' by affecting the fitness of relatives, with important consequences for inclusive fitness. However, inclusive fitness arguments have received limited consideration in this context, and definitions of 'direct' and 'indirect' fitness effects in this field are often inconsistent with those of inclusive fitness theory. Here, we use a sociobiology approach based on inclusive fitness theory to distinguish between direct and indirect fitness effects. We first consider direct effects: we review how competition leads to sexual conflict, and discuss the conditions under which repression of competition fosters sexual mutualism. We then clarify indirect effects, and show that greenbeard effects, kin recognition and population viscosity can all lead to episodes of indirect selection on sexual interactions creating potential for sexual altruism and spite. We argue that the integration of direct and indirect fitness effects within a sociobiology approach enables us to consider a more diverse spectrum of evolutionary outcomes of sexual interactions, and may help resolving current debates over sexual selection and sexual conflict.     

MATHEMATICS OF KIN‐ AND GROUP‐SELECTION: FORMALLY EQUIVALENT?

Evolutionary game theory is a general mathematical framework that describes the evolution of social traits. This framework forms the basis of many multilevel selection models and is also frequently used to model evolutionary dynamics on networks. Kin selection, which was initially restricted to describe social interactions between relatives, has also led to a broader mathematical approach, inclusive fitness, that can not only describe social evolution among relatives, but also in group structured populations or on social networks. It turns out that the underlying mathematics of game theory is fundamentally different from the approach of inclusive fitness. Thus, both approaches—evolutionary game theory and inclusive fitness—can be helpful to understand the evolution of social traits in group structured or spatially extended populations.     

Bipartite graphs as models of population structures in evolutionary multiplayer games

By combining evolutionary game theory and graph theory, "games on graphs" study the evolutionary dynamics of frequency-dependent selection in population structures modeled as geographical or social networks. Networks are usually represented by means of unipartite graphs, and social interactions by two-person games such as the famous prisoner's dilemma. Unipartite graphs have also been used for modeling interactions going beyond pairwise interactions. In this paper, we argue that bipartite graphs are a better alternative to unipartite graphs for describing population structures in evolutionary multiplayer games. To illustrate this point, we make use of bipartite graphs to investigate, by means of computer simulations, the evolution of cooperation under the conventional and the distributed N-person prisoner's dilemma. We show that several implicit assumptions arising from the standard approach based on unipartite graphs (such as the definition of replacement neighborhoods, the intertwining of individual and group diversity, and the large overlap of interaction neighborhoods) can have a large impact on the resulting evolutionary dynamics. Our work provides a clear example of the importance of construction procedures in games on graphs, of the suitability of bigraphs and hypergraphs for computational modeling, and of the importance of concepts from social network analysis such as centrality, centralization and bipartite clustering for the understanding of dynamical processes occurring on networked population structures.     

The concurrent evolution of cooperation and the population structures that support it

The evolution of cooperation often depends upon population structure, yet nearly all models of cooperation implicitly assume that this structure remains static. This is a simplifying assumption, because most organisms possess genetic traits that affect their population structure to some degree. These traits, such as a group size preference, affect the relatedness of interacting individuals and hence the opportunity for kin or group selection. We argue that models that do not explicitly consider their evolution cannot provide a satisfactory account of the origin of cooperation, because they cannot explain how the prerequisite population structures arise. Here, we consider the concurrent evolution of genetic traits that affect population structure, with those that affect social behavior. We show that not only does population structure drive social evolution, as in previous models, but that the opportunity for cooperation can in turn drive the creation of population structures that support it. This occurs through the generation of linkage disequilibrium between socio-behavioral and population-structuring traits, such that direct kin selection on social behavior creates indirect selection pressure on population structure. We illustrate our argument with a model of the concurrent evolution of group size preference and social behavior.     

Evolution and the classification of social behavior

Recent studies in the evolution of cooperation have shifted focus from altruistic to mutualistic cooperation. This change in focus is purported to reveal new explanations for the evolution of prosocial behavior. We argue that the common classification scheme for social behavior used to distinguish between altruistic and mutualistic cooperation is flawed because it fails to take into account dynamically relevant game-theoretic features. This leads some arguments about the evolution of cooperation to conflate dynamical scenarios that differ regarding the basic conditions on the emergence and maintenance of cooperation. We use the tools of evolutionary game theory to increase the resolution of the classification scheme and analyze what evolutionary inferences classifying social behavior can license.     

Negotiation of mutualism: rhizobia and legumes

The evolution and persistence of biological cooperation have been an important puzzle in evolutionary theory. Here, we suggest a new approach based on bargaining theory to tackle the question. We present a mechanistic model for negotiation of benefits between a nitrogen-fixing nodule and a legume plant. To that end, we first derive growth rates for the nodule and plant from metabolic models of each as a function of material fluxes between them. We use these growth rates as pay-off functions in the negotiation process, which is analogous to collective bargaining between a firm and a workers' union. Our model predicts that negotiations lead to the Nash bargaining solution, maximizing the product of players' pay-offs. This work introduces elements of cooperative game theory into the field of mutualistic interactions. In the discussion of the paper, we argue for the benefits of such an approach in studying the question of biological cooperation.     

Evolution of interactions and cooperation in the spatial prisoner's dilemma game

We study the evolution of cooperation in the spatial prisoner's dilemma game where players are allowed to establish new interactions with others. By employing a simple coevolutionary rule entailing only two crucial parameters, we find that different selection criteria for the new interaction partners as well as their number vitally affect the outcome of the game. The resolution of the social dilemma is most probable if the selection favors more successful players and if their maximally attainable number is restricted. While the preferential selection of the best players promotes cooperation irrespective of game parametrization, the optimal number of new interactions depends somewhat on the temptation to defect. Our findings reveal that the "making of new friends" may be an important activity for the successful evolution of cooperation, but also that partners must be selected carefully and their number limited.     

Similar preferences for ornamentation in opposite‐ and same‐sex choice experiments

Selection due to social interactions comprises competition over matings (sexual selection stricto sensu) plus other forms of social competition and cooperation. Sexual selection explains sex differences in ornamentation and in various other phenotypes, but does not easily explain cases where those phenotypes are similar in males and females. Understanding such similarities requires knowing how phenotypes influence nonsexual social interactions as well, which can be very important in gregarious animals, but whose role for phenotypic evolution has been overlooked. For example, ‘mate choice’ experiments often found preferences for ornamentation, but have not assessed whether those are strictly sexual or are general social preferences. Using choice experiments with a gregarious and mutually ornamented finch, the common waxbill (Estrilda astrild), we show that preferences for ornamentation in the opposite‐sex also extend to same‐sex interactions. Waxbills discriminated between opposite‐ and same‐sex individuals, but most preferences for colour traits were similar when interacting with either sex. Similar preferences in sexual and nonsexual associations may be widespread in nature, either as social adaptations or as by‐product of mate preferences. In either case, such preferences may set the stage for the evolution of mutual ornamentation and of various other similarities between the sexes.     

Sexual networks: measuring sexual selection in structured,polyandrous populations

Sexual selection is traditionally measured at the population level, assuming that populations lack structure. However, increasing evidence undermines this approach, indicating that intrasexual competition in natural populations often displays complex patterns of spatial and temporal structure. This complexity is due in part to the degree and mechanisms of polyandry within a population, which can influence the intensity and scale of both pre- and post-copulatory sexual competition. Attempts to measure selection at the local and global scale have been made through multi-level selection approaches. However, definitions of local scale are often based on physical proximity, providing a rather coarse measure of local competition, particularly in polyandrous populations where the local scale of pre- and post-copulatory competition may differ drastically from each other. These limitations can be solved by social network analysis, which allows us to define a unique sexual environment for each member of a population: ‘local scale’ competition, therefore, becomes an emergent property of a sexual network. Here, we first propose a novel quantitative approach to measure pre- and post-copulatory sexual selection, which integrates multi-level selection with information on local scale competition derived as an emergent property of networks of sexual interactions. We then use simple simulations to illustrate the ways in which polyandry can impact estimates of sexual selection. We show that for intermediate levels of polyandry, the proposed network-based approach provides substantially more accurate measures of sexual selection than the more traditional population-level approach. We argue that the increasing availability of fine-grained behavioural datasets provides exciting new opportunities to develop network approaches to study sexual selection in complex societies.     

Cooperation under indirect reciprocity and imitative trust

Indirect reciprocity, a key concept in behavioral experiments and evolutionary game theory, provides a mechanism that allows reciprocal altruism to emerge in a population of self-regarding individuals even when repeated interactions between pairs of actors are unlikely. Recent empirical evidence show that humans typically follow complex assessment strategies involving both reciprocity and social imitation when making cooperative decisions. However, currently, we have no systematic understanding of how imitation, a mechanism that may also generate negative effects via a process of cumulative advantage, affects cooperation when repeated interactions are unlikely or information about a recipient's reputation is unavailable. Here we extend existing evolutionary models, which use an image score for reputation to track how individuals cooperate by contributing resources, by introducing a new imitative-trust score, which tracks whether actors have been the recipients of cooperation in the past. We show that imitative trust can co-exist with indirect reciprocity mechanisms up to a threshold and then cooperation reverses -revealing the elusive nature of cooperation. Moreover, we find that when information about a recipient's reputation is limited, trusting the action of third parties towards her (i.e. imitating) does favor a higher collective cooperation compared to random-trusting and share-alike mechanisms. We believe these results shed new light on the factors favoring social imitation as an adaptive mechanism in populations of cooperating social actors.     

Modeling social and evolutionary games

When game theory was introduced to biology, the components of classic game theory models were replaced with elements more befitting evolutionary phenomena. The actions of intelligent agents are replaced by phenotypic traits; utility is replaced by fitness; rational deliberation is replaced by natural selection. In this paper, I argue that this classic conception of comprehensive reapplication is misleading, for it overemphasizes the discontinuity between human behavior and evolved traits. Explicitly considering the representational roles of evolutionary game theory brings to attention areas of overlap that are often neglected, and so a range of evolutionary possibilities that are often overlooked. The clarifications this analysis provides are well illustrated by-and particularly valuable for-game theoretic treatments of the evolution of social behavior.     

Sexual selection is a form of social selection

Social selection influences the evolution of weapons, ornaments and behaviour in both males and females. Thus, social interactions in both sexual and non-sexual contexts can have a powerful influence on the evolution of traits that would otherwise appear to be detrimental to survival. Although clearly outlined by West-Eberhard in the early 1980s, the idea that social selection is a comprehensive framework for the study of ornaments and weapons has largely been ignored. In West-Eberhard's view, sexual selection is a form of social selection-a concept supported by several lines of evidence. Darwin's distinction between natural and sexual selection has been useful, but recent confusion about the limits of sexual selection suggests that some traits are not easily categorized as naturally or sexually selected. Because social selection theory has much to offer the current debates about both sexual selection and reproductive competition in females, it is sometimes viewed, narrowly, to be most useful when considering female roles. However, social selection theory encompasses much more than female reproductive competition. Our goal here was to provide that broader perspective.     

Demographic and Social Predictors of Intimate Partner Violence in Colombia

Intimate partner violence (IPV) is a major health and human rights problem globally. However, empirical findings on the predictors of IPV cross-culturally are highly inconsistent, and the theory of IPV is underdeveloped. We propose a new analytical framework based on cooperative game theory in which IPV is a function of the power relations of the dyadic relationship, not simply the actors involved. Using data from the 2005 Colombian Demographic and Health Survey, we test the hypothesis that IPV is predicted by large asymmetries in dyadic power using a hierarchical generalized linear model. Results suggest that education, urban residence, age at sexual debut, whether the woman has other sexual partners, and the age difference between spouses have strong effects on the log-odds of a woman experiencing IPV. Cooperative game theory and social network analysis offer a general approach to the problem of intimate partner interactions which can be applied broadly cross-culturally.     

Kin selection-mutation balance: a model for the origin, maintenance, and consequences of social cheating

Social conflict, in the form of intraspecific selfish "cheating," has been observed in a number of natural systems. However, a formal, evolutionary genetic theory of social cheating that provides an explanatory, predictive framework for these observations is lacking. Here we derive the kin selection-mutation balance, which provides an evolutionary null hypothesis for the statics and dynamics of cheating. When social interactions have linear fitness effects and Hamilton's rule is satisfied, selection is never strong enough to eliminate recurrent cheater mutants from a population, but cheater lineages are transient and do not invade. Instead, cheating lineages are eliminated by kin selection but are constantly reintroduced by mutation, maintaining a stable equilibrium frequency of cheaters. The presence of cheaters at equilibrium creates a "cheater load" that selects for mechanisms of cheater control, such as policing. We find that increasing relatedness reduces the cheater load more efficiently than does policing the costs and benefits of cooperation. Our results provide new insight into the effects of genetic systems, mating systems, ecology, and patterns of sex-limited expression on social evolution. We offer an explanation for the widespread cheater/altruist polymorphism found in nature and suggest that the common fear of conflict-induced social collapse is unwarranted.     

Sexual selection

Sexual selection is a concept that has probably been misunderstood and misrepresented more than any other idea in evolutionary biology, confusion that continues to the present day. We are not entirely sure why this is, but sexual politics seems to have played its role, as does a failure to understand what sexual selection is and why it was initially invoked. While in some ways less intuitive than natural selection, sexual selection is conceptually identical to it, and evolution via either mechanism will occur given sufficient genetic variation. Recent claims that sexual selection theory is fundamentally flawed are simply wrong and ignore an enormous body of evidence that provides a bedrock of support for this major mechanism of organic evolution. In fact it is partly due to this solid foundation that current research has largely shifted from documenting whether or not sexual selection occurs, to addressing more complex evolutionary questions.     

A Theoretical Analysis of Temporal Difference Learning in the Iterated Prisoner’s Dilemma Game

Direct reciprocity is a chief mechanism of mutual cooperation in social dilemma. Agents cooperate if future interactions with the same opponents are highly likely. Direct reciprocity has been explored mostly by evolutionary game theory based on natural selection. Our daily experience tells, however, that real social agents including humans learn to cooperate based on experience. In this paper, we analyze a reinforcement learning model called temporal difference learning and study its performance in the iterated Prisoner’s Dilemma game. Temporal difference learning is unique among a variety of learning models in that it inherently aims at increasing future payoffs, not immediate ones. It also has a neural basis. We analytically and numerically show that learners with only two internal states properly learn to cooperate with retaliatory players and to defect against unconditional cooperators and defectors. Four-state learners are more capable of achieving a high payoff against various opponents. Moreover, we numerically show that four-state learners can learn to establish mutual cooperation for sufficiently small learning rates.     

The coevolution of long‐term pair bonds and cooperation

The evolution of social traits may not only depend on but also change the social structure of the population. In particular, the evolution of pairwise cooperation, such as biparental care, depends on the pair‐matching distribution of the population, and the latter often emerges as a collective outcome of individual pair‐bonding traits, which are also under selection. Here, we develop an analytical model and individual‐based simulations to study the coevolution of long‐term pair bonds and cooperation in parental care, where partners play a Snowdrift game in each breeding season. We illustrate that long‐term pair bonds may coevolve with cooperation when bonding cost is below a threshold. As long‐term pair bonds lead to assortative interactions through pair‐matching dynamics, they may promote the prevalence of cooperation. In addition to the pay‐off matrix of a single game, the evolutionarily stable equilibrium also depends on bonding cost and accidental divorce rate, and it is determined by a form of balancing selection because the benefit from pair‐bond maintenance diminishes as the frequency of cooperators increases. Our findings highlight the importance of ecological factors affecting social bonding cost and stability in understanding the coevolution of social behaviour and social structures, which may lead to the diversity of biological social systems.