Evolutionary dynamics of biological auctions

Many scenarios in the living world, where individual organisms compete for winning positions (or resources), have properties of auctions. Here we study the evolution of bids in biological auctions. For each auction, n individuals are drawn at random from a population of size N. Each individual makes a bid which entails a cost. The winner obtains a benefit of a certain value. Costs and benefits are translated into reproductive success (fitness). Therefore, successful bidding strategies spread in the population. We compare two types of auctions. In “biological all-pay auctions”, the costs are the bid for every participating individual. In “biological second price all-pay auctions”, the cost for everyone other than the winner is the bid, but the cost for the winner is the second highest bid. Second price all-pay auctions are generalizations of the “war of attrition” introduced by Maynard Smith. We study evolutionary dynamics in both types of auctions. We calculate pairwise invasion plots and evolutionarily stable distributions over the continuous strategy space. We find that the average bid in second price all-pay auctions is higher than in all-pay auctions, but the average cost for the winner is similar in both auctions. In both cases, the average bid is a declining function of the number of participants, n. The more individuals participate in an auction the smaller is the chance of winning, and thus expensive bids must be avoided.     

Partner's behavior, not reward distribution, determines success in an unequal cooperative task in capuchin monkeys

It was recently demonstrated that capuchin monkeys notice and respond to distributional inequity, a trait that has been proposed to support the evolution of cooperation in the human species. However, it is unknown how capuchins react to inequitable rewards in an unrestricted cooperative paradigm in which they may freely choose both whether to participate and, within the bounds of their partner's behavior, which reward they will receive for their participation. We tested capuchin monkeys with such a design, using a cooperative barpull, which has been used with great success in the past. Contrary to our expectations, the equity of the reward distribution did not affect success or pulling behavior. However, the behavior of the partner in an unequal situation did affect overall success rates: pairs that had a tendency to alternate which individual received the higher-value food in unequal reward situations were more than twice as successful in obtaining rewards than pairs in which one individual dominated the higher-value food. This ability to equitably distribute rewards in inherently biased cooperative situations has profound implications for activities such as group hunts, in which multiple individuals work together for a single, monopolizable reward.     

Responses to a simple barter task in chimpanzees, Pan troglodytes

Chimpanzees (Pan troglodytes) frequently participate in social exchange involving multiple goods and services of variable value, yet they have not been tested in a formalized situation to see whether they can barter using multiple tokens and rewards. We set up a simple barter economy with two tokens and two associated rewards and tested chimpanzees on their ability to obtain rewards by returning the matching token in situations in which their access to tokens was unlimited or limited. Chimpanzees easily learned to associate value with the tokens, as expected, and did barter, but followed a simple strategy of favoring the higher-value token, regardless of the reward proffered, instead of a more complex but more effective strategy of returning the token that matched the reward. This response is similar to that shown by capuchin monkeys in our previous study. We speculate that this response, while not ideal, may be sufficient to allow for stability of the social exchange system in these primates, and that the importance of social barter to both species may have led to this convergence of strategies.     

Mixed infections may promote diversification of mutualistic symbionts: why are there ineffective rhizobia?

While strategy variation is a key feature of symbiotic mutualisms, little work focuses on the origin of this diversity. Rhizobia strategies range from mutualistic nitrogen fixers to parasitic nonfixers that hoard plant resources to increase their own survival in soil. Host plants reward beneficial rhizobia with higher nodule growth rates, generating a trade‐off between reproduction in nodules and subsequent survival in soil. However, hosts might not discriminate between strains in mixed infections, allowing nonfixing strains to escape sanctions. We construct an adaptive dynamics model of symbiotic nitrogen‐fixation and find general situations where symbionts undergo adaptive diversification, but in most situations complete nonfixers do not evolve. Social conflict in mixed infections when symbionts face a survival–reproduction trade‐off can drive the origin of some coexisting symbiont strategies, where less mutualistic strains exploit benefits generated by better mutualists.     

Single units in the pigeon brain integrate reward amount and time-to-reward in an impulsive choice task

BACKGROUND: Animals prefer small over large rewards when the delays preceding large rewards exceed an individual tolerance limit. Such impulsive choice behavior occurs even in situations in which alternative strategies would yield more optimal outcomes. Behavioral research has shown that an animal's choice is guided by the alternative rewards' subjective values, which are a function of reward amount and time-to-reward. Despite increasing knowledge about the pharmacology and anatomy underlying impulsivity, it is still unknown how the brain combines reward amount and time-to-reward information to represent subjective reward value. RESULTS: We trained pigeons to choose between small, immediate rewards and large rewards delivered after gradually increasing delays. Single-cell recordings in the avian Nidopallium caudolaterale, the presumed functional analog of the mammalian prefrontal cortex, revealed that neural delay activation decreased with increasing delay length but also covaried with the expected reward amount. This integrated neural response was modulated by reward amount and delay, as predicted by a hyperbolical equation, of subjective reward value derived from behavioral studies. Furthermore, the neural activation pattern reflected the current reward preference and the time point of the shift from large to small rewards. CONCLUSIONS: The reported activity was modulated by the temporal devaluation of the anticipated reward in addition to reward amount. Our findings contribute to the understanding of neuropathologies such as drug addiction, pathological gambling, frontal lobe syndrome, and attention-deficit disorders, which are characterized by inappropriate temporal discounting and increased impulsiveness.     

The present values of delayed rewards are approximately additive

In two experiments, human subjects were asked to estimate their present values of single delayed rewards and their present values of temporal sequences of three rewards. Present values were solicited by asking subjects to indicate an amount of money v for which they would be indifferent between receiving v at the end of the session and receiving the delayed reward(s). A procedure was used for which responding the true value of v was the optimal strategy, and the actual payoff that each subject received was determined by one randomly selected trial. In Experiment 1 (n=29) each delayed reward was 9.90 dollars in cash. In Experiment 2 (n=19) the delayed rewards were dated 15 dollars gift certificates to a local restaurant. In both experiments, the present values of the sequences were approximately equal to the sums of the present values of their component rewards. The presence of outliers suggests that a few subjects may have valued sequences less than the sums of their single rewards. Effects of a preference for uniform sequences, if any, were too small to be detected. Discounting of sequences was well fit by a parallel hyperbolic discounting equation, consistent with Mazur's [Mazur, J.E., 1986. Choice between single and multiple delayed reinforcers. J. Exp. Anal. Behav. 46 (1), 67-77] results using multiple reinforcers.     

Probabilistic participation in public goods games

Voluntary participation in public goods games (PGGs) has turned out to be a simple but effective mechanism for promoting cooperation under full anonymity. Voluntary participation allows individuals to adopt a risk-aversion strategy, termed loner. A loner refuses to participate in unpromising public enterprises and instead relies on a small but fixed pay-off. This system leads to a cyclic dominance of three pure strategies, cooperators, defectors and loners, but at the same time, there remain two considerable restrictions: the addition of loners cannot stabilize the dynamics and the time average pay-off for each strategy remains equal to the pay-off of loners. Here, we introduce probabilistic participation in PGGs from the standpoint of diversification of risk, namely simple mixed strategies with loners, and prove the existence of a dynamical regime in which the restrictions ono longer hold. Considering two kinds of mixed strategies associated with participants (cooperators or defectors) and non-participants (loners), we can recover all basic evolutionary dynamics of the two strategies: dominance; coexistence; bistability; and neutrality, as special cases depending on pairs of probabilities. Of special interest is that the expected pay-off of each mixed strategy exceeds the pay-off of loners at some interior equilibrium in the coexistence region.     

Can Monkeys Choose Optimally When Faced with Noisy Stimuli and Unequal Rewards?

We review the leaky competing accumulator model for two-alternative forced-choice decisions with cued responses, and propose extensions to account for the influence of unequal rewards. Assuming that stimulus information is integrated until the cue to respond arrives and that firing rates of stimulus-selective neurons remain well within physiological bounds, the model reduces to an Ornstein-Uhlenbeck (OU) process that yields explicit expressions for the psychometric function that describes accuracy. From these we compute strategies that optimize the rewards expected over blocks of trials administered with mixed difficulty and reward contingencies. The psychometric function is characterized by two parameters: its midpoint slope, which quantifies a subject''s ability to extract signal from noise, and its shift, which measures the bias applied to account for unequal rewards. We fit these to data from two monkeys performing the moving dots task with mixed coherences and reward schedules. We find that their behaviors averaged over multiple sessions are close to optimal, with shifts erring in the direction of smaller penalties. We propose two methods for biasing the OU process to produce such shifts.     

Do kleptoparasites reduce their own foraging effort in order to detect kleptoparasitic opportunities?: An empirical test of a key assumption of kleptoparasitic models

Determining if, or when, individuals trade off time spent personally feeding against time spent monitoring others for kleptoparasitism opportunities is essential to an understanding of the evolution of scrounging and usurpation behaviours. We provide a first field test of whether kleptoparasites reduce their personal foraging effort in situations where the frequency and rewards of kleptoparasitism increase. We provided experimental food patches for wild European blackbirds that varied in the distribution of prey and that had a potentially high rate of kleptoparasitism within pairs of blackbirds feeding in them. Although individuals differed in their rate of kleptoparasitism, they did not vary in the size of the reward that they gained from kleptoparasitism. As prey became more clumped, kleptoparasitism rate and its reward per incident increased on average. There was, however, no evidence that individuals that were kleptoparasitising more quickly and/or at a higher frequency had lower personal foraging effort. In contrast, foraging effort increased in both birds compared to when they were foraging alone, independent of dominance, kleptoparasitic opportunity or reward. Our evidence suggests that in some circumstances a kleptoparasite can detect kleptoparasitic opportunities without compromising its own personal foraging rate.     

Social Influences on Inequity Aversion in Children

Adults and children are willing to sacrifice personal gain to avoid both disadvantageous and advantageous inequity. These two forms of inequity aversion follow different developmental trajectories, with disadvantageous inequity aversion emerging around 4 years and advantageous inequity aversion emerging around 8 years. Although inequity aversion is assumed to be specific to situations where resources are distributed among individuals, the role of social context has not been tested in children. Here, we investigated the influence of two aspects of social context on inequity aversion in 4- to 9-year-old children: (1) the role of the experimenter distributing rewards and (2) the presence of a peer with whom rewards could be shared. Experiment 1 showed that children rejected inequity at the same rate, regardless of whether the experimenter had control over reward allocations. This indicates that children’s decisions are based upon reward allocations between themselves and a peer and are not attempts to elicit more favorable distributions from the experimenter. Experiment 2 compared rejections of unequal reward allocations in children interacting with or without a peer partner. When faced with a disadvantageous distribution, children frequently rejected a smaller reward when a larger reward was visible, even if no partner would obtain the larger reward. This suggests that nonsocial factors partly explain disadvantageous inequity rejections. However, rejections of disadvantageous distributions were higher when the larger amount would go to a peer, indicating that social context enhances disadvantageous inequity aversion. By contrast, children rejected advantageous distributions almost exclusively in the social context. Therefore, advantageous inequity aversion appears to be genuinely social, highlighting its potential relevance for the development of fairness concerns. By comparing social and nonsocial factors, this study provides a detailed picture of the expression of inequity aversion in human ontogeny and raises questions about the function and evolution of inequity aversion in humans.     

Evolutionary games and two species population dynamics

Competition between species has long been modeled by population dynamics based on total numbers of each species. Recently, the evolution of strategy frequencies has been used successfully for competition models between individuals. In this paper, we illustrate that these two views of competition are compatible. It is shown that the rate of intra and interspecific competitions between individuals largely determines the population dynamics. Competition models over a single common resource and predator-prey models are developed from this individual competition approach. In particular, the equilibrium strategies in a co-evolving predator-prey system are shown to be more stable than the predicted strategy cycling of standard evolutionary game theory.     

The role of size-specific predation in the evolution and diversification of prey life histories

Some of the best empirical examples of life-history evolution involve responses to predation. Nevertheless, most life-history theory dealing with responses to predation has not been formulated within an explicit dynamic food-web context. In particular, most previous theory does not explicitly consider the coupled population dynamics of the focal species and its predators and resources. Here we present a model of life-history evolution that explores the evolutionary consequences of size-specific predation on small individuals when there is a trade-off between growth and reproduction. The model explicitly describes the population dynamics of a predator, the prey of interest, and its resource. The selective forces that cause life-history evolution in the prey species emerge from the ecological interactions embodied by this model and can involve important elements of frequency dependence. Our results demonstrate that the strength of the coupling between predator and prey in the community determines many aspects of life-history evolution. If the coupling is weak (as is implicitly assumed in many previous models), differences in resource productivity have no effect on the nature of life-history evolution. A single life-history strategy is favored that minimizes the equilibrium resource density (if possible). If the coupling is strong, then higher resource productivities select for faster growth into the predation size refuge. Moreover, under strong coupling it is also possible for natural selection to favor an evolutionary diversification of life histories, possibly resulting in two coexisting species with divergent life-history strategies.     

Games of age-dependent prevention of chronic infections by social distancing

Epidemiological games combine epidemic modelling with game theory to assess strategic choices in response to risks from infectious diseases. In most epidemiological games studied thus-far, the strategies of an individual are represented with a single choice parameter. There are many natural situations where strategies can not be represented by a single dimension, including situations where individuals can change their behavior as they age. To better understand how age-dependent variations in behavior can help individuals deal with infection risks, we study an epidemiological game in an SI model with two life-history stages where social distancing behaviors that reduce exposure rates are age-dependent. When considering a special case of the general model, we show that there is a unique Nash equilibrium when the infection pressure is a monotone function of aggregate exposure rates, but non-monotone effects can appear even in our special case. The non-monotone effects sometimes result in three Nash equilibria, two of which have local invasion potential simultaneously. Returning to a general case, we also describe a game with continuous age-structure using partial-differential equations, numerically identify some Nash equilibria, and conjecture about uniqueness.     

Parallel Representation of Value-Based and Finite State-Based Strategies in the Ventral and Dorsal Striatum

Previous theoretical studies of animal and human behavioral learning have focused on the dichotomy of the value-based strategy using action value functions to predict rewards and the model-based strategy using internal models to predict environmental states. However, animals and humans often take simple procedural behaviors, such as the “win-stay, lose-switch” strategy without explicit prediction of rewards or states. Here we consider another strategy, the finite state-based strategy, in which a subject selects an action depending on its discrete internal state and updates the state depending on the action chosen and the reward outcome. By analyzing choice behavior of rats in a free-choice task, we found that the finite state-based strategy fitted their behavioral choices more accurately than value-based and model-based strategies did. When fitted models were run autonomously with the same task, only the finite state-based strategy could reproduce the key feature of choice sequences. Analyses of neural activity recorded from the dorsolateral striatum (DLS), the dorsomedial striatum (DMS), and the ventral striatum (VS) identified significant fractions of neurons in all three subareas for which activities were correlated with individual states of the finite state-based strategy. The signal of internal states at the time of choice was found in DMS, and for clusters of states was found in VS. In addition, action values and state values of the value-based strategy were encoded in DMS and VS, respectively. These results suggest that both the value-based strategy and the finite state-based strategy are implemented in the striatum.     

Evolution of honest reward signal in flowers

Some flowering plants signal the abundance of their rewards by changing their flower colour, scent or other floral traits as rewards are depleted. These floral trait changes can be regarded as honest signals of reward states for pollinators. Previous studies have hypothesized that these signals are used to maintain plant-level attractiveness to pollinators, but the evolutionary conditions leading to the development of honest signals have not been well investigated from a theoretical basis. We examined conditions leading to the evolution of honest reward signals in flowers by applying a theoretical model that included pollinator response and signal accuracy. We assumed that pollinators learn floral traits and plant locations in association with reward states and use this information to decide which flowers to visit. While manipulating the level of associative learning, we investigated optimal flower longevity, the proportion of reward and rewardless flowers, and honest- and dishonest-signalling strategies. We found that honest signals are evolutionarily stable only when flowers are visited by pollinators with both high and low learning abilities. These findings imply that behavioural variation in learning within a pollinator community can lead to the evolution of an honest signal even when there is no contribution of rewardless flowers to pollinator attractiveness.     

Giving-up time variation in response to differences in nectar volume and concentration in the giant tropical ant,Paraponera clavata (Hymenoptera: Formicidae)

Giving-up times in resource patches by workers of the giant tropical ant,Paraponera clavata, are associated with travel time and reward volume but not reward concentration. The discovery of an artificial nectar reward stimulates local search which is centered around the initial reward site. Longer giving-up times increase the likelihood that a worker will find a second reward, but the search appears to be more effective for renewed rewards at the same location than for nearby rewards. When workers are near the colony, larger rewards cause the workers to stop searching and to initiate recruitment behavior. At patches distant from the nest, the threshold in reward volume for recruitment is much higher. These results are consistent with expectations for search strategies when energy expenditure in search is minimal, resources are renewable, and recruitment can occur.     

Selection on signal–reward correlation: limits and opportunities to the evolution of deceit in Turnera ulmifolia L

Because pollinators are unable to directly assess the amount of rewards offered by flowers, they rely on the information provided by advertising floral traits. Thus, having a lower intra-individual correlation between signal and reward (signal accuracy) than other plants in the population provides the opportunity to reduce investment in rewards and cheat pollinators. However, pollinators' cognitive capacities can impose a limit to the evolution of this plant cheating strategy if they can punish those plants with low signal accuracy. In this study, we examined the opportunity for cheating in the perennial weed Turnera ulmifolia L. evaluating the selective value of signal accuracy, floral display and reward production in a natural population. We found that plant reproductive success was positively related to signal accuracy and floral display, but not to nectar production. The intensity of selection on floral display was more than three times higher than on signal accuracy. The pattern of selection indicated that pollinators can select for signal accuracy provided by plants and suggests that learning abilities of pollinators can limit the evolution of deceptive strategies in T. ulmifolia.     

Adaptive dynamics of cooperation may prevent the coexistence of defectors and cooperators and even cause extinction

It has recently been demonstrated that ecological feedback mechanisms can facilitate the emergence and maintenance of cooperation in public goods interactions: the replicator dynamics of defectors and cooperators can result, for example, in the ecological coexistence of cooperators and defectors. Here we show that these results change dramatically if cooperation strategy is not fixed but instead is a continuously varying trait under natural selection. For low values of the factor with which the value of resources is multiplied before they are shared among all participants, evolution will always favour lower cooperation strategies until the population falls below an Allee threshold and goes extinct, thus evolutionary suicide occurs. For higher values of the factor, there exists a unique evolutionarily singular strategy, which is convergence stable. Because the fitness function is linear with respect to the strategy of the mutant, this singular strategy is neutral against mutant invasions. This neutrality disappears if a nonlinear functional response in receiving benefits is assumed. For strictly concave functional responses, singular strategies become uninvadable. Evolutionary branching, which could result in the evolutionary emergence of cooperators and defectors, can occur only with locally convex functional responses, but we illustrate that it can also result in coevolutionary extinction.     

Evolutionarily unstable fitness maxima and stable fitness minima of continuous traits

Summary We present models of adaptive change in continuous traits for the following situations: (1) adaptation of a single trait within a single population in which the fitness of a given individual depends on the population's mean trait value as well as its own trait value; (2) adaptation of two (or more) traits within a single population; (3) adaptation in two or more interacting species. We analyse a dynamic model of these adaptive scenarios in which the rate of change of the mean trait value is an increasing function of the fitness gradient (i.e. the rate of increase of individual fitness with the individual's trait value). Such models have been employed in evolutionary game theory and are often appropriate both for the evolution of quantitative genetic traits and for the behavioural adjustment of phenotypically plastic traits. The dynamics of the adaptation of several different ecologically important traits can result in characters that minimize individual fitness and can preclude evolution towards characters that maximize individual fitness. We discuss biological circumstances that are likely to produce such adaptive failures for situations involving foraging, predator avoidance, competition and coevolution. The results argue for greater attention to dynamical stability in models of the evolution of continuous traits.     

Differential Reward Learning for Self and Others Predicts Self-Reported Altruism

In social environments, decisions not only determine rewards for oneself but also for others. However, individual differences in pro-social behaviors have been typically studied through self-report. We developed a decision-making paradigm in which participants chose from card decks with differing rewards for themselves and charity; some decks gave similar rewards to both, while others gave higher rewards for one or the other. We used a reinforcement-learning model that estimated each participant''s relative weighting of self versus charity reward. As shown both in choices and model parameters, individuals who showed relatively better learning of rewards for charity – compared to themselves – were more likely to engage in pro-social behavior outside of a laboratory setting indicated by self-report. Overall rates of reward learning, however, did not predict individual differences in pro-social tendencies. These results support the idea that biases toward learning about social rewards are associated with one''s altruistic tendencies.