Learning rules for social foragers: implications for the producer-scrounger game and ideal free distribution theory

In population games, the optimal behaviour of a forager depends partly on courses of action selected by other individuals in the population. How individuals learn to allocate effort in foraging games involving frequency-dependent payoffs has been little examined. The performance of three different learning rules was investigated in several types of habitats in each of two population games. Learning rules allow individuals to weigh information about the past and the present and to choose among alternative patterns of behaviour. In the producer-scrounger game, foragers use producer to locate food patches and scrounger to exploit the food discoveries of others. In the ideal free distribution game, foragers that experience feeding interference from companions distribute themselves among heterogeneous food patches. In simulations of each population game, the use of different learning rules induced large variation in foraging behaviour, thus providing a tool to assess the relevance of each learning rule in experimental systems. Rare mutants using alternative learning rules often successfully invaded populations of foragers using other rules indicating that some learning rules are not stable when pitted against each other. Learning rules often closely approximated optimal behaviour in each population game suggesting that stimulus-response learning of contingencies created by foraging companions could be sufficient to perform at near-optimal level in two population games.     

On learning dynamics underlying the evolution of learning rules

In order to understand the development of non-genetically encoded actions during an animal’s lifespan, it is necessary to analyze the dynamics and evolution of learning rules producing behavior. Owing to the intrinsic stochastic and frequency-dependent nature of learning dynamics, these rules are often studied in evolutionary biology via agent-based computer simulations. In this paper, we show that stochastic approximation theory can help to qualitatively understand learning dynamics and formulate analytical models for the evolution of learning rules. We consider a population of individuals repeatedly interacting during their lifespan, and where the stage game faced by the individuals fluctuates according to an environmental stochastic process. Individuals adjust their behavioral actions according to learning rules belonging to the class of experience-weighted attraction learning mechanisms, which includes standard reinforcement and Bayesian learning as special cases. We use stochastic approximation theory in order to derive differential equations governing action play probabilities, which turn out to have qualitative features of mutator-selection equations. We then perform agent-based simulations to find the conditions where the deterministic approximation is closest to the original stochastic learning process for standard 2-action 2-player fluctuating games, where interaction between learning rules and preference reversal may occur. Finally, we analyze a simplified model for the evolution of learning in a producer–scrounger game, which shows that the exploration rate can interact in a non-intuitive way with other features of co-evolving learning rules. Overall, our analyses illustrate the usefulness of applying stochastic approximation theory in the study of animal learning.     

Evolution of learned strategy choice in a frequency-dependent game

In frequency-dependent games, strategy choice may be innate or learned. While experimental evidence in the producer-scrounger game suggests that learned strategy choice may be common, a recent theoretical analysis demonstrated that learning by only some individuals prevents learning from evolving in others. Here, however, we model learning explicitly, and demonstrate that learning can easily evolve in the whole population. We used an agent-based evolutionary simulation of the producer-scrounger game to test the success of two general learning rules for strategy choice. We found that learning was eventually acquired by all individuals under a sufficient degree of environmental fluctuation, and when players were phenotypically asymmetric. In the absence of sufficient environmental change or phenotypic asymmetries, the correct target for learning seems to be confounded by game dynamics, and innate strategy choice is likely to be fixed in the population. The results demonstrate that under biologically plausible conditions, learning can easily evolve in the whole population and that phenotypic asymmetry is important for the evolution of learned strategy choice, especially in a stable or mildly changing environment.     

Producer–Scrounger Games in a Spatially Explicit World: Tactic Use Influences Flock Geometry of Spice Finches

Group-foraging animals can either search for their food (producer) or search for opportunities to join the food discoveries of others (scrounger). To maximize food returns, producers should distance themselves from potential competitors whereas scroungers should increase proximity to potential producers. We investigated the extent to which playing one or the other tactic affected an individual's location in captive flocks of ground-feeding spice finches ( Lonchura punctulata ) as they foraged for hidden clumps of food on an aviary floor. We constrained some individuals to use the producer tactic by pre-training them to find food hidden under lids. Constrained producers foraged significantly further from the center of flocks than constrained scroungers. Flocks with many scroungers were significantly more compact than flocks with fewer scroungers. The results are consistent with published simulations of spatially explicit producer–scrounger models and suggest that the use of producer and scrounger foraging tactics be included as a factor that affects an individual's position within foraging groups.     

Experimental evidence that group foragers can converge on predicted producer-scrounger equilibria

When foraging together, animals are often observed to feed from food discoveries of others. The producer-scrounger (PS) game predicts how frequently this phenomenon of food parasitism should occur. The game assumes: (1) at any moment all individuals can unambiguously be categorized as either playing producer (searching for undiscovered food resources) or scrounger (searching for exploitation opportunities), and (2) the payoffs received from the scrounger tactic are negatively frequency dependent; a scrounger does better than a producer when the scrounger tactic is rare, but worse when it is common. No study to date has shown that the payoffs of producer and scrounger conform to the game's assumptions or that groups of foragers reach the predicted stable equilibrium frequency (SEF) of scrounger, whereby both tactics obtain the same payoff. The current study of three captive flocks of spice finches, Lonchura punctulata, provides the first test of the PS game using an apparatus in which both assumptions of the PS game are met. The payoffs to the scrounger, measured as feeding rate (seeds/s), were highly negatively frequency dependent on the frequency of scrounger. The feeding rate for scrounger declined linearly while the rate for producer either declined only slightly or not at all with increasing scrounger frequency. When given the opportunity to alternate between tactics, the birds changed their use of each, such that the group converged on the predicted SEF of scrounger after 5-8 days of testing. Individuals in this study, therefore, demonstrated sufficient plasticity in tactic use such that the flock foraged at the SEF of scrounger. Copyright 2000 The Association for the Study of Animal Behaviour.     

Risky decisions: a test of risk sensitivity in socially foraging flocks of Lonchura punctulata

Group foraging allows for individuals to exploit the food discoveriesof other group members. If searching for food and searchingfor exploitation opportunities within a group are mutually exclusivealternatives, the decision to use one or the other is modeledas a producer-scrounger game because the value of each alternativeis frequency dependent. Stochastic producer-scrounger modelsgenerally assume that producer provides a more variable anduncertain reward than does the scrounger and hence is a riskierforaging alternative. Socially foraging animals that are attemptingto reduce their risk of starvation should therefore alter theiruse of producer and scrounger alternatives in response to changesin energy budget. We observed flocks of nutmeg mannikins (L.punctulata) foraging in an indoor aviary to determine whethertheir use of producer and scrounger alternatives were risk sensitive.Analyses of the foraging rewards of three flocks of seven birdsconfirm that producer is a riskier foraging strategy than isscrounger, although the difference in risk is rather small.We then submitted two other flocks to two different energy budgetsand observed the foraging decision of four focal birds in eachflock. All but one bird increased their relative use of theriskier producer strategy in the low food reserve treatment,but the overall use of producer did not differ significantlybetween treatments, providing evidence for a small but consistenteffect.     

Geometry for a selfish foraging group: a genetic algorithm approach

The advantages of group living are not shared equally among all group members and these advantages may depend on the spatial position occupied by a forager within the group. For instance, it is thought that socially dominant individuals prefer the predator-safe central position of groups forcing subordinates to the periphery. Uneven spread of benefits among group members can occur when some animals (the scroungers) parasitically exploit the food findings of other foragers (the producers). Here we focus on how playing producer or scrounger affects an individual''s spatial position within a group. We model the movement of foraging animals playing scrounger or producer using a spatially explicit simulation and use a genetic algorithm to establish movement rules. We find that groups containing producers and scroungers are more compact compared to an equivalent group of producers only. Furthermore, the position occupied by strategies varies: scroungers are mainly found in central positions, while producers in the periphery, suggesting that the best position for strategies differs. Dominants, therefore, should prefer movement rules which lead to central positions because of the positional benefits provided to the scrounger strategy they use. Moreover, position within a group will introduce an asymmetry among otherwise phenotypically symmetric individuals.     

Increasing vulnerability to predation increases preference for the scrounger foraging tactic

When animals forage in groups, individuals can search for foodthemselves (producer tactic) or they can search for and joinother individuals that have located food (scrounger tactic).The scrounger tactic may provide greater antipredator benefitsthan the producer tactic because "scroungers" hop with theirheads up and tend to occupy central positions in a group, whereas"producers" hop with their heads down and tend to occupy edgepositions. We tested whether increasing an individual's vulnerabilityto predation (using wing-loading manipulations) causes an increasedpreference for the scrounger tactic in zebra finches (Taeniopygiaguttata). Wing-loading manipulations were effective at increasingfocal individuals' perception of vulnerability to predation;treatment individuals increased their total time allocated tovigilance, whereas control individuals did not. Treatment individualsalso increased their use of the scrounger tactic (proportionof hops with head up) and scrounged a greater proportion ofpatches, whereas control individuals exhibited no changes. Ourresults are consistent with the hypothesis that the scroungertactic confers greater antipredator benefits than the producertactic, although whether antipredator benefits are achievedthrough differences in head orientation, spatial position, orboth, remains unclear. Our finding that individuals adjust theiruse of the scrounger tactic according to changes in their phenotypeprovides evidence for phenotype-limited allocation strategiesin producer–scrounger games.     

Learning to Choose Among Social Foraging Strategies in Adult House Sparrows (Passer domesticus)

Social foragers may be regarded as being engaged in a producer–scrounger game in which they can search for food independently or join others who have discovered food. Research on the producer–scrounger game has focused mainly on the different factors influencing its evolutionarily stable strategy (ESS) solution, but very little is known about the actual mechanisms that shape players' decisions. Recent work has shown that early experience can affect producer–scrounger foraging tendencies in young house sparrows and that in nutmeg mannikins learning is involved in reaching the ESS. Here, we show that direct manipulation of the success rate experienced by adult sparrows when following others can change their strategy choice on the following day. We presented to live sparrows an experimental regime, where stuffed adult house sparrows in a feeding position were positioned on a foraging grid that included two reward regimes: a positive one, in which the stuffed models were placed near food, and a negative one, in which the models were placed away from food. There was a significant increase in joining behavior after the positive treatment (exhibited by 84% of the birds), but no change after the negative treatment. Further analysis demonstrated that sparrows more frequently used the strategy with which they were more successful (usually joining) and that differences in strategy use were correlated with differences in success. These results suggest that adult birds can monitor their success and learn to choose among social foraging strategies in the producer–scrounger game.     

Testing dynamic variance-sensitive foraging using individual differences in basal metabolic rates of zebra finches

Social foragers can alternate between searching for food (producer tactic), and searching for other individuals that have located food in order to join them (scrounger tactic). Both tactics yield equal rewards on average, but the rewards generated by producer are more variable. A dynamic variance-sensitive foraging model predicts that social foragers should increase their use of scrounger with increasing energy requirements and/or decreased food availability early in the foraging period. We tested whether natural variation in minimum energy requirements (basal metabolic rate or BMR) is associated with differences in the use of producer–scrounger foraging tactics in female zebra finches Taeniopygia guttata . As predicted by the dynamic variance-sensitive model, high BMR individuals had significantly greater use of the scrounger tactic compared with low BMR individuals. However, we observed no effect of food availability on tactic use, indicating that female zebra finches were not variance-sensitive foragers under our experimental conditions. This study is the first to report that variation in BMR within a species is associated with differences in foraging behaviour. BMR-related differences in scrounger tactic use are consistent with phenotype-dependent tactic use decisions. We suggest that BMR is correlated with another phenotypic trait which itself influences tactic use decisions.     

Phenotypic Correlates of Scrounging Behavior in Zebra Finches: Role of Foraging Efficiency and Dominance

In flocks, individuals can search for their own food using the producer tactic or exploit the discoveries of companions using the scrounger tactic. Models of the producer–scrounger game usually assume that tactic payoffs are independent of individual phenotypic traits. However, factors such as dominance status or foraging efficiency may constrain the use of tactics and lead to asymmetric tactic use among individuals. For instance, in flocks composed of foragers with unequal foraging efficiency, foragers that are less efficient at obtaining food are expected to rely on the scrounger tactic to a greater extent. I examined the role of foraging efficiency and dominance status as potential correlates of scrounging behavior in small aviary flocks of zebra finches (Taenopygia guttata). Individual foraging efficiency was documented in each flock in a treatment that prevented scrounging. In a subsequent treatment that allowed scrounging, higher levels of scrounging occurred as predicted in foragers with lower foraging efficiency. Dominance status was a poor predictor of tactic choice. Birds that arrived later on the foraging grid foraged less efficiently when scrounging was prevented and used scrounging to a greater extent when allowed, suggesting a link between boldness, foraging efficiency and the choice of foraging tactics in small flocks of zebra finches.     

Daily Patterns of Optimal Producer and Scrounger Use under Predation Hazard: A State-Dependent Dynamic Game Analysis

Feeding in groups often gives rise to joining: feeding from other's discoveries. The joining decision has been modeled as a producer-scrounger game where the producer strategy consists of searching for one's food and the scrounger strategy consists of searching for food discovered by others. Previous models revealed that the evolutionarily stable proportion of scrounging mostly depends on the fraction of each food patch available only to its producer. These early models are static and state independent and are therefore unable to explore whether the time of day, the animal's state, and the degree of predation hazard influence an individual's decision of whether to use the producer or scrounger strategy. To investigate these issues, we developed a state-dependent dynamic producer-scrounger game model. The model predicts that, early in the day, low reserves promote a preference for the scrounger strategy, while the same condition late in the day favors the use of the producer strategy. Under rich and clumped food, the availability of scrounging can improve the daily survival of any average group member. The model suggests only weak effects of predation hazard on the use of scrounging. Future developments should consider the effects of dominance asymmetries and allowing foragers a choice between foraging alone or in a group harboring an evolutionarily stable frequency of scrounger.     

Co-evolution of learning complexity and social foraging strategies

Variation in learning abilities within populations suggests that complex learning may not necessarily be more adaptive than simple learning. Yet, the high cost of complex learning cannot fully explain this variation without some understanding of why complex learning is too costly for some individuals but not for others. Here we propose that different social foraging strategies can favor different learning strategies (that learn the environment with high or low resolution), thereby maintaining variable learning abilities within populations. Using a genetic algorithm in an agent-based evolutionary simulation of a social foraging game (the producer-scrounger game) we demonstrate how an association evolves between a strategy based on independent search for food (playing a producer) and a complex (high resolution) learning rule, while a strategy that combines independent search and following others (playing a scrounger) evolves an association with a simple (low resolution) learning rule. The reason for these associations is that for complex learning to have an advantage, a large number of learning steps, normally not achieved by scroungers, are necessary. These results offer a general explanation for persistent variation in cognitive abilities that is based on co-evolution of learning rules and social foraging strategies.     

Incompatibility between antipredatory vigilance and scrounger tactic in nutmeg mannikins, Lonchura punctulata

Most social foragers must search for food while avoiding predators. Group-foraging nutmeg mannikins engaged in a producer-scrounger game search for their own food (play producer) by hopping with the head down and search for others' food discoveries (play scrounger) by hopping with the head up. If the scrounger tactic is compatible with antipredatory vigilance, then an increase in antipredatory vigilance should lead to the detection of more joining opportunities, and hence to more joining by foragers. We tested this prediction as well as the extent to which stationary birds use head up exclusively for antipredatory purposes and hopping birds use head up for foraging purposes only. We observed three flocks of nutmeg mannikins searching for hidden clumps of food in an indoor aviary. We used a 2×2 factorial design in which both the distance to a safe refuge and the food distribution were manipulated. The use of head up by stationary and eating birds increased significantly with increased distance to cover. Distance to cover, however, had no effect on the use of the scrounger tactic or on the level of joining. We found no evidence of compatibility between the scrounger tactic and antipredatory vigilance. Our results provide the first unambiguous evidence for the existence of two distinct and incompatible patterns of vigilance for predators and for conspecifics. Copyright 2003 Published by Elsevier Ltd on behalf of The Association for the Study of Animal Behaviour.      

Antibiotic resistance and the evolution of group-beneficial traits. II: a metapopulation model

Inspired by the evolution of antibiotic resistance in bacteria, we have developed a model that examines the evolution of "producers" (who secrete a substance that breaks down antibiotics) and non-producers. In a previous study, we found that frequency-dependent selection could favor an intermediate frequency of producers in a single, large population. Here we develop a metapopulation model that examines the evolution of producers and non-producers. Our results indicate that in a metapopulation with many groups, each of size N, the equilibrial frequency of producers decreases with group size. Even when N is high (e.g. 150 individuals/group), however, a significant frequency of producers is still predicted. We also found that the equilibrial frequency of producers increases as the minimum numbers of producers necessary to provide protection to non-producers increases. Lastly, increasing the benefit/cost ratio (b/c) for producers increases their equilibrial frequency.     

The effect of cheats on siderophore diversity in Pseudomonas aeruginosa

Cooperation can be maintained if cooperative behaviours are preferentially directed towards other cooperative individuals. Tag‐based cooperation (greenbeards) – where cooperation benefits individuals with the same tag as the actor – is one way to achieve this. Tag‐based cooperation can be exploited by individuals who maintain the specific tag but do not cooperate, and selection to escape this exploitation can result in the evolution of tag diversity. We tested key predictions crucial for the evolution of cheat‐mediated tag diversity using the production of iron‐scavenging pyoverdine by the opportunistic pathogen, Pseduomonas aeruginosa as a model system. Using two strains that produce different pyoverdine types and their respective cheats, we show that cheats outcompete their homologous pyoverdine producer, but are outcompeted by the heterologous producer in well‐mixed environments. As a consequence, co‐inoculating two types of pyoverdine producer and one type of pyoverdine cheat resulted in the pyoverdine type whose cheat was not present having a large fitness advantage. Theory suggests that in such interactions, cheats can maintain tag diversity in spatially structured environments, but that tag‐based cooperation will be lost in well‐mixed populations, regardless of tag diversity. We saw that when all pyoverdine producers and cheats were co‐inoculated in well‐mixed environments, both types of pyoverdine producers were outcompeted, whereas spatial structure (agar plates and compost microcosms), rather than maintaining diversity, resulted in the domination of one pyoverdine producer. These results suggest cheats may play a more limited role in the evolution of pyoverdine diversity than predicted.     

Mutation in evolutionary games can increase average fitness at equilibrium

We study game dynamical interactions between two strategies, A and B, and analyse whether the average fitness of the population at equilibrium can be increased by adding mutation from A to B. Classifying all two by two games with payoff matrix [(a,b),(c,d)], we show that mutation from A to B enhances the average fitness of the whole population (i) if both a and d are less than (b + c)/2 and (ii) if c is less than b. Furthermore, we study conditions for maximizing the productivity of strategy A, and we analyse the effect of mutations in both directions. Depending on the biological system, a mutation in an evolutionary game can be interpreted as a genetic alteration, a cellular differentiation, a change in gene expression, an accidental or deliberate modification in cultural transmission, or a learning error. In a cultural context, our results indicate that the equilibrium payoff of the population can be increased if players sometimes choose the strategy with lower payoff. In a genetic context, we have shown that for frequency-dependent selection mutation can enhance the average fitness of the population at equilibrium.     

The evolution of social learning rules: Payoff-biased and frequency-dependent biased transmission

Humans and other animals do not use social learning indiscriminately, rather, natural selection has favoured the evolution of social learning rules that make selective use of social learning to acquire relevant information in a changing environment. We present a gene-culture coevolutionary analysis of a small selection of such rules (unbiased social learning, payoff-biased social learning and frequency-dependent biased social learning, including conformism and anti-conformism) in a population of asocial learners where the environment is subject to a constant probability of change to a novel state. We define conditions under which each rule evolves to a genetically polymorphic equilibrium. We find that payoff-biased social learning may evolve under high levels of environmental variation if the fitness benefit associated with the acquired behaviour is either high or low but not of intermediate value. In contrast, both conformist and anti-conformist biases can become fixed when environment variation is low, whereupon the mean fitness in the population is higher than for a population of asocial learners. Our examination of the population dynamics reveals stable limit cycles under conformist and anti-conformist biases and some highly complex dynamics including chaos. Anti-conformists can out-compete conformists when conditions favour a low equilibrium frequency of the learned behaviour. We conclude that evolution, punctuated by the repeated successful invasion of different social learning rules, should continuously favour a reduction in the equilibrium frequency of asocial learning, and propose that, among competing social learning rules, the dominant rule will be the one that can persist with the lowest frequency of asocial learning.     

Coevolution of adaptive technology,maladaptive culture and population size in a producer–scrounger game

Technology (i.e. tools, methods of cultivation and domestication, systems of construction and appropriation, machines) has increased the vital rates of humans, and is one of the defining features of the transition from Malthusian ecological stagnation to a potentially perpetual rising population growth. Maladaptations, on the other hand, encompass behaviours, customs and practices that decrease the vital rates of individuals. Technology and maladaptations are part of the total stock of culture carried by the individuals in a population. Here, we develop a quantitative model for the coevolution of cumulative adaptive technology and maladaptive culture in a ‘producer–scrounger’ game, which can also usefully be interpreted as an ‘individual–social’ learner interaction. Producers (individual learners) are assumed to invent new adaptations and maladaptations by trial-and-error learning, insight or deduction, and they pay the cost of innovation. Scroungers (social learners) are assumed to copy or imitate (cultural transmission) both the adaptations and maladaptations generated by producers. We show that the coevolutionary dynamics of producers and scroungers in the presence of cultural transmission can have a variety of effects on population carrying capacity. From stable polymorphism, where scroungers bring an advantage to the population (increase in carrying capacity), to periodic cycling, where scroungers decrease carrying capacity, we find that selection-driven cultural innovation and transmission may send a population on the path of indefinite growth or to extinction.     

Fixation probabilities for the Moran process with three or more strategies: general and coupling results

We study fixation probabilities for the Moran stochastic process for the evolution of a population with three or more types of individuals and frequency-dependent fitnesses. Contrary to the case of populations with two types of individuals, in which fixation probabilities may be calculated by an exact formula, here we must solve a large system of linear equations. We first show that this system always has a unique solution. Other results are upper and lower bounds for the fixation probabilities obtained by coupling the Moran process with three strategies with birth–death processes with only two strategies. We also apply our bounds to the problem of evolution of cooperation in a population with three types of individuals already studied in a deterministic setting by Núñez Rodríguez and Neves (J Math Biol 73:1665–1690, 2016). We argue that cooperators will be fixated in the population with probability arbitrarily close to 1 for a large region of initial conditions and large enough population sizes.