Learning rules for social foragers: implications for the producer-scrounger game and ideal free distribution theory

In population games, the optimal behaviour of a forager depends partly on courses of action selected by other individuals in the population. How individuals learn to allocate effort in foraging games involving frequency-dependent payoffs has been little examined. The performance of three different learning rules was investigated in several types of habitats in each of two population games. Learning rules allow individuals to weigh information about the past and the present and to choose among alternative patterns of behaviour. In the producer-scrounger game, foragers use producer to locate food patches and scrounger to exploit the food discoveries of others. In the ideal free distribution game, foragers that experience feeding interference from companions distribute themselves among heterogeneous food patches. In simulations of each population game, the use of different learning rules induced large variation in foraging behaviour, thus providing a tool to assess the relevance of each learning rule in experimental systems. Rare mutants using alternative learning rules often successfully invaded populations of foragers using other rules indicating that some learning rules are not stable when pitted against each other. Learning rules often closely approximated optimal behaviour in each population game suggesting that stimulus-response learning of contingencies created by foraging companions could be sufficient to perform at near-optimal level in two population games.     

Co-evolution of learning complexity and social foraging strategies

Variation in learning abilities within populations suggests that complex learning may not necessarily be more adaptive than simple learning. Yet, the high cost of complex learning cannot fully explain this variation without some understanding of why complex learning is too costly for some individuals but not for others. Here we propose that different social foraging strategies can favor different learning strategies (that learn the environment with high or low resolution), thereby maintaining variable learning abilities within populations. Using a genetic algorithm in an agent-based evolutionary simulation of a social foraging game (the producer-scrounger game) we demonstrate how an association evolves between a strategy based on independent search for food (playing a producer) and a complex (high resolution) learning rule, while a strategy that combines independent search and following others (playing a scrounger) evolves an association with a simple (low resolution) learning rule. The reason for these associations is that for complex learning to have an advantage, a large number of learning steps, normally not achieved by scroungers, are necessary. These results offer a general explanation for persistent variation in cognitive abilities that is based on co-evolution of learning rules and social foraging strategies.     

The effect of social learning in a small population facing environmental change: an agent-based simulation

Learning is defined as behavioral modification due to experience, social or asocial. Social learning might be less costly than asocial learning and allow the rapid accumulation of learned traits across generations. However, the benefits of social learning in a small population of individuals relying on local interactions and experiencing environmental change are not well understood yet. In this study, we used agent-based simulations to address this issue by comparing the performance of social learning to asocial learning and innate behavior, in both a static and a changing environment. Learning was modeled using neural networks, and innate behavior was modeled using genetically coded behaviors. The performance of 10 mobile simulated agents was measured under three environmental scenarios: static, abrupt change and gradual change. We found that social learning allows for a better performance (in terms of survival) than asocial learning in static and abrupt-change scenarios. In contrast, when changes are gradual, social learning delays achieving the correct alternative, while asocial learning facilitates innovation; interestingly, a mixed population (social and asocial learners) performs the best.     

The coevolution of sexual imprinting by males and females

Sexual imprinting is the learning of a mate preference by direct observation of the phenotype of another member of the population. Sexual imprinting can be paternal, maternal, or oblique if individuals learn to prefer the phenotypes of their fathers, mothers, or other members of the population, respectively. Which phenotypes are learned can affect trait evolution and speciation rates. “Good genes” models of polygynous systems predict that females should evolve to imprint on their fathers, because paternal imprinting helps females to choose mates that will produce offspring that are both viable and sexy. Sexual imprinting by males has been observed in nature, but a theory for the evolution of sexual imprinting by males does not exist. We developed a good genes model to study the conditions under which sexual imprinting by males or by both sexes can evolve and to ask which sexual imprinting strategies maximize the fitness of the choosy sex. We found that when only males imprint, maternal imprinting is the most advantageous strategy. When both sexes imprint, it is most advantageous for both sexes to use paternal imprinting. Previous theory suggests that, in a given population, either males or females but not both will evolve choosiness in mating. We show how environmental change can lead to the evolution of sexual imprinting behavior by both sexes in the same population.     

The effect of learning on experimental evolution of resource preference in Drosophila melanogaster

Abstract Learning is thought to be adaptive in variable environments, whereas constant, predictable environments are supposed to favor unconditional, genetically fixed responses. A dichotomous view of behavior as either learned or innate ignores a potential evolutionary interaction between the learned and innate components of a behavioral response. We addressed this interaction in the context of oviposition substrate choice in Drosophila melanogaster, asking two main questions. First, will learning also evolve in a constant environment in which it always pays to show the same choice? Second, how does an opportunity to learn affect the evolution of the innate (genetic) component of oviposition substrate choice? We exposed experimental populations to four selection regimes, involving selection on oviposition substrate preference (an orange versus a pineapple medium). In two selection regimes the flies were selected for preference either for the orange medium, or for the pineapple medium. In the remaining two selection regimes the flies were also selected for preference for either orange or pineapple, but additionally could use past experience (aversion learning) to decide which medium it paid to avoid. Lines exposed to the latter selection regimes evolved improved learning ability, indicating that learning may be advantageous even if the same behavioral response is favored every generation. Furthermore, of the two selection regimes that favored oviposition on the pineapple medium, the regime that allowed for learning led to the evolution of a stronger innate preference for pineapple, than the regime that did not allow for learning. In contrast, of the two regimes that selected for oviposition on the orange medium, the one that allowed for learning led to a smaller evolutionary change of the innate preference. Thus, an opportunity to learn facilitated the evolution of innate preference under selection for preference for pineapple, but hindered it under selection for preference for orange. We discuss possible mechanisms for this effect.     

Frequency-dependent payoffs and sequential decision-making favour consistent tactic use

Although natural selection should have favoured individuals capable of adjusting the weight they give to personal and social information according to circumstances, individuals generally differ consistently in their individual weighting of both types of information. Such individual differences are correlated with personality traits, suggesting that personality could directly affect individuals' ability to collect personal or social information. Alternatively, the link between personality and information use could simply emerge as a by-product of the sequential decision-making process in a frequency-dependent context. Indeed, when the gains associated with behavioural options depend on the choices of others, an individual's sequence of arrival could constrain its choice of options leading to the emergence of correlated behaviours. Any factor such as personality that affects decision order could thus be correlated with information use. To test this new explanation, we developed an individual-based model that simulates a group of animals engaged in a game of sequential frequency-dependent decision: a producer-scrounger game. Our results confirm that the sequence of decision, in this case enforced by the order in which animals enter a foraging area, consistently influences their mean tactic use and their individual plasticity, an outcome reminiscent of the correlation reported between personality and social information use.     

Cooperation and Stability through Periodic Impulses

Basic games, where each individual chooses between two strategies, illustrate several issues that immediately emerge from the standard approach that applies strategic reasoning, based on rational decisions, to predict population behavior where no rationality is assumed. These include how mutual cooperation (which corresponds to the best outcome from the population perspective) can evolve when the only individually rational choice is to defect, illustrated by the Prisoner''s Dilemma (PD) game, and how individuals can randomize between two strategies when neither is individually rational, illustrated by the Battle of the Sexes (BS) game that models male-female conflict over parental investment in offspring. We examine these questions from an evolutionary perspective where the evolutionary dynamics includes an impulsive effect that models sudden changes in collective population behavior. For the PD game, we show analytically that cooperation can either coexist with defection or completely take over the population, depending on the strength of the impulse. By extending these results for the PD game, we also show that males and females each evolve to a single strategy in the BS game when the impulsive effect is strong and that weak impulses stabilize the randomized strategies of this game.     

Song learning as an indicator mechanism: modelling the developmental stress hypothesis

The ‘developmental stress hypothesis’ attempts to provide a functional explanation of the evolutionary maintenance of song learning in songbirds. It argues that song learning can be viewed as an indicator mechanism that allows females to use learned features of song as a window on a male's early development, a potentially stressful period that may have long-term phenotypic effects. In this paper we formally model this hypothesis for the first time, presenting a population genetic model that takes into account both the evolution of genetic learning preferences and cultural transmission of song. The models demonstrate that a preference for song types that reveal developmental stress can evolve in a population, and that cultural transmission of these song types can be stable, lending more support to the hypothesis.     

THE EVOLUTION OF STRATEGY VARIATION: WILL AN ESS EVOLVE?

Evolutionarily stable strategy (ESS) models are widely viewed as predicting the strategy of an individual that when monomorphic or nearly so prevents a mutant with any other strategy from entering the population. In fact, the prediction of some of these models is ambiguous when the predicted strategy is "mixed", as in the case of a sex ratio, which may be regarded as a mixture of the subtraits "produce a daughter" and "produce a son." Some models predict only that such a mixture be manifested by the population as a whole, that is, as an "evolutionarily stable state"; consequently, strategy monomorphism or polymorphism is consistent with the prediction. The hawk-dove game and the sex-ratio game in a panmictic population are models that make such a "degenerate" prediction. We show here that the incorporation of population finiteness into degenerate models has effects for and against the evolution of a monomorphism (an ESS) that are of equal order in the population size, so that no one effect can be said to predominate. Therefore, we used Monte Carlo simulations to determine the probability that a finite population evolves to an ESS as opposed to a polymorphism. We show that the probability that an ESS will evolve is generally much less than has been reported and that this probability depends on the population size, the type of competition among individuals, and the number of and distribution of strategies in the initial population. We also demonstrate how the strength of natural selection on strategies can increase as population size decreases. This inverse dependency underscores the incorrectness of Fisher's and Wright's assumption that there is just one qualitative relationship between population size and the intensity of natural selection.     

Daily Patterns of Optimal Producer and Scrounger Use under Predation Hazard: A State-Dependent Dynamic Game Analysis

Feeding in groups often gives rise to joining: feeding from other's discoveries. The joining decision has been modeled as a producer-scrounger game where the producer strategy consists of searching for one's food and the scrounger strategy consists of searching for food discovered by others. Previous models revealed that the evolutionarily stable proportion of scrounging mostly depends on the fraction of each food patch available only to its producer. These early models are static and state independent and are therefore unable to explore whether the time of day, the animal's state, and the degree of predation hazard influence an individual's decision of whether to use the producer or scrounger strategy. To investigate these issues, we developed a state-dependent dynamic producer-scrounger game model. The model predicts that, early in the day, low reserves promote a preference for the scrounger strategy, while the same condition late in the day favors the use of the producer strategy. Under rich and clumped food, the availability of scrounging can improve the daily survival of any average group member. The model suggests only weak effects of predation hazard on the use of scrounging. Future developments should consider the effects of dominance asymmetries and allowing foragers a choice between foraging alone or in a group harboring an evolutionarily stable frequency of scrounger.     

Recombination and the evolution of coordinated phenotypic expression in a frequency-dependent game

A long standing question in evolutionary biology concerns the maintenance of adaptive combinations of traits in the presence of recombination. This problem may be solved if positive epistasis selects for reducing the rate of recombination between such traits, but this requires sufficiently strong epistasis. Here we use a model that we developed previously to analyze a frequency-dependent strategy game in asexual populations, to study how adaptive combinations of traits may be maintained in the presence of recombination when epistasis is too weak to select for genetic linkage. Previously, in the asexual case, our model demonstrated the evolution of adaptive associations between social foraging strategies and learning rules. We verify that these adaptive associations, which are represented by different two-locus haplotypes, can easily be broken by genetic recombination. We also confirm that a modifier allele that reduces the rate of recombination fails to evolve (due to weak epistasis). However, we find that under the same conditions of weak epistasis, there is an alternative mechanism that allows an association between traits to evolve. This is based on a genetic switch that responds to the presence of one social foraging allele by activating one of the two alternative learning alleles that are carried by all individuals. We suggest that such coordinated phenotypic expression by genetic switches offers a general and robust mechanism for the evolution of adaptive combinations of traits in the presence of recombination.     

Learning in a game context: strategy choice by some keeps learning from evolving in others

Behavioural decisions in a social context commonly have frequency-dependent outcomes and so require analysis using evolutionary game theory. Learning provides a mechanism for tracking changing conditions and it has frequently been predicted to supplant fixed behaviour in shifting environments; yet few studies have examined the evolution of learning specifically in a game-theoretic context. We present a model that examines the evolution of learning in a frequency-dependent context created by a producer–scrounger game, where producers search for their own resources and scroungers usurp the discoveries of producers. We ask whether a learning mutant that can optimize its use of producer and scrounger to local conditions can invade a population of non-learning individuals that play producer and scrounger with fixed probabilities. We find that learning provides an initial advantage but never evolves to fixation. Once a stable equilibrium is attained, the population is always made up of a majority of fixed players and a minority of learning individuals. This result is robust to variation in the initial proportion of fixed individuals, the rate of within- and between-generation environmental change, and population size. Such learning polymorphisms will manifest themselves in a wide range of contexts, providing an important element leading to behavioural syndromes.     

On the evolution of omnivory in a community context

Omnivory is extremely common in animals, yet theory predicts that when given a choice of resources specialization should be favored over being generalist. The evolution of a feeding phenotype involves complex interactions with many factors other than resource choice alone, including environmental heterogeneity, resource quality, availability, and interactions with other organisms. We applied an evolutionary simulation model to examine how ecological conditions shape evolution of feeding phenotypes (e.g., omnivory), by varying the quality and availability (absolute and relative) of plant and animal (prey) resources. Resulting feeding phenotypes were defined by the relative contribution of plants and prey to diets of individuals. We characterized organisms using seven traits that were allowed to evolve freely in different simulated environments, and we asked which traits are important for different feeding phenotypes to evolve among interacting organisms. Carnivores, herbivores, and omnivores all coexisted without any requirement in the model for a synergistic effect of eating plant and animal prey. Omnivores were most prevalent when ratio of plants and animal prey was low, and to a lesser degree, when habitat productivity was high. A key result of the model is that omnivores evolved through many different combinations of trait values and environmental contexts. Specific combinations of traits tended to form emergent trait complexes, and under certain environmental conditions, are expressed as omnivorous feeding phenotypes. The results indicate that relative availabilities of plants and prey (over the quality of resources) determine an individual's feeding class and that feeding phenotypes are often the product of convergent evolution of emergent trait complexes under specific environmental conditions. Foraging outcomes appear to be consequences of degree and type of phenotypic specialization for plant and animal prey, navigation and exploitation of the habitat, reproduction, and interactions with other individuals in a heterogeneous environment. Omnivory should not be treated as a fixed strategy, but instead a pattern of phenotypic expression, emerging from diverse genetic sources and coevolving across a range of ecological contexts.     

Strain differences in a high response-cost daily time-place learning task

Previous research has shown that rats, unlike birds, do not readily demonstrate daily time-place learning (TPL). It has been suggested, however, that rats are more successful at these tasks if the response cost (RC) is increased. Widman et al. (2000) found that female Sprague Dawley (SD) rats learned a daily TPL task in which they were required to climb different towers depending on the time of day to find a food reward. Using a similar apparatus, we found that male SD rats learned the task, while male Long Evans rats did not. While all rats quickly learned to restrict the majority of their searching to the two towers that provided food, only the SD rats learned to go to the correct location at the correct time of day. Thus, there appears to be a strain difference in the effectiveness of a high RC task to promote learning. Tests of the timing strategies used revealed individual differences with one rat using a circadian strategy and another using an ordinal strategy. Post criterion decrements in performance did not allow sufficient testing to determine the timing strategies of the remaining rats. Possible interactions between strain, response cost, species typical behaviors and dependent measures are discussed.     

Eco-Evolutionary Feedback and the Tuning of Proto-Developmental Life Cycles

Multicellular organisms depend on developmental programs to coordinate growth and differentiation from single cells, but the origins of development are unclear. A possible starting point is stochastic phenotypic variation generated by molecular noise. Given appropriate environmental conditions, noise-driven differentiation could conceivably evolve so as to come under regulatory control; however, abiotic conditions are likely to be restrictive. Drawing from an experimental system, we present a model in which environmental fluctuations are coupled to population growth. We show that this coupling generates stable selection for a single optimal strategy that is largely insensitive to environmental conditions, including the number of competitors, carrying capacity of the environment, difference in growth rates among phenotypic variants, and population density. We argue that this optimal strategy establishes stabilizing conditions likely to improve the quality and reliability of information experienced by evolving organisms, thus increasing opportunity for the evolutionary emergence of developmental programs.     

Evolution of learning in fluctuating environments: when selection favors both social and exploratory individual learning

Cumulative cultural change requires organisms that are capable of both exploratory individual learning and faithful social learning. In our model, an organism's phenotype is initially determined innately (by its genotypic value) or by social learning (copying a phenotype from the parental generation), and then may or may not be modified by individual learning (exploration around the initial phenotype). The environment alternates periodically between two states, each defined as a certain range of phenotypes that can survive. These states may overlap, in which case the same phenotype can survive in both states, or they may not. We find that a joint social and exploratory individual learning strategy-the strategy that supports cumulative culture-is likely to spread when the environmental states do not overlap. In particular, when the environmental states are contiguous and mutation is allowed among the genotypic values, this strategy will spread in either moderately or highly stable environments, depending on the exact nature of the individual learning applied. On the other hand, natural selection often favors a social learning strategy without exploration when the environmental states overlap. We find only partial support for the "consensus" view, which holds that individual learning, social learning, and innate determination of behavior will evolve at short, intermediate, and long environmental periodicities, respectively.     

The evolution of conformist social learning can cause population collapse in realistically variable environments

Why do societies collapse? We use an individual-based evolutionary model to show that, in environmental conditions dominated by low-frequency variation (“red noise”), extirpation may be an outcome of the evolution of cultural capacity. Previous analytical models predicted an equilibrium between individual learners and social learners, or a contingent strategy in which individuals learn socially or individually depending on the circumstances. However, in red noise environments, whose main signature is that variation is concentrated in relatively large, relatively rare excursions, individual learning may be selected from the population. If the social learning system comes to lack sufficient individual learning or cognitively costly adaptive biases, behavior ceases tracking environmental variation. Then, when the environment does change, fitness declines and the population may collapse or even be extirpated. The modeled scenario broadly fits some human population collapses and might also explain nonhuman extirpations. Varying model parameters showed that the fixation of social learning is less likely when individual learning is less costly, when the environment is less red or more variable, with larger population sizes, and when learning is not conformist or is from parents rather than from the general population. Once social learning is fixed, extirpation is likely except when social learning is biased towards successful models. Thus, the risk of population collapse may be reduced by promoting individual learning and innovation over cultural conformity, or by preferential selection of relatively fit individuals as models for social learning.     

ASSORTMENT AND THE EVOLUTION OF GENERALIZED RECIPROCITY

Reciprocity is often invoked to explain cooperation. Reciprocity is cognitively demanding, and both direct and indirect reciprocity require that individuals store information about the propensity of their partners to cooperate. By contrast, generalized reciprocity, wherein individuals help on the condition that they received help previously, only relies on whether an individual received help in a previous encounter. Such anonymous information makes generalized reciprocity hard to evolve in a well‐mixed population, as the strategy will lose out to pure defectors. Here we analyze a model for the evolution of generalized reciprocity, incorporating assortment of encounters, to investigate the conditions under which it will evolve. We show that, in a well‐mixed population, generalized reciprocity cannot evolve. However, incorporating assortment of encounters can favor the evolution of generalized reciprocity in which indiscriminate cooperation and defection are both unstable. We show that generalized reciprocity can evolve under both the prisoner's dilemma and the snowdrift game.     

From reciprocity to unconditional altruism through signalling benefits

Cooperation among genetically unrelated individuals is commonly explained by the potential for future reciprocity or by the risk of being punished by group members. However, unconditional altruism is more difficult to explain. We demonstrate that unconditional altruism can evolve as a costly signal of individual quality (i.e. a handicap) as a consequence of reciprocal altruism. This is because the emergent correlation between altruism and individual quality in reciprocity games can facilitate the use of altruism as a quality indicator in a much wider context, outside the reciprocity game, thus affecting its further evolution through signalling benefits. Our model, based on multitype evolutionary game theory shows that, when the additive signalling benefit of donating help exceeds the cost for only some individuals (of high-quality state) but not for others (of low-quality state), the population possesses an evolutionarily stable strategy (ESS) profile wherein high-quality individuals cooperate unconditionally while low-quality individuals defect or play tit-for-tat (TfT). Hence, as predicted by Zahavi's handicap model, signalling benefits of altruistic acts can establish a stable generosity by high-quality individuals that no longer depends on the probability of future reciprocation or punishment.     

Harvesting strategies for Norwegian spring-spawning herring

The overfishing of an increasing number of fish populations has put focus on the need for development of robust sustainable harvest strategies that can be easily implemented. This requires estimates and modelling of the deterministic and stochastic components of the population dynamics as well as an evaluation of the contribution of different harvest strategies to future population fluctuations. Here we present an example of such an approach, using the collapse of Norwegian spring-spawning herring stock as a case. We demonstrate that the collapse probably was due to overfishing, and that the large influence of the environmental stochasticity could only influence the timing of the collapse. We suggest that a proportional threshold strategy with a threshold around 14 billion individuals (4 200 000 tons), combined with a harvest of 30–40% of the individuals above this threshold will give a sustainable yield with little annual variation. The choice of harvest strategy should also be strongly influenced by the uncertainty in the assessment of stock size. When the population stock is estimated with uncertainty, the proportional threshold strategy give a mean annual yield close to the optimum for known population size.