Meta-control of social learning strategies

Social learning, copying other’s behavior without actual experience, offers a cost-effective means of knowledge acquisition. However, it raises the fundamental question of which individuals have reliable information: successful individuals versus the majority. The former and the latter are known respectively as success-based and conformist social learning strategies. We show here that while the success-based strategy fully exploits the benign environment of low uncertainly, it fails in uncertain environments. On the other hand, the conformist strategy can effectively mitigate this adverse effect. Based on these findings, we hypothesized that meta-control of individual and social learning strategies provides effective and sample-efficient learning in volatile and uncertain environments. Simulations on a set of environments with various levels of volatility and uncertainty confirmed our hypothesis. The results imply that meta-control of social learning affords agents the leverage to resolve environmental uncertainty with minimal exploration cost, by exploiting others’ learning as an external knowledge base.     

How copying affects the amount, evenness and persistence of cultural knowledge: insights from the social learning strategies tournament

Darwinian processes should favour those individuals that deploy the most effective strategies for acquiring information about their environment. We organized a computer-based tournament to investigate which learning strategies would perform well in a changing environment. The most successful strategies relied almost exclusively on social learning (here, learning a behaviour performed by another individual) rather than asocial learning, even when environments were changing rapidly; moreover, successful strategies focused learning effort on periods of environmental change. Here, we use data from tournament simulations to examine how these strategies might affect cultural evolution, as reflected in the amount of culture (i.e. number of cultural traits) in the population, the distribution of cultural traits across individuals, and their persistence through time. We found that high levels of social learning are associated with a larger amount of more persistent knowledge, but a smaller amount of less persistent expressed behaviour, as well as more uneven distributions of behaviour, as individuals concentrated on exploiting a smaller subset of behaviour patterns. Increased rates of environmental change generated increases in the amount and evenness of behaviour. These observations suggest that copying confers on cultural populations an adaptive plasticity, allowing them to respond to changing environments rapidly by drawing on a wider knowledge base.     

Simulated Trust: A cheap social learning strategy

Animals use heuristic strategies to determine from which conspecifics to learn socially. This leads to directed social learning. Directed social learning protects them from copying non-adaptive information. So far, the strategies of animals, leading to directed social learning, are assumed to rely on (possibly indirect) inferences about the demonstrator’s success. As an alternative to this assumption, we propose a strategy that only uses self-established estimates of the pay-offs of behavior. We evaluate the strategy in a number of agent-based simulations. Critically, the strategy’s success is warranted by the inclusion of an incremental learning mechanism. Our findings point out new theoretical opportunities to regulate social learning for animals. More broadly, our simulations emphasize the need to include a realistic learning mechanism in game-theoretic studies of social learning strategies, and call for re-evaluation of previous findings.     

Bayesian approximations and extensions: Optimal decisions for small brains and possibly big ones too

We compared the performance of Bayesian learning strategies and approximations to such strategies, which are far less computationally demanding, in a setting requiring individuals to make binary decisions based on experience. Extending Bayesian updating schemes, we compared the different strategies while allowing for various implementations of memory and knowledge about the environment. The dynamics of the observable variables was modeled through basic probability distributions and convolution. This theoretical framework was applied to the problem of male fruit flies who have to decide which females they should court. Computer simulations indicated that, for most parameter values, approximations to the Bayesian strategy performed as well as the full Bayesian one. The linear approximation, reminiscent of the linear operator, was notably successful, and, without innate knowledge, the only successful learning strategy. Besides being less demanding in computation and thus realistic for small brains, the linear approximation was also successful at limited memory, which would translate into robustness in rapidly changing environments. Knowledge about the environment boosted the performance of the various learning strategies with maximal performance at large utilization of memory. Only for limited memory capacities, intermediate knowledge was most successful. We conclude that many animals may rely on algorithms that involve approximations rather than full Bayesian calculations because such approximations achieve high levels of performance with only a fraction of the computational requirements, in particular for extensions of Bayesian updating schemes, which can represent universal and realistic environments.     

Co-evolution of learning complexity and social foraging strategies

Variation in learning abilities within populations suggests that complex learning may not necessarily be more adaptive than simple learning. Yet, the high cost of complex learning cannot fully explain this variation without some understanding of why complex learning is too costly for some individuals but not for others. Here we propose that different social foraging strategies can favor different learning strategies (that learn the environment with high or low resolution), thereby maintaining variable learning abilities within populations. Using a genetic algorithm in an agent-based evolutionary simulation of a social foraging game (the producer-scrounger game) we demonstrate how an association evolves between a strategy based on independent search for food (playing a producer) and a complex (high resolution) learning rule, while a strategy that combines independent search and following others (playing a scrounger) evolves an association with a simple (low resolution) learning rule. The reason for these associations is that for complex learning to have an advantage, a large number of learning steps, normally not achieved by scroungers, are necessary. These results offer a general explanation for persistent variation in cognitive abilities that is based on co-evolution of learning rules and social foraging strategies.     

Tolerance of understory plants subject to herbivory by roe deer

Classical foraging theory states that animals feeding in a patchy environment can maximise their long term prey capture rates by quitting food patches when they have depleted prey to a certain threshold level. Theory suggests that social foragers may be better able to do this if all individuals in a group have access to the prey capture information of all other group members. This will allow all foragers to make a more accurate estimation of the patch quality over time and hence enable them to quit patches closer to the optimal prey threshold level. We develop a model to examine the foraging efficiency of three strategies that could be used by a cohesive foraging group to initiate quitting a patch, where foragers do not use such information, and compare these with a fourth strategy in which foragers use public information of all prey capture events made by the group. We carried out simulations in six different prey environments, in which we varied the mean number of prey per patch and the variance of prey number between patches. Groups sharing public information were able to consistently quit patches close to the optimal prey threshold level, and obtained constant prey capture rates, in groups of all sizes. In contrast all groups not sharing public information quit patches progressively earlier than the optimal prey threshold value, and experienced decreasing prey capture rates, as group size increased. This is more apparent as the variance in prey number between patches increases. Thus in a patchy environment, where uncertainty is high, although public information use does not increase the foraging efficiency of groups over that of a lone forager, it certainly offers benefits over groups which do not, and particularly where group size is large.     

Information processing in multigame environments modeling the evolution of sex via punctuated equilibria

Uncertain environments are properly described by probability distributions which, as usual, can be collapsed or conditioned into distributions with reduced uncertainty through the processing of environmental information. Organisms which force this collapse gain evolutionary advantage by being able to employ strategies in a known environment rather than in a merely probable one. The accrued benefit gained from processing information can be precisely quantified by comparing benefits returned using distributions prior to, and after collapse, and these often large and immediate benefits can amply justify the evolutionary cost of information processing systems. More importantly, the evolution of information processing systems must necessarily occur in a predictable evolutionary sequence from less complex to more complex. Practical applications include modeling the evolution of sex modeled here as a sequence from asexual reproduction, to single gene exchange, to gene packet exchange, to same species packet exchange, and finally to sexual reproduction, sexual selection, Red Queen contests and so on. Modeling this sequence requires extensions to game theory originally designed to model a single game, to allow the simultaneous operation of many games. This extension is called a multigame environment. The dynamical evolution of the development sequence shows punctuated equilibria.     

Strategies for integrating disparate social information

Social information use is widespread in the animal kingdom, helping individuals rapidly acquire useful knowledge and adjust to novel circumstances. In humans, the highly interconnected world provides ample opportunities to benefit from social information but also requires navigating complex social environments with people holding disparate or conflicting views. It is, however, still largely unclear how people integrate information from multiple social sources that (dis)agree with them, and among each other. We address this issue in three steps. First, we present a judgement task in which participants could adjust their judgements after observing the judgements of three peers. We experimentally varied the distribution of this social information, systematically manipulating its variance (extent of agreement among peers) and its skewness (peer judgements clustering either near or far from the participant''s judgement). As expected, higher variance among peers reduced their impact on behaviour. Importantly, observing a single peer confirming a participant''s own judgement markedly decreased the influence of other—more distant—peers. Second, we develop a framework for modelling the cognitive processes underlying the integration of disparate social information, combining Bayesian updating with simple heuristics. Our model accurately accounts for observed adjustment strategies and reveals that people particularly heed social information that confirms personal judgements. Moreover, the model exposes strong inter-individual differences in strategy use. Third, using simulations, we explore the possible implications of the observed strategies for belief updating. These simulations show how confirmation-based weighting can hamper the influence of disparate social information, exacerbate filter bubble effects and deepen group polarization. Overall, our results clarify what aspects of the social environment are, and are not, conducive to changing people''s minds.     

Efficient or Inaccurate? Analytical and Numerical Modelling of Random Search Strategies

A large number of observational and theoretical studies have investigated animal movement strategies for finding randomly located food items. Many of these studies have claimed that a particular strategy is advantageous over other strategies or that the spatial distribution of the food items affects the search efficiency. Here, we study a deliberately idealised problem, in which a blind forager searches for re-visitable food items. We show analytically that the forager’s efficiency is completely independent of both its movement strategy and the spatial pattern of the food items and depends only on the density of food in the environment. However, in some cases, apparent optima in search strategies can arise as artefacts of inappropriate and inaccurate numerical simulations. We discuss modifications to the idealised foraging problem that can confer an advantage on certain strategies, including when the forager has some memory or knowledge of the environment; when the food items are non-revisitable; and when the problem is viewed in an evolutionary context.     

Coevolution of honest signaling and cooperative norms by cultural group selection

Evolution of cooperative norms is studied in a population where individual and group level selection are both in operation. Individuals play indirect reciprocity game within their group and follow second order norms. Individuals are norm-followers, and imitate their successful group mates. Aside from direct observation individuals can be informed about the previous actions and reputations by information transferred by others. A potential donor estimates the reputation of a potential receiver either by her own observation or by the opinion of the majority of others (indirect observation). Following a previous study (Scheuring, 2009) we assume that norms determine only the probabilities of actions, and mutants can differ in these probabilities. Similarly, we assume that individuals follow a stochastic information transfer strategy. The central question is whether cooperative norm and honest social information transfer can emerge in a population where initially only non-cooperative norms were present, and the transferred information was not sufficiently honest.It is shown that evolution can lead to a cooperative state where information transferred in a reliable manner, where generous cooperative strategies are dominant. This cooperative state emerges along a sharp transition of norms. We studied the characteristics of actions and strategies in this transition by classifying the stochastic norms, and found that a series of more and more judging strategies invade each other before the stabilization of the so-called generous judging strategy. Numerical experiments on the coevolution of social parameters (e.g. probability of direct observation and the number of indirect observers) reveal that it is advantageous to lean on indirect observation even if information transfer is much noisier than for direct observation, which is because to follow the majorities’ opinion suppresses information noise meaningfully.     

An experimental comparison of human social learning strategies: payoff-biased social learning is adaptive but underused

Analytical models have identified a set of social learning strategies that are predicted to be adaptive relative to individual (asocial) learning. In the present study, human participants engaged in an ecologically valid artifact-design task with the opportunity to engage in a range of social learning strategies: payoff bias, conformity, averaging and random copying. The artifact (an arrowhead) was composed of multiple continuous and discrete attributes which jointly generated a complex multimodal adaptive landscape that likely reflects actual cultural fitness environments. Participants exhibited a mix of individual learning and payoff-biased social learning, with negligible frequencies of the other social learning strategies. This preference for payoff-biased social learning was evident from the initial trials, suggesting that participants came into the study with an intrinsic preference for this strategy. There was also a small but significant increase in the frequency of payoff-biased social learning over sessions, suggesting that strategy choice may itself be subject to learning. Frequency of payoff-biased social learning predicted both absolute and relative success in the task, especially in a multimodal (rather than unimodal) fitness environment. This effect was driven by a minority of hardcore social learners who copied the best group member on more than half of trials. These hardcore social learners were also above-average individual learners, suggesting a link between individual and social learning ability. The lower-than-expected frequency of social learning may reflect the existence of information producer–scrounger dynamics in human populations.     

Exploiting Social Networks to Mitigate the Obesity Epidemic

Despite significant efforts, obesity continues to be a major public health problem, and there are surprisingly few effective strategies for its prevention and treatment. We now realize that healthy diet and activity patterns are difficult to maintain in the current physical environment. Recently, it was suggested that the social environment also contributes to obesity. Therefore, using network‐based interaction models, we simulate how obesity spreads along social networks and predict the effectiveness of large‐scale weight management interventions. For a wide variety of conditions and networks, we show that individuals with similar BMIs will cluster together into groups, and if left unchecked, current social forces will drive these groups toward increasing obesity. Our simulations show that many traditional weight management interventions fail because they target overweight and obese individuals without consideration of their surrounding cluster and wider social network. The popular strategy for dieting with friends is shown to be an ineffective long‐term weight loss strategy, whereas dieting with friends of friends can be somewhat more effective by forcing a shift in cluster boundaries. Fortunately, our simulations also show that interventions targeting well‐connected and/or normal weight individuals at the edges of a cluster may quickly halt the spread of obesity. Furthermore, by changing social forces and altering the behavior of a small but random assortment of both obese and normal weight individuals, highly effective network‐driven strategies can reverse current trends and return large segments of the population to a healthier weight.     

Chemo-enzymatic synthesis of the exocyclic olefin isomer of thymidine monophosphate

Exocyclic olefin variants of thymidylate (dTMP) recently have been proposed as reaction intermediates for the thymidyl biosynthesis enzymes found in many pathogenic organisms, yet synthetic reports on these materials are lacking. Here we report two strategies to prepare the exocyclic olefin isomer of dTMP, which is a putative reaction intermediate in pathogenic thymidylate biosynthesis and a novel nucleotide analog. Our most effective strategy involves preserving the existing glyosidic bond of thymidine and manipulating the base to generate the exocyclic methylene moiety. We also report a successful enzymatic deoxyribosylation of a non-aromatic nucleobase isomer of thymine, which provides an additional strategy to access nucleotide analogs with disrupted ring conjugation or with reduced heterocyclic bases. The strategies reported here are straightforward and extendable towards the synthesis of various pyrimidine nucleotide analogs, which could lead to compounds of value in studies of enzyme reaction mechanisms or serve as templates for rational drug design.     

Win-stay, lose-shift strategies for repeated games-memory length, aspiration levels and noise.

Win-stay, lose-shift, the principle to retain a successful action is a simple and general learning rule that can be applied to all types of repeated decision problems. In this paper I consider win-stay, lose-shift strategies with diverse memory sizes and strategies that adapt their aspiration levels, i.e. the payoff level considered as "success". I study their evolution for the Prisoner's Dilemma, as well as in a rapidly changing environment, where a randomly selected game is assigned to the players. For win-stay, lose-shift strategies with memory one the average payoffs are computed and their evolutionary stability is discussed. Using computer simulations I show that the win-stay, lose-shift strategies with longer memory are very successful both for the Prisoner's Dilemma, where cooperation dominates even for high noise levels, and the randomly assigned games, where the players achieve nearly the expected Pareto optimal payoffs. I discuss the impact of noise and show that the memory length of the players increases with the noise level. These results indicate that the win-stay, lose-shift principle is a very successful strategy in repeated games with noise.     

Generalized walks-based centrality measures for complex biological networks

A strategy for zooming in and out the topological environment of a node in a complex network is developed. This approach is applied here to generalize the subgraph centrality of nodes in complex networks. In this case the zooming in strategy is based on the use of some known matrix functions which allow focusing locally on the environment of a node. When a zooming out strategy is applied new matrix functions are introduced, which give a more global picture of the topological surrounds of a node. These indices permit a modulation of the scales at which the environment of a node influences its centrality. We apply them to the study of 10 protein-protein interaction (PPI) networks. We illustrate the similarities and differences between the generalized subgraph centrality indices as well as among them and some classical centrality measures. We show here that the use of centrality indices based on the zooming in strategy identifies a larger number of essential proteins in the yeast PPI network than any of the other centrality measures studied.     

Evolutionary dynamics of prey exploitation in a metapopulation of predators

In well-mixed populations of predators and prey, natural selection favors predators with high rates of prey consumption and population growth. When spatial structure prevents the populations from being well mixed, such predators may have a selective disadvantage because they do not make full use of the prey's growth capacity and hence produce fewer propagules. The best strategy then depends on the degree to which predators can monopolize the exploitation of local prey populations, which in turn depends on the spatial structure, the number of migrants, and, in particular, the stochastic nature of the colonization process. To analyze the evolutionary dynamics of predators in a spatially structured predator-prey system, we performed simulations with a metapopulation model that has explicit local dynamics of nonpersistent populations, keeps track of the number of emigrants entering the migration pool, assumes individuals within local populations as well as within the migration pool to be well mixed, and takes stochastic colonization into account. We investigated which of the predator's exploitation strategies are evolutionarily stable and whether these strategies minimize the overall density of prey, as is the case in Lotka-Volterra-type models of competitive exclusion. This was analyzed by pairwise invasibility plots based on short-term simulations and tested by long-term simulation experiments of competition between resident and mutant predator-types that differed in one of the following parameters: the prey-to-predator conversion efficiency, the per capita prey consumption rate, or the per capita emigration rate from local populations. In addition, we asked which of these three strategies are most likely to evolve. Our simulations showed that under selection for conversion efficiency the predator-prey system always goes globally extinct yet persists under selection for consumption or emigration rates and that the evolutionarily stable (ES) exploitation strategies do not maximize local population growth rates. The most successful exploitation strategy minimizes the overall density of prey but does not make it settle exactly at the minimum. The system did not settle at the point where the mean time to co-invasion (i.e., immigration of a second predator in a local prey population) equals the mean local interaction time (an idea borne out from studies on host exploitation strategies in host-pathogen systems) but rather where the mean time to co-invasion was larger. The ES exploitation strategies represent more prudent strategies than the ones that minimize prey density. Finally, we show that-compared to consumption-emigration is a more likely target for selection to achieve prudent exploitation and that prudent exploitation strategies can evolve only provided the prey-to-predator conversion efficiency is subject to constraints.     

Friendship, cliquishness, and the emergence of cooperation

The evolution of cooperation is a central problem in biology and the social sciences. While theoretical work using the iterated prisoner's dilemma (IPD) has shown that cooperation among non-kin can be sustained among reciprocal strategies (i.e. tit-for-tat), these results are sensitive to errors in strategy execution, cyclical invasions by free riders, and the specific ecology of strategies. Moreover, the IPD assumes that a strategy's probability of playing the PD game with other individuals is independent of the decisions made by others. Here, we remove the assumption of independent pairing by studying a more plausible cooperative dilemma in which players can preferentially interact with a limited set of known partners and also deploy longer-term accounting strategies that can counteract the effects of random errors. We show that cooperative strategies readily emerge and persist in a range of noisy environments, with successful cooperative strategies (henceforth, cliquers) maintaining medium-term memories for partners and low thresholds for acceptable cooperation (i.e. forgiveness). The success of these strategies relies on their cliquishness-a propensity to defect with strangers if they already have an adequate number of partners. Notably, this combination of medium-term accounting, forgiveness, and cliquishness fits with empirical studies of friendship and other long-term relationships among humans.     

Development of ‘strategies’ utilized by albino rats in operant conditioning

A study concerning an operant conditioning of albino rats over a period of 6 months showed that different subjects progressively adopted a highly stable behaviour which could be called a ‘strategy’. Each one of these ‘strategies’ enables the subject to obtain, in a defined manner, a certain number of reinforcements within a given period; despite their different efficiencies, each one of these ‘strategies’ can be considered as ‘correct’. We have shown here that the duration of the conditioning session and the mean yield per stimulus depend on the choice of a given ‘strategy’.     

Developments in Bioremediation of Soils and Sediments Polluted with Metals and Radionuclides: 2. Field Research on Bioremediation of Metals and Radionuclides

Bioremediation of metals and radionuclides has had many field tests, demonstrations, and full-scale implementations in recent years. Field research in this area has occurred for many different metals and radionuclides using a wide array of strategies. These strategies can be generally characterized in six major categories: biotransformation, bioaccumulation/bisorption, biodegradation of chelators, volatilization, treatment trains, and natural attenuation. For all field applications there are a number of critical biogeochemical issues that most be addressed for the successful field application. Monitoring and characterization parameters that are enabling to bioremediation of metals and radionuclides are presented here. For each of the strategies a case study is presented to demonstrate a field application that uses this strategy.     

THE EVOLUTION OF STRATEGY VARIATION: WILL AN ESS EVOLVE?

Evolutionarily stable strategy (ESS) models are widely viewed as predicting the strategy of an individual that when monomorphic or nearly so prevents a mutant with any other strategy from entering the population. In fact, the prediction of some of these models is ambiguous when the predicted strategy is "mixed", as in the case of a sex ratio, which may be regarded as a mixture of the subtraits "produce a daughter" and "produce a son." Some models predict only that such a mixture be manifested by the population as a whole, that is, as an "evolutionarily stable state"; consequently, strategy monomorphism or polymorphism is consistent with the prediction. The hawk-dove game and the sex-ratio game in a panmictic population are models that make such a "degenerate" prediction. We show here that the incorporation of population finiteness into degenerate models has effects for and against the evolution of a monomorphism (an ESS) that are of equal order in the population size, so that no one effect can be said to predominate. Therefore, we used Monte Carlo simulations to determine the probability that a finite population evolves to an ESS as opposed to a polymorphism. We show that the probability that an ESS will evolve is generally much less than has been reported and that this probability depends on the population size, the type of competition among individuals, and the number of and distribution of strategies in the initial population. We also demonstrate how the strength of natural selection on strategies can increase as population size decreases. This inverse dependency underscores the incorrectness of Fisher's and Wright's assumption that there is just one qualitative relationship between population size and the intensity of natural selection.