Evolving learning rules and emergence of cooperation in spatial prisoner's dilemma

In the evolutionary Prisoner's dilemma (PD) game, agents play with each other and update their strategies in every generation according to some microscopic dynamical rule. In its spatial version, agents do not play with every other but, instead, interact only with their neighbours, thus mimicking the existing of a social or contact network that defines who interacts with whom. In this work, we explore evolutionary, spatial PD systems consisting of two types of agents, each with a certain update (reproduction, learning) rule. We investigate two different scenarios: in the first case, update rules remain fixed for the entire evolution of the system; in the second case, agents update both strategy and update rule in every generation. We show that in a well-mixed population the evolutionary outcome is always full defection. We subsequently focus on two-strategy competition with nearest-neighbour interactions on the contact network and synchronised update of strategies. Our results show that, for an important range of the parameters of the game, the final state of the system is largely different from that arising from the usual setup of a single, fixed dynamical rule. Furthermore, the results are also very different if update rules are fixed or evolve with the strategies. In these respect, we have studied representative update rules, finding that some of them may become extinct while others prevail. We describe the new and rich variety of final outcomes that arise from this co-evolutionary dynamics. We include examples of other neighbourhoods and asynchronous updating that confirm the robustness of our conclusions. Our results pave the way to an evolutionary rationale for modelling social interactions through game theory with a preferred set of update rules.     

Synchronous and asynchronous updating in cellular automata.

We analyze the properties of a synchronous and of various asynchronous methods to iterate cellular automata. Asynchronous methods in which the time variable is not explicitly defined, operate by specifying an updating order of the cells. The statistical properties of this order have significant consequences for the dynamics and the patterns generated by the cellular automata. Stronger correlations between consecutive steps in the updating order result in more, artificial structure in the patterns. Among these step-driven methods, using random choice with replacement to pick the next cell for updating, yields results that are least influenced by the updating method. We also analyse a time-driven method in which the state transitions of single cells are governed by a probability per unit time that determines an exponential distribution of the waiting time until the next transition. The statistical properties of this method are completely independent of the size of the grid. Consecutive updating steps therefore show no correlation at all. The stationary states of a cellular automaton do not depend on whether a synchronous or asynchronous updating method is used. Their basins of attraction might, however, be vastly different under synchronous and asynchronous iteration. Cyclic dynamics occur only with synchronous updating.     

Comparing alternative models to empirical data: cognitive models of western scrub-jay foraging behavior

Animals often select one item from a set of candidates, as when choosing a foraging site or mate, and are expected to possess accurate and efficient rules for acquiring information and making decisions. Little is known, however, about the decision rules animals use. We compare patterns of information sampling by western scrub-jays (Aphelocoma californica) when choosing a nut with three decision rules: best of n (BN), flexible threshold (FT), and comparative Bayes (CB). First, we use a null hypothesis testing approach and find that the CB decision rule, in which individuals use past experiences to make nonrandom assessment and choice decisions, produces patterns of behavior that more closely correspond to observed patterns of nut sampling in scrub-jays than the other two rules. This approach does not allow us to quantify how much better CB is at predicting scrub-jay behavior than the other decision rules. Second, we use a model selection approach that uses Akaike Information Criteria to quantify how well alternative models approximate observed data. We find that the CB rule is much more likely to produce the observed patterns of scrub-jay behavior than the other rules. This result provides some of the best empirical evidence of the use of Bayesian information updating by a nonhuman animal.     

Learning to recognize objects

Several aspects of systems for learning pattern or object recognition rules are discussed. First, how are recognition rules developed and to what extent is structural pattern information embedded into these recognition rules. Second, how are these rules applied to the recognition of complex patterns such as objects embedded in scenes and how is evidence from different rules combined into a single evidence vector. Third, how can learned rules be improved through performance evaluation and feedback to rule generation stages.     

Differential neural activation for updating rule versus stimulus information in working memory

Establishing what information is actively maintained in working memory (WM) and how it is represented and controlled is essential to understanding how such information guides future behavior. WM has traditionally been investigated in terms of the maintenance of stimulus-specific information, such as locations or words. More recently, investigators have emphasized the importance of rules that establish relationships between those stimuli and the pending response. The current study used a mental arithmetic task with fMRI to test whether updating of numbers (i.e., stimuli) and updating of mathematical operations (i.e., rules) in WM relies on the same neural system. Results indicate that, while a common network is activated by both types of updating, rule updating preferentially activates prefrontal cortex while number updating preferentially activates parietal cortex. The results suggest that both numbers and rules are maintained in WM but that they are different types of information that are controlled independently.     

Two-category model of task allocation with application to ant societies

In many network models of interacting units such as cells or insects, the coupling coefficients between units are independent of the state of the units. Here we analyze the temporal behavior of units that can switch between two 'category' states according to rules that involve category-dependent coupling coefficients. The behaviors of the category populations resulting from the asynchronous random updating of units are first classified according to the signs of the coupling coefficients using numerical simulations. They range from isolated fixed points to lines of fixed points and stochastic attractors. These behaviors are then explained analytically using iterated function systems and birth-death jump processes. The main inspiration for our work comes from studies of non-hierarchical task allocation in, e.g., harvester ant colonies where temporal fluctuations in the numbers of ants engaged in various tasks occur as circumstances require and depend on interactions between ants. We identify interaction types that produce quick recovery from perturbations to an asymptotic behavior whose characteristics are function of the coupling coefficients between ants as well as between ants and their environment. We also compute analytically the probability density of the population numbers, and show that perturbations in our model decay twice as fast as in a model with random switching dynamics. A subset of the interaction types between ants yields intrinsic stochastic asymptotic behaviors which could account for some of the experimentally observed fluctuations. Such noisy trajectories are shown to be random walks with state-dependent biases in the 'category population' phase space. With an external stimulus, the parameters of the category-switching rules become time-dependent. Depending on the growth rate of the stimulus in comparison to its population-dependent decay rate, the dynamics may qualitatively differ from the case without stimulus. Our simple two-category model provides a framework for understanding the rich variety of behaviors in network dynamics with state-dependent coupling coefficients, and especially in task allocation processes with many tasks.     

Stability of complex spike timing-dependent plasticity in cerebellar learning

Dynamics of spike-timing dependent synaptic plasticity are analyzed for excitatory and inhibitory synapses onto cerebellar Purkinje cells. The purpose of this study is to place theoretical constraints on candidate synaptic learning rules that determine the changes in synaptic efficacy due to pairing complex spikes with presynaptic spikes in parallel fibers and inhibitory interneurons. Constraints are derived for the timing between complex spikes and presynaptic spikes, constraints that result from the stability of the learning dynamics of the learning rule. Potential instabilities in the parallel fiber synaptic learning rule are found to be stabilized by synaptic plasticity at inhibitory synapses if the inhibitory learning rules are stable, and conditions for stability of inhibitory plasticity are given. Combining excitatory with inhibitory plasticity provides a mechanism for minimizing the overall synaptic input. Stable learning rules are shown to be able to sculpt simple-spike patterns by regulating the excitability of neurons in the inferior olive that give rise to climbing fibers.     

Learning rules for optimal selection in a varying environment: mate choice revisited

The quality of a chosen partner can be one of the most significantfactors affecting an animal's long-term reproductive success.We investigate optimal mate choice rules in an environment wherethere is both local variation in the quality of potential mateswithin each local mating pool and spatial (or temporal) variationin the average quality of the pools themselves. In such a situation,a robust rule that works well across a variety of environmentswill confer a significant reproductive advantage. We formulatea full Bayesian model for updating information in such a varyingenvironment and derive the form of the rule that maximizes expectedreward in a spatially varying environment. We compare the theoreticalperformance of our optimal learning rule against both fixedthreshold rules and simpler near-optimal learning rules andshow that learning is most advantageous when both the localand environmental variances are large. We consider how optimalsimple learning rules might evolve and compare their evolutionwith that of fixed threshold rules using genetic algorithmsas minimal models of the relevant genetics. Our analysis pointsup the variety of ways in which a near-optimal rule can be expressed.Finally, we describe how our results extend to the case of temporallyvarying environments.     

Complexity of environment and parsimony of decision rules in insect societies

This paper shows how colonies of social insects process information and solve problems in a complex environment, while keeping some parsimony at the level of the individuals' decision rules. Two studies on ant foraging reveal the diversity of adaptive colony-level patterns that can be generated through self-organization, based on the same individual-level recruitment rules. Regarding prey scavenging, the "ability to retrieve the prey" rule accounts for changes in foraging patterns, with increasing prey size, that show all stages intermediate between an individual and a mass exploitation of food resources. Regarding liquid food foraging, the "ability to ingest a desired volume" rule enables a colony to adjust the number of tending ants to the honeydew production of aphids. In both cases, decision rules are based on intelligent criteria that intrinsically integrate information on multiple variables that are relevant to the ants. Furthermore, the environment can contribute directly to the emergence of collective patterns, independently of any individual behavioral changes. Each environmental factor, including abiotic ones, that alters the dynamics of information transfer in group-living animals should be reconsidered not simply as a constraint but also as a part of the decision-making process and as a agent that shapes the collective pattern.     

Win-stay-lose-learn promotes cooperation in the spatial prisoner's dilemma game

Holding on to one's strategy is natural and common if the later warrants success and satisfaction. This goes against widespread simulation practices of evolutionary games, where players frequently consider changing their strategy even though their payoffs may be marginally different than those of the other players. Inspired by this observation, we introduce an aspiration-based win-stay-lose-learn strategy updating rule into the spatial prisoner's dilemma game. The rule is simple and intuitive, foreseeing strategy changes only by dissatisfied players, who then attempt to adopt the strategy of one of their nearest neighbors, while the strategies of satisfied players are not subject to change. We find that the proposed win-stay-lose-learn rule promotes the evolution of cooperation, and it does so very robustly and independently of the initial conditions. In fact, we show that even a minute initial fraction of cooperators may be sufficient to eventually secure a highly cooperative final state. In addition to extensive simulation results that support our conclusions, we also present results obtained by means of the pair approximation of the studied game. Our findings continue the success story of related win-stay strategy updating rules, and by doing so reveal new ways of resolving the prisoner's dilemma.     

Sequential spiking neural P systems with exhaustive use of rules

Spiking neural P systems (SN P systems, for short) are a class of distributed parallel computing devices inspired from the way neurons communicate by means of spikes, where neurons work in parallel in the sense that each neuron that can fire should fire, but the work in each neuron is sequential in the sense that at most one rule can be applied at each computation step. In this work, we consider SN P systems with the restriction that at most one neuron can fire at each step, and each neuron works in an exhaustive manner (a kind of local parallelism - an applicable rule in a neuron is used as many times as possible). Such SN P systems are called sequential SN P systems with exhaustive use of rules. The computation power of sequential SN P systems with exhaustive use of rules is investigated. Specifically, characterizations of Turing computability and of semilinear sets of numbers are obtained, as well as a strict superclass of semilinear sets is generated. The results show that the computation power of sequential SN P systems with exhaustive use of rules is closely related with the types of spiking rules in neurons.     

Effects of patch number and dispersal patterns on population dynamics and synchrony

In this paper, we examine the effects of patch number and different dispersal patterns on dynamics of local populations and on the level of synchrony between them. Local population renewal is governed by the Ricker model and we also consider asymmetrical dispersal as well as the presence of environmental heterogeneity. Our results show that both population dynamics and the level of synchrony differ markedly between two and a larger number of local populations. For two patches different dispersal rules give very versatile dynamics. However, for a larger number of local populations the dynamics are similar irrespective of the dispersal rule. For example, for the parameter values yielding stable or periodic dynamics in a single population, the dynamics do not change when the patches are coupled with dispersal. High intensity of dispersal does not guarantee synchrony between local populations. The level of synchrony depends also on dispersal rule, the number of local populations, and the intrinsic rate of increase. In our study, the effects of density-independent and density-dependent dispersal rules do not show any consistent difference. The results call for caution when drawing general conclusions from models of only two interacting populations and question the applicability of a large number of theoretical papers dealing with two local populations.     

Designing boosting ensemble of relational fuzzy systems

A method frequently used in classification systems for improving classification accuracy is to combine outputs of several classifiers. Among various types of classifiers, fuzzy ones are tempting because of using intelligible fuzzy if-then rules. In the paper we build an AdaBoost ensemble of relational neuro-fuzzy classifiers. Relational fuzzy systems bond input and output fuzzy linguistic values by a binary relation; thus, fuzzy rules have additional, comparing to traditional fuzzy systems, weights - elements of a fuzzy relation matrix. Thanks to this the system is better adjustable to data during learning. In the paper an ensemble of relational fuzzy systems is proposed. The problem is that such an ensemble contains separate rule bases which cannot be directly merged. As systems are separate, we cannot treat fuzzy rules coming from different systems as rules from the same (single) system. In the paper, the problem is addressed by a novel design of fuzzy systems constituting the ensemble, resulting in normalization of individual rule bases during learning. The method described in the paper is tested on several known benchmarks and compared with other machine learning solutions from the literature.     

Swidden, Sawah, and In-Between: Agricultural Transformation in Borneo

The change from swidden to sawah cultivation in Tara'n Dayak villages in West Kalimantan, Indonesia, is presented as a long-term, complex incremental process in which distinct, unstable, and often confusing production technologies figure as transitional forms. The transitional phases are discussed in terms of their efficiency and sustainability. It is argued that the failure to perceive and understand long-term processes of agricultural change may result in both misinterpretation of technological patterns and environmental variables, as well as of rules of labor and resource sharing.     

Species co-occurrences and neutral models: reassessing J. M. Diamond's assembly rules

The question whether species co-occurrence patterns are non-random has intrigued ecology for more than two decades. Recently Gotelli and McCabe used meta-analysis to show that natural assemblages indeed tend to have non-random species co-occurrence patterns and that these patterns are in line with the predictions of Diamond's assembly rule model. Here I show that neutral ecological drift models are able to generate patterns in line with Diamond's assembly rules and very similar to the empirical results in Gotelli and McCabe. Ecological drift shifted species co-occurrence patterns (measured by C-scores, checkerboard scores and species combination scores) of model species placed into a grid of the 25 cells (sites; metacommunity sizes 5 to 25 species with 100 individuals each) significantly from an initial random pattern towards a pattern predicted by the assembly rule model of Diamond. These findings imply that instead of asking whether natural communities are structured according to some assembly rules we should ask whether these non-random patterns are generated by species interactions or by stochastic drift processes.     

Bergmann's rule; a concept cluster?

The science behind ecology has been contested for years, partially because of the misuse and misrepresentation of concepts within ecology. This paper discusses the use of Bergmann's rule, a fundamental rule of biogeography. The rule was proposed by Carl Bergmann in 1847 and was published only in German; therefore, the majority of researchers have relied on a single translation by Mayr suggesting that races from cooler climates tend to be larger in species of warm-blooded vertebrates than races of the same species living in warmer climates. That many scientists cannot go back to the original source of information because it has not been published in English has resulted in relying on others for interpretation and led to several problems, the largest of which is whether the definition of the rule should include the mechanism, which had been proposed by Bergmann. There has been a large field of research on the subject, but few tests of the mechanisms behind the observed phenomenon. We conducted a review of the literature on Bergmann's rule, and from this suggest (1) Bergmann's original rule be maintained (a direct translation is provided), (2) mechanism is inherent in Bergmann's rule and is required for a rule to be of scientific value; patterns should be labelled as trends, not rules, (3) the focus should be on falsifying hypothesized mechanisms rather than simply describing patterns, and (4) to truly evaluate Bergmann's rule in a scientific manner the original German source should be translated and made available to the scientific public.     

Attractor analysis of asynchronous Boolean models of signal transduction networks

Prior work on the dynamics of Boolean networks, including analysis of the state space attractors and the basin of attraction of each attractor, has mainly focused on synchronous update of the nodes’ states. Although the simplicity of synchronous updating makes it very attractive, it fails to take into account the variety of time scales associated with different types of biological processes. Several different asynchronous update methods have been proposed to overcome this limitation, but there have not been any systematic comparisons of the dynamic behaviors displayed by the same system under different update methods. Here we fill this gap by combining theoretical analysis such as solution of scalar equations and Markov chain techniques, as well as numerical simulations to carry out a thorough comparative study on the dynamic behavior of a previously proposed Boolean model of a signal transduction network in plants. Prior evidence suggests that this network admits oscillations, but it is not known whether these oscillations are sustained. We perform an attractor analysis of this system using synchronous and three different asynchronous updating schemes both in the case of the unperturbed (wild-type) and perturbed (node-disrupted) systems. This analysis reveals that while the wild-type system possesses an update-independent fixed point, any oscillations eventually disappear unless strict constraints regarding the timing of certain processes and the initial state of the system are satisfied. Interestingly, in the case of disruption of a particular node all models lead to an extended attractor. Overall, our work provides a roadmap on how Boolean network modeling can be used as a predictive tool to uncover the dynamic patterns of a biological system under various internal and environmental perturbations.     

Motor Demands Constrain Cognitive Rule Structures

Study of human executive function focuses on our ability to represent cognitive rules independently of stimulus or response modality. However, recent findings suggest that executive functions cannot be modularized separately from perceptual and motor systems, and that they instead scaffold on top of motor action selection. Here we investigate whether patterns of motor demands influence how participants choose to implement abstract rule structures. In a learning task that requires integrating two stimulus dimensions for determining appropriate responses, subjects typically structure the problem hierarchically, using one dimension to cue the task-set and the other to cue the response given the task-set. However, the choice of which dimension to use at each level can be arbitrary. We hypothesized that the specific structure subjects adopt would be constrained by the motor patterns afforded within each rule. Across four independent data-sets, we show that subjects create rule structures that afford motor clustering, preferring structures in which adjacent motor actions are valid within each task-set. In a fifth data-set using instructed rules, this bias was strong enough to counteract the well-known task switch-cost when instructions were incongruent with motor clustering. Computational simulations confirm that observed biases can be explained by leveraging overlap in cortical motor representations to improve outcome prediction and hence infer the structure to be learned. These results highlight the importance of sensorimotor constraints in abstract rule formation and shed light on why humans have strong biases to invent structure even when it does not exist.     

Information-driven self-organization: the dynamical system approach to autonomous robot behavior

In recent years, information theory has come into the focus of researchers interested in the sensorimotor dynamics of both robots and living beings. One root for these approaches is the idea that living beings are information processing systems and that the optimization of these processes should be an evolutionary advantage. Apart from these more fundamental questions, there is much interest recently in the question how a robot can be equipped with an internal drive for innovation or curiosity that may serve as a drive for an open-ended, self-determined development of the robot. The success of these approaches depends essentially on the choice of a convenient measure for the information. This article studies in some detail the use of the predictive information (PI), also called excess entropy or effective measure complexity, of the sensorimotor process. The PI of a process quantifies the total information of past experience that can be used for predicting future events. However, the application of information theoretic measures in robotics mostly is restricted to the case of a finite, discrete state-action space. This article aims at applying the PI in the dynamical systems approach to robot control. We study linear systems as a first step and derive exact results for the PI together with explicit learning rules for the parameters of the controller. Interestingly, these learning rules are of Hebbian nature and local in the sense that the synaptic update is given by the product of activities available directly at the pertinent synaptic ports. The general findings are exemplified by a number of case studies. In particular, in a two-dimensional system, designed at mimicking embodied systems with latent oscillatory locomotion patterns, it is shown that maximizing the PI means to recognize and amplify the latent modes of the robotic system. This and many other examples show that the learning rules derived from the maximum PI principle are a versatile tool for the self-organization of behavior in complex robotic systems.     

The spatiotemporal learning rule and its efficiency in separating spatiotemporal patterns

The hippocampus plays an important role in the course of establishing long-term memory, i.e., to make short-term memory of spatially and temporally associated input information. In 1996 (Tsukada et al. 1996), the spatiotemporal learning rule was proposed based on differences observed in hippocampal long-term potentiation (LTP) induced by various spatiotemporal pattern stimuli. One essential point of this learning rule is that the change of synaptic weight depends on both spatial coincidence and the temporal summation of input pulses. We applied this rule to a single-layered neural network and compared its ability to separate spatiotemporal patterns with that of other rules, including the Hebbian learning rule and its extended rules. The simulated results showed that the spatiotemporal learning rule had the highest efficiency in discriminating spatiotemporal pattern sequences, while the Hebbian learning rule (including its extended rules) was sensitive to differences in spatial patterns.