Perceptual classification in a rapidly changing environment

Humans and monkeys can learn to classify perceptual information in a statistically optimal fashion if the functional groupings remain stable over many hundreds of trials, but little is known about categorization when the environment changes rapidly. Here, we used a combination of computational modeling and functional neuroimaging to understand how humans classify visual stimuli drawn from categories whose mean and variance jumped unpredictably. Models based on optimal learning (Bayesian model) and a cognitive strategy (working memory model) both explained unique variance in choice, reaction time, and brain activity. However, the working memory model was the best predictor of performance in volatile environments, whereas statistically optimal performance emerged in periods of relative stability. Bayesian and working memory models predicted decision-related activity in distinct regions of the prefrontal cortex and midbrain. These findings suggest that perceptual category judgments, like value-guided choices, may be guided by multiple controllers.     

The role of additive neurogenesis and synaptic plasticity in a hippocampal memory model with grid-cell like input

Recently, we presented a study of adult neurogenesis in a simplified hippocampal memory model. The network was required to encode and decode memory patterns despite changing input statistics. We showed that additive neurogenesis was a more effective adaptation strategy compared to neuronal turnover and conventional synaptic plasticity as it allowed the network to respond to changes in the input statistics while preserving representations of earlier environments. Here we extend our model to include realistic, spatially driven input firing patterns in the form of grid cells in the entorhinal cortex. We compare network performance across a sequence of spatial environments using three distinct adaptation strategies: conventional synaptic plasticity, where the network is of fixed size but the connectivity is plastic; neuronal turnover, where the network is of fixed size but units in the network may die and be replaced; and additive neurogenesis, where the network starts out with fewer initial units but grows over time. We confirm that additive neurogenesis is a superior adaptation strategy when using realistic, spatially structured input patterns. We then show that a more biologically plausible neurogenesis rule that incorporates cell death and enhanced plasticity of new granule cells has an overall performance significantly better than any one of the three individual strategies operating alone. This adaptation rule can be tailored to maximise performance of the network when operating as either a short- or long-term memory store. We also examine the time course of adult neurogenesis over the lifetime of an animal raised under different hypothetical rearing conditions. These growth profiles have several distinct features that form a theoretical prediction that could be tested experimentally. Finally, we show that place cells can emerge and refine in a realistic manner in our model as a direct result of the sparsification performed by the dentate gyrus layer.     

How copying affects the amount, evenness and persistence of cultural knowledge: insights from the social learning strategies tournament

Darwinian processes should favour those individuals that deploy the most effective strategies for acquiring information about their environment. We organized a computer-based tournament to investigate which learning strategies would perform well in a changing environment. The most successful strategies relied almost exclusively on social learning (here, learning a behaviour performed by another individual) rather than asocial learning, even when environments were changing rapidly; moreover, successful strategies focused learning effort on periods of environmental change. Here, we use data from tournament simulations to examine how these strategies might affect cultural evolution, as reflected in the amount of culture (i.e. number of cultural traits) in the population, the distribution of cultural traits across individuals, and their persistence through time. We found that high levels of social learning are associated with a larger amount of more persistent knowledge, but a smaller amount of less persistent expressed behaviour, as well as more uneven distributions of behaviour, as individuals concentrated on exploiting a smaller subset of behaviour patterns. Increased rates of environmental change generated increases in the amount and evenness of behaviour. These observations suggest that copying confers on cultural populations an adaptive plasticity, allowing them to respond to changing environments rapidly by drawing on a wider knowledge base.     

Learning and imprinting in stationary and non-stationary environment

The performance of the extended Bush-Mosteller learning and imprinting scheme developed previously is studied for stationary and non-stationary stochastic environments. As a performance criterion the average missing information level is chosen. For a stationary environment the approximate time course of the latter is derived and discussed, an exact symmetry in the performance of learning and imprinting schemes is proved, and the biological advantage of imprinting processes, with respect to energy consumption, is pointed out. For a non-stationary environment the performance of proper learning schemes is shown to be superior to imprinting processes, as the adaptability of the latter to novel environmental properties decreases exponentially in time. The optimal memory range of a learning system is calculated as a function of the time span during which the environment changes significantly and of the mean amplitude with which these changes occur.Supported by the Deutsche Forschungsgemeinschaft and the Humboldt Foundation.Fellow of the Humboldt Foundation; on leave of absence from the University of Poona, India.     

Towards expanding relevance vector machines to large scale datasets

In this paper we develop and analyze methods for expanding automated learning of Relevance Vector Machines (RVM) to large scale text sets. RVM rely on Bayesian inference learning and while maintaining state-of-the-art performance, offer sparse and probabilistic solutions. However, efforts towards applying RVM to large scale sets have met with limited success in the past, due to computational constraints. We propose a diversified set of divide-and-conquer approaches where decomposition techniques promote the definition of smaller working sets that permit the use of all training examples. The rationale is that by exploring incremental, ensemble and boosting strategies, it is possible to improve classification performance, taking advantage of the large training set available. Results on Reuters-21578 and RCV1 are presented, showing performance gains and maintaining sparse solutions that can be deployed in distributed environments.     

Improved discretisation and linearisation of active tension in strongly coupled cardiac electro-mechanics simulations

Mathematical models of cardiac electro-mechanics typically consist of three tightly coupled parts: systems of ordinary differential equations describing electro-chemical reactions and cross-bridge dynamics in the muscle cells, a system of partial differential equations modelling the propagation of the electrical activation through the tissue and a nonlinear elasticity problem describing the mechanical deformations of the heart muscle. The complexity of the mathematical model motivates numerical methods based on operator splitting, but simple explicit splitting schemes have been shown to give severe stability problems for realistic models of cardiac electro-mechanical coupling. The stability may be improved by adopting semi-implicit schemes, but these give rise to challenges in updating and linearising the active tension. In this paper we present an operator splitting framework for strongly coupled electro-mechanical simulations and discuss alternative strategies for updating and linearising the active stress component. Numerical experiments demonstrate considerable performance increases from an update method based on a generalised Rush–Larsen scheme and a consistent linearisation of active stress based on the first elasticity tensor.     

Smart,smarter, smartest: foraging information states and coexistence

Animals possess different abilities to gain and use information about the foraging patches they exploit. When ignorant of the qualities of encountered patches, a smart forager should leave all patches after the same amount of fixed search time. A smarter forager can be Bayesian by using information on cumulative harvest and time spent searching a patch to better inform its patch‐departure decision. The smartest forager has immediate and continuous knowledge about patch quality, and can make a perfect decision about when to leave each patch. Here we let each of these three strategies harvest resources from a slowly regenerating environment. Eventually a steady‐state distribution of prey among patches arises where the environment‐wide resource renewal just balances the environment‐wide harvest of the foragers. The fixed time forager creates a distribution with the highest mean and highest variance of patch qualities, followed by the Bayesian and the prescient in that order. The less informed strategies promote distributions with both more resources and more exploitable information than the more informed strategies. While it is true that a better‐informed strategy will always out‐perform a less well‐informed, its increase in performance may not compensate it for any costs associated with being better informed. We imagine that the fixed time strategy may be least expensive and the prescient strategy most expensive in terms of sensory organs and associated assess and respond capabilities. To consider competition between such strategies with varying costs, we introduced a single individual of each of the strategies into the environments created by populations of the other strategies. There are threshold costs associated with the better‐informed strategy such that it can or cannot outcompete a less‐informed strategy. However, over a relatively narrow range of foraging costs, less‐informed and better‐informed strategies will coexist. Furthermore, for the prescient and the Bayesian strategies, some combinations of foraging costs produce alternate stable states – whichever strategy establishes first remains safe from invasion by the other.     

Carving the cognitive niche: optimal learning strategies in homogeneous and heterogeneous environments

A model learning system is constructed, in which an organism samples behaviors from a behavioral repertoire in response to a stimulus and selects the behavior with the highest payoff. The stimulus and most rewarding behavior may be kept in the organism's long-term memory and reused if the stimulus is encountered again. The value of the memory depends on the reliability of the stimulus, that is, how the corresponding payoffs of behaviors change over time. We describe how the inclusion of memory can increase the optimal sampling size in environments with some stimulus reliability. In addition to using memory to guide behavior, our organism may use information in its memory to choose the stimulus to which it reacts. This choice is influenced by both the organism's memory state and how many stimuli the organism can observe (its sensory capability). The number of sampled behaviors, memory length, and sensory capability are the variables that define the learning strategy. When all stimuli have the same reliability, there appears to be only a single optimal learning strategy. However, when there is heterogeneity in stimulus reliability, multiple locally optimal strategies may exist.     

The Development of Metaphor Comprehension and Its Relationship with Relational Verbal Reasoning and Executive Function

Our main objective was to analyse the different contributions of relational verbal reasoning (analogical and class inclusion) and executive functioning to metaphor comprehension across development. We postulated that both relational reasoning and executive functioning should predict individual and developmental differences. However, executive functioning would become increasingly involved when metaphor comprehension is highly demanding, either because of the metaphors’ high difficulty (relatively novel metaphors in the absence of a context) or because of the individual’s special processing difficulties, such as low levels of reading experience or low semantic knowledge. Three groups of participants, 11-year-olds, 15-year-olds and young adults, were assessed in different relational verbal reasoning tasks—analogical and class-inclusion—and in executive functioning tasks—updating information in working memory, inhibition, and shifting. The results revealed clear progress in metaphor comprehension between ages 11 and 15 and between ages 15 and 21. However, the importance of executive function in metaphor comprehension was evident by age 15 and was restricted to updating information in working memory and cognitive inhibition. Participants seemed to use two different strategies to interpret metaphors: relational verbal reasoning and executive functioning. This was clearly shown when comparing the performance of the "more efficient" participants in metaphor interpretation with that of the "less efficient” ones. Whereas in the first case none of the executive variables or those associated with relational verbal reasoning were significantly related to metaphor comprehension, in the latter case, both groups of variables had a clear predictor effect.     

Path planning versus cue responding: a bio-inspired model of switching between navigation strategies

In this article, we describe a new computational model of switching between path-planning and cue-guided navigation strategies. It is based on three main assumptions: (i) the strategies are mediated by separate memory systems that learn independently and in parallel; (ii) the learning algorithms are different in the two memory systems—the cue-guided strategy uses a temporal-difference (TD) learning rule to approach a visible goal, whereas the path-planning strategy relies on a place-cell-based graph-search algorithm to learn the location of a hidden goal; (iii) a strategy selection mechanism uses TD-learning rule to choose the most successful strategy based on past experience. We propose a novel criterion for strategy selection based on the directions of goal-oriented movements suggested by the different strategies. We show that the selection criterion based on this “common currency” is capable of choosing the best among TD-learning and planning strategies and can be used to solve navigational tasks in continuous state and action spaces. The model has been successfully applied to reproduce rat behavior in two water-maze tasks in which the two strategies were shown to interact. The model was used to analyze competitive and cooperative interactions between different strategies during these tasks as well as relative influence of different types of sensory cues.     

The effectiveness and limitations of immune memory: understanding protective immune responses

Immune memory is the foundation of the practise of vaccination. Research on the molecular and cellular events leading to generation and development of memory T and B lymphocytes explain why there are heightened secondary immune responses after an initial encounter with antigen. In this review, we discuss how clonal expansion, targeted tissue localisation, more efficient antigen recognition and more proficient effector functions contribute to the improved effectiveness of memory cells. Despite the enhanced efficacy of memory cells and the recall immune response, there are numerous experimental and empirical examples in which protection provided by vaccines are short-lived, particularly against pathogens that replicate and cause pathology at their site of entry. In the absence of active immune effector activities, the ability of memory cells to respond quickly enough to control this type of infection is limited. The protective efficacy of bovine herpes virus-1 vaccines in experimental and field challenge conditions are used to illustrate the concept that full protection from disease conferred by vaccination requires the presence of active immune effector mechanisms. Thus, regardless of the many successful technological advances in vaccine design and better understanding of mechanisms underlining induction of memory responses by vaccination, we should recognise that vaccine immunoprophylaxis has limitations. Expectations for vaccines should be realistic and linked to the understanding of host immune responses and knowledge regarding the pathogen and disease pathogenesis.     

Win-stay, lose-shift strategies for repeated games-memory length, aspiration levels and noise.

Win-stay, lose-shift, the principle to retain a successful action is a simple and general learning rule that can be applied to all types of repeated decision problems. In this paper I consider win-stay, lose-shift strategies with diverse memory sizes and strategies that adapt their aspiration levels, i.e. the payoff level considered as "success". I study their evolution for the Prisoner's Dilemma, as well as in a rapidly changing environment, where a randomly selected game is assigned to the players. For win-stay, lose-shift strategies with memory one the average payoffs are computed and their evolutionary stability is discussed. Using computer simulations I show that the win-stay, lose-shift strategies with longer memory are very successful both for the Prisoner's Dilemma, where cooperation dominates even for high noise levels, and the randomly assigned games, where the players achieve nearly the expected Pareto optimal payoffs. I discuss the impact of noise and show that the memory length of the players increases with the noise level. These results indicate that the win-stay, lose-shift principle is a very successful strategy in repeated games with noise.     

Hippocampus, ageing, and taste memories

Previous studies have shown that ageing may induce deficits in hippocampal-dependent learning and memory tasks, the spatial task being most extensively applied in rats. It is proposed that taste learning and memory tasks may assist in understanding the ageing of memory systems, giving access to a more complete picture. Taste learning tasks allow us to explore a variety of learning phenomena in safe and aversive memories using similar behavioral procedures. In demanding the same sensory, response, and motivational requirements, this approach provides reliable comparisons between the performance of hippocampal lesioned and aged rats in different types of memory. Present knowledge on the effect of both ageing and hippocampal damage in complex taste learning phenomena is reviewed. Besides inducing deficits in hippocampal-dependent phenomena, such as blocking of conditioned taste aversion, while at the same time leaving intact nonhippocampal-dependent effects, such as latent inhibition, ageing is also associated with an increased neophobia by previous aversive taste memories and enhanced taste aversion conditioning which cannot be explained by age-related changes in taste or visceral distress sensitivity. In all, the results indicate a peculiar organization of the memory systems during aging that cannot be explained by a general cognitive decline or exclusively by the decay of the hippocampal function.     

Learning rules for optimal selection in a varying environment: mate choice revisited

The quality of a chosen partner can be one of the most significantfactors affecting an animal's long-term reproductive success.We investigate optimal mate choice rules in an environment wherethere is both local variation in the quality of potential mateswithin each local mating pool and spatial (or temporal) variationin the average quality of the pools themselves. In such a situation,a robust rule that works well across a variety of environmentswill confer a significant reproductive advantage. We formulatea full Bayesian model for updating information in such a varyingenvironment and derive the form of the rule that maximizes expectedreward in a spatially varying environment. We compare the theoreticalperformance of our optimal learning rule against both fixedthreshold rules and simpler near-optimal learning rules andshow that learning is most advantageous when both the localand environmental variances are large. We consider how optimalsimple learning rules might evolve and compare their evolutionwith that of fixed threshold rules using genetic algorithmsas minimal models of the relevant genetics. Our analysis pointsup the variety of ways in which a near-optimal rule can be expressed.Finally, we describe how our results extend to the case of temporallyvarying environments.     

Reasoning, learning, and creativity: frontal lobe function and human decision-making

The frontal lobes subserve decision-making and executive control--that is, the selection and coordination of goal-directed behaviors. Current models of frontal executive function, however, do not explain human decision-making in everyday environments featuring uncertain, changing, and especially open-ended situations. Here, we propose a computational model of human executive function that clarifies this issue. Using behavioral experiments, we show that unlike others, the proposed model predicts human decisions and their variations across individuals in naturalistic situations. The model reveals that for driving action, the human frontal function monitors up to three/four concurrent behavioral strategies and infers online their ability to predict action outcomes: whenever one appears more reliable than unreliable, this strategy is chosen to guide the selection and learning of actions that maximize rewards. Otherwise, a new behavioral strategy is tentatively formed, partly from those stored in long-term memory, then probed, and if competitive confirmed to subsequently drive action. Thus, the human executive function has a monitoring capacity limited to three or four behavioral strategies. This limitation is compensated by the binary structure of executive control that in ambiguous and unknown situations promotes the exploration and creation of new behavioral strategies. The results support a model of human frontal function that integrates reasoning, learning, and creative abilities in the service of decision-making and adaptive behavior.     

Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings

Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating.     

Long-term moderate elevation of corticosterone facilitates avian food-caching behaviour and enhances spatial memory

It is widely assumed that chronic stress and corresponding chronic elevations of glucocorticoid levels have deleterious effects on animals' brain functions such as learning and memory. Some animals, however, appear to maintain moderately elevated levels of glucocorticoids over long periods of time under natural energetically demanding conditions, and it is not clear whether such chronic but moderate elevations may be adaptive. I implanted wild-caught food-caching mountain chickadees (Poecile gambeli), which rely at least in part on spatial memory to find their caches, with 90-day continuous time-release corticosterone pellets designed to approximately double the baseline corticosterone levels. Corticosterone-implanted birds cached and consumed significantly more food and showed more efficient cache recovery and superior spatial memory performance compared with placebo-implanted birds. Thus, contrary to prevailing assumptions, long-term moderate elevations of corticosterone appear to enhance spatial memory in food-caching mountain chickadees. These results suggest that moderate chronic elevation of corticosterone may serve as an adaptation to unpredictable environments by facilitating feeding and food-caching behaviour and by improving cache-retrieval efficiency in food-caching birds.     

Machine learning in quantitative histopathology

The role of expert systems functioning as process controllers in learning image understanding systems is discussed. Numeric learning systems already have found a number of applications in cytologic and histopathologic diagnosis. Depending on the required capabilities, systems of increasing complexity are needed. Expert systems to guide scene segmentation in histopathologic imagery require model-based reasoning. Diagnostic image interpretation with learning capability demands a full model of the human expert's competence, including a considerable variety of knowledge representation schemes and inference strategies, coordinated by a meta-process controller.