Circuits,hormones, and learning: Vocal behavior in songbirds

Species-typical vocal patterns subserve species identification and communication for individual organisms. Only a few groups of organisms learn the sounds used for vocal communication, including songbirds, humans, and cetaceans. Vocal learning in songbirds has come to serve as a model system for the study of brain-behavior relationships and neural mechanisms of learning and memory. Songbirds learn specific vocal patterns during a sensitive period of development via a complex assortment of neurobehavioral mechanisms. In many species of songbirds, the production of vocal behavior by adult males is used to defend territories and attract females, and both males and females must perceive vocal patterns and respond to them. In both juveniles and adults, specific types of auditory experience are necessary for initial song learning as well as the maintenance of stable song patterns. External sources of experience such as acoustic cues must be integrated with internal regulatory factors such as hormones, neurotransmitters, and cytokines for vocal patterns to be learned and produced. Thus, vocal behavior in songbirds is a culturally acquired trait that is regulated by multiple intrinsic as well as extrinsic factors. Here, we focus on functional relationships between circuitry and behavior in male songbirds. In that context, we consider in particular the influence of sex hormones on vocal behavior and its underlying circuitry, as well as the regulatory and functional mechanisms suggested by morphologic changes in the neural substrate for song control. We describe new data on the architecture of the song system that suggests strong similarities between the songbird vocal control system and neural circuits for memory, cognition, and use-dependent plasticity in the mammalian brain. © 1997 John Wiley & Sons, Inc. J Neurobiol 33: 602–618, 1997     

Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings

Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating.     

The evolution of cultural adaptations: Fijian food taboos protect against dangerous marine toxins

The application of evolutionary theory to understanding the origins of our species'' capacities for social learning has generated key insights into cultural evolution. By focusing on how our psychology has evolved to adaptively extract beliefs and practices by observing others, theorists have hypothesized how social learning can, over generations, give rise to culturally evolved adaptations. While much field research documents the subtle ways in which culturally transmitted beliefs and practices adapt people to their local environments, and much experimental work reveals the predicted patterns of social learning, little research connects real-world adaptive cultural traits to the patterns of transmission predicted by these theories. Addressing this gap, we show how food taboos for pregnant and lactating women in Fiji selectively target the most toxic marine species, effectively reducing a woman''s chances of fish poisoning by 30 per cent during pregnancy and 60 per cent during breastfeeding. We further analyse how these taboos are transmitted, showing support for cultural evolutionary models that combine familial transmission with selective learning from locally prestigious individuals. In addition, we explore how particular aspects of human cognitive processes increase the frequency of some non-adaptive taboos. This case demonstrates how evolutionary theory can be deployed to explain both adaptive and non-adaptive behavioural patterns.     

The adaptive importance of cognitive efficiency: an alternative theory of why we have beliefs and desires

Finding out why we have beliefs and desires is important for a thorough understanding of the nature of our minds (and those of other animals). It is therefore unsurprising that several accounts have been presented that are meant to answer this question. At least in the philosophical literature, the most widely accepted of these are due to Kim Sterelny and Peter Godfrey-Smith, who argue that beliefs and desires evolved due to their enabling us to be behaviourally flexible in a way that reflexes do not—which, they claim, is beneficial in epistemically complex environments. However, as I try to make clear in this paper, upon closer consideration, this kind of account turns out to be theoretically implausible. In the main, this is because it fails to give due credit to the powers of reflex-driven organisms, which can in fact be just as flexible in their behaviour as ones that are belief/desire-driven. In order to improve on this account, I therefore propose that beliefs and desires evolved, not due to their enabling us to do something completely different from what reflexive organisms can do, but rather due to their enabling us to do the same things better. Specifically, I argue that beliefs and desires evolved for making the generation of behaviour more efficient, since they can simplify the necessary cognitive labour considerably. I end by considering various implications of this account.     

Changing Museum Visitors’ Conceptions of Evolution

We examined whether a single visit to an evolution exhibition contributed to conceptual change in adult (n?=?30), youth, and child (n?=?34) museum visitors?? reasoning about evolution. The exhibition included seven current research projects in evolutionary science, each focused on a different organism. To frame this study, we integrated a developmental model of visitors?? understanding of evolution, which incorporates visitors?? intuitive beliefs, with a model of free-choice learning that includes personal, sociocultural, and contextual variables. Using pre- and post-measures, we assessed how visitors?? causal explanations about biological change, drawn from three reasoning patterns (evolutionary, intuitive, and creationist), were modified as a result of visiting the exhibition. Whatever their age, background beliefs, or prior intuitive reasoning patterns, visitors significantly increased their use of explanations from the evolutionary reasoning pattern across all measures and extended this reasoning across diverse organisms. Visitors also increased their use of one intuitive reasoning pattern, need-based (goal-directed) explanations, which, we argue, may be a step toward evolutionary reasoning. Nonetheless, visitors continued to use mixed reasoning (endorsing all three reasoning patterns) in explaining biological change. The personal, socio-cultural, and contextual variables were found to be related to these reasoning patterns in predictable ways. These findings are used to examine the structure of visitors?? reasoning patterns and those aspects of the exhibition that may have contributed to the gains in museum visitors?? understanding of evolution.     

Structure Learning in a Sensorimotor Association Task

Learning is often understood as an organism''s gradual acquisition of the association between a given sensory stimulus and the correct motor response. Mathematically, this corresponds to regressing a mapping between the set of observations and the set of actions. Recently, however, it has been shown both in cognitive and motor neuroscience that humans are not only able to learn particular stimulus-response mappings, but are also able to extract abstract structural invariants that facilitate generalization to novel tasks. Here we show how such structure learning can enhance facilitation in a sensorimotor association task performed by human subjects. Using regression and reinforcement learning models we show that the observed facilitation cannot be explained by these basic models of learning stimulus-response associations. We show, however, that the observed data can be explained by a hierarchical Bayesian model that performs structure learning. In line with previous results from cognitive tasks, this suggests that hierarchical Bayesian inference might provide a common framework to explain both the learning of specific stimulus-response associations and the learning of abstract structures that are shared by different task environments.     

A multiorganism based method for Bayesian gene network estimation

The primary goal of this article is to infer genetic interactions based on gene expression data. A new method for multiorganism Bayesian gene network estimation is presented based on multitask learning. When the input datasets are sparse, as is the case in microarray gene expression data, it becomes difficult to separate random correlations from true correlations that would lead to actual edges when modeling the gene interactions as a Bayesian network. Multitask learning takes advantage of the similarity between related tasks, in order to construct a more accurate model of the underlying relationships represented by the Bayesian networks. The proposed method is tested on synthetic data to illustrate its validity. Then it is iteratively applied on real gene expression data to learn the genetic regulatory networks of two organisms with homologous genes.     

Evolution of Associative Learning in Chemical Networks

Organisms that can learn about their environment and modify their behaviour appropriately during their lifetime are more likely to survive and reproduce than organisms that do not. While associative learning – the ability to detect correlated features of the environment – has been studied extensively in nervous systems, where the underlying mechanisms are reasonably well understood, mechanisms within single cells that could allow associative learning have received little attention. Here, using in silico evolution of chemical networks, we show that there exists a diversity of remarkably simple and plausible chemical solutions to the associative learning problem, the simplest of which uses only one core chemical reaction. We then asked to what extent a linear combination of chemical concentrations in the network could approximate the ideal Bayesian posterior of an environment given the stimulus history so far? This Bayesian analysis revealed the ‘memory traces’ of the chemical network. The implication of this paper is that there is little reason to believe that a lack of suitable phenotypic variation would prevent associative learning from evolving in cell signalling, metabolic, gene regulatory, or a mixture of these networks in cells.     

A Theoretical Analysis of Temporal Difference Learning in the Iterated Prisoner’s Dilemma Game

Direct reciprocity is a chief mechanism of mutual cooperation in social dilemma. Agents cooperate if future interactions with the same opponents are highly likely. Direct reciprocity has been explored mostly by evolutionary game theory based on natural selection. Our daily experience tells, however, that real social agents including humans learn to cooperate based on experience. In this paper, we analyze a reinforcement learning model called temporal difference learning and study its performance in the iterated Prisoner’s Dilemma game. Temporal difference learning is unique among a variety of learning models in that it inherently aims at increasing future payoffs, not immediate ones. It also has a neural basis. We analytically and numerically show that learners with only two internal states properly learn to cooperate with retaliatory players and to defect against unconditional cooperators and defectors. Four-state learners are more capable of achieving a high payoff against various opponents. Moreover, we numerically show that four-state learners can learn to establish mutual cooperation for sufficiently small learning rates.     

Does women's greater fear of snakes and spiders originate in infancy?

Previous studies with adult humans and nonhuman animals revealed more rapid fear learning for spiders and snakes than for mushrooms and flowers. The current experiments tested whether 11-month-olds show a similar effect in learning associative pairings between facial emotions and fear-relevant and fear-irrelevant stimuli. Consistent with the greater incidence of snake and spider phobias in women, results show that female but not male infants learn rapidly to associate negative facial emotions with fear-relevant stimuli. No difference was found between the sexes for fear-irrelevant stimuli. The results are discussed in relation to fear learning, phobias, and a specialized evolved fear mechanism in humans.     

Observing the observer (II): deciding when to decide

In a companion paper [1], we have presented a generic approach for inferring how subjects make optimal decisions under uncertainty. From a Bayesian decision theoretic perspective, uncertain representations correspond to "posterior" beliefs, which result from integrating (sensory) information with subjective "prior" beliefs. Preferences and goals are encoded through a "loss" (or "utility") function, which measures the cost incurred by making any admissible decision for any given (hidden or unknown) state of the world. By assuming that subjects make optimal decisions on the basis of updated (posterior) beliefs and utility (loss) functions, one can evaluate the likelihood of observed behaviour. In this paper, we describe a concrete implementation of this meta-Bayesian approach (i.e. a Bayesian treatment of Bayesian decision theoretic predictions) and demonstrate its utility by applying it to both simulated and empirical reaction time data from an associative learning task. Here, inter-trial variability in reaction times is modelled as reflecting the dynamics of the subjects' internal recognition process, i.e. the updating of representations (posterior densities) of hidden states over trials while subjects learn probabilistic audio-visual associations. We use this paradigm to demonstrate that our meta-Bayesian framework allows for (i) probabilistic inference on the dynamics of the subject's representation of environmental states, and for (ii) model selection to disambiguate between alternative preferences (loss functions) human subjects could employ when dealing with trade-offs, such as between speed and accuracy. Finally, we illustrate how our approach can be used to quantify subjective beliefs and preferences that underlie inter-individual differences in behaviour.     

The Evil Eye (għajn) in Malta: Grappling with Skinner’s pigeons and rehabilitating lame ducks1

In this paper, I construct a plea for superstition and examine the ways in which contemporary scholars use the term to denote irrational belief. ‘Superstition’ has, throughout history, been used as a derogatory term denoting inferior and dangerous beliefs. Examining the process whereby people continue to believe that which they deem irrational, I adopt a reflexive and phenomenological approach. Focussing on the evil eye (g?ajn) in Malta and the Mediterranean, I redefine ‘superstition’ as the product of an intrasubjective antinomy between orthodoxy and its subversion.     

Causal inference in biomolecular pathways using a Bayesian network approach and an Implicit method

We introduce here the concept of Implicit networks which provide, like Bayesian networks, a graphical modelling framework that encodes the joint probability distribution for a set of random variables within a directed acyclic graph. We show that Implicit networks, when used in conjunction with appropriate statistical techniques, are very attractive for their ability to understand and analyze biological data. Particularly, we consider here the use of Implicit networks for causal inference in biomolecular pathways. In such pathways, an Implicit network encodes dependencies among variables (proteins, genes), can be trained to learn causal relationships (regulation, interaction) between them and then used to predict the biological response given the status of some key proteins or genes in the network. We show that Implicit networks offer efficient methodologies for learning from observations without prior knowledge and thus provide a good alternative to classical inference in Bayesian networks when priors are missing. We illustrate our approach by an application to simulated data for a simplified signal transduction pathway of the epidermal growth factor receptor (EGFR) protein.     

Collective Learning and Optimal Consensus Decisions in Social Animal Groups

Learning has been studied extensively in the context of isolated individuals. However, many organisms are social and consequently make decisions both individually and as part of a collective. Reaching consensus necessarily means that a single option is chosen by the group, even when there are dissenting opinions. This decision-making process decouples the otherwise direct relationship between animals'' preferences and their experiences (the outcomes of decisions). Instead, because an individual''s learned preferences influence what others experience, and therefore learn about, collective decisions couple the learning processes between social organisms. This introduces a new, and previously unexplored, dynamical relationship between preference, action, experience and learning. Here we model collective learning within animal groups that make consensus decisions. We reveal how learning as part of a collective results in behavior that is fundamentally different from that learned in isolation, allowing grouping organisms to spontaneously (and indirectly) detect correlations between group members'' observations of environmental cues, adjust strategy as a function of changing group size (even if that group size is not known to the individual), and achieve a decision accuracy that is very close to that which is provably optimal, regardless of environmental contingencies. Because these properties make minimal cognitive demands on individuals, collective learning, and the capabilities it affords, may be widespread among group-living organisms. Our work emphasizes the importance and need for theoretical and experimental work that considers the mechanism and consequences of learning in a social context.     

Social learning of fear and safety is determined by the demonstrator's racial group

Social learning offers an efficient route through which humans and other animals learn about potential dangers in the environment. Such learning inherently relies on the transmission of social information and should imply selectivity in what to learn from whom. Here, we conducted two observational learning experiments to assess how humans learn about danger and safety from members (‘demonstrators'') of an other social group than their own. We show that both fear and safety learning from a racial in-group demonstrator was more potent than learning from a racial out-group demonstrator.     

From Birdsong to Human Speech Recognition: Bayesian Inference on a Hierarchy of Nonlinear Dynamical Systems

Our knowledge about the computational mechanisms underlying human learning and recognition of sound sequences, especially speech, is still very limited. One difficulty in deciphering the exact means by which humans recognize speech is that there are scarce experimental findings at a neuronal, microscopic level. Here, we show that our neuronal-computational understanding of speech learning and recognition may be vastly improved by looking at an animal model, i.e., the songbird, which faces the same challenge as humans: to learn and decode complex auditory input, in an online fashion. Motivated by striking similarities between the human and songbird neural recognition systems at the macroscopic level, we assumed that the human brain uses the same computational principles at a microscopic level and translated a birdsong model into a novel human sound learning and recognition model with an emphasis on speech. We show that the resulting Bayesian model with a hierarchy of nonlinear dynamical systems can learn speech samples such as words rapidly and recognize them robustly, even in adverse conditions. In addition, we show that recognition can be performed even when words are spoken by different speakers and with different accents—an everyday situation in which current state-of-the-art speech recognition models often fail. The model can also be used to qualitatively explain behavioral data on human speech learning and derive predictions for future experiments.     

Crocodile bites and traditional beliefs in Korogwe District,Tanzania.

OBJECTIVES--To investigate why fatal crocodile bites are increasing in a Tanzanian district and the importance of traditional beliefs and superstitions in determining the residents'' response to the crocodiles. DESIGN--Information about beliefs was obtained by interview of Korogwe residents. Human and crocodile fatality statistics were obtained from the Korogwe Department of Natural Resources. SETTING--Villages within Korogwe District. SUBJECTS--Population of Korogwe District. RESULTS--Crocodiles have been responsible for 51 deaths in the 52 months from January 1990 to April 1994. Of these, 18 deaths occurred in the first four months of 1994. CONCLUSIONS--Local beliefs and superstitions about crocodiles include those about the taming of animals, with implications concerning the choice of victim and the penalties that may ensue if a crocodile is killed. The recent rise in human fatalities is thought to relate to increasing river pollution reducing the fish supply, together with a change in social mores at the riverside which has increased the crocodiles'' displeasure. A reliable pumped water supply would reduce the need to draw water and bathe in the river, and eradication of superstition would empower the villagers in the fight against a common enemy.     

A Biophysical Model of Synaptic Delay Learning and Temporal Pattern Recognition in a Cerebellar Purkinje Cell

It has been suggested that information in the brain is encoded in temporal spike patterns which are decoded by a combination of time delays and coincidence detection. Here, we show how a multi-compartmental model of a cerebellar Purkinje cell can learn to recognise temporal parallel fibre activity patterns by adapting latencies of calcium responses after activation of metabotropic glutamate receptors (mGluRs). In each compartment of our model, the mGluR signalling cascade is represented by a set of differential equations that reflect the underlying biochemistry. Phosphorylation of the mGluRs changes the concentration of receptors which are available for activation by glutamate and thereby adjusts the time delay between mGluR stimulation and voltage response. The adaptation of a synaptic delay as opposed to a weight represents a novel non-Hebbian learning mechanism that can also implement the adaptive timing of the classically conditioned eye-blink response.     

A Bayesian Developmental Approach to Robotic Goal-Based Imitation Learning

A fundamental challenge in robotics today is building robots that can learn new skills by observing humans and imitating human actions. We propose a new Bayesian approach to robotic learning by imitation inspired by the developmental hypothesis that children use self-experience to bootstrap the process of intention recognition and goal-based imitation. Our approach allows an autonomous agent to: (i) learn probabilistic models of actions through self-discovery and experience, (ii) utilize these learned models for inferring the goals of human actions, and (iii) perform goal-based imitation for robotic learning and human-robot collaboration. Such an approach allows a robot to leverage its increasing repertoire of learned behaviors to interpret increasingly complex human actions and use the inferred goals for imitation, even when the robot has very different actuators from humans. We demonstrate our approach using two different scenarios: (i) a simulated robot that learns human-like gaze following behavior, and (ii) a robot that learns to imitate human actions in a tabletop organization task. In both cases, the agent learns a probabilistic model of its own actions, and uses this model for goal inference and goal-based imitation. We also show that the robotic agent can use its probabilistic model to seek human assistance when it recognizes that its inferred actions are too uncertain, risky, or impossible to perform, thereby opening the door to human-robot collaboration.     

A Comparative Study of Relational Learning Capacity in Honeybees (Apis mellifera) and Stingless Bees (Melipona rufiventris)

Background

Learning of arbitrary relations is the capacity to acquire knowledge about associations between events or stimuli that do not share any similarities, and use this knowledge to make behavioural choices. This capacity is well documented in humans and vertebrates, and there is some evidence it exists in the honeybee (Apis mellifera). However, little is known about whether the ability for relational learning extends to other invertebrates, although many insects have been shown to possess excellent learning capacities in spite of their small brains.

Methodology/Principal Findings

Using a symbolic matching-to-sample procedure, we show that the honeybee Apis mellifera rapidly learns arbitrary relations between colours and patterns, reaching 68.2% correct choice for pattern-colour relations and 73.3% for colour-pattern relations. However, Apis mellifera does not transfer this knowledge to the symmetrical relations when the stimulus order is reversed. A second bee species, the stingless bee Melipona rufiventris from Brazil, seems unable to learn the same arbitrary relations between colours and patterns, although it exhibits excellent discrimination learning.

Conclusions/Significance

Our results confirm that the capacity for learning arbitrary relations is not limited to vertebrates, but even insects with small brains can perform this learning task. Interestingly, it seems to be a species-specific ability. The disparity in relational learning performance between the two bee species we tested may be linked to their specific foraging and recruitment strategies, which evolved in adaptation to different environments.