Causal inference in multisensory perception

Perceptual events derive their significance to an animal from their meaning about the world, that is from the information they carry about their causes. The brain should thus be able to efficiently infer the causes underlying our sensory events. Here we use multisensory cue combination to study causal inference in perception. We formulate an ideal-observer model that infers whether two sensory cues originate from the same location and that also estimates their location(s). This model accurately predicts the nonlinear integration of cues by human subjects in two auditory-visual localization tasks. The results show that indeed humans can efficiently infer the causal structure as well as the location of causes. By combining insights from the study of causal inference with the ideal-observer approach to sensory cue combination, we show that the capacity to infer causal structure is not limited to conscious, high-level cognition; it is also performed continually and effortlessly in perception.     

Dual Reward Prediction Components Yield Pavlovian Sign- and Goal-Tracking

Reinforcement learning (RL) has become a dominant paradigm for understanding animal behaviors and neural correlates of decision-making, in part because of its ability to explain Pavlovian conditioned behaviors and the role of midbrain dopamine activity as reward prediction error (RPE). However, recent experimental findings indicate that dopamine activity, contrary to the RL hypothesis, may not signal RPE and differs based on the type of Pavlovian response (e.g. sign- and goal-tracking responses). In this study, we address this discrepancy by introducing a new neural correlate for learning reward predictions; the correlate is called “cue-evoked reward”. It refers to a recall of reward evoked by the cue that is learned through simple cue-reward associations. We introduce a temporal difference learning model, in which neural correlates of the cue itself and cue-evoked reward underlie learning of reward predictions. The animal''s reward prediction supported by these two correlates is divided into sign and goal components respectively. We relate the sign and goal components to approach responses towards the cue (i.e. sign-tracking) and the food-tray (i.e. goal-tracking) respectively. We found a number of correspondences between simulated models and the experimental findings (i.e. behavior and neural responses). First, the development of modeled responses is consistent with those observed in the experimental task. Second, the model''s RPEs were similar to dopamine activity in respective response groups. Finally, goal-tracking, but not sign-tracking, responses rapidly emerged when RPE was restored in the simulated models, similar to experiments with recovery from dopamine-antagonist. These results suggest two complementary neural correlates, corresponding to the cue and its evoked reward, form the basis for learning reward predictions in the sign- and goal-tracking rats.     

Quantifying individual variation in the propensity to attribute incentive salience to reward cues

If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience") to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878) subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience), to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers") vs. others that approached the location of reward delivery ("goal-trackers"). This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.     

Behavioural Flexibility and Personality in Zebra Finches

Consistent individual differences in behaviour of animals, that is, personalities, are both widespread and widely studied, but very few studies also include cognitive traits in this context. Animal personality has recently been integrated into the pace‐of‐life‐syndrome hypothesis, relating individual behavioural traits to life history. Variation in cognitive traits could be explained well by this theoretical framework. A risk‐reward trade‐off might lead to different cognitive types: Active birds that learn fast, take risks and probably have a fast lifestyle and less active, slow learning birds that are risk averse but thereby perform better in reversal learning as they probably pay more attention to external cues. We investigated the performance of zebra finches (Taeniopygia guttata) in a cognitively challenging reversal learning task and linked this to two personality traits: activity and fearfulness. Male birds were better in reversal learning than females. While no personality‐related differences occurred in the initial learning of our task, more active and fearful birds relearned the cue–reward association faster. While birds of different sex might have revealed different risk‐taking strategies in the training, our findings do not reveal the expected direction of a risk‐reward trade‐off in the reversal learning. It seems likely that a more general and personality‐related cognitive ability might improve performance across different tasks. The linkage between personality and cognition documented here could hence suggest that cognitive traits are indeed part of an overall pace‐of‐life syndrome.     

Reinforcement Learning or Active Inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain.     

Second-Order Conditioning and Conditioned Inhibition: Influences of Speed versus Accuracy on Human Causal Learning

In human causal learning, excitatory and inhibitory learning effects can sometimes be found in the same paradigm by altering the learning conditions. This study aims to explore whether learning in the feature negative paradigm can be dissociated by emphasising speed over accuracy. In two causal learning experiments, participants were given a feature negative discrimination in which the outcome caused by one cue was prevented by the addition of another. Participants completed training trials either in a self-paced fashion with instructions emphasising accuracy, or under strict time constraints with instructions emphasising speed. Using summation tests in which the preventative cue was paired with another causal cue, participants in the accuracy groups correctly rated the preventative cue as if it reduced the probability of the outcome. However, participants in the speed groups rated the preventative cue as if it increased the probability of the outcome. In Experiment 1, both speed and accuracy groups later judged the same cue to be preventative in a reasoned inference task. Experiment 2 failed to find evidence of similar dissociations in retrospective revaluation (release from overshadowing vs. mediated extinction) or learning about a redundant cue (blocking vs. augmentation). However in the same experiment, the tendency for the accuracy group to show conditioned inhibition and the speed group to show second-order conditioning was consistent even across sub-sets of the speed and accuracy groups with equivalent accuracy in training, suggesting that second-order conditioning is not merely a consequence of poorer acquisition. This dissociation mirrors the trade-off between second-order conditioning and conditioned inhibition observed in animal conditioning when training is extended.     

Cue integration in categorical tasks: insights from audio-visual speech perception

Previous cue integration studies have examined continuous perceptual dimensions (e.g., size) and have shown that human cue integration is well described by a normative model in which cues are weighted in proportion to their sensory reliability, as estimated from single-cue performance. However, this normative model may not be applicable to categorical perceptual dimensions (e.g., phonemes). In tasks defined over categorical perceptual dimensions, optimal cue weights should depend not only on the sensory variance affecting the perception of each cue but also on the environmental variance inherent in each task-relevant category. Here, we present a computational and experimental investigation of cue integration in a categorical audio-visual (articulatory) speech perception task. Our results show that human performance during audio-visual phonemic labeling is qualitatively consistent with the behavior of a Bayes-optimal observer. Specifically, we show that the participants in our task are sensitive, on a trial-by-trial basis, to the sensory uncertainty associated with the auditory and visual cues, during phonemic categorization. In addition, we show that while sensory uncertainty is a significant factor in determining cue weights, it is not the only one and participants' performance is consistent with an optimal model in which environmental, within category variability also plays a role in determining cue weights. Furthermore, we show that in our task, the sensory variability affecting the visual modality during cue-combination is not well estimated from single-cue performance, but can be estimated from multi-cue performance. The findings and computational principles described here represent a principled first step towards characterizing the mechanisms underlying human cue integration in categorical tasks.     

A simple spatial alternation task for assessing memory function in zebrafish

A series of studies was initiated to examine learning and memory function in the zebrafish (Danio rerio) by using a simple spatial alternation paradigm for a food reward. Fish were fed on alternating sides of a divided fish tank, with a red card displayed on one side serving as a visual means of orientation. Although responses were recorded at cue (light tap on the tank), 5 s after cue (as food was delivered), and 5 s after food delivery, the learning test was choice of a correct side of the tank to receive food. Therefore, an accurate level of an animal's achievement of the spatial task was represented by responses at food delivery. Data collected from 11 separate experiments indicated that zebrafish learned to alternate for a food reward. Further, statistical analysis showed that the zebrafish learned the task in the first half of the experiment as exhibited by a calculated t1/2 of 13.9 trials. Zebrafish could recall the task after a short period of 10 days with no testing. The alternating behavior was extinguished by withholding the food reward. Thus, the spatial alternation task can be learned easily by zebrafish, and may be useful in addressing learning and memory functions in vertebrate animals using zebrafish as a model organism.     

Structural drift: the population dynamics of sequential learning

We introduce a theory of sequential causal inference in which learners in a chain estimate a structural model from their upstream "teacher" and then pass samples from the model to their downstream "student". It extends the population dynamics of genetic drift, recasting Kimura's selectively neutral theory as a special case of a generalized drift process using structured populations with memory. We examine the diffusion and fixation properties of several drift processes and propose applications to learning, inference, and evolution. We also demonstrate how the organization of drift process space controls fidelity, facilitates innovations, and leads to information loss in sequential learning with and without memory.     

Neural integration of reward, arousal, and feeding: Recruitment of VTA, lateral hypothalamus, and ventral striatal neurons

The ability to control neuronal activity using light pulses and optogenetic tools has revealed new properties of neural circuits and established causal relationships between activation of a single genetically defined population of neurons and complex behaviors. Here, we briefly review the causal effect of activity of six genetically defined neural circuits on behavior, including the dopaminergic neurons DA in the ventral tegmental area (VTA); the two main populations of medium-sized spiny neurons (D1- and D2-positive) in the striatum; the giant Cholinergic interneurons in the ventral striatum; and the hypocretin- and MCH- expressing neurons in the lateral hypothalamus. We argue that selective spatiotemporal recruitment and coordinated spiking activity among these cell type-specific neural circuits may underlie the neural integration of reward, learning, arousal and feeding.     

Sensorimotor Learning Biases Choice Behavior: A Learning Neural Field Model for Decision Making

According to a prominent view of sensorimotor processing in primates, selection and specification of possible actions are not sequential operations. Rather, a decision for an action emerges from competition between different movement plans, which are specified and selected in parallel. For action choices which are based on ambiguous sensory input, the frontoparietal sensorimotor areas are considered part of the common underlying neural substrate for selection and specification of action. These areas have been shown capable of encoding alternative spatial motor goals in parallel during movement planning, and show signatures of competitive value-based selection among these goals. Since the same network is also involved in learning sensorimotor associations, competitive action selection (decision making) should not only be driven by the sensory evidence and expected reward in favor of either action, but also by the subject''s learning history of different sensorimotor associations. Previous computational models of competitive neural decision making used predefined associations between sensory input and corresponding motor output. Such hard-wiring does not allow modeling of how decisions are influenced by sensorimotor learning or by changing reward contingencies. We present a dynamic neural field model which learns arbitrary sensorimotor associations with a reward-driven Hebbian learning algorithm. We show that the model accurately simulates the dynamics of action selection with different reward contingencies, as observed in monkey cortical recordings, and that it correctly predicted the pattern of choice errors in a control experiment. With our adaptive model we demonstrate how network plasticity, which is required for association learning and adaptation to new reward contingencies, can influence choice behavior. The field model provides an integrated and dynamic account for the operations of sensorimotor integration, working memory and action selection required for decision making in ambiguous choice situations.     

Attention controls multisensory perception via two distinct mechanisms at different levels of the cortical hierarchy

To form a percept of the multisensory world, the brain needs to integrate signals from common sources weighted by their reliabilities and segregate those from independent sources. Previously, we have shown that anterior parietal cortices combine sensory signals into representations that take into account the signals’ causal structure (i.e., common versus independent sources) and their sensory reliabilities as predicted by Bayesian causal inference. The current study asks to what extent and how attentional mechanisms can actively control how sensory signals are combined for perceptual inference. In a pre- and postcueing paradigm, we presented observers with audiovisual signals at variable spatial disparities. Observers were precued to attend to auditory or visual modalities prior to stimulus presentation and postcued to report their perceived auditory or visual location. Combining psychophysics, functional magnetic resonance imaging (fMRI), and Bayesian modelling, we demonstrate that the brain moulds multisensory inference via two distinct mechanisms. Prestimulus attention to vision enhances the reliability and influence of visual inputs on spatial representations in visual and posterior parietal cortices. Poststimulus report determines how parietal cortices flexibly combine sensory estimates into spatial representations consistent with Bayesian causal inference. Our results show that distinct neural mechanisms control how signals are combined for perceptual inference at different levels of the cortical hierarchy.

A combination of psychophysics, computational modelling and fMRI reveals novel insights into how the brain controls the binding of information across the senses, such as the voice and lip movements of a speaker.     

A unified mechanism for innate and learned visual landmark guidance in the insect central complex

Insects can navigate efficiently in both novel and familiar environments, and this requires flexiblity in how they are guided by sensory cues. A prominent landmark, for example, can elicit strong innate behaviours (attraction or menotaxis) but can also be used, after learning, as a specific directional cue as part of a navigation memory. However, the mechanisms that allow both pathways to co-exist, interact or override each other are largely unknown. Here we propose a model for the behavioural integration of innate and learned guidance based on the neuroanatomy of the central complex (CX), adapted to control landmark guided behaviours. We consider a reward signal provided either by an innate attraction to landmarks or a long-term visual memory in the mushroom bodies (MB) that modulates the formation of a local vector memory in the CX. Using an operant strategy for a simulated agent exploring a simple world containing a single visual cue, we show how the generated short-term memory can support both innate and learned steering behaviour. In addition, we show how this architecture is consistent with the observed effects of unilateral MB lesions in ants that cause a reversion to innate behaviour. We suggest the formation of a directional memory in the CX can be interpreted as transforming rewarding (positive or negative) sensory signals into a mapping of the environment that describes the geometrical attractiveness (or repulsion). We discuss how this scheme might represent an ideal way to combine multisensory information gathered during the exploration of an environment and support optimal cue integration.     

Causal inference regulates audiovisual spatial recalibration via its influence on audiovisual perception

To obtain a coherent perception of the world, our senses need to be in alignment. When we encounter misaligned cues from two sensory modalities, the brain must infer which cue is faulty and recalibrate the corresponding sense. We examined whether and how the brain uses cue reliability to identify the miscalibrated sense by measuring the audiovisual ventriloquism aftereffect for stimuli of varying visual reliability. To adjust for modality-specific biases, visual stimulus locations were chosen based on perceived alignment with auditory stimulus locations for each participant. During an audiovisual recalibration phase, participants were presented with bimodal stimuli with a fixed perceptual spatial discrepancy; they localized one modality, cued after stimulus presentation. Unimodal auditory and visual localization was measured before and after the audiovisual recalibration phase. We compared participants’ behavior to the predictions of three models of recalibration: (a) Reliability-based: each modality is recalibrated based on its relative reliability—less reliable cues are recalibrated more; (b) Fixed-ratio: the degree of recalibration for each modality is fixed; (c) Causal-inference: recalibration is directly determined by the discrepancy between a cue and its estimate, which in turn depends on the reliability of both cues, and inference about how likely the two cues derive from a common source. Vision was hardly recalibrated by audition. Auditory recalibration by vision changed idiosyncratically as visual reliability decreased: the extent of auditory recalibration either decreased monotonically, peaked at medium visual reliability, or increased monotonically. The latter two patterns cannot be explained by either the reliability-based or fixed-ratio models. Only the causal-inference model of recalibration captures the idiosyncratic influences of cue reliability on recalibration. We conclude that cue reliability, causal inference, and modality-specific biases guide cross-modal recalibration indirectly by determining the perception of audiovisual stimuli.     

Coexistence of Reward and Unsupervised Learning During the Operant Conditioning of Neural Firing Rates

A fundamental goal of neuroscience is to understand how cognitive processes, such as operant conditioning, are performed by the brain. Typical and well studied examples of operant conditioning, in which the firing rates of individual cortical neurons in monkeys are increased using rewards, provide an opportunity for insight into this. Studies of reward-modulated spike-timing-dependent plasticity (RSTDP), and of other models such as R-max, have reproduced this learning behavior, but they have assumed that no unsupervised learning is present (i.e., no learning occurs without, or independent of, rewards). We show that these models cannot elicit firing rate reinforcement while exhibiting both reward learning and ongoing, stable unsupervised learning. To fix this issue, we propose a new RSTDP model of synaptic plasticity based upon the observed effects that dopamine has on long-term potentiation and depression (LTP and LTD). We show, both analytically and through simulations, that our new model can exhibit unsupervised learning and lead to firing rate reinforcement. This requires that the strengthening of LTP by the reward signal is greater than the strengthening of LTD and that the reinforced neuron exhibits irregular firing. We show the robustness of our findings to spike-timing correlations, to the synaptic weight dependence that is assumed, and to changes in the mean reward. We also consider our model in the differential reinforcement of two nearby neurons. Our model aligns more strongly with experimental studies than previous models and makes testable predictions for future experiments.     

Reward Maximization Justifies the Transition from Sensory Selection at Childhood to Sensory Integration at Adulthood

In a multisensory task, human adults integrate information from different sensory modalities -behaviorally in an optimal Bayesian fashion- while children mostly rely on a single sensor modality for decision making. The reason behind this change of behavior over age and the process behind learning the required statistics for optimal integration are still unclear and have not been justified by the conventional Bayesian modeling. We propose an interactive multisensory learning framework without making any prior assumptions about the sensory models. In this framework, learning in every modality and in their joint space is done in parallel using a single-step reinforcement learning method. A simple statistical test on confidence intervals on the mean of reward distributions is used to select the most informative source of information among the individual modalities and the joint space. Analyses of the method and the simulation results on a multimodal localization task show that the learning system autonomously starts with sensory selection and gradually switches to sensory integration. This is because, relying more on modalities -i.e. selection- at early learning steps (childhood) is more rewarding than favoring decisions learned in the joint space since, smaller state-space in modalities results in faster learning in every individual modality. In contrast, after gaining sufficient experiences (adulthood), the quality of learning in the joint space matures while learning in modalities suffers from insufficient accuracy due to perceptual aliasing. It results in tighter confidence interval for the joint space and consequently causes a smooth shift from selection to integration. It suggests that sensory selection and integration are emergent behavior and both are outputs of a single reward maximization process; i.e. the transition is not a preprogrammed phenomenon.     

Imitative behavior. A theoretical view

This article reviews the results of experimental studies on imitative behavior reported by various investigators, and then discusses the possible brain mechanisms responsible for this behavior. It was found that human infants in their first hours of life were already capable of spontaneous imitation of simple motor acts demonstrated by an adult, without previous training or reward; these observations suggest that imitative behavior is an innate process that can be considered an unconditional reflex of imitation. It was also found that satiated animals resumed eating when they saw their companions eating. In the latter case, the imitative reflex triggered the previously acquired feeding behavior. Similar mechanisms could be responsible for the phenomenon of eating more in the presence of companions than in their absence, as well as that of preferring the food chosen by companions. When followed by a reward, the imitative act can be learned--that is, transformed into an instrumental conditional response; learning by imitation of simple motor acts was observed in animals, and that of complex motor acts was observed in children who had already achieved a certain developmental stage. In animals, learning complex motor tasks was facilitated by previous observation of a companion performing this task. In this case, the presence of the observer during the session could lead to habituation of the experimental situation and production of associations between this situation and stimuli or emotions related to the reward or punishment, and might result in more efficient learning later. The imitative behavior can be inhibited by stimuli producing responses antagonistic to the act of imitation.     

Can Honey Bees Learn the Removal of a Stimulus as a Conditioning Cue?

Experiments investigated a Pavlovian conditioning situation where the presence and absence of the stimulus are reversed temporally with respect to the presentation of a reward. Instead of a conditioned stimulus (e.g. odor) signaling the presence of a reward, the stimulus (e.g. odor) is present in the environment except just prior to the presence of the reward. Thus, the absence of the stimulus, or offset of the stimulus (e.g. absence of odor), serves as a conditioned stimulus and is the reward cue. Honey bees (Apis mellifera) were used as a model invertebrate system, and the proboscis‐conditioning paradigm was used as the test procedure. Using both simple Pavlovian conditioning and discrimination‐learning protocols, animals learned to associate the onset of an odor as conditioned stimuli when paired with a sucrose reward. They could also learn to associate the onset of a puff of air with a sucrose reward. However, bees could not associate the offset of an order stimulus with the presentation of a sucrose reward in either a simple conditioning or a discrimination‐learning situation. These results support the model that a very different cognitive architecture is used by invertebrates to deal with certain environmental situations, including signaled avoidance.     

Bayesian action-perception computational model: interaction of production and recognition of cursive letters

In this paper, we study the collaboration of perception and action representations involved in cursive letter recognition and production. We propose a mathematical formulation for the whole perception-action loop, based on probabilistic modeling and bayesian inference, which we call the Bayesian Action-Perception (BAP) model. Being a model of both perception and action processes, the purpose of this model is to study the interaction of these processes. More precisely, the model includes a feedback loop from motor production, which implements an internal simulation of movement. Motor knowledge can therefore be involved during perception tasks. In this paper, we formally define the BAP model and show how it solves the following six varied cognitive tasks using bayesian inference: i) letter recognition (purely sensory), ii) writer recognition, iii) letter production (with different effectors), iv) copying of trajectories, v) copying of letters, and vi) letter recognition (with internal simulation of movements). We present computer simulations of each of these cognitive tasks, and discuss experimental predictions and theoretical developments.     

Multisensory Oddity Detection as Bayesian Inference

A key goal for the perceptual system is to optimally combine information from all the senses that may be available in order to develop the most accurate and unified picture possible of the outside world. The contemporary theoretical framework of ideal observer maximum likelihood integration (MLI) has been highly successful in modelling how the human brain combines information from a variety of different sensory modalities. However, in various recent experiments involving multisensory stimuli of uncertain correspondence, MLI breaks down as a successful model of sensory combination. Within the paradigm of direct stimulus estimation, perceptual models which use Bayesian inference to resolve correspondence have recently been shown to generalize successfully to these cases where MLI fails. This approach has been known variously as model inference, causal inference or structure inference. In this paper, we examine causal uncertainty in another important class of multi-sensory perception paradigm – that of oddity detection and demonstrate how a Bayesian ideal observer also treats oddity detection as a structure inference problem. We validate this approach by showing that it provides an intuitive and quantitative explanation of an important pair of multi-sensory oddity detection experiments – involving cues across and within modalities – for which MLI previously failed dramatically, allowing a novel unifying treatment of within and cross modal multisensory perception. Our successful application of structure inference models to the new ‘oddity detection’ paradigm, and the resultant unified explanation of across and within modality cases provide further evidence to suggest that structure inference may be a commonly evolved principle for combining perceptual information in the brain.