Representations of uncertainty in sensorimotor control

Uncertainty is ubiquitous in our sensorimotor interactions, arising from factors such as sensory and motor noise and ambiguity about the environment. Setting it apart from previous theories, a quintessential property of the Bayesian framework for making inference about the state of world so as to select actions, is the requirement to represent the uncertainty associated with inferences in the form of probability distributions. In the context of sensorimotor control and learning, the Bayesian framework suggests that to respond optimally to environmental stimuli the central nervous system needs to construct estimates of the sensorimotor transformations, in the form of internal models, as well as represent the structure of the uncertainty in the inputs, outputs and in the transformations themselves. Here we review Bayesian inference and learning models that have been successful in demonstrating the sensitivity of the sensorimotor system to different forms of uncertainty as well as recent studies aimed at characterizing the representation of the uncertainty at different computational levels.     

Efficient probabilistic model personalization integrating uncertainty on data and parameters: Application to Eikonal-Diffusion models in cardiac electrophysiology

Biophysical models are increasingly used for medical applications at the organ scale. However, model predictions are rarely associated with a confidence measure although there are important sources of uncertainty in computational physiology methods. For instance, the sparsity and noise of the clinical data used to adjust the model parameters (personalization), and the difficulty in modeling accurately soft tissue physiology. The recent theoretical progresses in stochastic models make their use computationally tractable, but there is still a challenge in estimating patient-specific parameters with such models. In this work we propose an efficient Bayesian inference method for model personalization using polynomial chaos and compressed sensing. This method makes Bayesian inference feasible in real 3D modeling problems. We demonstrate our method on cardiac electrophysiology. We first present validation results on synthetic data, then we apply the proposed method to clinical data. We demonstrate how this can help in quantifying the impact of the data characteristics on the personalization (and thus prediction) results. Described method can be beneficial for the clinical use of personalized models as it explicitly takes into account the uncertainties on the data and the model parameters while still enabling simulations that can be used to optimize treatment. Such uncertainty handling can be pivotal for the proper use of modeling as a clinical tool, because there is a crucial requirement to know the confidence one can have in personalized models.     

Inferring on the Intentions of Others by Hierarchical Bayesian Learning

Inferring on others'' (potentially time-varying) intentions is a fundamental problem during many social transactions. To investigate the underlying mechanisms, we applied computational modeling to behavioral data from an economic game in which 16 pairs of volunteers (randomly assigned to “player” or “adviser” roles) interacted. The player performed a probabilistic reinforcement learning task, receiving information about a binary lottery from a visual pie chart. The adviser, who received more predictive information, issued an additional recommendation. Critically, the game was structured such that the adviser''s incentives to provide helpful or misleading information varied in time. Using a meta-Bayesian modeling framework, we found that the players'' behavior was best explained by the deployment of hierarchical learning: they inferred upon the volatility of the advisers'' intentions in order to optimize their predictions about the validity of their advice. Beyond learning, volatility estimates also affected the trial-by-trial variability of decisions: participants were more likely to rely on their estimates of advice accuracy for making choices when they believed that the adviser''s intentions were presently stable. Finally, our model of the players'' inference predicted the players'' interpersonal reactivity index (IRI) scores, explicit ratings of the advisers'' helpfulness and the advisers'' self-reports on their chosen strategy. Overall, our results suggest that humans (i) employ hierarchical generative models to infer on the changing intentions of others, (ii) use volatility estimates to inform decision-making in social interactions, and (iii) integrate estimates of advice accuracy with non-social sources of information. The Bayesian framework presented here can quantify individual differences in these mechanisms from simple behavioral readouts and may prove useful in future clinical studies of maladaptive social cognition.     

Effects of branch length uncertainty on Bayesian posterior probabilities for phylogenetic hypotheses

In Bayesian phylogenetics, confidence in evolutionary relationships is expressed as posterior probability--the probability that a tree or clade is true given the data, evolutionary model, and prior assumptions about model parameters. Model parameters, such as branch lengths, are never known in advance; Bayesian methods incorporate this uncertainty by integrating over a range of plausible values given an assumed prior probability distribution for each parameter. Little is known about the effects of integrating over branch length uncertainty on posterior probabilities when different priors are assumed. Here, we show that integrating over uncertainty using a wide range of typical prior assumptions strongly affects posterior probabilities, causing them to deviate from those that would be inferred if branch lengths were known in advance; only when there is no uncertainty to integrate over does the average posterior probability of a group of trees accurately predict the proportion of correct trees in the group. The pattern of branch lengths on the true tree determines whether integrating over uncertainty pushes posterior probabilities upward or downward. The magnitude of the effect depends on the specific prior distributions used and the length of the sequences analyzed. Under realistic conditions, however, even extraordinarily long sequences are not enough to prevent frequent inference of incorrect clades with strong support. We found that across a range of conditions, diffuse priors--either flat or exponential distributions with moderate to large means--provide more reliable inferences than small-mean exponential priors. An empirical Bayes approach that fixes branch lengths at their maximum likelihood estimates yields posterior probabilities that more closely match those that would be inferred if the true branch lengths were known in advance and reduces the rate of strongly supported false inferences compared with fully Bayesian integration.     

Reconstructing ancestral ecologies: challenges and possible solutions

There are several ways to extract information about the evolutionary ecology of clades from their phylogenies. Of these, character state optimization and 'ancestor reconstruction' are perhaps the most widely used despite their being fraught with assumptions and potential pitfalls. Requirements for robust inferences of ancestral traits in general (i.e. those applicable to all types of characters) include accurate and robust phylogenetic hypotheses, complete species-level sampling and the appropriate choice of optimality criterion. Ecological characters, however, also require careful consideration of methods for accounting for intraspecific variability. Such methods include 'Presence Coding' and 'Polymorphism Coding' for discrete ecological characters, and 'Range Coding' and 'MaxMin Coding' for continuously variable characters. Ultimately, however, historical inferences such as these are, as with phylogenetic inference itself, associated with a degree of uncertainty. Statistically based uncertainty estimates are available within the context of model-based inference (e.g. maximum likelihood and Bayesian); however, these measures are only as reliable as the chosen model is appropriate. Although generally thought to preclude the possibility of measuring relative uncertainty or support for alternative possible reconstructions, certain useful non-statistical support measures (i.e. 'Sharkey support' and 'Parsimony support') are applicable to parsimony reconstructions.     

A computational framework for the study of confidence in humans and animals

Confidence judgements, self-assessments about the quality of a subject's knowledge, are considered a central example of metacognition. Prima facie, introspection and self-report appear the only way to access the subjective sense of confidence or uncertainty. Contrary to this notion, overt behavioural measures can be used to study confidence judgements by animals trained in decision-making tasks with perceptual or mnemonic uncertainty. Here, we suggest that a computational approach can clarify the issues involved in interpreting these tasks and provide a much needed springboard for advancing the scientific understanding of confidence. We first review relevant theories of probabilistic inference and decision-making. We then critically discuss behavioural tasks employed to measure confidence in animals and show how quantitative models can help to constrain the computational strategies underlying confidence-reporting behaviours. In our view, post-decision wagering tasks with continuous measures of confidence appear to offer the best available metrics of confidence. Since behavioural reports alone provide a limited window into mechanism, we argue that progress calls for measuring the neural representations and identifying the computations underlying confidence reports. We present a case study using such a computational approach to study the neural correlates of decision confidence in rats. This work shows that confidence assessments may be considered higher order, but can be generated using elementary neural computations that are available to a wide range of species. Finally, we discuss the relationship of confidence judgements to the wider behavioural uses of confidence and uncertainty.     

Comparing Families of Dynamic Causal Models

Mathematical models of scientific data can be formally compared using Bayesian model evidence. Previous applications in the biological sciences have mainly focussed on model selection in which one first selects the model with the highest evidence and then makes inferences based on the parameters of that model. This “best model” approach is very useful but can become brittle if there are a large number of models to compare, and if different subjects use different models. To overcome this shortcoming we propose the combination of two further approaches: (i) family level inference and (ii) Bayesian model averaging within families. Family level inference removes uncertainty about aspects of model structure other than the characteristic of interest. For example: What are the inputs to the system? Is processing serial or parallel? Is it linear or nonlinear? Is it mediated by a single, crucial connection? We apply Bayesian model averaging within families to provide inferences about parameters that are independent of further assumptions about model structure. We illustrate the methods using Dynamic Causal Models of brain imaging data.     

Identification of connectivity in neural networks.

Analytical and experimental methods are provided for estimating synaptic connectivities from simultaneous recordings of multiple neurons. The results are based on detailed, yet flexible neuron models in which spike trains are modeled as general doubly stochastic point processes. The expressions derived can be used with nonstationary or stationary records, and can be readily extended from pairwise to multineuron estimates. Furthermore, we show analytically how the estimates are improved as more neurons are sampled, and derive the appropriate normalizations to eliminate stimulus-related correlations. Finally, we illustrate the use and interpretation of the analytical expressions on simulated spike trains and neural networks, and give explicit confidence measures on the estimates.     

Estimating divergence times from DNA sequences

The patterns of genetic variation within and among individuals and populations can be used to make inferences about the evolutionary forces that generated those patterns. Numerous population genetic approaches have been developed in order to infer evolutionary history. Here, we present the “Two-Two (TT)” and the “Two-Two-outgroup (TTo)” methods; two closely related approaches for estimating divergence time based in coalescent theory. They rely on sequence data from two haploid genomes (or a single diploid individual) from each of two populations. Under a simple population-divergence model, we derive the probabilities of the possible sample configurations. These probabilities form a set of equations that can be solved to obtain estimates of the model parameters, including population split times, directly from the sequence data. This transparent and computationally efficient approach to infer population divergence time makes it possible to estimate time scaled in generations (assuming a mutation rate), and not as a compound parameter of genetic drift. Using simulations under a range of demographic scenarios, we show that the method is relatively robust to migration and that the TTo method can alleviate biases that can appear from drastic ancestral population size changes. We illustrate the utility of the approaches with some examples, including estimating split times for pairs of human populations as well as providing further evidence for the complex relationship among Neandertals and Denisovans and their ancestors.     

Bayesian characterization of uncertainty in species interaction strengths

Considerable effort has been devoted to the estimation of species interaction strengths. This effort has focused primarily on statistical significance testing and obtaining point estimates of parameters that contribute to interaction strength magnitudes, leaving the characterization of uncertainty associated with those estimates unconsidered. We consider a means of characterizing the uncertainty of a generalist predator’s interaction strengths by formulating an observational method for estimating a predator’s prey-specific per capita attack rates as a Bayesian statistical model. This formulation permits the explicit incorporation of multiple sources of uncertainty. A key insight is the informative nature of several so-called non-informative priors that have been used in modeling the sparse data typical of predator feeding surveys. We introduce to ecology a new neutral prior and provide evidence for its superior performance. We use a case study to consider the attack rates in a New Zealand intertidal whelk predator, and we illustrate not only that Bayesian point estimates can be made to correspond with those obtained by frequentist approaches, but also that estimation uncertainty as described by 95% intervals is more useful and biologically realistic using the Bayesian method. In particular, unlike in bootstrap confidence intervals, the lower bounds of the Bayesian posterior intervals for attack rates do not include zero when a predator–prey interaction is in fact observed. We conclude that the Bayesian framework provides a straightforward, probabilistic characterization of interaction strength uncertainty, enabling future considerations of both the deterministic and stochastic drivers of interaction strength and their impact on food webs.     

Estimating biomarker-based HIV incidence using prevalence data in high risk groups with missing outcomes

The novel two-step serologic sensitive/less sensitive testing algorithm for detecting recent HIV seroconversion (STARHS) provides a simple and practical method to estimate HIV-1 incidence using cross-sectional HIV seroprevalence data. STARHS has been used increasingly in epidemiologic studies. However, the uncertainty of incidence estimates using this algorithm has not been well described, especially for high risk groups or when missing data is present because a fraction of sensitive enzyme immunoassay (EIA) positive specimens are not tested by the less sensitive EIA. Ad hoc methods used in practice provide incorrect confidence limits and thus may jeopardize statistical inference. In this report, we propose maximum likelihood and Bayesian methods for correctly estimating the uncertainty in incidence estimates obtained using prevalence data with a fraction missing, and extend the methods to regression settings. Using a study of injection drug users participating in a drug detoxification program in New York city as an example, we demonstrated the impact of underestimating the uncertainty in incidence estimates using ad hoc methods. Our methods can be applied to estimate the incidence of other diseases from prevalence data using similar testing algorithms when missing data is present.     

Moment fitting for parameter inference in repeatedly and partially observed stochastic biological models

The inference of reaction rate parameters in biochemical network models from time series concentration data is a central task in computational systems biology. Under the assumption of well mixed conditions the network dynamics are typically described by the chemical master equation, the Fokker Planck equation, the linear noise approximation or the macroscopic rate equation. The inverse problem of estimating the parameters of the underlying network model can be approached in deterministic and stochastic ways, and available methods often compare individual or mean concentration traces obtained from experiments with theoretical model predictions when maximizing likelihoods, minimizing regularized least squares functionals, approximating posterior distributions or sequentially processing the data. In this article we assume that the biological reaction network can be observed at least partially and repeatedly over time such that sample moments of species molecule numbers for various time points can be calculated from the data. Based on the chemical master equation we furthermore derive closed systems of parameter dependent nonlinear ordinary differential equations that predict the time evolution of the statistical moments. For inferring the reaction rate parameters we suggest to not only compare the sample mean with the theoretical mean prediction but also to take the residual of higher order moments explicitly into account. Cost functions that involve residuals of higher order moments may form landscapes in the parameter space that have more pronounced curvatures at the minimizer and hence may weaken or even overcome parameter sloppiness and uncertainty. As a consequence both deterministic and stochastic parameter inference algorithms may be improved with respect to accuracy and efficiency. We demonstrate the potential of moment fitting for parameter inference by means of illustrative stochastic biological models from the literature and address topics for future research.     

Fundamental Identifiability Limits in Molecular Epidemiology

Viral phylogenies provide crucial information on the spread of infectious diseases, and many studies fit mathematical models to phylogenetic data to estimate epidemiological parameters such as the effective reproduction ratio (R_e) over time. Such phylodynamic inferences often complement or even substitute for conventional surveillance data, particularly when sampling is poor or delayed. It remains generally unknown, however, how robust phylodynamic epidemiological inferences are, especially when there is uncertainty regarding pathogen prevalence and sampling intensity. Here, we use recently developed mathematical techniques to fully characterize the information that can possibly be extracted from serially collected viral phylogenetic data, in the context of the commonly used birth-death-sampling model. We show that for any candidate epidemiological scenario, there exists a myriad of alternative, markedly different, and yet plausible “congruent” scenarios that cannot be distinguished using phylogenetic data alone, no matter how large the data set. In the absence of strong constraints or rate priors across the entire study period, neither maximum-likelihood fitting nor Bayesian inference can reliably reconstruct the true epidemiological dynamics from phylogenetic data alone; rather, estimators can only converge to the “congruence class” of the true dynamics. We propose concrete and feasible strategies for making more robust epidemiological inferences from viral phylogenetic data.     

Accounting for uncertainty in dormant life stages in stochastic demographic models

Dormant life stages are often critical for population viability in stochastic environments, but accurate field data characterizing them are difficult to collect. Such limitations may translate into uncertainties in demographic parameters describing these stages, which then may propagate errors in the examination of population‐level responses to environmental variation. Expanding on current methods, we 1) apply data‐driven approaches to estimate parameter uncertainty in vital rates of dormant life stages and 2) test whether such estimates provide more robust inferences about population dynamics. We built integral projection models (IPMs) for a fire‐adapted, carnivorous plant species using a Bayesian framework to estimate uncertainty in parameters of three vital rates of dormant seeds – seed‐bank ingression, stasis and egression. We used stochastic population projections and elasticity analyses to quantify the relative sensitivity of the stochastic population growth rate (log λs) to changes in these vital rates at different fire return intervals. We then ran stochastic projections of log λs for 1000 posterior samples of the three seed‐bank vital rates and assessed how strongly their parameter uncertainty propagated into uncertainty in estimates of log λs and the probability of quasi‐extinction, P_q(t). Elasticity analyses indicated that changes in seed‐bank stasis and egression had large effects on log λs across fire return intervals. In turn, uncertainty in the estimates of these two vital rates explained > 50% of the variation in log λs estimates at several fire‐return intervals. Inferences about population viability became less certain as the time between fires widened, with estimates of P_q(t) potentially > 20% higher when considering parameter uncertainty. Our results suggest that, for species with dormant stages, where data is often limited, failing to account for parameter uncertainty in population models may result in incorrect interpretations of population viability.     

Mixture models for estimating the size of a closed population when capture rates vary among individuals

We develop a parameterization of the beta-binomial mixture that provides sensible inferences about the size of a closed population when probabilities of capture or detection vary among individuals. Three classes of mixture models (beta-binomial, logistic-normal, and latent-class) are fitted to recaptures of snowshoe hares for estimating abundance and to counts of bird species for estimating species richness. In both sets of data, rates of detection appear to vary more among individuals (animals or species) than among sampling occasions or locations. The estimates of population size and species richness are sensitive to model-specific assumptions about the latent distribution of individual rates of detection. We demonstrate using simulation experiments that conventional diagnostics for assessing model adequacy, such as deviance, cannot be relied on for selecting classes of mixture models that produce valid inferences about population size. Prior knowledge about sources of individual heterogeneity in detection rates, if available, should be used to help select among classes of mixture models that are to be used for inference.     

Permutation inference methods for multivariate meta-analysis

Multivariate meta-analysis is gaining prominence in evidence synthesis research because it enables simultaneous synthesis of multiple correlated outcome data, and random-effects models have generally been used for addressing between-studies heterogeneities. However, coverage probabilities of confidence regions or intervals for standard inference methods for random-effects models (eg, restricted maximum likelihood estimation) cannot retain their nominal confidence levels in general, especially when the number of synthesized studies is small because their validities depend on large sample approximations. In this article, we provide permutation-based inference methods that enable exact joint inferences for average outcome measures without large sample approximations. We also provide accurate marginal inference methods under general settings of multivariate meta-analyses. We propose effective approaches for permutation inferences using optimal weighting based on the efficient score statistic. The effectiveness of the proposed methods is illustrated via applications to bivariate meta-analyses of diagnostic accuracy studies for airway eosinophilia in asthma and a network meta-analysis for antihypertensive drugs on incident diabetes, as well as through simulation experiments. In numerical evaluations performed via simulations, our methods generally provided accurate confidence regions or intervals under a broad range of settings, whereas the current standard inference methods exhibited serious undercoverage properties.     

Task-induced neural covariability as a signature of approximate Bayesian learning and inference

Perception is often characterized computationally as an inference process in which uncertain or ambiguous sensory inputs are combined with prior expectations. Although behavioral studies have shown that observers can change their prior expectations in the context of a task, robust neural signatures of task-specific priors have been elusive. Here, we analytically derive such signatures under the general assumption that the responses of sensory neurons encode posterior beliefs that combine sensory inputs with task-specific expectations. Specifically, we derive predictions for the task-dependence of correlated neural variability and decision-related signals in sensory neurons. The qualitative aspects of our results are parameter-free and specific to the statistics of each task. The predictions for correlated variability also differ from predictions of classic feedforward models of sensory processing and are therefore a strong test of theories of hierarchical Bayesian inference in the brain. Importantly, we find that Bayesian learning predicts an increase in so-called “differential correlations” as the observer’s internal model learns the stimulus distribution, and the observer’s behavioral performance improves. This stands in contrast to classic feedforward encoding/decoding models of sensory processing, since such correlations are fundamentally information-limiting. We find support for our predictions in data from existing neurophysiological studies across a variety of tasks and brain areas. Finally, we show in simulation how measurements of sensory neural responses can reveal information about a subject’s internal beliefs about the task. Taken together, our results reinterpret task-dependent sources of neural covariability as signatures of Bayesian inference and provide new insights into their cause and their function.     

Observers exploit stochastic models of sensory change to help judge the passage of time

Sensory stimulation can systematically bias the perceived passage of time, but why and how this happens is mysterious. In this report, we provide evidence that such biases may ultimately derive from an innate and adaptive use of stochastically evolving dynamic stimuli to help refine estimates derived from internal timekeeping mechanisms. A simplified statistical model based on probabilistic expectations of stimulus change derived from the second-order temporal statistics of the natural environment makes three predictions. First, random noise-like stimuli whose statistics violate natural expectations should induce timing bias. Second, a previously unexplored obverse of this effect is that similar noise stimuli with natural statistics should reduce the variability of timing estimates. Finally, this reduction in variability should scale with the interval being timed, so as to preserve the overall Weber law of interval timing. All three predictions are borne out experimentally. Thus, in the context of our novel theoretical framework, these results suggest that observers routinely rely on sensory input to augment their sense of the passage of time, through a process of Bayesian inference based on expectations of change in the natural environment.     

Uncertainty, neuromodulation, and attention

Uncertainty in various forms plagues our interactions with the environment. In a Bayesian statistical framework, optimal inference and prediction, based on unreliable observations in changing contexts, require the representation and manipulation of different forms of uncertainty. We propose that the neuromodulators acetylcholine and norepinephrine play a major role in the brain's implementation of these uncertainty computations. Acetylcholine signals expected uncertainty, coming from known unreliability of predictive cues within a context. Norepinephrine signals unexpected uncertainty, as when unsignaled context switches produce strongly unexpected observations. These uncertainty signals interact to enable optimal inference and learning in noisy and changeable environments. This formulation is consistent with a wealth of physiological, pharmacological, and behavioral data implicating acetylcholine and norepinephrine in specific aspects of a range of cognitive processes. Moreover, the model suggests a class of attentional cueing tasks that involve both neuromodulators and shows how their interactions may be part-antagonistic, part-synergistic.     

Reproductive decision-making by female peacock wrasses: flexible versus fixed behavioral rules in variable environments

Because environments are temporally variable, animals may oftenestimate the current environmental state to inform their behavioralchoices. However, using experience may cause behavior to lagbehind the current state of the environment, and estimates maysuffer from sampling errors. We used stochastic dynamic modelsto examine the environmental conditions that favor flexible ratherthan fixed estimates and behaviors. The examination was conductedin the context of reproductive decisions made by the femalepeacock wrasse (Symphodus tinca), a nearshore Mediterraneanfish. Female peacock wrasses can choose to spawn in a nest,with males that defend these nests within territories, or outof a nest, with males that defend neither territories nor nests.A female must expend effort and time to find nesting males,and the profitability of this search, relative to spawning with nonnestingmales, changes with the density of nests and relative hatching successof eggs in and out of nests. A female can increase her fitnessby estimating the environment's state and matching her reproductivedecisions to the current environment. These estimates can beflexible and formed by experience, or fixed and formed by selection.We found that flexible estimates based on experience do betterwhen there is variance within and between seasons and when thereis greater uncertainty. The optimal rate for forgetting experiencesis set by the rate of environmental change. Comparisons of predictedfemale behavior using flexible and fixed estimates with observed behaviorsuggest that females use estimates that are updated by experience.