Risk-Sensitivity in Bayesian Sensorimotor Integration

Information processing in the nervous system during sensorimotor tasks with inherent uncertainty has been shown to be consistent with Bayesian integration. Bayes optimal decision-makers are, however, risk-neutral in the sense that they weigh all possibilities based on prior expectation and sensory evidence when they choose the action with highest expected value. In contrast, risk-sensitive decision-makers are sensitive to model uncertainty and bias their decision-making processes when they do inference over unobserved variables. In particular, they allow deviations from their probabilistic model in cases where this model makes imprecise predictions. Here we test for risk-sensitivity in a sensorimotor integration task where subjects exhibit Bayesian information integration when they infer the position of a target from noisy sensory feedback. When introducing a cost associated with subjects'' response, we found that subjects exhibited a characteristic bias towards low cost responses when their uncertainty was high. This result is in accordance with risk-sensitive decision-making processes that allow for deviations from Bayes optimal decision-making in the face of uncertainty. Our results suggest that both Bayesian integration and risk-sensitivity are important factors to understand sensorimotor integration in a quantitative fashion.     

A Bayesian Attractor Model for Perceptual Decision Making

Even for simple perceptual decisions, the mechanisms that the brain employs are still under debate. Although current consensus states that the brain accumulates evidence extracted from noisy sensory information, open questions remain about how this simple model relates to other perceptual phenomena such as flexibility in decisions, decision-dependent modulation of sensory gain, or confidence about a decision. We propose a novel approach of how perceptual decisions are made by combining two influential formalisms into a new model. Specifically, we embed an attractor model of decision making into a probabilistic framework that models decision making as Bayesian inference. We show that the new model can explain decision making behaviour by fitting it to experimental data. In addition, the new model combines for the first time three important features: First, the model can update decisions in response to switches in the underlying stimulus. Second, the probabilistic formulation accounts for top-down effects that may explain recent experimental findings of decision-related gain modulation of sensory neurons. Finally, the model computes an explicit measure of confidence which we relate to recent experimental evidence for confidence computations in perceptual decision tasks.     

Robust optimization for resource-constrained project scheduling with uncertain activity durations

The purpose of this paper is to propose models for project scheduling when there is considerable uncertainty in the activity durations, to the extent that the decision maker cannot with confidence associate probabilities with the possible outcomes of a decision. Our modeling techniques stem from robust discrete optimization, which is a theoretical framework that enables the decision maker to produce solutions that will have a reasonably good objective value under any likely input data scenario. We develop and implement a scenario-relaxation algorithm and a scenario-relaxation-based heuristic. The first algorithm produces optimal solutions but requires excessive running times even for medium-sized instances; the second algorithm produces high-quality solutions for medium-sized instances and outperforms two benchmark heuristics.     

A bayesian approach to the design of phase II clinical trials

A new strategy for the design of Phase II clinical trials is presented which utilizes the information provided by the prior distribution of the response rate, the costs of treating a patient, and the losses or gains resulting from the decisions taken at the completion of the study. A risk function is derived from which one may determine the optimal Bayes sampling plan. The decision theoretic/Bayesian approach is shown to provide a formal justification for the sample sizes often used in practice and shows the conditions under which such sample sizes are clearly inappropriate.     

Adaptive Sampling of Information in Perceptual Decision-Making

In many perceptual and cognitive decision-making problems, humans sample multiple noisy information sources serially, and integrate the sampled information to make an overall decision. We derive the optimal decision procedure for two-alternative choice tasks in which the different options are sampled one at a time, sources vary in the quality of the information they provide, and the available time is fixed. To maximize accuracy, the optimal observer allocates time to sampling different information sources in proportion to their noise levels. We tested human observers in a corresponding perceptual decision-making task. Observers compared the direction of two random dot motion patterns that were triggered only when fixated. Observers allocated more time to the noisier pattern, in a manner that correlated with their sensory uncertainty about the direction of the patterns. There were several differences between the optimal observer predictions and human behaviour. These differences point to a number of other factors, beyond the quality of the currently available sources of information, that influences the sampling strategy.     

Continuous evolution of statistical estimators for optimal decision-making

In many everyday situations, humans must make precise decisions in the presence of uncertain sensory information. For example, when asked to combine information from multiple sources we often assign greater weight to the more reliable information. It has been proposed that statistical-optimality often observed in human perception and decision-making requires that humans have access to the uncertainty of both their senses and their decisions. However, the mechanisms underlying the processes of uncertainty estimation remain largely unexplored. In this paper we introduce a novel visual tracking experiment that requires subjects to continuously report their evolving perception of the mean and uncertainty of noisy visual cues over time. We show that subjects accumulate sensory information over the course of a trial to form a continuous estimate of the mean, hindered only by natural kinematic constraints (sensorimotor latency etc.). Furthermore, subjects have access to a measure of their continuous objective uncertainty, rapidly acquired from sensory information available within a trial, but limited by natural kinematic constraints and a conservative margin for error. Our results provide the first direct evidence of the continuous mean and uncertainty estimation mechanisms in humans that may underlie optimal decision making.     

Characterizing the Impact of Category Uncertainty on Human Auditory Categorization Behavior

Categorization is an important cognitive process. However, the correct categorization of a stimulus is often challenging because categories can have overlapping boundaries. Whereas perceptual categorization has been extensively studied in vision, the analogous phenomenon in audition has yet to be systematically explored. Here, we test whether and how human subjects learn to use category distributions and prior probabilities, as well as whether subjects employ an optimal decision strategy when making auditory-category decisions. We asked subjects to classify the frequency of a tone burst into one of two overlapping, uniform categories according to the perceived tone frequency. We systematically varied the prior probability of presenting a tone burst with a frequency originating from one versus the other category. Most subjects learned these changes in prior probabilities early in testing and used this information to influence categorization. We also measured each subject''s frequency-discrimination thresholds (i.e., their sensory uncertainty levels). We tested each subject''s average behavior against variations of a Bayesian model that either led to optimal or sub-optimal decision behavior (i.e. probability matching). In both predicting and fitting each subject''s average behavior, we found that probability matching provided a better account of human decision behavior. The model fits confirmed that subjects were able to learn category prior probabilities and approximate forms of the category distributions. Finally, we systematically explored the potential ways that additional noise sources could influence categorization behavior. We found that an optimal decision strategy can produce probability-matching behavior if it utilized non-stationary category distributions and prior probabilities formed over a short stimulus history. Our work extends previous findings into the auditory domain and reformulates the issue of categorization in a manner that can help to interpret the results of previous research within a generative framework.     

The Sense of Confidence during Probabilistic Learning: A Normative Account

Learning in a stochastic environment consists of estimating a model from a limited amount of noisy data, and is therefore inherently uncertain. However, many classical models reduce the learning process to the updating of parameter estimates and neglect the fact that learning is also frequently accompanied by a variable “feeling of knowing” or confidence. The characteristics and the origin of these subjective confidence estimates thus remain largely unknown. Here we investigate whether, during learning, humans not only infer a model of their environment, but also derive an accurate sense of confidence from their inferences. In our experiment, humans estimated the transition probabilities between two visual or auditory stimuli in a changing environment, and reported their mean estimate and their confidence in this report. To formalize the link between both kinds of estimate and assess their accuracy in comparison to a normative reference, we derive the optimal inference strategy for our task. Our results indicate that subjects accurately track the likelihood that their inferences are correct. Learning and estimating confidence in what has been learned appear to be two intimately related abilities, suggesting that they arise from a single inference process. We show that human performance matches several properties of the optimal probabilistic inference. In particular, subjective confidence is impacted by environmental uncertainty, both at the first level (uncertainty in stimulus occurrence given the inferred stochastic characteristics) and at the second level (uncertainty due to unexpected changes in these stochastic characteristics). Confidence also increases appropriately with the number of observations within stable periods. Our results support the idea that humans possess a quantitative sense of confidence in their inferences about abstract non-sensory parameters of the environment. This ability cannot be reduced to simple heuristics, it seems instead a core property of the learning process.     

Forward and Backward Inference in Spatial Cognition

This paper shows that the various computations underlying spatial cognition can be implemented using statistical inference in a single probabilistic model. Inference is implemented using a common set of ‘lower-level’ computations involving forward and backward inference over time. For example, to estimate where you are in a known environment, forward inference is used to optimally combine location estimates from path integration with those from sensory input. To decide which way to turn to reach a goal, forward inference is used to compute the likelihood of reaching that goal under each option. To work out which environment you are in, forward inference is used to compute the likelihood of sensory observations under the different hypotheses. For reaching sensory goals that require a chaining together of decisions, forward inference can be used to compute a state trajectory that will lead to that goal, and backward inference to refine the route and estimate control signals that produce the required trajectory. We propose that these computations are reflected in recent findings of pattern replay in the mammalian brain. Specifically, that theta sequences reflect decision making, theta flickering reflects model selection, and remote replay reflects route and motor planning. We also propose a mapping of the above computational processes onto lateral and medial entorhinal cortex and hippocampus.     

Nudging Cooperation in a Crowd Experiment

We examine the hypothesis that driven by a competition heuristic, people don''t even reflect or consider whether a cooperation strategy may be better. As a paradigmatic example of this behavior we propose the zero-sum game fallacy, according to which people believe that resources are fixed even when they are not. We demonstrate that people only cooperate if the competitive heuristic is explicitly overridden in an experiment in which participants play two rounds of a game in which competition is suboptimal. The observed spontaneous behavior for most players was to compete. Then participants were explicitly reminded that the competing strategy may not be optimal. This minor intervention boosted cooperation, implying that competition does not result from lack of trust or willingness to cooperate but instead from the inability to inhibit the competition bias. This activity was performed in a controlled laboratory setting and also as a crowd experiment. Understanding the psychological underpinnings of these behaviors may help us improve cooperation and thus may have vast practical consequences to our society.     

THE MEASUREMENT OF TASTE DISCRIMINATION WITH THE SAME-DIFFERENT TASK: A DETECTION-THEORY ANALYSIS

The biases to which the same-different task is prone can be accounted for by having subjects rate their confidence that two stimuli are the same or different. The rating method of detection theory was therefore used to study the discriminability of two concentrations of a fruit drink. A model in which the decision variable was the difference in sensory strength of the two samples provided a satisfactory fit to the Receiver Operating Characteristics of four of the five subjects. The bias-free index of discriminability, d', was estimated for each subject. A combined operating characteristic, derived from jackknifing the data of the individual subjects, revealed an asymmetry characteristic of the differencing model. The results suggest that the same-different task, which is readily understood by subjects, can provide an unbiased measure of the discriminability of foods or beverages.     

Two-phase strategy of controlling motor coordination determined by task performance optimality

A quantitative model of optimal coordination between hand transport and grip aperture has been derived in our previous studies of reach-to-grasp movements without utilizing explicit knowledge of the optimality criterion or motor plant dynamics. The model’s utility for experimental data analysis has been demonstrated. Here we show how to generalize this model for a broad class of reaching-type, goal-directed movements. The model allows for measuring the variability of motor coordination and studying its dependence on movement phase. The experimentally found characteristics of that dependence imply that execution noise is low and does not affect motor coordination significantly. From those characteristics it is inferred that the cost of neural computations required for information acquisition and processing is included in the criterion of task performance optimality as a function of precision demand for state estimation and decision making. The precision demand is an additional optimized control variable that regulates the amount of neurocomputational resources activated dynamically. It is shown that an optimal control strategy in this case comprises two different phases. During the initial phase, the cost of neural computations is significantly reduced at the expense of reducing the demand for their precision, which results in speed-accuracy tradeoff violation and significant inter-trial variability of motor coordination. During the final phase, neural computations and thus motor coordination are considerably more precise to reduce the cost of errors in making a contact with the target object. The generality of the optimal coordination model and the two-phase control strategy is illustrated on several diverse examples.     

Generation and reshaping of sequences in neural systems

The generation of informational sequences and their reorganization or reshaping is one of the most intriguing subjects for both neuroscience and the theory of autonomous intelligent systems. In spite of the diversity of sequential activities of sensory, motor, and cognitive neural systems, they have many similarities from the dynamical point of view. In this review we discus the ideas, models, and mathematical image of sequence generation and reshaping on different levels of the neural hierarchy, i.e., the role of a sensory network dynamics in the generation of a motor program (hunting swimming of marine mollusk Clione), olfactory dynamical coding, and sequential learning and decision making. Analysis of these phenomena is based on the winnerless competition principle. The considered models can be a basis for the design of biologically inspired autonomous intelligent systems.     

Observing the observer (I): meta-bayesian models of learning and decision-making

In this paper, we present a generic approach that can be used to infer how subjects make optimal decisions under uncertainty. This approach induces a distinction between a subject's perceptual model, which underlies the representation of a hidden "state of affairs" and a response model, which predicts the ensuing behavioural (or neurophysiological) responses to those inputs. We start with the premise that subjects continuously update a probabilistic representation of the causes of their sensory inputs to optimise their behaviour. In addition, subjects have preferences or goals that guide decisions about actions given the above uncertain representation of these hidden causes or state of affairs. From a Bayesian decision theoretic perspective, uncertain representations are so-called "posterior" beliefs, which are influenced by subjective "prior" beliefs. Preferences and goals are encoded through a "loss" (or "utility") function, which measures the cost incurred by making any admissible decision for any given (hidden) state of affair. By assuming that subjects make optimal decisions on the basis of updated (posterior) beliefs and utility (loss) functions, one can evaluate the likelihood of observed behaviour. Critically, this enables one to "observe the observer", i.e. identify (context- or subject-dependent) prior beliefs and utility-functions using psychophysical or neurophysiological measures. In this paper, we describe the main theoretical components of this meta-Bayesian approach (i.e. a Bayesian treatment of Bayesian decision theoretic predictions). In a companion paper ('Observing the observer (II): deciding when to decide'), we describe a concrete implementation of it and demonstrate its utility by applying it to simulated and real reaction time data from an associative learning task.     

Reward Maximization Justifies the Transition from Sensory Selection at Childhood to Sensory Integration at Adulthood

In a multisensory task, human adults integrate information from different sensory modalities -behaviorally in an optimal Bayesian fashion- while children mostly rely on a single sensor modality for decision making. The reason behind this change of behavior over age and the process behind learning the required statistics for optimal integration are still unclear and have not been justified by the conventional Bayesian modeling. We propose an interactive multisensory learning framework without making any prior assumptions about the sensory models. In this framework, learning in every modality and in their joint space is done in parallel using a single-step reinforcement learning method. A simple statistical test on confidence intervals on the mean of reward distributions is used to select the most informative source of information among the individual modalities and the joint space. Analyses of the method and the simulation results on a multimodal localization task show that the learning system autonomously starts with sensory selection and gradually switches to sensory integration. This is because, relying more on modalities -i.e. selection- at early learning steps (childhood) is more rewarding than favoring decisions learned in the joint space since, smaller state-space in modalities results in faster learning in every individual modality. In contrast, after gaining sufficient experiences (adulthood), the quality of learning in the joint space matures while learning in modalities suffers from insufficient accuracy due to perceptual aliasing. It results in tighter confidence interval for the joint space and consequently causes a smooth shift from selection to integration. It suggests that sensory selection and integration are emergent behavior and both are outputs of a single reward maximization process; i.e. the transition is not a preprogrammed phenomenon.     

Pregnancy wantedness: attitude stability over time

In this analysis we have compared two attitudinal reports, from interviews two years apart, of the same event for consistency over time. We have looked at women's self-reports, and at their proxy reports for partners' attitudes as well. The inconsistent reports of women's own views tended to shift slightly toward more favorable reports at the second interview (15 percent more positive versus 10 percent more negative). More specifically, 7.5 percent of women who reported at Time 1 that their most recent births had been unwanted switched to more favorable reports at Time 2, and about the same percentage of women who reported Table 4, the findings would suggest that some of the groups typically considered to be most "at risk" were those who were also most apt to report inconsistently. This findings should signal some concern in the policy community. Net of other factors, there is no effect of income, but a number of at-risk groups remained significantly more likely than others to change their reports over time. Of particular concern is the finding that women who reported their pregnancies as mistimed or unwanted were so much more likely to change their reports over time than were women who initially said that their pregnancies had been well timed. When we look at the direction of change, it is clear that, with some exceptions, the more "at risk" groups were more likely to shift in a more negative direction, while the less "at risk" groups were less likely to do so; and at times they were more likely to report more favorably at the second interview. Although somewhat hampered by small sample sizes in the analysis of Time 1 mistimed and unwanted reports, results confirmed that certain subgroups were not only more apt to report inconsistently, they were also more likely to do so in specific directions. To the extent that future survey questions can emphasize to respondents that they should focus on the time of conception, and not on current status, inconsistency across groups may be reduced.     

Bayesian estimation and testing of gene frequencies

This article explains estimation of gene frequencies from a Bayesian viewpoint using prior information. How to obtain Bayes estimators and the highest posterior density credible sets (Bayesian counterpart to classical confidence intervals) for gene frequencies is described. Tests of hypotheses are also discussed. A readily available mathematical application package is used to demonstrate the mathematical computations.     

A computational framework for the study of confidence in humans and animals

Confidence judgements, self-assessments about the quality of a subject's knowledge, are considered a central example of metacognition. Prima facie, introspection and self-report appear the only way to access the subjective sense of confidence or uncertainty. Contrary to this notion, overt behavioural measures can be used to study confidence judgements by animals trained in decision-making tasks with perceptual or mnemonic uncertainty. Here, we suggest that a computational approach can clarify the issues involved in interpreting these tasks and provide a much needed springboard for advancing the scientific understanding of confidence. We first review relevant theories of probabilistic inference and decision-making. We then critically discuss behavioural tasks employed to measure confidence in animals and show how quantitative models can help to constrain the computational strategies underlying confidence-reporting behaviours. In our view, post-decision wagering tasks with continuous measures of confidence appear to offer the best available metrics of confidence. Since behavioural reports alone provide a limited window into mechanism, we argue that progress calls for measuring the neural representations and identifying the computations underlying confidence reports. We present a case study using such a computational approach to study the neural correlates of decision confidence in rats. This work shows that confidence assessments may be considered higher order, but can be generated using elementary neural computations that are available to a wide range of species. Finally, we discuss the relationship of confidence judgements to the wider behavioural uses of confidence and uncertainty.     

Boring's formulation: a scheme for identifying functional neuron groups in a sensory system

In 1935 Edwin Boring proposed that each attribute of sensation reflects the activity of a different neural circuit. If this idea is valid, it could facilitate both psychophysical and neurophysiological research on sensory systems. We think it likely that Boring's formulation is correct for three reasons: 1) Different sensory attributes reflect conscious information about different parameters of a stimulus. To be measured by any device, each of these parameters must be individually computed. Different neural circuits would appear to be necessary for the nervous system to carry out these different computations. 2) Perceived information about different sensory attributes can be made to diverge by appropriate manipulations of the stimuli. If there is a rigorous relationship between conscious sensory experience and neural activity, such a divergence implies that different sensory attributes are served by different neural circuits. 3) Accurate information about a sensory attribute requires that a human observer's attention be focused on that attribute. Changes in direction of attention are thought to involve a process of switching from one neural circuit to another, and provide another way to cause perceived information about different sensory attributes to diverge.     

Optimal experimental design for a nonlinear response in environmental toxicology

A start-stop experiment in environmental toxicology provides a backdrop for this design discussion. The basic problem is to decide when to sample a nonlinear response in order to minimize the generalized variance of the estimated parameters. An easily coded heuristic optimization strategy can be applied to this problem to obtain optimal or nearly optimal designs. The efficiency of the heuristic approach allows a straightforward exploration of the sensitivity of the suggested design with respect to such problem-specific concerns as variance heterogeneity, time-grid resolution, design criteria, and interval specification of planning values for parameters. A second illustration of design optimization is briefly presented in the context of concentration spacing for a reproductive toxicity study.