Observing the observer (I): meta-bayesian models of learning and decision-making

In this paper, we present a generic approach that can be used to infer how subjects make optimal decisions under uncertainty. This approach induces a distinction between a subject's perceptual model, which underlies the representation of a hidden "state of affairs" and a response model, which predicts the ensuing behavioural (or neurophysiological) responses to those inputs. We start with the premise that subjects continuously update a probabilistic representation of the causes of their sensory inputs to optimise their behaviour. In addition, subjects have preferences or goals that guide decisions about actions given the above uncertain representation of these hidden causes or state of affairs. From a Bayesian decision theoretic perspective, uncertain representations are so-called "posterior" beliefs, which are influenced by subjective "prior" beliefs. Preferences and goals are encoded through a "loss" (or "utility") function, which measures the cost incurred by making any admissible decision for any given (hidden) state of affair. By assuming that subjects make optimal decisions on the basis of updated (posterior) beliefs and utility (loss) functions, one can evaluate the likelihood of observed behaviour. Critically, this enables one to "observe the observer", i.e. identify (context- or subject-dependent) prior beliefs and utility-functions using psychophysical or neurophysiological measures. In this paper, we describe the main theoretical components of this meta-Bayesian approach (i.e. a Bayesian treatment of Bayesian decision theoretic predictions). In a companion paper ('Observing the observer (II): deciding when to decide'), we describe a concrete implementation of it and demonstrate its utility by applying it to simulated and real reaction time data from an associative learning task.     

Active inference and agency: optimal control without cost functions

This paper describes a variational free-energy formulation of (partially observable) Markov decision problems in decision making under uncertainty. We show that optimal control can be cast as active inference. In active inference, both action and posterior beliefs about hidden states minimise a free energy bound on the negative log-likelihood of observed states, under a generative model. In this setting, reward or cost functions are absorbed into prior beliefs about state transitions and terminal states. Effectively, this converts optimal control into a pure inference problem, enabling the application of standard Bayesian filtering techniques. We then consider optimal trajectories that rest on posterior beliefs about hidden states in the future. Crucially, this entails modelling control as a hidden state that endows the generative model with a representation of agency. This leads to a distinction between models with and without inference on hidden control states; namely, agency-free and agency-based models, respectively.     

The statistical basis of public policy: a paradigm shift is overdue.

The recent controversy over the increased risk of venous thrombosis with third generation oral contraceptives illustrates the public policy dilemma that can be created by relying on conventional statistical tests and estimates: case-control studies showed a significant increase in risk and forced a decision either to warn or not to warn. Conventional statistical tests are an improper basis for such decisions because they dichotomise results according to whether they are or are not significant and do not allow decision makers to take explicit account of additional evidence--for example, of biological plausibility or of biases in the studies. A Bayesian approach overcomes both these problems. A Bayesian analysis starts with a "prior" probability distribution for the value of interest (for example, a true relative risk)--based on previous knowledge--and adds the new evidence (via a model) to produce a "posterior" probability distribution. Because different experts will have different prior beliefs sensitivity analyses are important to assess the effects on the posterior distributions of these differences. Sensitivity analyses should also examine the effects of different assumptions about biases and about the model which links the data with the value of interest. One advantage of this method is that it allows such assumptions to be handled openly and explicitly. Data presented as a series of posterior probability distributions would be a much better guide to policy, reflecting the reality that degrees of belief are often continuous, not dichotomous, and often vary from one person to another in the face of inconclusive evidence.     

Process evaluation of appreciative inquiry to translate pain management evidence into pediatric nursing practice

Background

Current healthcare systems have extended the evidence-based medicine (EBM) approach to health policy and delivery decisions, such as access-to-care, healthcare funding and health program continuance, through attempts to integrate valid and reliable evidence into the decision making process. These policy decisions have major impacts on society and have high personal and financial costs associated with those decisions. Decision models such as these function under a shared assumption of rational choice and utility maximization in the decision-making process.

Discussion

We contend that health policy decision makers are generally unable to attain the basic goals of evidence-based decision making (EBDM) and evidence-based policy making (EBPM) because humans make decisions with their naturally limited, faulty, and biased decision-making processes. A cognitive information processing framework is presented to support this argument, and subtle cognitive processing mechanisms are introduced to support the focal thesis: health policy makers' decisions are influenced by the subjective manner in which they individually process decision-relevant information rather than on the objective merits of the evidence alone. As such, subsequent health policy decisions do not necessarily achieve the goals of evidence-based policy making, such as maximizing health outcomes for society based on valid and reliable research evidence.

Summary

In this era of increasing adoption of evidence-based healthcare models, the rational choice, utility maximizing assumptions in EBDM and EBPM, must be critically evaluated to ensure effective and high-quality health policy decisions. The cognitive information processing framework presented here will aid health policy decision makers by identifying how their decisions might be subtly influenced by non-rational factors. In this paper, we identify some of the biases and potential intervention points and provide some initial suggestions about how the EBDM/EBPM process can be improved.     

A nonparametric Bayesian basket trial design

Targeted therapies on the basis of genomic aberrations analysis of the tumor have shown promising results in cancer prognosis and treatment. Regardless of tumor type, trials that match patients to targeted therapies for their particular genomic aberrations have become a mainstream direction of therapeutic management of patients with cancer. Therefore, finding the subpopulation of patients who can most benefit from an aberration‐specific targeted therapy across multiple cancer types is important. We propose an adaptive Bayesian clinical trial design for patient allocation and subpopulation identification. We start with a decision theoretic approach, including a utility function and a probability model across all possible subpopulation models. The main features of the proposed design and population finding methods are the use of a flexible nonparametric Bayesian survival regression based on a random covariate‐dependent partition of patients, and decisions based on a flexible utility function that reflects the requirement of the clinicians appropriately and realistically, and the adaptive allocation of patients to their superior treatments. Through extensive simulation studies, the new method is demonstrated to achieve desirable operating characteristics and compares favorably against the alternatives.     

Internal Representations of Temporal Statistics and Feedback Calibrate Motor-Sensory Interval Timing

Humans have been shown to adapt to the temporal statistics of timing tasks so as to optimize the accuracy of their responses, in agreement with the predictions of Bayesian integration. This suggests that they build an internal representation of both the experimentally imposed distribution of time intervals (the prior) and of the error (the loss function). The responses of a Bayesian ideal observer depend crucially on these internal representations, which have only been previously studied for simple distributions. To study the nature of these representations we asked subjects to reproduce time intervals drawn from underlying temporal distributions of varying complexity, from uniform to highly skewed or bimodal while also varying the error mapping that determined the performance feedback. Interval reproduction times were affected by both the distribution and feedback, in good agreement with a performance-optimizing Bayesian observer and actor model. Bayesian model comparison highlighted that subjects were integrating the provided feedback and represented the experimental distribution with a smoothed approximation. A nonparametric reconstruction of the subjective priors from the data shows that they are generally in agreement with the true distributions up to third-order moments, but with systematically heavier tails. In particular, higher-order statistical features (kurtosis, multimodality) seem much harder to acquire. Our findings suggest that humans have only minor constraints on learning lower-order statistical properties of unimodal (including peaked and skewed) distributions of time intervals under the guidance of corrective feedback, and that their behavior is well explained by Bayesian decision theory.     

A Bayesian Attractor Model for Perceptual Decision Making

Even for simple perceptual decisions, the mechanisms that the brain employs are still under debate. Although current consensus states that the brain accumulates evidence extracted from noisy sensory information, open questions remain about how this simple model relates to other perceptual phenomena such as flexibility in decisions, decision-dependent modulation of sensory gain, or confidence about a decision. We propose a novel approach of how perceptual decisions are made by combining two influential formalisms into a new model. Specifically, we embed an attractor model of decision making into a probabilistic framework that models decision making as Bayesian inference. We show that the new model can explain decision making behaviour by fitting it to experimental data. In addition, the new model combines for the first time three important features: First, the model can update decisions in response to switches in the underlying stimulus. Second, the probabilistic formulation accounts for top-down effects that may explain recent experimental findings of decision-related gain modulation of sensory neurons. Finally, the model computes an explicit measure of confidence which we relate to recent experimental evidence for confidence computations in perceptual decision tasks.     

Detecting recombination in 4-taxa DNA sequence alignments with Bayesian hidden Markov models and Markov chain Monte Carlo

This article presents a statistical method for detecting recombination in DNA sequence alignments, which is based on combining two probabilistic graphical models: (1) a taxon graph (phylogenetic tree) representing the relationship between the taxa, and (2) a site graph (hidden Markov model) representing interactions between different sites in the DNA sequence alignments. We adopt a Bayesian approach and sample the parameters of the model from the posterior distribution with Markov chain Monte Carlo, using a Metropolis-Hastings and Gibbs-within-Gibbs scheme. The proposed method is tested on various synthetic and real-world DNA sequence alignments, and we compare its performance with the established detection methods RECPARS, PLATO, and TOPAL, as well as with two alternative parameter estimation schemes.     

Moral concepts set decision strategies to abstract values

Persons have different value preferences. Neuroimaging studies where value-based decisions in actual conflict situations were investigated suggest an important role of prefrontal and cingulate brain regions. General preferences, however, reflect a superordinate moral concept independent of actual situations as proposed in psychological and socioeconomic research. Here, the specific brain response would be influenced by abstract value systems and moral concepts. The neurobiological mechanisms underlying such responses are largely unknown. Using functional magnetic resonance imaging (fMRI) with a forced-choice paradigm on word pairs representing abstract values, we show that the brain handles such decisions depending on the person's superordinate moral concept. Persons with a predominant collectivistic (altruistic) value system applied a "balancing and weighing" strategy, recruiting brain regions of rostral inferior and intraparietal, and midcingulate and frontal cortex. Conversely, subjects with mainly individualistic (egocentric) value preferences applied a "fight-and-flight" strategy by recruiting the left amygdala. Finally, if subjects experience a value conflict when rejecting an alternative congruent to their own predominant value preference, comparable brain regions are activated as found in actual moral dilemma situations, i.e., midcingulate and dorsolateral prefrontal cortex. Our results demonstrate that superordinate moral concepts influence the strategy and the neural mechanisms in decision processes, independent of actual situations, showing that decisions are based on general neural principles. These findings provide a novel perspective to future sociological and economic research as well as to the analysis of social relations by focusing on abstract value systems as triggers of specific brain responses.     

Allocation Variable-Based Probabilistic Algorithm to Deal with Label Switching Problem in Bayesian Mixture Models

The label switching problem occurs as a result of the nonidentifiability of posterior distribution over various permutations of component labels when using Bayesian approach to estimate parameters in mixture models. In the cases where the number of components is fixed and known, we propose a relabelling algorithm, an allocation variable-based (denoted by AVP) probabilistic relabelling approach, to deal with label switching problem. We establish a model for the posterior distribution of allocation variables with label switching phenomenon. The AVP algorithm stochastically relabel the posterior samples according to the posterior probabilities of the established model. Some existing deterministic and other probabilistic algorithms are compared with AVP algorithm in simulation studies, and the success of the proposed approach is demonstrated in simulation studies and a real dataset.     

Agency Modulates the Lateral and Medial Prefrontal Cortex Responses in Belief-Based Decision Making

Many real-life decisions in complex and changing environments are guided by the decision maker’s beliefs, such as her perceived control over decision outcomes (i.e., agency), leading to phenomena like the “illusion of control”. However, the neural mechanisms underlying the “agency” effect on belief-based decisions are not well understood. Using functional imaging and a card guessing game, we revealed that the agency manipulation (i.e., either asking the subjects (SG) or the computer (CG) to guess the location of the winning card) not only affected the size of subjects’ bets, but also their “world model” regarding the outcome dependency. Functional imaging results revealed that the decision-related activation in the lateral and medial prefrontal cortex (PFC) was significantly modulated by agency and previous outcome. Specifically, these PFC regions showed stronger activation when subjects made decisions after losses than after wins under the CG condition, but the pattern was reversed under the SG condition. Furthermore, subjects with high external attribution of negative events were more affected by agency at the behavioral and neural levels. These results suggest that the prefrontal decision-making system can be modulated by abstract beliefs, and are thus vulnerable to factors such as false agency and attribution.     

A bayesian approach to the design of phase II clinical trials

A new strategy for the design of Phase II clinical trials is presented which utilizes the information provided by the prior distribution of the response rate, the costs of treating a patient, and the losses or gains resulting from the decisions taken at the completion of the study. A risk function is derived from which one may determine the optimal Bayes sampling plan. The decision theoretic/Bayesian approach is shown to provide a formal justification for the sample sizes often used in practice and shows the conditions under which such sample sizes are clearly inappropriate.     

ToPS: A Framework to Manipulate Probabilistic Models of Sequence Data

Discrete Markovian models can be used to characterize patterns in sequences of values and have many applications in biological sequence analysis, including gene prediction, CpG island detection, alignment, and protein profiling. We present ToPS, a computational framework that can be used to implement different applications in bioinformatics analysis by combining eight kinds of models: (i) independent and identically distributed process; (ii) variable-length Markov chain; (iii) inhomogeneous Markov chain; (iv) hidden Markov model; (v) profile hidden Markov model; (vi) pair hidden Markov model; (vii) generalized hidden Markov model; and (viii) similarity based sequence weighting. The framework includes functionality for training, simulation and decoding of the models. Additionally, it provides two methods to help parameter setting: Akaike and Bayesian information criteria (AIC and BIC). The models can be used stand-alone, combined in Bayesian classifiers, or included in more complex, multi-model, probabilistic architectures using GHMMs. In particular the framework provides a novel, flexible, implementation of decoding in GHMMs that detects when the architecture can be traversed efficiently.

This is a PLOS Computational Biology Software Article.

     

The theory of functional systems and purposeful behavior

The role of probability forecasting in the purposive behavior under conditions of subjective uncertainty is considered in terms of the theory of functional systems. Participation of the probability forecasting in the afferent synthesis, goal formation, formation of the acceptor of action result and action program, and, finally, in the action program actualization is substantiated. The model of behavior under conditions of subjective uncertainty is advanced. It includes all the classical elements of the model of behavioral act developed by P.K. Anokhin. In order to take into account the probability aspects of behavior, the role of probability forecasting is emphasized at every stage of the system functioning. In addition to the classical elements, two novel components are introduced. These are the "memory buffer" (results of searching reactions) and the apparatus of probability decisions about changes in the action program. By the memory buffer an apparatus is meant, which gathers and stores the information about the results of many behavioral acts performed during the actualization of the action program. This information is used in the process of making a probability decision as whether to alter or not the action program after each specific behavioral act. Such an approach integrates the probability forecasting and the theory of functional systems. The theory becomes universal, i.e., applicable not only to deterministic but also to probabilistic environments.     

Bayes' theorem and its applications in animal behaviour

Bayesian decision theory can be used to model animal behaviour. In this paper we give an overview of the theoretical concepts in such models. We also review the biological contexts in which Bayesian models have been applied, and outline some directions where future studies would be useful. Bayesian decision theory, when applied to animal behaviour, is based on the assumption that the individual has some sort of "prior opinion" of the possible states of the world. This may, for example, be a previously experienced distribution of qualities of food patches, or qualities of potential mates. The animal is then assumed to be able use sampling information to arrive at a "posterior opinion", concerning e.g. the quality of a given food patch, or the average qualities of mates in a year. A correctly formulated Bayesian model predicts how animals may combine previous experience with sampling information to make optimal decisions. We argue that the assumption that animals may have "prior opinions" is reasonable. Their priors may come from one or both of two sources: either from their own individual experience, gained while sampling the environment, or from an adaptation to the environment experienced by previous generations. This means that we should often expect to see "Bayesian-like" decision-making in nature.     

Dynamic Modeling of Vaccinating Behavior as a Function of Individual Beliefs

Individual perception of vaccine safety is an important factor in determining a person's adherence to a vaccination program and its consequences for disease control. This perception, or belief, about the safety of a given vaccine is not a static parameter but a variable subject to environmental influence. To complicate matters, perception of risk (or safety) does not correspond to actual risk. In this paper we propose a way to include the dynamics of such beliefs into a realistic epidemiological model, yielding a more complete depiction of the mechanisms underlying the unraveling of vaccination campaigns. The methodology proposed is based on Bayesian inference and can be extended to model more complex belief systems associated with decision models. We found the method is able to produce behaviors which approximate what has been observed in real vaccine and disease scare situations. The framework presented comprises a set of useful tools for an adequate quantitative representation of a common yet complex public-health issue. These tools include representation of beliefs as Bayesian probabilities, usage of logarithmic pooling to combine probability distributions representing opinions, and usage of natural conjugate priors to efficiently compute the Bayesian posterior. This approach allowed a comprehensive treatment of the uncertainty regarding vaccination behavior in a realistic epidemiological model.     

From Sensory Signals to Modality-Independent Conceptual Representations: A Probabilistic Language of Thought Approach

People learn modality-independent, conceptual representations from modality-specific sensory signals. Here, we hypothesize that any system that accomplishes this feat will include three components: a representational language for characterizing modality-independent representations, a set of sensory-specific forward models for mapping from modality-independent representations to sensory signals, and an inference algorithm for inverting forward models—that is, an algorithm for using sensory signals to infer modality-independent representations. To evaluate this hypothesis, we instantiate it in the form of a computational model that learns object shape representations from visual and/or haptic signals. The model uses a probabilistic grammar to characterize modality-independent representations of object shape, uses a computer graphics toolkit and a human hand simulator to map from object representations to visual and haptic features, respectively, and uses a Bayesian inference algorithm to infer modality-independent object representations from visual and/or haptic signals. Simulation results show that the model infers identical object representations when an object is viewed, grasped, or both. That is, the model’s percepts are modality invariant. We also report the results of an experiment in which different subjects rated the similarity of pairs of objects in different sensory conditions, and show that the model provides a very accurate account of subjects’ ratings. Conceptually, this research significantly contributes to our understanding of modality invariance, an important type of perceptual constancy, by demonstrating how modality-independent representations can be acquired and used. Methodologically, it provides an important contribution to cognitive modeling, particularly an emerging probabilistic language-of-thought approach, by showing how symbolic and statistical approaches can be combined in order to understand aspects of human perception.