Observing the observer (II): deciding when to decide

In a companion paper [1], we have presented a generic approach for inferring how subjects make optimal decisions under uncertainty. From a Bayesian decision theoretic perspective, uncertain representations correspond to "posterior" beliefs, which result from integrating (sensory) information with subjective "prior" beliefs. Preferences and goals are encoded through a "loss" (or "utility") function, which measures the cost incurred by making any admissible decision for any given (hidden or unknown) state of the world. By assuming that subjects make optimal decisions on the basis of updated (posterior) beliefs and utility (loss) functions, one can evaluate the likelihood of observed behaviour. In this paper, we describe a concrete implementation of this meta-Bayesian approach (i.e. a Bayesian treatment of Bayesian decision theoretic predictions) and demonstrate its utility by applying it to both simulated and empirical reaction time data from an associative learning task. Here, inter-trial variability in reaction times is modelled as reflecting the dynamics of the subjects' internal recognition process, i.e. the updating of representations (posterior densities) of hidden states over trials while subjects learn probabilistic audio-visual associations. We use this paradigm to demonstrate that our meta-Bayesian framework allows for (i) probabilistic inference on the dynamics of the subject's representation of environmental states, and for (ii) model selection to disambiguate between alternative preferences (loss functions) human subjects could employ when dealing with trade-offs, such as between speed and accuracy. Finally, we illustrate how our approach can be used to quantify subjective beliefs and preferences that underlie inter-individual differences in behaviour.     

Monkeys Are More Patient in a Foraging Task than in a Standard Intertemporal Choice Task

Studies of animal impulsivity generally find steep subjective devaluation, or discounting, of delayed rewards – often on the order of a 50% reduction in value in a few seconds. Because such steep discounting is highly disfavored in evolutionary models of time preference, we hypothesize that discounting tasks provide a poor measure of animals’ true time preferences. One prediction of this hypothesis is that estimates of time preferences based on these tasks will lack external validity, i.e. fail to predict time preferences in other contexts. We examined choices made by four rhesus monkeys in a computerized patch-leaving foraging task interleaved with a standard intertemporal choice task. Monkeys were significantly more patient in the foraging task than in the intertemporal choice task. Patch-leaving behavior was well fit by parameter-free optimal foraging equations but poorly fit by the hyperbolic discount parameter obtained from the intertemporal choice task. Day-to-day variation in time preferences across the two tasks was uncorrelated with each other. These data are consistent with the conjecture that seemingly impulsive behavior in animals is an artifact of their difficulty understanding the structure of intertemporal choice tasks, and support the idea that animals are more efficient rate maximizers in the multi-second range than intertemporal choice tasks would suggest.     

Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings

Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating.     

Environmental Statistics and Optimal Regulation

Any organism is embedded in an environment that changes over time. The timescale for and statistics of environmental change, the precision with which the organism can detect its environment, and the costs and benefits of particular protein expression levels all will affect the suitability of different strategies–such as constitutive expression or graded response–for regulating protein levels in response to environmental inputs. We propose a general framework–here specifically applied to the enzymatic regulation of metabolism in response to changing concentrations of a basic nutrient–to predict the optimal regulatory strategy given the statistics of fluctuations in the environment and measurement apparatus, respectively, and the costs associated with enzyme production. We use this framework to address three fundamental questions: (i) when a cell should prefer thresholding to a graded response; (ii) when there is a fitness advantage to implementing a Bayesian decision rule; and (iii) when retaining memory of the past provides a selective advantage. We specifically find that: (i) relative convexity of enzyme expression cost and benefit influences the fitness of thresholding or graded responses; (ii) intermediate levels of measurement uncertainty call for a sophisticated Bayesian decision rule; and (iii) in dynamic contexts, intermediate levels of uncertainty call for retaining memory of the past. Statistical properties of the environment, such as variability and correlation times, set optimal biochemical parameters, such as thresholds and decay rates in signaling pathways. Our framework provides a theoretical basis for interpreting molecular signal processing algorithms and a classification scheme that organizes known regulatory strategies and may help conceptualize heretofore unknown ones.     

LiCl-induced sickness modulates rat gustatory cortical responses

Gustatory cortex (GC), a structure deeply involved in the making of consumption decisions, presumably performs this function by integrating information about taste, experiences, and internal states related to the animal’s health, such as illness. Here, we investigated this assertion, examining whether illness is represented in GC activity, and how this representation impacts taste responses and behavior. We recorded GC single-neuron activity and local field potentials (LFPs) from healthy rats and rats made ill (via LiCl injection). We show (consistent with the extant literature) that the onset of illness-related behaviors arises contemporaneously with alterations in 7 to 12 Hz LFP power at approximately 12 min following injection. This process was accompanied by reductions in single-neuron taste response magnitudes and discriminability, and with enhancements in palatability-relatedness—a result reflecting the collapse of responses toward a simple “good-bad” code visible in the entire sample, but focused on a specific subset of GC neurons. Overall, our data show that a state (illness) that profoundly reduces consumption changes basic properties of the sensory cortical response to tastes, in a manner that can easily explain illness’ impact on consumption.

Sickness is an internal state that impacts consumption, and so could be expected to influence the neural processing of tastes. This study shows that onset of illness changes basic properties of gustatory cortical network processing and taste responses, such that activity comes more purely to reflect the "goodness" or "badness" of tastes.     

Task-induced neural covariability as a signature of approximate Bayesian learning and inference

Perception is often characterized computationally as an inference process in which uncertain or ambiguous sensory inputs are combined with prior expectations. Although behavioral studies have shown that observers can change their prior expectations in the context of a task, robust neural signatures of task-specific priors have been elusive. Here, we analytically derive such signatures under the general assumption that the responses of sensory neurons encode posterior beliefs that combine sensory inputs with task-specific expectations. Specifically, we derive predictions for the task-dependence of correlated neural variability and decision-related signals in sensory neurons. The qualitative aspects of our results are parameter-free and specific to the statistics of each task. The predictions for correlated variability also differ from predictions of classic feedforward models of sensory processing and are therefore a strong test of theories of hierarchical Bayesian inference in the brain. Importantly, we find that Bayesian learning predicts an increase in so-called “differential correlations” as the observer’s internal model learns the stimulus distribution, and the observer’s behavioral performance improves. This stands in contrast to classic feedforward encoding/decoding models of sensory processing, since such correlations are fundamentally information-limiting. We find support for our predictions in data from existing neurophysiological studies across a variety of tasks and brain areas. Finally, we show in simulation how measurements of sensory neural responses can reveal information about a subject’s internal beliefs about the task. Taken together, our results reinterpret task-dependent sources of neural covariability as signatures of Bayesian inference and provide new insights into their cause and their function.     

Phenotypic variability in unicellular organisms: from calcium signalling to social behaviour

Historically, research has focused on the mean and often neglected the variance. However, variability in nature is observable at all scales: among cells within an individual, among individuals within a population and among populations within a species. A fundamental quest in biology now is to find the mechanisms that underlie variability. Here, we investigated behavioural variability in a unique unicellular organism, Physarum polycephalum. We combined experiments and models to show that variability in cell signalling contributes to major differences in behaviour underpinning some aspects of social interactions. First, following thousands of cells under various contexts, we identified distinct behavioural phenotypes: ‘slow–regular–social’, ‘fast–regular–social’ and ‘fast–irregular–asocial’. Second, coupling chemical analysis and behavioural assays we found that calcium signalling is responsible for these behavioural phenotypes. Finally, we show that differences in signalling and behaviour led to alternative social strategies. Our results have considerable implications for our understanding of the emergence of variability in living organisms.     

Advance Care Planning in Dementia: Do Family Carers Know the Treatment Preferences of People with Early Dementia?

MethodsA cross-sectional study interviewing 60 dyads (a person with early dementia and preserved capacity and their family carer) each completing a modified LSPQ. We assessed how closely carers’ choices resembled the PWD’s preferences for treatment in three proposed health states: the here and now; severe stroke with coma; terminal cancer. Agreement between the PWD and their family carer responses was assessed using Kappa and Prevalence-Adjusted Bias-Adjusted Kappa (PABAK) statistics. We examined whether carer burden and distress, and relationship quality, influenced agreement.ResultsIn interviews PWD were able to indicate their treatment preferences across all three scenarios. In the here-and-now most wanted antibiotics (98%), fewer cardio-pulmonary resuscitation (CPR) (50%) and tube feeding (47%). In severe stroke and coma antibiotics remained the more preferred treatment (88%), followed by CPR (57%) and tube feeding (30%). In advanced cancer PWD expressed lower preferences for all treatments (antibiotics 68%; CPR 50%; tube feeding 37%). Carers’ choices were similar to the PWDs’ preferences in the here-and-now (71% (k = 0.03; PABAK = 0.4) with less agreement for future hypothetical health states. In severe stroke and coma carers tended wrongly to suggest that the PWD preferred more intervention (antibiotic, 67%; k = -0.022; PABAK = -0.60; CPR, 73%; k = 0.20; PABAK = -0.20, tube feeding, 66%; k = 0.25; PABAK = -0.12). In advanced cancer the agreement between PWD and carers was low (antibiotics; k = -0.03; PABAK = -0.52; CPR, k = -0.07; PABAK = -0.45; tube feeding; k = 0.20; PABAK = -0.22). However, both PWD and carers showed marked uncertainty about their preferences for end of life treatment choices. Relationship quality, carer distress and burden had no influence on agreement.ConclusionsThis study is the first to have used the LSPQ with PWD in the UK to consider treatment options in hypothetical illness scenarios. Key finding are that family carers had a low to moderate agreement with PWD on preferences for end of life treatment. This underscores how planning for care at the end of life is beset with uncertainty, even when the carer and PWD perceive the care-giving/receiving relationship is good. Families affected by dementia may benefit from early and ongoing practical and emotional support to prepare for potential changes and aid decision making in the context of the realities of care towards the end of life.     

Multisensory Oddity Detection as Bayesian Inference

A key goal for the perceptual system is to optimally combine information from all the senses that may be available in order to develop the most accurate and unified picture possible of the outside world. The contemporary theoretical framework of ideal observer maximum likelihood integration (MLI) has been highly successful in modelling how the human brain combines information from a variety of different sensory modalities. However, in various recent experiments involving multisensory stimuli of uncertain correspondence, MLI breaks down as a successful model of sensory combination. Within the paradigm of direct stimulus estimation, perceptual models which use Bayesian inference to resolve correspondence have recently been shown to generalize successfully to these cases where MLI fails. This approach has been known variously as model inference, causal inference or structure inference. In this paper, we examine causal uncertainty in another important class of multi-sensory perception paradigm – that of oddity detection and demonstrate how a Bayesian ideal observer also treats oddity detection as a structure inference problem. We validate this approach by showing that it provides an intuitive and quantitative explanation of an important pair of multi-sensory oddity detection experiments – involving cues across and within modalities – for which MLI previously failed dramatically, allowing a novel unifying treatment of within and cross modal multisensory perception. Our successful application of structure inference models to the new ‘oddity detection’ paradigm, and the resultant unified explanation of across and within modality cases provide further evidence to suggest that structure inference may be a commonly evolved principle for combining perceptual information in the brain.     

Clarifying the relationship between body size and extinction risk in amphibians by complete mapping of model space

In vertebrates, large body size is often a key diagnostic feature of species threatened with extinction. However, in amphibians the link between body size and extinction risk is highly uncertain, with previous studies suggesting positive, negative, u-shaped, or no relationship. Part of the reason for this uncertainty is ‘researcher degrees of freedom’: the subjectivity and selectivity in choices associated with specifying and fitting models. Here, I clarify the size–threat association in amphibians using Specification Curve Analysis, an analytical approach from the social sciences that attempts to minimize this problem by complete mapping of model space. I find strong support for prevailing negative associations between body size and threat status, the opposite of patterns typical in other vertebrates. This pattern is largely explained by smaller species having smaller geographic ranges, but smaller amphibian species also appear to lack some of the life-history advantages (e.g. higher reproductive output) that are often assumed to ‘protect’ small species in other taxa. These results highlight the need for a renewed conservation focus on the smallest species of the world''s most threatened class of vertebrates, as aquatic habitats become increasingly degraded by human activity.     

Resolution of uncertainty in prefrontal cortex

Making optimal decisions in the face of uncertain or incomplete information arises as a common problem in everyday behavior, but the neural processes underlying this ability remain poorly understood. A typical case is navigation, in which a subject has to search for a known goal from an unknown location. Navigating under uncertain conditions requires making decisions on the basis of the current belief about location and updating that belief based on incoming information. Here, we use functional magnetic resonance imaging during a maze navigation task to study neural activity relating to the resolution of uncertainty as subjects make sequential decisions to reach a goal. We show that distinct regions of prefrontal cortex are engaged in specific computational functions that are well described by a Bayesian model of decision making. This permits efficient goal-oriented navigation and provides new insights into decision making by humans.     

People’s desire to be in nature and how they experience it are partially heritable

Nature experiences have been linked to mental and physical health. Despite the importance of understanding what determines individual variation in nature experience, the role of genes has been overlooked. Here, using a twin design (TwinsUK, number of individuals = 2,306), we investigate the genetic and environmental contributions to a person’s nature orientation, opportunity (living in less urbanized areas), and different dimensions of nature experience (frequency and duration of public nature space visits and frequency and duration of garden visits). We estimate moderate heritability of nature orientation (46%) and nature experiences (48% for frequency of public nature space visits, 34% for frequency of garden visits, and 38% for duration of garden visits) and show their genetic components partially overlap. We also find that the environmental influences on nature experiences are moderated by the level of urbanization of the home district. Our study demonstrates genetic contributions to individuals’ nature experiences, opening a new dimension for the study of human–nature interactions.

Nature experiences have been linked to mental and physical health. This twin study reveals genetic influences on an individual’s orientation towards nature and nature experiences, opening a new dimension to understanding human-nature interactions.     

Assessing agreement between multiple raters with missing rating information, applied to breast cancer tumour grading

Background

We consider the problem of assessing inter-rater agreement when there are missing data and a large number of raters. Previous studies have shown only ‘moderate’ agreement between pathologists in grading breast cancer tumour specimens. We analyse a large but incomplete data-set consisting of 24177 grades, on a discrete 1–3 scale, provided by 732 pathologists for 52 samples.

Methodology/Principal Findings

We review existing methods for analysing inter-rater agreement for multiple raters and demonstrate two further methods. Firstly, we examine a simple non-chance-corrected agreement score based on the observed proportion of agreements with the consensus for each sample, which makes no allowance for missing data. Secondly, treating grades as lying on a continuous scale representing tumour severity, we use a Bayesian latent trait method to model cumulative probabilities of assigning grade values as functions of the severity and clarity of the tumour and of rater-specific parameters representing boundaries between grades 1–2 and 2–3. We simulate from the fitted model to estimate, for each rater, the probability of agreement with the majority. Both methods suggest that there are differences between raters in terms of rating behaviour, most often caused by consistent over- or under-estimation of the grade boundaries, and also considerable variability in the distribution of grades assigned to many individual samples. The Bayesian model addresses the tendency of the agreement score to be biased upwards for raters who, by chance, see a relatively ‘easy’ set of samples.

Conclusions/Significance

Latent trait models can be adapted to provide novel information about the nature of inter-rater agreement when the number of raters is large and there are missing data. In this large study there is substantial variability between pathologists and uncertainty in the identity of the ‘true’ grade of many of the breast cancer tumours, a fact often ignored in clinical studies.     

Subjective optimality in finite sequential decision-making

Many decisions in life are sequential and constrained by a time window. Although mathematically derived optimal solutions exist, it has been reported that humans often deviate from making optimal choices. Here, we used a secretary problem, a classic example of finite sequential decision-making, and investigated the mechanisms underlying individuals’ suboptimal choices. Across three independent experiments, we found that a dynamic programming model comprising subjective value function explains individuals’ deviations from optimality and predicts the choice behaviors under fewer and more opportunities. We further identified that pupil dilation reflected the levels of decision difficulty and subsequent choices to accept or reject the stimulus at each opportunity. The value sensitivity, a model-based estimate that characterizes each individual’s subjective valuation, correlated with the extent to which individuals’ physiological responses tracked stimuli information. Our results provide model-based and physiological evidence for subjective valuation in finite sequential decision-making, rediscovering human suboptimality in subjectively optimal decision-making processes.     

Neural signatures of betrayal aversion: an fMRI study of trust

Decisions are said to be ‘risky’ when they are made in environments with uncertainty caused by nature. By contrast, a decision is said to be ‘trusting’ when its outcome depends on the uncertain decisions of another person. A rapidly expanding literature reveals economically important differences between risky and trusting decisions, and further suggests these differences are due to ‘betrayal aversion’. While its neural foundations have not been previously illuminated, the prevailing hypothesis is that betrayal aversion stems from a desire to avoid negative emotions that arise from learning one''s trust was betrayed. Here, we provide evidence from an fMRI study that supports this hypothesis. In particular, our data indicate that the anterior insula modulates trusting decisions that involve the possibility of betrayal.     

Neural Correlates of Advantageous and Disadvantageous Inequity in Sharing Decisions

Humans have a strong preference for fair distributions of resources. Neuroimaging studies have shown that being treated unfairly coincides with activation in brain regions involved in signaling conflict and negative affect. Less is known about neural responses involved in violating a fairness norm ourselves. Here, we investigated the neural patterns associated with inequity, where participants were asked to choose between an equal split of money and an unequal split that could either maximize their own (advantageous inequity) or another person’s (disadvantageous inequity) earnings. Choosing to divide money unequally, irrespective who benefited from the unequal distribution, was associated with activity in the dorsal anterior cingulate cortex, anterior insula and the dorsolateral prefrontal cortex. Inequity choices that maximized another person’s profits were further associated with activity in the ventral striatum and ventromedial prefrontal cortex. Taken together, our findings show evidence of a common neural pattern associated with both advantageous and disadvantageous inequity in sharing decisions and additional recruitment of neural circuitry previously linked to the computation of subjective value and reward when violating a fairness norm at the benefit of someone else.     

Humans can adopt optimal discounting strategy under real-time constraints

Critical to our many daily choices between larger delayed rewards, and smaller more immediate rewards, are the shape and the steepness of the function that discounts rewards with time. Although research in artificial intelligence favors exponential discounting in uncertain environments, studies with humans and animals have consistently shown hyperbolic discounting. We investigated how humans perform in a reward decision task with temporal constraints, in which each choice affects the time remaining for later trials, and in which the delays vary at each trial. We demonstrated that most of our subjects adopted exponential discounting in this experiment. Further, we confirmed analytically that exponential discounting, with a decay rate comparable to that used by our subjects, maximized the total reward gain in our task. Our results suggest that the particular shape and steepness of temporal discounting is determined by the task that the subject is facing, and question the notion of hyperbolic reward discounting as a universal principle.     

A High Throughput In Vivo Assay for Taste Quality and Palatability

Taste quality and palatability are two of the most important properties measured in the evaluation of taste stimuli. Human panels can report both aspects, but are of limited experimental flexibility and throughput capacity. Relatively efficient animal models for taste evaluation have been developed, but each of them is designed to measure either taste quality or palatability as independent experimental endpoints. We present here a new apparatus and method for high throughput quantification of both taste quality and palatability using rats in an operant taste discrimination paradigm. Cohorts of four rats were trained in a modified operant chamber to sample taste stimuli by licking solutions from a 96-well plate that moved in a randomized pattern beneath the chamber floor. As a rat’s tongue entered the well it disrupted a laser beam projecting across the top of the 96-well plate, consequently producing two retractable levers that operated a pellet dispenser. The taste of sucrose was associated with food reinforcement by presses on a sucrose-designated lever, whereas the taste of water and other basic tastes were associated with the alternative lever. Each disruption of the laser was counted as a lick. Using this procedure, rats were trained to discriminate 100 mM sucrose from water, quinine, citric acid, and NaCl with 90-100% accuracy. Palatability was determined by the number of licks per trial and, due to intermediate rates of licking for water, was quantifiable along the entire spectrum of appetitiveness to aversiveness. All 96 samples were evaluated within 90 minute test sessions with no evidence of desensitization or fatigue. The technology is capable of generating multiple concentration–response functions within a single session, is suitable for in vivo primary screening of tastant libraries, and potentially can be used to evaluate stimuli for any taste system.     

Functional annotation signatures of disease susceptibility loci improve SNP association analysis

Background

Genetic association studies are conducted to discover genetic loci that contribute to an inherited trait, identify the variants behind these associations and ascertain their functional role in determining the phenotype. To date, functional annotations of the genetic variants have rarely played more than an indirect role in assessing evidence for association. Here, we demonstrate how these data can be systematically integrated into an association study’s analysis plan.

Results

We developed a Bayesian statistical model for the prior probability of phenotype–genotype association that incorporates data from past association studies and publicly available functional annotation data regarding the susceptibility variants under study. The model takes the form of a binary regression of association status on a set of annotation variables whose coefficients were estimated through an analysis of associated SNPs in the GWAS Catalog (GC). The functional predictors examined included measures that have been demonstrated to correlate with the association status of SNPs in the GC and some whose utility in this regard is speculative: summaries of the UCSC Human Genome Browser ENCODE super–track data, dbSNP function class, sequence conservation summaries, proximity to genomic variants in the Database of Genomic Variants and known regulatory elements in the Open Regulatory Annotation database, PolyPhen–2 probabilities and RegulomeDB categories. Because we expected that only a fraction of the annotations would contribute to predicting association, we employed a penalized likelihood method to reduce the impact of non–informative predictors and evaluated the model’s ability to predict GC SNPs not used to construct the model. We show that the functional data alone are predictive of a SNP’s presence in the GC. Further, using data from a genome–wide study of ovarian cancer, we demonstrate that their use as prior data when testing for association is practical at the genome–wide scale and improves power to detect associations.

Conclusions

We show how diverse functional annotations can be efficiently combined to create ‘functional signatures’ that predict the a priori odds of a variant’s association to a trait and how these signatures can be integrated into a standard genome–wide–scale association analysis, resulting in improved power to detect truly associated variants.

Electronic supplementary material

The online version of this article (doi:10.1186/1471-2164-15-398) contains supplementary material, which is available to authorized users.