Dynamic integration of reward and stimulus information in perceptual decision-making

In perceptual decision-making, ideal decision-makers should bias their choices toward alternatives associated with larger rewards, and the extent of the bias should decrease as stimulus sensitivity increases. When responses must be made at different times after stimulus onset, stimulus sensitivity grows with time from zero to a final asymptotic level. Are decision makers able to produce responses that are more biased if they are made soon after stimulus onset, but less biased if they are made after more evidence has been accumulated? If so, how close to optimal can they come in doing this, and how might their performance be achieved mechanistically? We report an experiment in which the payoff for each alternative is indicated before stimulus onset. Processing time is controlled by a “go” cue occurring at different times post stimulus onset, requiring a response within msec. Reward bias does start high when processing time is short and decreases as sensitivity increases, leveling off at a non-zero value. However, the degree of bias is sub-optimal for shorter processing times. We present a mechanistic account of participants'' performance within the framework of the leaky competing accumulator model [1], in which accumulators for each alternative accumulate noisy information subject to leakage and mutual inhibition. The leveling off of accuracy is attributed to mutual inhibition between the accumulators, allowing the accumulator that gathers the most evidence early in a trial to suppress the alternative. Three ways reward might affect decision making in this framework are considered. One of the three, in which reward affects the starting point of the evidence accumulation process, is consistent with the qualitative pattern of the observed reward bias effect, while the other two are not. Incorporating this assumption into the leaky competing accumulator model, we are able to provide close quantitative fits to individual participant data.     

Accumulation of continuously time-varying sensory evidence constrains neural and behavioral responses in human collision threat detection

Evidence accumulation models provide a dominant account of human decision-making, and have been particularly successful at explaining behavioral and neural data in laboratory paradigms using abstract, stationary stimuli. It has been proposed, but with limited in-depth investigation so far, that similar decision-making mechanisms are involved in tasks of a more embodied nature, such as movement and locomotion, by directly accumulating externally measurable sensory quantities of which the precise, typically continuously time-varying, magnitudes are important for successful behavior. Here, we leverage collision threat detection as a task which is ecologically relevant in this sense, but which can also be rigorously observed and modelled in a laboratory setting. Conventionally, it is assumed that humans are limited in this task by a perceptual threshold on the optical expansion rate–the visual looming–of the obstacle. Using concurrent recordings of EEG and behavioral responses, we disprove this conventional assumption, and instead provide strong evidence that humans detect collision threats by accumulating the continuously time-varying visual looming signal. Generalizing existing accumulator model assumptions from stationary to time-varying sensory evidence, we show that our model accounts for previously unexplained empirical observations and full distributions of detection response. We replicate a pre-response centroparietal positivity (CPP) in scalp potentials, which has previously been found to correlate with accumulated decision evidence. In contrast with these existing findings, we show that our model is capable of predicting the onset of the CPP signature rather than its buildup, suggesting that neural evidence accumulation is implemented differently, possibly in distinct brain regions, in collision detection compared to previously studied paradigms.     

Lateralized Readiness Potentials Reveal Properties of a Neural Mechanism for Implementing a Decision Threshold

Many perceptual decision making models posit that participants accumulate noisy evidence over time to improve the accuracy of their decisions, and that in free response tasks, participants respond when the accumulated evidence reaches a decision threshold. Research on the neural correlates of these models'' components focuses primarily on evidence accumulation. Far less attention has been paid to the neural correlates of decision thresholds, reflecting the final commitment to a decision. Inspired by a model of bistable neural activity that implements a decision threshold, we reinterpret human lateralized readiness potentials (LRPs) as reflecting the crossing of a decision threshold. Interestingly, this threshold crossing preserves signatures of a drift-diffusion process of evidence accumulation that feeds in to the threshold mechanism. We show that, as our model predicts, LRP amplitudes and growth rates recorded while participants performed a motion discrimination task correlate with individual differences in behaviorally-estimated prior beliefs, decision thresholds and evidence accumulation rates. As such LRPs provide a useful measure to test dynamical models of both evidence accumulation and decision commitment processes non-invasively.     

Decisions reduce sensitivity to subsequent information

Behavioural studies over half a century indicate that making categorical choices alters beliefs about the state of the world. People seem biased to confirm previous choices, and to suppress contradicting information. These choice-dependent biases imply a fundamental bound of human rationality. However, it remains unclear whether these effects extend to lower level decisions, and only little is known about the computational mechanisms underlying them. Building on the framework of sequential-sampling models of decision-making, we developed novel psychophysical protocols that enable us to dissect quantitatively how choices affect the way decision-makers accumulate additional noisy evidence. We find robust choice-induced biases in the accumulation of abstract numerical (experiment 1) and low-level perceptual (experiment 2) evidence. These biases deteriorate estimations of the mean value of the numerical sequence (experiment 1) and reduce the likelihood to revise decisions (experiment 2). Computational modelling reveals that choices trigger a reduction of sensitivity to subsequent evidence via multiplicative gain modulation, rather than shifting the decision variable towards the chosen alternative in an additive fashion. Our results thus show that categorical choices alter the evidence accumulation mechanism itself, rather than just its outcome, rendering the decision-maker less sensitive to new information.     

Decomposing Effects of Time on Task Reveals an Anteroposterior Gradient of Perceptual Decision Regions

In perceptual decision making, the selection of an appropriate action depends critically on an organism’s ability to use sensory inputs to accumulate evidence for a decision. However, differentiating decision-related processes from effects of “time on task” can be difficult. Here we combine the response signal paradigm, in which the experimenter rather than the subject dictates the time of the response, and independent components analysis (ICA) to search for signatures consistent with time on task and decision making, respectively, throughout the brain. Using this novel approach, we identify two such independent components from BOLD activity related to a random dot motion task: one sensitive to the main effect of stimulus duration, and one to both the main effect of motion coherence and its interaction with duration. Furthermore, we demonstrate that these two components are expressed differently throughout the brain, with activity in occipital regions most reflective of the former, activity within intraparietal sulcus modulated by both factors, and more anterior regions including the anterior insula, pre-SMA, and inferior frontal sulcus driven almost exclusively by the latter. Consistent with these ICA findings, cluster analysis identifies a posterior-to-anterior gradient that differentiates regions sensitive to time on task from regions whose activity is strongly tied to motion coherence. Together, these findings demonstrate that progressively more anterior regions are likely to participate in progressively more proximate decision-related processes.     

Evidence for social role in a dolphin social network

Social animals have to take into consideration the behaviour of conspecifics when making decisions to go by their daily lives. These decisions affect their fitness and there is therefore an evolutionary pressure to try making the right choices. In many instances individuals will make their own choices and the behaviour of the group will be a democratic integration of everyone’s decision. However, in some instances it can be advantageous to follow the choice of a few individuals in the group if they have more information regarding the situation that has arisen. Here I provide early evidence that decisions about shifts in activity states in a population of bottlenose dolphin follow such a decision-making process. This unshared consensus is mediated by a non-vocal signal, which can be communicated globally within the dolphin school. These signals are emitted by individuals that tend to have more information about the behaviour of potential competitors because of their position in the social network. I hypothesise that this decision-making process emerged from the social structure of the population and the need to maintain mixed-sex schools.     

Counterfactual choice and learning in a neural network centered on human lateral frontopolar cortex

Decision making and learning in a real-world context require organisms to track not only the choices they make and the outcomes that follow but also other untaken, or counterfactual, choices and their outcomes. Although the neural system responsible for tracking the value of choices actually taken is increasingly well understood, whether a neural system tracks counterfactual information is currently unclear. Using a three-alternative decision-making task, a Bayesian reinforcement-learning algorithm, and fMRI, we investigated the coding of counterfactual choices and prediction errors in the human brain. Rather than representing evidence favoring multiple counterfactual choices, lateral frontal polar cortex (lFPC), dorsomedial frontal cortex (DMFC), and posteromedial cortex (PMC) encode the reward-based evidence favoring the best counterfactual option at future decisions. In addition to encoding counterfactual reward expectations, the network carries a signal for learning about counterfactual options when feedback is available-a counterfactual prediction error. Unlike other brain regions that have been associated with the processing of counterfactual outcomes, counterfactual prediction errors within the identified network cannot be related to regret theory. Furthermore, individual variation in counterfactual choice-related activity and prediction error-related activity, respectively, predicts variation in the propensity to switch to profitable choices in the future and the ability to learn from hypothetical feedback. Taken together, these data provide both neural and behavioral evidence to support the existence of a previously unidentified neural system responsible for tracking both counterfactual choice options and their outcomes.     

Activity in Inferior Parietal and Medial Prefrontal Cortex Signals the Accumulation of Evidence in a Probability Learning Task

In an uncertain environment, probabilities are key to predicting future events and making adaptive choices. However, little is known about how humans learn such probabilities and where and how they are encoded in the brain, especially when they concern more than two outcomes. During functional magnetic resonance imaging (fMRI), young adults learned the probabilities of uncertain stimuli through repetitive sampling. Stimuli represented payoffs and participants had to predict their occurrence to maximize their earnings. Choices indicated loss and risk aversion but unbiased estimation of probabilities. BOLD response in medial prefrontal cortex and angular gyri increased linearly with the probability of the currently observed stimulus, untainted by its value. Connectivity analyses during rest and task revealed that these regions belonged to the default mode network. The activation of past outcomes in memory is evoked as a possible mechanism to explain the engagement of the default mode network in probability learning. A BOLD response relating to value was detected only at decision time, mainly in striatum. It is concluded that activity in inferior parietal and medial prefrontal cortex reflects the amount of evidence accumulated in favor of competing and uncertain outcomes.     

Decision making in recurrent neuronal circuits

Decision making has recently emerged as a central theme in neurophysiological studies of cognition, and experimental and computational work has led to the proposal of a cortical circuit mechanism of elemental decision computations. This mechanism depends on slow recurrent synaptic excitation balanced by fast feedback inhibition, which not only instantiates attractor states for forming categorical choices but also long transients for gradually accumulating evidence in favor of or against alternative options. Such a circuit endowed with reward-dependent synaptic plasticity is able to produce adaptive choice behavior. While decision threshold is a core concept for reaction time tasks, it can be dissociated from a general decision rule. Moreover, perceptual decisions and value-based economic choices are described within a unified framework in which probabilistic choices result from irregular neuronal activity as well as iterative interactions of a decision maker with an uncertain environment or other unpredictable decision makers in a social group.     

Initiation and regulation of water deficit-induced abscisic acid accumulation in maize leaves and roots: cellular volume and water relations

Water deficit-induced ABA accumulation in relation to cellular water relations was investigated in maize root and leaf tissues. While polyethylene glycol (PEG) treatment led to a significant increase of ABA content in both root and leaf tissues, ethylene glycol (EG), a permeable monomer of PEG, had no effect on ABA accumulation at similar or much lower osmotic potentials. A rapid and massive accumulation of ABA in leaf tissues occurred at a specific threshold of PEG 6000 concentration, about 20% (w/v), and closely coincided with the start of the tissue weight loss and the obvious decrease of cellular osmotic potential. Pretreatment with EG lowered the cell sap osmotic potential and also lowered the capability of both root and leaf tissues to accumulate ABA in response to further air-drying or PEG treatment. When samples were dehydrated and incubated under pressure, a method to maintain high water potential and pressure potential during dehydration, ABA accumulation was similar to those dehydrated and incubated under atmospheric pressure. Such results suggest that both the absolute water potential and pressure potential per se had no direct effects on the dehydration-induced ABA accumulation. The results have provided evidence that the initiation of ABA accumulation is related to the weight loss of tissues or changes in cellular volume rather than the cell water relation parameters, and the capability of ABA accumulation can be regulated by cellular osmotic potential.     

Probabilistic Decision Making with Spikes: From ISI Distributions to Behaviour via Information Gain

Computational theories of decision making in the brain usually assume that sensory ''evidence'' is accumulated supporting a number of hypotheses, and that the first accumulator to reach threshold triggers a decision in favour of its associated hypothesis. However, the evidence is often assumed to occur as a continuous process whose origins are somewhat abstract, with no direct link to the neural signals - action potentials or ''spikes'' - that must ultimately form the substrate for decision making in the brain. Here we introduce a new variant of the well-known multi-hypothesis sequential probability ratio test (MSPRT) for decision making whose evidence observations consist of the basic unit of neural signalling - the inter-spike interval (ISI) - and which is based on a new form of the likelihood function. We dub this mechanism s-MSPRT and show its precise form for a range of realistic ISI distributions with positive support. In this way we show that, at the level of spikes, the refractory period may actually facilitate shorter decision times, and that the mechanism is robust against poor choice of the hypothesized data distribution. We show that s-MSPRT performance is related to the Kullback-Leibler divergence (KLD) or information gain between ISI distributions, through which we are able to link neural signalling to psychophysical observation at the behavioural level. Thus, we find the mean information needed for a decision is constant, thereby offering an account of Hick''s law (relating decision time to the number of choices). Further, the mean decision time of s-MSPRT shows a power law dependence on the KLD offering an account of Piéron''s law (relating reaction time to stimulus intensity). These results show the foundations for a research programme in which spike train analysis can be made the basis for predictions about behavior in multi-alternative choice tasks.     

Integration of Sensory and Reward Information during Perceptual Decision-Making in Lateral Intraparietal Cortex (LIP) of the Macaque Monkey

Single neurons in cortical area LIP are known to carry information relevant to both sensory and value-based decisions that are reported by eye movements. It is not known, however, how sensory and value information are combined in LIP when individual decisions must be based on a combination of these variables. To investigate this issue, we conducted behavioral and electrophysiological experiments in rhesus monkeys during performance of a two-alternative, forced-choice discrimination of motion direction (sensory component). Monkeys reported each decision by making an eye movement to one of two visual targets associated with the two possible directions of motion. We introduced choice biases to the monkeys'' decision process (value component) by randomly interleaving balanced reward conditions (equal reward value for the two choices) with unbalanced conditions (one alternative worth twice as much as the other). The monkeys'' behavior, as well as that of most LIP neurons, reflected the influence of all relevant variables: the strength of the sensory information, the value of the target in the neuron''s response field, and the value of the target outside the response field. Overall, detailed analysis and computer simulation reveal that our data are consistent with a two-stage drift diffusion model proposed by Diederich and Bussmeyer [1] for the effect of payoffs in the context of sensory discrimination tasks. Initial processing of payoff information strongly influences the starting point for the accumulation of sensory evidence, while exerting little if any effect on the rate of accumulation of sensory evidence.     

Interactive effects of three core goal pursuit processes on brain control systems: goal maintenance, performance monitoring, and response inhibition

Goal attainment relies in part on one's ability to maintain a cognitive representation of the desired goal (goal maintenance), monitor the current state vis-à-vis the targeted end state and remain vigilant for lapses in progress (performance monitoring), and inhibit counter-goal behaviors (response inhibition). Because neurocognitive studies have typically examined these three processes in isolation from one another, little is known regarding if and how they interact during goal pursuit. However, these processes frequently co-occur during online, real-world goal pursuit. The present study employed a novel task to investigate how goal maintenance, performance monitoring, and response inhibition interact with one another. We identified functional activations distinct to each of the processes that correspond to results of prior investigations. In addition, we report interactive effects between response inhibition and goal maintenance in the dorsal anterior cingulate cortex and between performance monitoring and goal maintenance in the superior frontal gyrus and supramarginal gyrus. Implications for studying the neural systems of in situ goals include the need for both experimental designs that distinguish between process, but also more complex, realistic tasks to begin to map interactions among these neurocognitive processes and how they are altered by the presence or absence of one another.     

Appearance matters: neural correlates of food choice and packaging aesthetics

Neuro-imaging holds great potential for predicting choice behavior from brain responses. In this study we used both traditional mass-univariate and state-of-the-art multivariate pattern analysis to establish which brain regions respond to preferred packages and to what extent neural activation patterns can predict realistic low-involvement consumer choices. More specifically, this was assessed in the context of package-induced binary food choices. Mass-univariate analyses showed that several regions, among which the bilateral striatum, were more strongly activated in response to preferred food packages. Food choices could be predicted with an accuracy of up to 61.2% by activation patterns in brain regions previously found to be involved in healthy food choices (superior frontal gyrus) and visual processing (middle occipital gyrus). In conclusion, this study shows that mass-univariate analysis can detect small package-induced differences in product preference and that MVPA can successfully predict realistic low-involvement consumer choices from functional MRI data.     

Temporal integration of olfactory perceptual evidence in human orbitofrontal cortex

Given a noisy sensory world, the nervous system integrates perceptual evidence over time to optimize decision-making. Neurophysiological accumulation of sensory information is well-documented in the animal visual system, but how such mechanisms are instantiated in the human brain remains poorly understood. Here we combined psychophysical techniques, drift-diffusion modeling, and functional magnetic resonance imaging (fMRI) to establish that odor evidence integration in the human olfactory system enhances discrimination on a two-alternative forced-choice task. Model-based measures of fMRI brain activity highlighted a ramp-like increase in orbitofrontal cortex (OFC) that peaked at the time of decision, conforming to predictions derived from an integrator model. Combined behavioral and fMRI data further suggest that decision bounds are not fixed but collapse over time, facilitating choice behavior in the presence of low-quality evidence. These data highlight a key role for the orbitofrontal cortex in resolving sensory uncertainty and provide substantiation for accumulator models of human perceptual decision-making.     

Pupil-Linked Arousal Determines Variability in Perceptual Decision Making

Decision making between several alternatives is thought to involve the gradual accumulation of evidence in favor of each available choice. This process is profoundly variable even for nominally identical stimuli, yet the neuro-cognitive substrates that determine the magnitude of this variability are poorly understood. Here, we demonstrate that arousal state is a powerful determinant of variability in perceptual decision making. We measured pupil size, a highly sensitive index of arousal, while human subjects performed a motion-discrimination task, and decomposed task behavior into latent decision making parameters using an established computational model of the decision process. In direct contrast to previous theoretical accounts specifying a role for arousal in several discrete aspects of decision making, we found that pupil diameter was uniquely related to a model parameter representing variability in the rate of decision evidence accumulation: Periods of increased pupil size, reflecting heightened arousal, were characterized by greater variability in accumulation rate. Pupil diameter also correlated trial-by-trial with specific patterns of behavior that collectively are diagnostic of changing accumulation rate variability, and explained substantial individual differences in this computational quantity. These findings provide a uniquely clear account of how arousal state impacts decision making, and may point to a relationship between pupil-linked neuromodulation and behavioral variability. They also pave the way for future studies aimed at augmenting the precision with which people make decisions.     

Consistent Implementation of Decisions in the Brain

Despite the complexity and variability of decision processes, motor responses are generally stereotypical and independent of decision difficulty. How is this consistency achieved? Through an engineering analogy we consider how and why a system should be designed to realise not only flexible decision-making, but also consistent decision implementation. We specifically consider neurobiologically-plausible accumulator models of decision-making, in which decisions are made when a decision threshold is reached. To trade-off between the speed and accuracy of the decision in these models, one can either adjust the thresholds themselves or, equivalently, fix the thresholds and adjust baseline activation. Here we review how this equivalence can be implemented in such models. We then argue that manipulating baseline activation is preferable as it realises consistent decision implementation by ensuring consistency of motor inputs, summarise empirical evidence in support of this hypothesis, and suggest that it could be a general principle of decision making and implementation. Our goal is therefore to review how neurobiologically-plausible models of decision-making can manipulate speed-accuracy trade-offs using different mechanisms, to consider which of these mechanisms has more desirable decision-implementation properties, and then review the relevant neuroscientific data on which mechanism brains actually use.     

Comparative evaluation and its implications for mate choice

Experiments on decision making by humans show that the choices that we make can be very labile. The magnitude of our preferences, and even our rank ordering of options, can vary according to the number and type of alternatives available for comparison. This apparent irrationality has been argued to result from our use of decision heuristics that have evolved to enable us to choose quickly and efficiently between options differing in multiple attributes. Here, we argue that, because there is also selective pressure for animals to make mating decisions quickly, and because potential mates also differ in multiple attributes, similar decision heuristics might have evolved for mate choice. Following this reasoning, the attractiveness of a given mate will depend on the others with whom he or she is being compared, rather than being an absolute function of his or her underlying quality. We describe some of the ramifications of such comparative evaluation, and argue that it could offer new insights into some of the biggest outstanding problems in mate choice and sexual selection.