Rapid processing of both reward probability and reward uncertainty in the human anterior cingulate cortex

Reward probability and uncertainty are two fundamental parameters of decision making. Whereas reward probability indicates the prospect of winning, reward uncertainty, measured as the variance of probability, indicates the degree of risk. Several lines of evidence have suggested that the anterior cingulate cortex (ACC) plays an important role in reward processing. What is lacking is a quantitative analysis of the encoding of reward probability and uncertainty in the human ACC. In this study, we addressed this issue by analyzing the feedback-related negativity (FRN), an event-related potential (ERP) component that reflects the ACC activity, in a simple gambling task in which reward probability and uncertainty were parametrically manipulated through predicting cues. Results showed that at the outcome evaluation phase, while both win and loss-related FRN amplitudes increased as the probability of win or loss decreased, only the win-related FRN was modulated by reward uncertainty. This study demonstrates the rapid encoding of reward probability and uncertainty in the human ACC and offers new insights into the functions of the ACC.     

Individual differences in the perception of probability

In recent studies of humans estimating non-stationary probabilities, estimates appear to be unbiased on average, across the full range of probability values to be estimated. This finding is surprising given that experiments measuring probability estimation in other contexts have often identified conservatism: individuals tend to overestimate low probability events and underestimate high probability events. In other contexts, repulsive biases have also been documented, with individuals producing judgments that tend toward extreme values instead. Using extensive data from a probability estimation task that produces unbiased performance on average, we find substantial biases at the individual level; we document the coexistence of both conservative and repulsive biases in the same experimental context. Individual biases persist despite extensive experience with the task, and are also correlated with other behavioral differences, such as individual variation in response speed and adjustment rates. We conclude that the rich computational demands of our task give rise to a variety of behavioral patterns, and that the apparent unbiasedness of the pooled data is an artifact of the aggregation of heterogeneous biases.     

Dynamic integration of reward and stimulus information in perceptual decision-making

In perceptual decision-making, ideal decision-makers should bias their choices toward alternatives associated with larger rewards, and the extent of the bias should decrease as stimulus sensitivity increases. When responses must be made at different times after stimulus onset, stimulus sensitivity grows with time from zero to a final asymptotic level. Are decision makers able to produce responses that are more biased if they are made soon after stimulus onset, but less biased if they are made after more evidence has been accumulated? If so, how close to optimal can they come in doing this, and how might their performance be achieved mechanistically? We report an experiment in which the payoff for each alternative is indicated before stimulus onset. Processing time is controlled by a “go” cue occurring at different times post stimulus onset, requiring a response within msec. Reward bias does start high when processing time is short and decreases as sensitivity increases, leveling off at a non-zero value. However, the degree of bias is sub-optimal for shorter processing times. We present a mechanistic account of participants'' performance within the framework of the leaky competing accumulator model [1], in which accumulators for each alternative accumulate noisy information subject to leakage and mutual inhibition. The leveling off of accuracy is attributed to mutual inhibition between the accumulators, allowing the accumulator that gathers the most evidence early in a trial to suppress the alternative. Three ways reward might affect decision making in this framework are considered. One of the three, in which reward affects the starting point of the evidence accumulation process, is consistent with the qualitative pattern of the observed reward bias effect, while the other two are not. Incorporating this assumption into the leaky competing accumulator model, we are able to provide close quantitative fits to individual participant data.     

Motivational state and reward content determine choice behavior under risk in mice

Risk is a ubiquitous feature of the environment for most organisms, who must often choose between a small and certain reward and a larger but less certain reward. To study choice behavior under risk in a genetically well characterized species, we trained mice (C57BL/6) on a discrete trial, concurrent-choice task in which they must choose between two levers. Pressing one lever (safe choice) is always followed by a small reward. Pressing the other lever (risky choice) is followed by a larger reward, but only on some of the trials. The overall payoff is the same on both levers. When mice were not food deprived, they were indifferent to risk, choosing both levers with equal probability regardless of the level of risk. In contrast, following food or water deprivation, mice earning 10% sucrose solution were risk-averse, though the addition of alcohol to the sucrose solution dose-dependently reduced risk aversion, even before the mice became intoxicated. Our results falsify the budget rule in optimal foraging theory often used to explain behavior under risk. Instead, they suggest that the overall demand or desired amount for a particular reward determines risk preference. Changes in motivational state or reward identity affect risk preference by changing demand. Any manipulation that increases the demand for a reward also increases risk aversion, by selectively increasing the frequency of safe choices without affecting frequency of risky choices.     

Inverse probability weighting estimation of the volume under the ROC surface in the presence of verification bias

In diagnostic medicine, the volume under the receiver operating characteristic (ROC) surface (VUS) is a commonly used index to quantify the ability of a continuous diagnostic test to discriminate between three disease states. In practice, verification of the true disease status may be performed only for a subset of subjects under study since the verification procedure is invasive, risky, or expensive. The selection for disease examination might depend on the results of the diagnostic test and other clinical characteristics of the patients, which in turn can cause bias in estimates of the VUS. This bias is referred to as verification bias. Existing verification bias correction in three‐way ROC analysis focuses on ordinal tests. We propose verification bias‐correction methods to construct ROC surface and estimate the VUS for a continuous diagnostic test, based on inverse probability weighting. By applying U‐statistics theory, we develop asymptotic properties for the estimator. A Jackknife estimator of variance is also derived. Extensive simulation studies are performed to evaluate the performance of the new estimators in terms of bias correction and variance. The proposed methods are used to assess the ability of a biomarker to accurately identify stages of Alzheimer's disease.     

Dissecting the links between reward and loss,decision-making,and self-reported affect using a computational approach

Links between affective states and risk-taking are often characterised using summary statistics from serial decision-making tasks. However, our understanding of these links, and the utility of decision-making as a marker of affect, needs to accommodate the fact that ongoing (e.g., within-task) experience of rewarding and punishing decision outcomes may alter future decisions and affective states. To date, the interplay between affect, ongoing reward and punisher experience, and decision-making has received little detailed investigation. Here, we examined the relationships between reward and loss experience, affect, and decision-making in humans using a novel judgement bias task analysed with a novel computational model. We demonstrated the influence of within-task favourability on decision-making, with more risk-averse/‘pessimistic’ decisions following more positive previous outcomes and a greater current average earning rate. Additionally, individuals reporting more negative affect tended to exhibit greater risk-seeking decision-making, and, based on our model, estimated time more poorly. We also found that individuals reported more positive affective valence during periods of the task when prediction errors and offered decision outcomes were more positive. Our results thus provide new evidence that (short-term) within-task rewarding and punishing experiences determine both future decision-making and subjectively experienced affective states.     

Under pressure: response urgency modulates striatal and insula activity during decision-making under risk

When deciding whether to bet in situations that involve potential monetary loss or gain (mixed gambles), a subjective sense of pressure can influence the evaluation of the expected utility associated with each choice option. Here, we explored how gambling decisions, their psychophysiological and neural counterparts are modulated by an induced sense of urgency to respond. Urgency influenced decision times and evoked heart rate responses, interacting with the expected value of each gamble. Using functional MRI, we observed that this interaction was associated with changes in the activity of the striatum, a critical region for both reward and choice selection, and within the insula, a region implicated as the substrate of affective feelings arising from interoceptive signals which influence motivational behavior. Our findings bridge current psychophysiological and neurobiological models of value representation and action-programming, identifying the striatum and insular cortex as the key substrates of decision-making under risk and urgency.     

Reputation management: in autism, generosity is its own reward

A recent study has found that autistic people donate the same to charity regardless of whether they are observed. This is not because they are oblivious to others, but because they are free of hypocrisy.     

A concept of welfare based on reward evaluating mechanisms in the brain: anticipatory behaviour as an indicator for the state of reward systems

In this review we attempt to link the efficiency by which animals behave (economy of animal behaviour) to a neuronal substrate and subjective states to arrive at a definition of animal welfare which broadens the scope of its study. Welfare is defined as the balance between positive (reward, satisfaction) and negative (stress) experiences or affective states. The state of this balance may range from positive (good welfare) to negative (poor welfare). These affective states are momentary or transient states which occur against the background of and are integrated with the state of this balancing system. As will be argued the efficiency in behaviour requires that, for instance, satisfaction is like a moving target: reward provides the necessary feedback to guide behaviour; it is a not steady-state which can be maintained for long. This balancing system is reflected in the brain by the concerted action of opioid and mesolimbic dopaminergic systems. The state of this system reflects the coping capacity of the animal and is determined by previous events. In other words, this integrative approach of behavioural biology and neurobiology aims at understanding how the coping capacity of animals may be affected and measured. We argue that this balancing system underlies the economy of behaviour. Furthermore we argue that among other techniques anticipation in Pavlovian conditioning is an easy and useful tool to assess the state of this balancing system: for estimating the state of an animal in terms of welfare we focus on the conditions when an animal is facing a challenge.     

Nutritional value of whole date waste and evaluating its application in ostrich diets

Although date waste products have been used as an alternative feed source in the diets of poultry for a long time, there is no quantitative information available regarding date waste used in ostrich diets. Therefore, two experiments were performed to evaluate the feeding value of whole date waste (WDW) as a feed ingredient in ostrich diets. In the first experiment, apparent metabolizable energy corrected to zero nitrogen balance (AMEn) of WDW was determined using 12 young ostriches (6 months old). The treatments included a reference diet and a test diet consisting of 60% of the reference diet and 40% of WDW. The AMEn of the WDW determined by total collection was 3 216 kcal/kg. In the second study, four groups of eight growing ostriches (seven month old), with almost similar BW (60.4 ± 1.6 kg), were individually housed in outdoor paddocks of ≈ 24 m² and were tested from 7 to 9 months of age. The groups were fed four isocaloric (2 420 kcal of AMEn/kg) and isonitrogenous (16.4% CP) diets containing 0, 10, 20, and 30% WDW. The results demonstrated that there were no significant differences among treatments in average daily feed intake, average daily gain, feed conversion ratio, and apparent total tract digestibility coefficients of DM, organic matter, energy, ether extract, ash, nitrogen-free extract, calcium, and phosphorus. In contrast, birds fed 0, 10, and 20% WDW diets had similar CP digestibility and this was significantly (P < 0.001) higher than that of birds on 30% WDW diet. The least crude fibre digestibility (P = 0.003) was also observed in birds fed 30% WDW diet. Blood RBC count, lymphocyte percentage, glucose concentration, and glutathione peroxidase activity increased linearly (P < 0.01), whereas heterophil percentage and heterophil-to-lymphocyte ratio decreased linearly (P = 0.002), in response to dietary inclusion of WDW. It can be concluded that WDW can be incorporated into the diets of ostrich chicks at levels of up to 30% without compromising growth performance. These results also suggest that WDW could be used as a feed ingredient for growing ostriches to improve stress-related variables and antioxidant status.     

Health risk perception in Canada II: Worldviews,attitudes and opinions

In this article, we present the results of a national survey of 1500 Canadians on their attitudes and opinions about health risks. Ratings of perceived risk, sources of information on health risks and responsibility for risk management were also investigated, with findings reported separately. A high degree of concern about health risks was associated with industrial pollution and chemical products (with the exception of medicines), with almost complete agreement that the land, air and water are more contaminated than ever. In addition, there was widespread belief that a risk‐free environment was an achievable goal, and an unwillingness to accept some health risks to improve the economy. Lifestyle factors such as diet, exercise, and tobacco smoking were perceived to be important modifiers of health risk. On the other hand, many respondents endorsed the idea that they had little control over the risks to their health.     

Individual typological differences in human behavior under conditions of reward choice assoicated with risk

A choice between probability (100, 75, 50, 25, 10, 0%) and valuable reward were investigated in adult. In behavioral experiments, adult subjects put in a situation of a choice between greater, but risky prize and smaller, but received always, made the decision according to situational factors and specific features of character such as propensity to risk and care. In a situation of choice of behavioral strategy, subjects could be divided in "inclined to risk" and "careful". Tipologycal differences between groups were the greatest under conditions of 25% probability of getting a valuable reward. Probability prognosis was more efficient in persons "tended to risk", than in "careful" subjects. According to psychological tests, "impulsive" people appeared to be more inclined to risk than to cautiousness. For the group of persons that tended to cautiousness under conditions of 25% probability of getting a valuable reward, synchronism of EEG rhythms in the alpha and beta bands was revealed.     

State estimation bias induced by optimization under uncertainty and error cost asymmetry is likely reflected in perception

It is well known from numerous studies that perception can be significantly affected by intended action in many everyday situations, indicating that perception and related decision-making is not a simple, one-way sequence, but a complex iterative cognitive process. However, the underlying functional mechanisms are yet unclear. Based on an optimality approach, a quantitative computational model of one such mechanism has been developed in this study. It is assumed in the model that significant uncertainty about task-related parameters of the environment results in parameter estimation errors and an optimal control system should minimize the cost of such errors in terms of the optimality criterion. It is demonstrated that, if the cost of a parameter estimation error is significantly asymmetrical with respect to error direction, the tendency to minimize error cost creates a systematic deviation of the optimal parameter estimate from its maximum likelihood value. Consequently, optimization of parameter estimate and optimization of control action cannot be performed separately from each other under parameter uncertainty combined with asymmetry of estimation error cost, thus making the certainty equivalence principle non-applicable under those conditions. A hypothesis that not only the action, but also perception itself is biased by the above deviation of parameter estimate is supported by ample experimental evidence. The results provide important insights into the cognitive mechanisms of interaction between sensory perception and planning an action under realistic conditions. Implications for understanding related functional mechanisms of optimal control in the CNS are discussed.     

Study of the preference of high probability of attaining the goal or its subjective value

The authors used testimonies of students on the dominant motives of joining a higher education institution as the behavioral model that prefers a high probability of the attainment of a goal or its subjective value. They compared the features of the motivation sphere related to the choice of profession with the psychophysiological data of 38 students of a sports higher education institution. The motivation of this choice was determined using a test questionnaire that permitted the authors to measure the subjective value (importance) and accessibility (probability of attainment) of the selected profession. The psychophysiological features were evaluated on two levels, i.e., formal dynamic (temperamental characteristics) and physiological (frequency and amplitude characteristics of EEG). The athlete students’ motivating factor of the choice of profession was shown to form under the influence of individual and typological features, which manifest themselves in psychomotor activity and total ergicity (working capacity), while accessibility experiences a negative influence of the temperamental characteristics in the intellectual sphere, as well as the total ergicity level. The background EEG was more activated in those who chose sports as their profession because of deeming it more valuable. In students who were attracted by the accessibility of this field, a lower cerebral activation level was observed, especially in the frontal zones of the left hemisphere.     

Patients, their doctors, nonsteroidal anti-inflammatory drugs and the perception of risk

This article is about risk. Risk is probably the most misunderstood component in determining therapeutic intervention; however, it is probably the most relevant issue to consider in the context of expected benefit. The rarity of quantitative risk–benefit assessment and the lack of comparative risk–benefit when alternative therapies exist for a given condition leads to inadequate decisions. Without some quantitation of the risks associated with specific therapies, doctors and patients cannot make optimal risk–benefit calculations. Patients may abandon effective therapies for which benefits may still outweigh risks, or opt for therapies with less well-publicized potential adverse events of even greater frequency or severity. When only small incremental benefits accrue to patients from the use of a given therapy, on the other hand, even very rare serious events may play a role in decision-making by patients, by their health care providers and by regulatory authorities.