Choice between probability and value of alimentary reinforcement as means for revealing individual typological features of dog behavior

Individual typological features of behavior of dogs were investigated by the method of choice between the low-valuable food available constantly and food of high quality presented with low probability. Animals were subjected to instrumental conditioning with the same conditioned stimuli but different types of reinforcement. Depression of a white pedal was always reinforced with meat-bread-crumb mixture, depression of a black pedal was reinforced with two pieces of liver (with probabilities of 100, 40, 33, 20, or 0%). The choice of reinforcement depended on probability of valuable food and individual typological features of the nervous system of a dog. Decreasing the probability of the reinforcement value to 40-20% revealed differences in behavior of dogs. Dogs of the first group, presumably with the weak type of the nervous system, more frequently pressed the white pedal (always reinforced) than the black pedal thus "avoiding a situation of risk" to receive an empty cup. They displayed symptoms of neurosis: whimper, refusals of food or of the choice of reinforcement, and obtrusive movements. Dogs of the second group, presumably with the strong type of the nervous system, more frequently pressed the black pedal (more valuable food) for the low-probability reward until they obtained the valuable food. They did not show neurosis symptoms and were not afraid of "situation of risk". A decrease in probability of the valuable reinforcement increased a percentage of long-latency depressions of pedals. It can be probably suggested that this phenomenon was associated with increasing involvement of cognitive processes, when contributions of the assessments of probability and value of the reinforcement to decision making became approximately equal. Choice between the probability and value of alimentary reinforcement is a good method for revealing individual typological features of dogs.     

Sensitivity and bias in decision-making under risk: evaluating the perception of reward, its probability and value

Background

There are few clinical tools that assess decision-making under risk. Tests that characterize sensitivity and bias in decisions between prospects varying in magnitude and probability of gain may provide insights in conditions with anomalous reward-related behaviour.

Objective

We designed a simple test of how subjects integrate information about the magnitude and the probability of reward, which can determine discriminative thresholds and choice bias in decisions under risk.

Design/Methods

Twenty subjects were required to choose between two explicitly described prospects, one with higher probability but lower magnitude of reward than the other, with the difference in expected value between the two prospects varying from 3 to 23%.

Results

Subjects showed a mean threshold sensitivity of 43% difference in expected value. Regarding choice bias, there was a ‘risk premium’ of 38%, indicating a tendency to choose higher probability over higher reward. An analysis using prospect theory showed that this risk premium is the predicted outcome of hypothesized non-linearities in the subjective perception of reward value and probability.

Conclusions

This simple test provides a robust measure of discriminative value thresholds and biases in decisions under risk. Prospect theory can also make predictions about decisions when subjective perception of reward or probability is anomalous, as may occur in populations with dopaminergic or striatal dysfunction, such as Parkinson''s disease and schizophrenia.     

Significance of motivation balance for a choice of dog's behavior under conditions of environmental uncertainty

Two experimental models with a choice between two reinforcements were used for assessment of individual typological features of dogs. In the first model dogs were given the choice of homogeneous food reinforcements: between less valuable constantly delivered reinforcement and more valuable reinforcement but delivered with low probabilities. In the second model the dogs had the choice of heterogeneous reinforcements: between performing alimentary and defensive reactions. Under conditions of rise of uncertainty owing to a decrease in probability of getting the valuable food, two dogs continued to prefer the valuable reinforcement, while the third animal gradually shifted its behavior from the choice of a highly valuable but infrequent reward to a less valuable but easily achieved reinforcement. Under condition of choice between the valuable food reinforcement and avoidance of electrocutaneous stimulation, the first two dogs preferred food, whereas the third animal which had been previously oriented to the choice of the low-valuable constant reinforcement, steadily preferred the avoidance behavior. The data obtained are consistent with the hypothesis according to which the individual typological characteristics of animals's (human's) behavior substantially depend on two parameters: extent of environmental uncertainty and subjective features of reinforcement assessment.     

Dynamics of betting behavior under flat reward condition

One of the missions of the cognitive process of animals, including humans, is to make reasonable judgments and decisions in the presence of uncertainty. The balance between exploration and exploitation investigated in the reinforcement-learning paradigm is one of the key factors in this process. Recently, following the pioneering work in behavioral economics, growing attention has been directed to human behaviors exhibiting deviations from the simple maximization of external reward. Here we study the dynamics of betting behavior in a simple game, where the probability of reward and the magnitude of reward are designed to give a "zero" expected net reward ("flat reward condition"). No matter how the subject behaves, there is on average no change in one's resources, and therefore every possible sequence of action has the same value. Even in such a situation, the subjects are found to behave not in a random manner, but in ways showing characteristic tendencies, reflecting the dynamics of brain's reward system. Our results suggest that brain's reward system is characterized by a rich and complex dynamics only loosely coupled with external reward structure.     

What chimpanzees (Pan troglodytes) learn in a cooperative task

To examine the development of cooperation in a captive group of chimpanzees (Pan troglodytes), we designed an apparatus which required the simultaneous traction by two animals to get a reward. Two chimpanzees, an adult male and an infant female in a group of six, produced most of the successful responses (pulling on two handles simultaneously). Visual behavior was used to try to determine what chimpanzees learned about the cooperative task. Propositions were made to investigate what kind of learning could be attributed to chimpanzees and were confronted with results. Both subjects learned the link between the presence of fruits on the apparatus and the possibility of getting a fruit. They also learned the importance of the partner at the apparatus to make a successful response. Only the adult male learned to take into account the behavior of the partner at the apparatus before pulling a handle. From a methodological point of view, the glances made by the animals can constitute a useful behavioral indicator of what the subjects learned in a given social situation.     

Motivational state and reward content determine choice behavior under risk in mice

Risk is a ubiquitous feature of the environment for most organisms, who must often choose between a small and certain reward and a larger but less certain reward. To study choice behavior under risk in a genetically well characterized species, we trained mice (C57BL/6) on a discrete trial, concurrent-choice task in which they must choose between two levers. Pressing one lever (safe choice) is always followed by a small reward. Pressing the other lever (risky choice) is followed by a larger reward, but only on some of the trials. The overall payoff is the same on both levers. When mice were not food deprived, they were indifferent to risk, choosing both levers with equal probability regardless of the level of risk. In contrast, following food or water deprivation, mice earning 10% sucrose solution were risk-averse, though the addition of alcohol to the sucrose solution dose-dependently reduced risk aversion, even before the mice became intoxicated. Our results falsify the budget rule in optimal foraging theory often used to explain behavior under risk. Instead, they suggest that the overall demand or desired amount for a particular reward determines risk preference. Changes in motivational state or reward identity affect risk preference by changing demand. Any manipulation that increases the demand for a reward also increases risk aversion, by selectively increasing the frequency of safe choices without affecting frequency of risky choices.     

Contributions of the amygdala to reward expectancy and choice signals in human prefrontal cortex

The prefrontal cortex (PFC) receives substantial anatomical input from the amygdala, and these two structures have long been implicated in reward-related learning and decision making. Yet little is known about how these regions interact, especially in humans. We investigated the contribution of the amygdala to reward-related signals in PFC by scanning two rare subjects with focal bilateral amygdala lesions using fMRI. The subjects performed a reversal learning task in which they first had to learn which of two choices was the more rewarding, and then flexibly switch their choices when contingencies changed. Compared with healthy controls, both amygdala lesion subjects showed a profound change in ventromedial prefrontal cortex (vmPFC) activity associated with reward expectation and behavioral choice. These findings support a critical role for the human amygdala in establishing expected reward representations in PFC, which in turn may be used to guide behavioral choice.     

Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value

Risky decision-making is altered in humans and animals with damage to the orbitofrontal cortex. However, the cellular function of the intact orbitofrontal cortex in processing information relevant for risky decisions is unknown. We recorded responses of single orbitofrontal neurons while monkeys viewed visual cues representing the key decision parameters, reward risk and value. Risk was defined as the mathematical variance of binary symmetric probability distributions of reward magnitudes; value was defined as non-risky reward magnitude. Monkeys displayed graded behavioral preferences for risky outcomes, as they did for value. A population of orbitofrontal neurons showed a distinctive risk signal: their cues and reward responses covaried monotonically with the variance of the different reward distributions without monotonically coding reward value. Furthermore, a small but statistically significant fraction of risk responses also coded reward value. These risk signals may provide physiological correlates for the role of the orbitofrontal cortex in risk processing.     

The results of conducting a rapid assessment of the situation in relation to intravenous narcotic abuse in the city of Rostov-on-Don

The main aspects of the spread and intravenous use of drugs in the city, contributing to the transmission of HIV infection among users of intravenous drugs (UID) have been studied. Anonymous questionnaires were distributed among 61 UID; of these, 50 were males (82%) and 11 were females (18%), mainly at the age of 19-35 years. In addition, sociological study was carried out among 582 persons; 61% of them were under 18 years of age, 18% were aged 19-25 years, 13% were aged 26-35 years. The drug users regarded their individual risk of getting HIV infection as insignificant in 23% of cases, slight in 21% of cases, high in 15% of cases, and 15% believed they were at no risk. As shown by the results of the sociological study, 91% of the respondents believed that the problem of drug addiction was topical in the city. The analysis of data obtained from physicians treating drug addicts and from the Head Office of Public Health made it possible to calculate that the number of UID in the city was about 9,000-10,000 persons. Taking into account the results of the rapid evaluation of the situation, the development of the draft program of "diminishing the harm" of the intravenous use of drugs is now in progress.     

Electrophysiological study of the choice motivation in man

In 70 Ss EEG coherence, amplitude and frequency characteristics as well as formal-dynamic (temperamental) properties were compared with the two dimensions of motivation involved in the choice of profession--its subjective "value" and the probability of its achievement ("accessability"). It was shown that the persons orienting themselves to the subjective value of the future profession tend to possess those features of temperament that testify to the stronger, more mobile and more labile nervous system as distinct from the subjects orienting themselves to the higher level of accessability. It was also shown that the persons with a higher level of the subjective value had a much higher level of EEG cortical activation in both hemispheres of the brain. The subjects with higher magnitudes of "accessability" were characterized by a lower level of brain activation.     

Undermatching in humans to amount of reinforcement

Human subjects were exposed to a concurrent-chain procedure in which amount of reinforcement in the terminal links was varied. The experimental procedure was designed to resemble as closely as possible animal operant procedures: verbal instructions were eliminated, the key-press operant response was shaped, and a “consummatory” response was required to receive reward. In addition to varying amount of reward, three different pairs of initial-link values in the concurrent chain were studied. The human subjects showed undermatching to amount of reinforcement (as do animal subjects). Moreover, the degree of undermatching tended to increase as the values of the initial links increased, consistent with Fantino's delay reduction hypothesis (1977) that choice for a larger reward decreases as the length of the initial link increases.     

Modeling the AIDS epidemic in Mexico City

To study the future course of the AIDS epidemic in Mexico City, we use an open compartmental model to forecast new AIDS cases among homosexual and bisexual males and among heterosexual males and females. For each group three compartments are defined: uninfected persons, infected but asymptomatic persons, and persons diagnosed with AIDS. It is assumed that the AIDS epidemic will follow the propagation of infectious disease model, where spread of infection is proportional to the product of the number of healthy persons and the number of infected ones. The compartmental model is represented by a system of nonlinear differential equations describing the rate of change in the number of persons in each compartment. The impact of preventive measures is explored by decreasing the probability of HIV transmission, which is one of the model parameters representing behavioral patterns. By April 1989, 491 AIDS cases had been reported in Mexico City and classified as sexually related. Our model predicts that the AIDS incidence will continue to rise in Mexico City for the foreseeable future and will spread among the heterosexual population. Decreasing the transmission probability by 10% in all groups (through education programs) will result in a decrease of 18.1% in the number of accumulated cases over a 5-year period. A 20% decrease would prevent more than 31% of the cases. We conclude that mathematical models can be valuable in predicting the spread of the AIDS epidemic and the impact of behavioral change on its spread.     

Deep brain stimulation of nucleus accumbens region in alcoholism affects reward processing

The influence of bilateral deep brain stimulation (DBS) of the nucleus nucleus (NAcc) on the processing of reward in a gambling paradigm was investigated using H(2)[(15)O]-PET (positron emission tomography) in a 38-year-old man treated for severe alcohol addiction. Behavioral data analysis revealed a less risky, more careful choice behavior under active DBS compared to DBS switched off. PET showed win- and loss-related activations in the paracingulate cortex, temporal poles, precuneus and hippocampus under active DBS, brain areas that have been implicated in action monitoring and behavioral control. Except for the temporal pole these activations were not seen when DBS was deactivated. These findings suggest that DBS of the NAcc may act partially by improving behavioral control.     

Asymmetric and adaptive reward coding via normalized reinforcement learning

Learning is widely modeled in psychology, neuroscience, and computer science by prediction error-guided reinforcement learning (RL) algorithms. While standard RL assumes linear reward functions, reward-related neural activity is a saturating, nonlinear function of reward; however, the computational and behavioral implications of nonlinear RL are unknown. Here, we show that nonlinear RL incorporating the canonical divisive normalization computation introduces an intrinsic and tunable asymmetry in prediction error coding. At the behavioral level, this asymmetry explains empirical variability in risk preferences typically attributed to asymmetric learning rates. At the neural level, diversity in asymmetries provides a computational mechanism for recently proposed theories of distributional RL, allowing the brain to learn the full probability distribution of future rewards. This behavioral and computational flexibility argues for an incorporation of biologically valid value functions in computational models of learning and decision-making.     

Effects of Voluntary Alcohol Intake on Risk Preference and Behavioral Flexibility during Rat Adolescence

Alcohol use is common in adolescence, with a large portion of intake occurring during episodes of binging. This pattern of alcohol consumption coincides with a critical period for neurocognitive development and may impact decision-making and reward processing. Prior studies have demonstrated alterations in adult decision-making following adolescent usage, but it remains to be seen if these alterations exist in adolescence, or are latent until adulthood. Here, using a translational model of voluntary binge alcohol consumption in adolescents, we assess the impact of alcohol intake on risk preference and behavioral flexibility during adolescence. During adolescence (postnatal day 30–50), rats were given 1-hour access to either a 10% alcohol gelatin mixture (EtOH) or a calorie equivalent gelatin (Control) at the onset of the dark cycle. EtOH consuming rats were classified as either High or Low consumers based on intake levels. Adolescent rats underwent behavioral testing once a day, with one group performing a risk preference task, and a second group performing a reversal-learning task during the 20-day period of gelatin access. EtOH-High rats showed increases in risk preference compared to Control rats, but not EtOH-Low animals. However, adolescent rats did a poor job of matching their behavior to optimize outcomes, suggesting that adolescents may adopt a response bias. In addition, adolescent ethanol exposure did not affect the animals'' ability to flexibly adapt behavior to changing reward contingencies during reversal learning. These data support the view that adolescent alcohol consumption can have short-term detrimental effects on risk-taking when examined during adolescence, which does not seem to be attributable to an inability to flexibly encode reward contingencies on behavioral responses.     

Identifying the Basal Ganglia Network Model Markers for Medication-Induced Impulsivity in Parkinson's Disease Patients

Impulsivity, i.e. irresistibility in the execution of actions, may be prominent in Parkinson''s disease (PD) patients who are treated with dopamine precursors or dopamine receptor agonists. In this study, we combine clinical investigations with computational modeling to explore whether impulsivity in PD patients on medication may arise as a result of abnormalities in risk, reward and punishment learning. In order to empirically assess learning outcomes involving risk, reward and punishment, four subject groups were examined: healthy controls, ON medication PD patients with impulse control disorder (PD-ON ICD) or without ICD (PD-ON non-ICD), and OFF medication PD patients (PD-OFF). A neural network model of the Basal Ganglia (BG) that has the capacity to predict the dysfunction of both the dopaminergic (DA) and the serotonergic (5HT) neuromodulator systems was developed and used to facilitate the interpretation of experimental results. In the model, the BG action selection dynamics were mimicked using a utility function based decision making framework, with DA controlling reward prediction and 5HT controlling punishment and risk predictions. The striatal model included three pools of Medium Spiny Neurons (MSNs), with D1 receptor (R) alone, D2R alone and co-expressing D1R-D2R. Empirical studies showed that reward optimality was increased in PD-ON ICD patients while punishment optimality was increased in PD-OFF patients. Empirical studies also revealed that PD-ON ICD subjects had lower reaction times (RT) compared to that of the PD-ON non-ICD patients. Computational modeling suggested that PD-OFF patients have higher punishment sensitivity, while healthy controls showed comparatively higher risk sensitivity. A significant decrease in sensitivity to punishment and risk was crucial for explaining behavioral changes observed in PD-ON ICD patients. Our results highlight the power of computational modelling for identifying neuronal circuitry implicated in learning, and its impairment in PD. The results presented here not only show that computational modelling can be used as a valuable tool for understanding and interpreting clinical data, but they also show that computational modeling has the potential to become an invaluable tool to predict the onset of behavioral changes during disease progression.     

Effect of Replacing Sugar with Non-Caloric Sweeteners in Beverages on the Reward Value after Repeated Exposure

Background

The reward value of food is partly dependent on learned associations. It is not yet known whether replacing sugar with non-caloric sweeteners in food is affecting long-term acceptance.

Objective

To determine the effect of replacing sugar with non-caloric sweeteners in a nutrient-empty drink (soft drink) versus nutrient-rich drink (yoghurt drink) on reward value after repeated exposure.

Design

We used a randomized crossover design whereby forty subjects (15 men, 25 women) with a mean±SD age of 21±2 y and BMI of 21.5±1.7 kg/m² consumed a fixed portion of a non-caloric sweetened (NS) and sugar sweetened (SS) versions of either a soft drink or a yoghurt drink (counterbalanced) for breakfast which were distinguishable by means of colored labels. Each version of a drink was offered 10 times in semi-random order. Before and after conditioning the reward value of the drinks was assessed using behavioral tasks on wanting, liking, and expected satiety. In a subgroup (n=18) fMRI was performed to assess brain reward responses to the drinks.

Results

Outcomes of both the behavioral tasks and fMRI showed that conditioning did not affect the reward value of the NS and SS versions of the drinks significantly. Overall, subjects preferred the yoghurt drinks to the soft drinks and the ss drinks to the NS drinks. In addition, they expected the yoghurt drinks to be more satiating, they reduced hunger more, and delayed the first eating episode more. Conditioning did not influence these effects.

Conclusion

Our study showed that repeated consumption of a non-caloric sweetened beverage, instead of a sugar sweetened version, appears not to result in changes in the reward value. It cannot be ruled out that learned associations between sensory attributes and food satiating capacity which developed preceding the conditioning period, during lifetime, affected the reward value of the drinks.     

The defensive behavioral patterns of captive white-lipped and collared peccary (Mammalia,Tayassuidae): an approach for conservation of the species

Defensive behavioral patterns in response to human-induced rapid environmental change can affect animals’ fitness and may play a role in species conservation status. To test this hypothesis, we compared the risk assessment and defensive behavioral responses of captive white-lipped peccary (WLP; Tayassu pecari) and collared peccary (CP; Pecari tajacu), which retain different conservation status; WLP are considered vulnerable and CP of least concern. We used an adapted paradigm of the mouse defense test battery (MDTB) comprising four consecutive tests. Two of these tests simulated a novel environment, while the other two stimulated the expression of defensive behavioral patterns. Besides differences in risk assessment and defensive threat/attack behavioral patterns between species, we compared flight initiation distance, flight speed, and plasma glucocorticoid concentrations. When facing a novel environment and risk challenges from humans’ predator-like cues, the white-lipped peccary showed more exploratory and defensive threat/attack behavioral patterns, shorter flight initiation distances, and lower flight speeds, whereas the collared peccaries showed more cautious and retreat patterns, longer flight initiation distances, and higher flight speeds. There were also correlations between physiological and behavioral parameters. We confirmed our hypothesis that the collared peccary’s cautiousness may help to prevent a decrease in its population, while the white-lipped peccary’s exploratory and confrontational behavioral patterns in overhunted areas, together with other simultaneous factors as forest fragmentation, might contribute to placing this species in the vulnerable category.