Sex chromosome complement influences operant responding for a palatable food in mice

The procurement and consumption of palatable, calorie‐dense foods is influenced by the nutritional and hedonic value of foods. Although many factors can influence the control over behavior by foods rich in sugar and fat, emerging evidence indicates that biological sex may play a particularly crucial role in the types of foods individuals seek out, as well as the level of motivation individuals will exert to obtain those foods. However, a systematic investigation of food‐seeking and consumption that disentangles the effects of the major sex‐biasing factors, including sex chromosome complement and organizational and activational effects of sex hormones, has yet to be conducted. Using the four core genotypes mouse model system, we separated and quantified the effects of sex chromosome complement and gonadal sex on consumption of and motivation to obtain a highly palatable solution [sweetened condensed milk (SCM)]. Gonadectomized mice with an XY sex chromosome complement, compared with those with two X chromosomes, independent of gonadal sex, appeared to be more sensitive to the reward value of the SCM solution and were more motivated to expend effort to obtain it, as evidenced by their dramatically greater expended effort in an instrumental task with progressively larger response‐to‐reward ratios. Gonadal sex independently affected free consumption of the solution but not motivation to obtain it. These data indicate that gonadal and chromosomal sex effects independently influence reward‐related behaviors, contributing to sexually dimorphic patterns of behavior related to the pursuit and consumption of rewards.     

Insulin, leptin, and food reward: update 2008

The hormones insulin and leptin have been demonstrated to act in the central nervous system (CNS) as regulators of energy homeostasis at medial hypothalamic sites. In a previous review, we described new research demonstrating that, in addition to these direct homeostatic actions at the hypothalamus, CNS circuitry that subserves reward and motivation is also a direct and an indirect target for insulin and leptin action. Specifically, insulin and leptin can decrease food reward behaviors and modulate the function of neurotransmitter systems and neural circuitry that mediate food reward, i.e., midbrain dopamine and opioidergic pathways. Here we summarize new behavioral, systems, and cellular evidence in support of this hypothesis and in the context of research into the homeostatic roles of both hormones in the CNS. We discuss some current issues in the field that should provide additional insight into this hypothetical model. The understanding of neuroendocrine modulation of food reward, as well as food reward modulation by diet and obesity, may point to new directions for therapeutic approaches to overeating or eating disorders.     

Reward magnitude and timing in pigeons

We investigated the interaction of motivation and timing by manipulating the expected reward magnitude during a peak procedure. Four pigeons were tested with three different reward magnitudes, operationalized as duration of food access. Each stimulus predicted a different reward magnitude on a 5 s fixed-interval schedule. Trials with different reward magnitudes were randomly intermingled in a session. Most pigeons responded less often and started responding later on peak trials when a smaller reward was expected, but showed no differences in response termination or peak times. Reward magnitude was independently corroborated through unreinforced choice trials, when pigeons chose between the three stimuli presented simultaneously. These results contribute to a growing body of evidence that the expected reward magnitude influences the decision to start anticipatory responding in tasks where the reward becomes available after a fixed interval, but does not alter peak times, nor the decision to stop responding on peak trials.     

Ghrelin and food reward: the story of potential underlying substrates

The incidence of obesity is increasing at an alarming rate and this worldwide epidemic represents a significant decrease in life span and quality of life of a large part of the affected population. Therefore an understanding of mechanisms underlying food overconsumption and obesity development is urgent and essential to find potential treatments. Research investigating mechanisms underlying obesity and the control of food intake has recently experienced a major shift in focus, from the brain's hypothalamus to additional important neural circuits controlling emotion, cognition and motivated behavior. Among them, the mesolimbic system, and the changes in reward and motivated behavior for food, emerge as new promising treatment targets. Furthermore, there is also growing appreciation of the impact of peripheral hormones that signal nutrition status to the mesolimbic areas, and especially the only known circulating orexigenic hormone, ghrelin. This review article provides a synthesis of recent evidence concerning the impact of manipulation of ghrelin and its receptor on models of food reward/food motivation behavior and the mesolimbic circuitry. Particular attention is given to the potential neurocircuitry and neurotransmitter systems downstream of ghrelin's effects on food reward.     

The Roles of Dopamine and Hypocretin in Reward: A Electroencephalographic Study

The proper functioning of the mesolimbic reward system is largely dependent on the neurotransmitter dopamine. Recent evidence suggests that the hypocretin system has significant projections to this reward system. We examined the distinct effects of reduced dopamine or reduced hypocretin levels on reward activity in patients with Parkinson’s disease, dopamine deficient, as well as patients with narcolepsy-cataplexy, hypocretin depleted, and healthy controls. Participants performed a simple game-like task while high-density electroencephalography was recorded. Topography and timing of event-related potentials for both reward cue, and reward feedback was examined across the entire dataset. While response to reward cue was similar in all groups, two distinct time points were found to distinguish patients and controls for reward feedback. Around 160ms both patient groups had reduced ERP amplitude compared to controls. Later at 250ms, both patient groups also showed a clear event-related potential (ERP), which was absent in controls. The initial differences show that both patient groups show a similar, blunted response to reward delivery. The second potential corresponds to the classic feedback-related negativity (FRN) potential which relies on dopamine activity and reflects reward prediction-error signaling. In particular the mismatch between predicted reward and reward subsequently received was significantly higher in PD compared to NC, independent of reward magnitude and valence. The intermediate FRN response in NC highlights the contribution of hypocretin in reward processing, yet also shows that this is not as detrimental to the reward system as in Parkinson’s. Furthermore, the inability to generate accurate predictions in NC may explain why hypocretin deficiency mediates cataplexy triggered by both positive and negative emotions.     

How the value of the environment controls persistence in visual search

Classic foraging theory predicts that humans and animals aim to gain maximum reward per unit time. However, in standard instrumental conditioning tasks individuals adopt an apparently suboptimal strategy: they respond slowly when the expected value is low. This reward-related bias is often explained as reduced motivation in response to low rewards. Here we present evidence this behavior is associated with a complementary increased motivation to search the environment for alternatives. We trained monkeys to search for reward-related visual targets in environments with different values. We found that the reward-related bias scaled with environment value, was consistent with persistent searching after the target was already found, and was associated with increased exploratory gaze to objects in the environment. A novel computational model of foraging suggests that this search strategy could be adaptive in naturalistic settings where both environments and the objects within them provide partial information about hidden, uncertain rewards.     

Studying Food Reward and Motivation in Humans

A key challenge in studying reward processing in humans is to go beyond subjective self-report measures and quantify different aspects of reward such as hedonics, motivation, and goal value in more objective ways. This is particularly relevant for the understanding of overeating and obesity as well as their potential treatments. In this paper are described a set of measures of food-related motivation using handgrip force as a motivational measure. These methods can be used to examine changes in food related motivation with metabolic (satiety) and pharmacological manipulations and can be used to evaluate interventions targeted at overeating and obesity. However to understand food-related decision making in the complex food environment it is essential to be able to ascertain the reward goal values that guide the decisions and behavioral choices that people make. These values are hidden but it is possible to ascertain them more objectively using metrics such as the willingness to pay and a method for this is described. Both these sets of methods provide quantitative measures of motivation and goal value that can be compared within and between individuals.     

Is there a 24-hour rhythm in alcohol craving and does it vary by sleep/circadian timing?

ABSTRACT

Increasing evidence implicates sleep/circadian factors in alcohol use; however, the role of such factors in alcohol craving has received scant attention. Prior research suggests a 24-hour rhythm in related processes (e.g., reward motivation), but more research directly investigating a rhythm in craving is needed. Moreover, prior evidence is ambiguous whether such a rhythm in alcohol craving may vary by sleep/circadian timing. To examine these possibilities, 36 late adolescents (18–22 years of age; 61% female) with regular alcohol use but without a current alcohol use disorder were recruited to complete smartphone reports of alcohol craving intensity six times a day for two weeks. During these two weeks, participants wore wrist actigraphs and completed two in-lab assessments (on Thursday and Sunday) of dim light melatonin onset (DLMO). Average actigraphically derived midpoint of sleep on weekends and average DLMO were used as indicators of sleep and circadian timing, respectively. Multilevel cosinor analysis revealed a 24-hour rhythm in alcohol craving. Findings across the sleep and circadian timing variables converged to suggest that sleep/circadian timing moderated the 24-hour rhythm in alcohol craving. Specifically, people with later sleep/circadian timing had later timing of peak alcohol craving. These findings add to the growing evidence of potential circadian influences on reward-related phenomena and suggest that greater consideration of sleep and circadian influences on alcohol craving may be useful for understanding alcohol use patterns and advancing related interventions.     

Mixing pleasures: Review of the effects of drugs on sex behavior in humans and animal models

Drugs of abuse act on the brain circuits mediating motivation and reward associated with natural behaviors. There is ample evidence that drugs of abuse impact male and female sexual behavior. First, the current review discusses the effect of drugs of abuse on sexual motivation and performance in male and female humans. In particular, we discuss the effects of commonly abused drugs including psychostimulants, opiates, marijuana/THC, and alcohol. In general, drug use affects sexual motivation, arousal, and performance and is commonly associated with increased sexual risk behaviors. Second, studies on effects of systemic administration of drugs of abuse on sexual behavior in animals are reviewed. These studies analyze the effects on sexual performance and motivation but do not investigate the effects of drugs on risk-taking behavior, creating a disconnect between human and animal studies. For this reason, we discuss two studies that focus on the effects of alcohol and methamphetamine on inhibition of maladaptive sex-seeking behaviors in rodents. Third, this review discusses potential brain areas where drugs of abuse may be exerting their effect on sexual behavior with a focus on the mesolimbic system as the site of action. Finally, we discuss recent studies that have brought to light that sexual experience in turn can affect drug responsiveness, including a sensitized locomotor response to amphetamine in female and male rodents as well as enhanced drug reward in male rats.     

Conditioning and time representation in long short-term memory networks

Dopaminergic models based on the temporal-difference learning algorithm usually do not differentiate trace from delay conditioning. Instead, they use a fixed temporal representation of elapsed time since conditioned stimulus onset. Recently, a new model was proposed in which timing is learned within a long short-term memory (LSTM) artificial neural network representing the cerebral cortex (Rivest et al. in J Comput Neurosci 28(1):107–130, 2010). In this paper, that model’s ability to reproduce and explain relevant data, as well as its ability to make interesting new predictions, are evaluated. The model reveals a strikingly different temporal representation between trace and delay conditioning since trace conditioning requires working memory to remember the past conditioned stimulus while delay conditioning does not. On the other hand, the model predicts no important difference in DA responses between those two conditions when trained on one conditioning paradigm and tested on the other. The model predicts that in trace conditioning, animal timing starts with the conditioned stimulus offset as opposed to its onset. In classical conditioning, it predicts that if the conditioned stimulus does not disappear after the reward, the animal may expect a second reward. Finally, the last simulation reveals that the buildup of activity of some units in the networks can adapt to new delays by adjusting their rate of integration. Most importantly, the paper shows that it is possible, with the proposed architecture, to acquire discharge patterns similar to those observed in dopaminergic neurons and in the cerebral cortex on those tasks simply by minimizing a predictive cost function.     

Using Effort to Measure Reward Value of Faces in Children with Autism

According to one influential account, face processing atypicalities in autism reflect reduced reward value of faces, which results in limited attention to faces during development and a consequent failure to acquire face expertise. Surprisingly, however, there is a paucity of work directly investigating the reward value of faces for individuals with autism and the evidence for diminished face rewards in this population remains equivocal. In the current study, we measured how hard children with autism would work to view faces, using an effortful key-press sequence, and whether they were sensitive to the differential reward value of attractive and unattractive faces. Contrary to expectations, cognitively able children with autism did not differ from typically developing children of similar age and ability in their willingness to work to view faces. Moreover, the effort expended was strongly positively correlated with facial attractiveness ratings in both groups of children. There was also no evidence of atypical reward values for other, less social categories (cars and inverted faces) in the children with autism. These results speak against the possibility that face recognition difficulties in autism are explained by atypical reward value of faces.     

Uncertainty–guided learning with scaled prediction errors in the basal ganglia

To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We show that the new model has an advantage over conventional reinforcement learning models in a value tracking task, and approaches a theoretic limit of performance provided by the Kalman filter. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. In the proposed network, dopaminergic neurons encode reward prediction errors scaled by standard deviation of rewards. We show that such scaling may arise if the striatal neurons learn the standard deviation of rewards and modulate the activity of dopaminergic neurons. The model is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.     

Quantifying Social Motivation in Mice Using Operant Conditioning

In this protocol, social motivation is measured in mice through a pair of operant conditioning paradigms. To conduct the experiments, two-chambered shuttle boxes were equipped with two operant levers (left and right) and a food receptacle in one chamber, which was then divided from the second chamber by an automated guillotine door covered by a wire grid. Different stimulus mice, rotated across testing days, served as a social stimulus behind the wire grid, and were only visible following the opening of the guillotine door. Test mice were trained to lever press in order to open the door and gain access to the stimulus partner for 15 sec. The number of lever presses required to obtain the social reward progressively increased on a fixed schedule of 3. Testing sessions ended after test mice stopped lever pressing for 5 consecutive minutes. The last reinforced ratio or breakpoint can be used as a quantitative measure of social motivation. For the second paradigm, test mice were trained to discriminate between left and right lever presses in order to obtain either a food reward or the social reward. Mice were rewarded for every 3 presses of each respective lever. The number of food and social rewards can be compared as a measurement of the value placed upon each reward. The ratio of each reward type can also be compared between mouse strains and the change in this ratio can be monitored within testing sessions to measure satiation with a given reward type. Both of these operant conditioning paradigms are highly useful for the quantification of social motivation in mouse models of autism and other disorders of social behavior.     

Enhancement of performances in a learning task in suckler calves after weaning and relocation: Motivational versus cognitive control?: A pilot study

Weaning in suckler calves influences performance in a learning task. The aim of the present study was to investigate whether the improved performance after weaning, including relocation, is due to differences in motivation for the reward or in learning abilities. Forty Aubrac calves were used; half of them were weaned from their dams at around eight months, the other half were weaned one month later. After weaning, calves were housed in groups of four in a new setting. From the day after weaning of the last group of calves, the animals were subjected to two tests: (1) an arena test, (2) a T-maze test where one arm led to either a social or a food reward. The T-maze test consisted of three sessions: in Session 1, trials were conducted until the animal acquired the task (i.e. did not take the unrewarded arm on three consecutive trials); in Session 2, the motivation for the reward was assessed via the walking time of the animal to reach the reward; in Session 3, the place of reward was reversed and the animals were trained until they acquired the new task.

Calves weaned for one day explored more (P < 0.05) and had lower heart rates during the arena test (P < 0.05) compared to the ones weaned for one month. During the T-maze test, calves weaned for one month versus one day did not differ in their capacities to learn the initial route (Session 1) or in their motivation for either the social or food reward (Session 2). Calves weaned for one day learned significantly faster (P < 0.05) the reversed route (Session 3) than calves weaned for one month. Hence, the better performances at reversal in the T-maze by calves that have just been weaned cannot be accounted for by a higher motivation for the reward. A better cognitive control of their behaviour due to a lower stress state is suggested by our results.     

Repeated Cocaine Exposure Facilitates the Expression of Incentive Motivation and Induces Habitual Control in Rats

There is growing evidence that mere exposure to drugs can induce long-term alterations in the neural systems that mediate reward processing, motivation, and behavioral control, potentially causing the pathological pursuit of drugs that characterizes the addicted state. The incentive sensitization theory proposes that drug exposure potentiates the influence of reward-paired cues on behavior. It has also been suggested that drug exposure biases action selection towards the automatic execution of habits and away from more deliberate goal-directed control. The current study investigated whether rats given repeated exposure to peripherally administered cocaine would show alterations in incentive motivation (assayed using the Pavlovian-to-instrumental transfer (PIT) paradigm) or habit formation (assayed using sensitivity to reward devaluation). After instrumental and Pavlovian training for food pellet rewards, rats were given 6 daily injections of cocaine (15 mg/kg, IP) or saline, followed by a 10-d period of rest. Consistent with the incentive sensitization theory, cocaine-treated rats showed stronger cue-evoked lever pressing than saline-treated rats during the PIT test. The same rats were then trained on a new instrumental action with a new food pellet reward before undergoing a reward devaluation testing. Although saline-treated rats exhibited sensitivity to reward devaluation, indicative of goal-directed performance, cocaine-treated rats were insensitive to this treatment, suggesting a reliance on habitual processes. These findings, when taken together, indicate that repeated exposure to cocaine can cause broad alterations in behavioral control, spanning both motivational and action selection processes, and could therefore help explain aberrations of decision-making that underlie drug addiction.     

下丘脑食欲素在药物成瘾中的作用

下丘脑是调控自然奖赏的重要脑区,它能特异性地表达一种神经肽——食欲素(orexin),这种神经肽在药物奖赏中的作用受到广泛关注。在成瘾研究中,发现不同脑区中的食欲素神经元对奖赏和动机行为的调节作用是不相同的:围穹窿区(PFA)和背内侧下丘脑区(DMH)的食欲素神经元主要参与激活应激系统,而外侧下丘脑(LH)的食欲素神经元主要通过激活与奖赏学习相关的大脑环路参与奖赏行为的调控。提示食欲素系统可在延长戒断防止复吸发生中成为新的研究目标,食欲素受体可以作为治疗药物成瘾的一种新的治疗靶标。     

Tentative evidence for inequity aversion to unequal work-effort but not to unequal reward distribution in Goffin's cockatoos

One hallmark in the evolution of cooperation is the ability to evaluate one's own payoff for a task against that of another person. To trace its evolutionary history, there has recently been a surge in comparative studies across different species. In non-human animals, evidence of inequity aversion has so far been identified in several primate species, dogs, and rats. Research in birds revealed mixed findings so far: among corvids, crows and ravens did react sensitively to unequal payoffs and work-effort, while New Caledonian crows did not. Among psittacids, kea were studied so far: Yet, despite the fact that they live in large, hierarchically organized social groups that show complex interactions, they did not show a significant reaction to inequitable payoffs. Here we tested for the first time a Cacatua, the Goffin's cockatoo, using a standardized token exchange paradigm in which first the partner and then the subject could exchange a token for a food reward. Our results show that subjects did not react to unequal reward distributions. However, in comparison to the Equity Condition, the likelihood to exchange was lower in the condition in which the partner received the same reward as a gift (without having to work for it) whereas the subject had to perform a task involving substantial work-effort, suggesting that the Goffin's cockatoos do react aversively to work-effort inequity. In a follow-up experiment, subjects never received a reward but observed a conspecific receive a high-quality reward depending on condition. We found again no evidence for an aversion for the unequal reward distribution, but only that, independent of condition, subjects quickly lost their motivation to participate due to not receiving a reward. In summary, Goffins showed some sensitivity to increased unequal work-effort, but did not react to unequal reward distribution.     

Neural differentiation of expected reward and risk in human subcortical structures

In decision-making under uncertainty, economic studies emphasize the importance of risk in addition to expected reward. Studies in neuroscience focus on expected reward and learning rather than risk. We combined functional imaging with a simple gambling task to vary expected reward and risk simultaneously and in an uncorrelated manner. Drawing on financial decision theory, we modeled expected reward as mathematical expectation of reward, and risk as reward variance. Activations in dopaminoceptive structures correlated with both mathematical parameters. These activations differentiated spatially and temporally. Temporally, the activation related to expected reward was immediate, while the activation related to risk was delayed. Analyses confirmed that our paradigm minimized confounds from learning, motivation, and salience. These results suggest that the primary task of the dopaminergic system is to convey signals of upcoming stochastic rewards, such as expected reward and risk, beyond its role in learning, motivation, and salience.     

Conditioned place preference for cocaine and methylphenidate in female mice from lines selectively bred for high voluntary wheel-running behavior

Behavioral addictions can come in many forms, including overeating, gambling and overexercising. All addictions share a common mechanism involving activation of the natural reward circuit and reinforcement learning, but the extent to which motivation for natural and drug rewards share similar neurogenetic mechanisms remains unknown. A unique mouse genetic model in which four replicate lines of female mice were selectively bred (>76 generations) for high voluntary wheel running (High Runner or HR lines) alongside four non-selected control (C) lines were used to test the hypothesis that high motivation for exercise is associated with greater reward for cocaine (20 mg/kg) and methylphenidate (10 mg/kg) using the conditioned place preference (CPP) test. HR mice run ~three times as many revolutions/day as C mice, but the extent to which they have increased motivation for other rewards is unknown. Both HR and C mice displayed significant CPP for cocaine and methylphenidate, but with no statistical difference between linetypes for either drug. Taken together, results suggest that selective breeding for increased voluntary running has modified the reward circuit in the brain in a way that increases motivation for running without affecting cocaine or methylphenidate reward.