Multiple reward signals in the brain

The fundamental biological importance of rewards has created an increasing interest in the neuronal processing of reward information. The suggestion that the mechanisms underlying drug addiction might involve natural reward systems has also stimulated interest. This article focuses on recent neurophysiological studies in primates that have revealed that neurons in a limited number of brain structures carry specific signals about past and future rewards. This research provides the first step towards an understanding of how rewards influence behaviour before they are received and how the brain might use reward information to control learning and goal-directed behaviour.     

Angiopoietin-2 and rat brain capillary remodeling during adaptation and deadaptation to prolonged mild hypoxia.

Angiogenesis is a crucial component of rat brain adaptation to prolonged hypoxia, but it is not known whether this structural change is permanent or reversed on return to normoxia. Also, the intrinsic mechanisms controlling brain microvascular plasticity in response to oxygen availability remains unclear. Our results indicate that capillary density in the rat cerebral cortex increased by 60% after 3 wk of hypoxia and that it progressively decreased to prehypoxic values after 3 wk of normoxic recovery (deadaptation). Angiopoietin-2 (Ang2) expression in the capillary endothelium was induced between 6 h and 14 days of hypoxia but fell to control levels at 21 days of hypoxia. During deadaptation, Ang2 levels were elevated at 1-14 days but decreased to baseline at 21 days. In contrast, the constitutive expression of Ang1 and Tie2 was not affected during hypoxia or deadaptation. TUNEL-positive endothelial cells and caspase-3 activation were observed at 7 and 14 days of deadaptation. These data suggest that Ang2 might modulate both angiogenesis and vascular regression in the rat brain and that capillary regression occurring during deadaptation involves activation of apoptosis.     

Metabolic adaptation to prolonged exercise.

A study was undertaken to evaluate and to examine the role of substrate supply in 50 healthy subjects after long distance events, such as 10 km, 25 km, and marathon races. The metabolic, variables of carbohydrate metabolism were greatest in 10-km runners, with the highest increase in glucose, lactate, and pyruvate, while in marathon runners only moderate changes were observed. Marathon competitors gave the greatest decrease in insulin concentration whereas glucagon and cortisol showed a contrary tendency. As for lipid concentrations, the most remarkable point was that after the marathon competition the best runners had the highest increase in free fatty acids; the longer the race, the higher were the beta-hydroxybutyrate and acetoacetate levels after the competition. It is important to emphasize that the limiting factor up to 90 min duration is the competitor's ability to deplete the stores of glycogen. Beyond 90 min (or 25 km) the decrease in insulin, the rise in cortisol and the higher concentration of ketnne bodies found indicate a change in metabnlic response.     

Learning from sensory and reward prediction errors during motor adaptation

Voluntary motor commands produce two kinds of consequences. Initially, a sensory consequence is observed in terms of activity in our primary sensory organs (e.g., vision, proprioception). Subsequently, the brain evaluates the sensory feedback and produces a subjective measure of utility or usefulness of the motor commands (e.g., reward). As a result, comparisons between predicted and observed consequences of motor commands produce two forms of prediction error. How do these errors contribute to changes in motor commands? Here, we considered a reach adaptation protocol and found that when high quality sensory feedback was available, adaptation of motor commands was driven almost exclusively by sensory prediction errors. This form of learning had a distinct signature: as motor commands adapted, the subjects altered their predictions regarding sensory consequences of motor commands, and generalized this learning broadly to neighboring motor commands. In contrast, as the quality of the sensory feedback degraded, adaptation of motor commands became more dependent on reward prediction errors. Reward prediction errors produced comparable changes in the motor commands, but produced no change in the predicted sensory consequences of motor commands, and generalized only locally. Because we found that there was a within subject correlation between generalization patterns and sensory remapping, it is plausible that during adaptation an individual''s relative reliance on sensory vs. reward prediction errors could be inferred. We suggest that while motor commands change because of sensory and reward prediction errors, only sensory prediction errors produce a change in the neural system that predicts sensory consequences of motor commands.     

Fetal breathing adaptation to prolonged hypoxaemia in sheep

Prolonged (6 days) fetal hypoxaemia was produced by placing pregnant ewes in an environmental chamber. A constant flow of N2 into the chamber reduced the fraction of inspired oxygen (Fi02) to 0.139 +/- 0.001, simulating an altitude of 4270 m. This reduced maternal PaO2 by about 39 mmHg and PaCO2 by nearly 5 mmHg, which produced a hypocapnic (delta PaCO2 = -5 mmHg) hypoxaemia (delta PaO2 = -8 mmHg) in the fetus. An analysis of the first 4 h of breathing recorded each day (1800-2200 h; start of hypoxaemia: 1200 h) showed that the incidence (12 +/- 2.0 min/day) during the first day of hypoxaemia was significantly less (P less than 0.05) than that (24 +/- 3.1 min/h) during the same time of the control day. By the second day, breathing had returned to normal. Further analysis indicated that a normal incidence of breathing may have occurred as early as 14 h after starting hypoxaemia. These results suggest that fetal breathing movements adapt rather quickly to this degree of hypocapnic hypoxaemia.     

Role of brain dopamine in food reward and reinforcement

The ability of food to establish and maintain response habits and conditioned preferences depends largely on the function of brain dopamine systems. While dopaminergic transmission in the nucleus accumbens appears sufficient for some forms of reward, the role of dopamine in food reward does not appear to be restricted to this region. Dopamine plays an important role in both the ability to energize feeding and to reinforce food-seeking behaviour; the role in energizing feeding is secondary to the prerequisite role in reinforcement. Dopaminergic activation is triggered by the auditory and visual as well as the tactile, olfactory, and gustatory stimuli of foods. While dopamine plays a central role in the feeding and food-seeking of normal animals, some food rewarded learning can be seen in genetically engineered dopamine-deficient mice.     

Endogenous opioids and ventilatory adaptation to prolonged hypoxia in goats

To investigate whether endogenous opioid peptides mediate time-dependent changes in ventilatory control during prolonged hypoxia, we studied four adult goats at rest during 14 days at simulated high altitude in a hypobaric chamber (PB approximately 450 Torr). Arterial PCO2 fell during the first several hours of hypoxia, remained stable over the next 7 days, and then rose slightly (but without statistical significance) by day 14. Ventilatory responsiveness to CO2 increased during the first week of hypoxia. By day 14, while still greater than control, the ventilatory response to CO2 was less than that observed on day 7. Immunoactive beta-endorphin levels in plasma and CSF did not change during the 14-day period. Administration of naloxone on day 14 did not restore the ventilatory response to CO2 to the level observed during the first week of acclimatization. We conclude that in adult goats, time-dependent changes in ventilatory response to CO2 during acclimatization to prolonged hypoxia are not primarily attributable to alterations in endogenous opioid peptide activity.     

Cortisol alters reward processing in the human brain

Dysfunctional reward processing is known to play a central role for the development of psychiatric disorders. Glucocorticoids that are secreted in response to stress have been shown to attenuate reward sensitivity and thereby might promote the onset of psychopathology. However, the underlying neurobiological mechanisms mediating stress hormone effects on reward processing as well as potential sex differences remain elusive. In this neuroimaging study, we administered 30 mg cortisol or a placebo to 30 men and 30 women and subsequently tested them in the Monetary Incentive Delay Task. Cortisol attenuated anticipatory neural responses to a verbal and a monetary reward in the left pallidum and the right anterior parahippocampal gyrus. Furthermore, in men, activation in the amygdala, the precuneus, the anterior cingulate, and in hippocampal regions was reduced under cortisol, whereas in cortisol-treated women a signal increase was observed in these regions. Behavioral performance also indicated that reward learning in men is impaired under high cortisol concentrations, while it is augmented in women. These findings illustrate that the stress hormone cortisol substantially diminishes reward anticipation and provide first evidence that cortisol effects on the neural reward system are sensitive to sex differences, which might translate into different vulnerabilities for psychiatric disorders.     

Multi-neuronal refractory period adapts centrally generated behaviour to reward

Oscillating neuronal circuits, known as central pattern generators (CPGs), are responsible for generating rhythmic behaviours such as walking, breathing and chewing. The CPG model alone however does not account for the ability of animals to adapt their future behaviour to changes in the sensory environment that signal reward. Here, using multi-electrode array (MEA) recording in an established experimental model of centrally generated rhythmic behaviour we show that the feeding CPG of Lymnaea stagnalis is itself associated with another, and hitherto unidentified, oscillating neuronal population. This extra-CPG oscillator is characterised by high population-wide activity alternating with population-wide quiescence. During the quiescent periods the CPG is refractory to activation by food-associated stimuli. Furthermore, the duration of the refractory period predicts the timing of the next activation of the CPG, which may be minutes into the future. Rewarding food stimuli and dopamine accelerate the frequency of the extra-CPG oscillator and reduce the duration of its quiescent periods. These findings indicate that dopamine adapts future feeding behaviour to the availability of food by significantly reducing the refractory period of the brain's feeding circuitry.     

Aerobic exercise reduces neuronal responses in food reward brain regions

Acute exercise suppresses ad libitum energy intake, but little is known about the effects of exercise on food reward brain regions. After an overnight fast, 30 (17 men, 13 women), healthy, habitually active (age = 22.2 ± 0.7 yr, body mass index = 23.6 ± 0.4 kg/m(2), Vo(2peak) = 44.2 ± 1.5 ml·kg(-1)·min(-1)) individuals completed 60 min of exercise on a cycle ergometer or 60 min of rest (no-exercise) in a counterbalanced, crossover fashion. After each condition, blood oxygen level-dependent responses to high-energy food, low-energy food, and control visual cues, were measured by functional magnetic resonance imaging. Exercise, compared with no-exercise, significantly (P < 0.005) reduced the neuronal response to food (high and low food) cues vs. control cues in the insula (-0.37 ± 0.13 vs. +0.07 ± 0.18%), putamen (-0.39 ± 0.10 vs. -0.10 ± 0.09%), and rolandic operculum (-0.37 ± 0.17 vs. 0.17 ± 0.12%). Exercise alone significantly (P < 0.005) reduced the neuronal response to high food vs. control and low food vs. control cues in the inferior orbitofrontal cortex (-0.94 ± 0.33%), insula (-0.37 ± 0.13%), and putamen (-0.41 ± 0.10%). No-exercise alone significantly (P < 0.005) reduced the neuronal response to high vs. control and low vs. control cues in the middle (-0.47 ± 0.15%) and inferior occipital gyrus (-1.00 ± 0.23%). Exercise reduced neuronal responses in brain regions consistent with reduced pleasure of food, reduced incentive motivation to eat, and reduced anticipation and consumption of food. Reduced neuronal response in these food reward brain regions after exercise is in line with the paradigm that acute exercise suppresses subsequent energy intake.     

The significance of action potential bursting in the brain reward circuit

The brain reward circuit consists of specialized cortical and subcortical structural components that code for various cognitive aspects of goal-directed behavior. These components include the prefrontal cortex (PFC), amygdala (AMY), nucleus accumbens (Nac), subiculum (SUB) of the hippocampal formation, and the dopamine (DA) neurons in the ventral tegmental area (VTA). Both serial and parallel processing in the different components of the circuit code the various aspects of reward-related behavior. Individual neurons within each component have developed specialized intrinsic membrane properties that have led them to be typically defined as either single spiking or high frequency burst-firing neurons. However, a strict definition based on the output mode may not be appropriate. Under the right conditions, neurons can switch between bursting and single-spiking modes, therefore providing a conditional output state. The preferred mode of each individual neuron depends on a combination of different plastic neuronal properties such as, dendritic architecture, neuromodulation, intracellular calcium (Ca(++)) buffering, excitatory and inhibitory synaptic strength, and the spatial distribution and density of voltage and ligand-gated channels. It is likely that, in vivo, most neurons in the circuit, despite variations in intrinsic membrane properties, are conditional output neurons equipped with the versatility of switching between output modes under appropriate conditions. Bursting mode may be used to boost the gain of neural signaling of important or novel events by enhancing transmitter release and enhancing dendritic depolarization, thereby increasing synaptic potentiation. Conversely, single spiking mode may be used to dampen neuronal signaling and may be associated with habituation to unimportant events. Mode switching may provide flexibility to the circuit allowing different sets of neurons to conditionally code for the various aspects of reward-related memory and behavior.     

Hedging your bets by learning reward correlations in the human brain

Human subjects are proficient at tracking the mean and variance of rewards and updating these via prediction errors. Here, we addressed whether humans can also learn about higher-order relationships between distinct environmental outcomes, a defining ecological feature of contexts where multiple sources of rewards are available. By manipulating the degree to which distinct outcomes are correlated, we show that subjects implemented an explicit model-based strategy to learn the associated outcome correlations and were adept in using that information to dynamically adjust their choices in a task that required a minimization of outcome variance. Importantly, the experimentally generated outcome correlations were explicitly represented neuronally in right midinsula with a learning prediction error signal expressed in rostral anterior cingulate cortex. Thus, our data show that the human brain represents higher-order correlation structures between rewards, a core adaptive ability whose immediate benefit is optimized sampling.     

Effects of fat adaptation and carbohydrate restoration on prolonged endurance exercise.

We determined the effect of fat adaptation on metabolism and performance during 5 h of cycling in seven competitive athletes who consumed a standard carbohydrate (CHO) diet for 1 day and then either a high-CHO diet (11 g. kg(-1)x day(-1) CHO, 1 g x kg(-1) x day(-1) fat; HCHO) or an isoenergetic high-fat diet (2.6 g x kg(-1) x day(-1) CHO, 4.6 g x kg(-1) x day(-1) fat; fat-adapt) for 6 days. On day 8, subjects consumed a high-CHO diet and rested. On day 9, subjects consumed a preexercise meal and then cycled for 4 h at 65% peak O(2) uptake, followed by a 1-h time trial (TT). Compared with baseline, 6 days of fat-adapt reduced respiratory exchange ratio (RER) with cycling at 65% peak O(2) uptake [0.78 +/- 0.01 (SE) vs. 0.85 +/- 0.02; P < 0.05]. However, RER was restored by 1 day of high-CHO diet, preexercise meal, and CHO ingestion (0.88 +/- 0.01; P < 0.05). RER was higher after HCHO than fat-adapt (0.85 +/- 0.01, 0.89 +/- 0.01, and 0.93 +/- 0.01 for days 2, 8, and 9, respectively; P < 0.05). Fat oxidation during the 4-h ride was greater (171 +/- 32 vs. 119 +/- 38 g; P < 0.05) and CHO oxidation lower (597 +/- 41 vs. 719 +/- 46 g; P < 0.05) after fat-adapt. Power output was 11% higher during the TT after fat-adapt than after HCHO (312 +/- 15 vs. 279 +/- 20 W; P = 0.11). In conclusion, compared with a high-CHO diet, fat oxidation during exercise increased after fat-adapt and remained elevated above baseline even after 1 day of a high-CHO diet and increased CHO availability. However, this study failed to detect a significant benefit of fat adaptation to performance of a 1-h TT undertaken after 4 h of cycling.     

Toward an understanding of the brain substrates of reward in humans

A network of brain regions has been implicated in food-reward processing. now provide evidence that this network is differentially modulated by anticipation versus receipt of a food reward and suggest an additional effect of valence of the stimulus.     

Functional and metabolic adaptation of the heart to prolonged thyroid hormone treatment

In heart failure, thyroid hormone (TH) treatment improves cardiac performance. The long-term effects of TH on cardiac function and metabolism, however, are incompletely known. To investigate the effects of up to 28 days of TH treatment, male Wistar rats received 3,3',5-triiodo-l-thyronine (200 microg/kg sc per day) leading to a 2.5-fold rise in plasma fatty acid (FA) level and progressive cardiac hypertrophy (+47% after 28 days) (P < 0.001). Ejection fraction (echocardiography) was increased (+12%; P < 0.05) between 7 and 14 days and declined thereafter. Neither cardiac FA oxidation, glycolytic capacity (homogenates) per unit muscle mass, nor mRNA levels of proteins involved in FA and glucose uptake and metabolism (Northern blots and microarray) were altered. After 28 days of treatment, mRNA levels of uncoupling proteins (UCP) 2 and 3 and atrial natriuretic factor were increased (P < 0.05). This indicates that TH-induced hypertrophy is associated with an initial increase in cardiac performance, followed by a decline in cardiac function and increased expression of UCPs and atrial natriuretic factor, suggesting that detrimental effects eventually prevail.     

Molecular adaptations underlying susceptibility and resistance to social defeat in brain reward regions

While stressful life events are an important cause of psychopathology, most individuals exposed to adversity maintain normal psychological functioning. The molecular mechanisms underlying such resilience are poorly understood. Here, we demonstrate that an inbred population of mice subjected to social defeat can be separated into susceptible and unsusceptible subpopulations that differ along several behavioral and physiological domains. By a combination of molecular and electrophysiological techniques, we identify signature adaptations within the mesolimbic dopamine circuit that are uniquely associated with vulnerability or insusceptibility. We show that molecular recapitulations of three prototypical adaptations associated with the unsusceptible phenotype are each sufficient to promote resistant behavior. Our results validate a multidisciplinary approach to examine the neurobiological mechanisms of variations in stress resistance, and illustrate the importance of plasticity within the brain's reward circuits in actively maintaining an emotional homeostasis.     

Body temperature and seawater adaptation in farmed Atlantic salmon and rainbow trout during prolonged chilling

The core temperature of the rainbow trout Oncorhynchus mykiss (3·5 kg) dropped to 1·0° C during the first 6 h of chilling at 0·5° C, remained stable until 24 h, and dropped significantly to 0·7° C after 39 h. Blood plasma osmolality increased and muscle moisture content decreased gradually with increasing chilling time. After 39 h of chilling, the rainbow trout experienced 40 mosmol l^-1 higher blood plasma osmolality and 2·8% less muscle moisture content compared with initial values. In the Atlantic salmon Salmo salar (5·3 kg), core temperature dropped to 1·3° C and blood plasma osmolality increased significantly during the first 6 h of chilling at 0·5° C, but remained relatively stable throughout the rest of the experimental period. After 39 h of chilling, the salmon experienced 20 mosmol l^-1 higher blood plasma osmolality and 0·5% less muscle moisture content compared with initial values. In rainbow trout muscle moisture content was inversely related to blood plasma osmolality indicating reduced seawater adaptation with increasing hours of chilling. No such relationship was observed in the Atlantic salmon. Hence, changes in plasma osmolality and muscle moisture in the Atlantic salmon do not indicate osmoregulatory failure since the new levels, once established, were maintained throughout the chilling time.