At What Stage of Neural Processing Does Cocaine Act to Boost Pursuit of Rewards?

Dopamine-containing neurons have been implicated in reward and decision making. One element of the supporting evidence is that cocaine, like other drugs that increase dopaminergic neurotransmission, powerfully potentiates reward seeking. We analyze this phenomenon from a novel perspective, introducing a new conceptual framework and new methodology for determining the stage(s) of neural processing at which drugs, lesions and physiological manipulations act to influence reward-seeking behavior. Cocaine strongly boosts the proclivity of rats to work for rewarding electrical brain stimulation. We show that the conventional conceptual framework and methods do not distinguish between three conflicting accounts of how the drug produces this effect: increased sensitivity of brain reward circuitry, increased gain, or decreased subjective reward costs. Sensitivity determines the stimulation strength required to produce a reward of a given intensity (a measure analogous to the KM of an enzyme) whereas gain determines the maximum intensity attainable (a measure analogous to the vmax of an enzyme-catalyzed reaction). To distinguish sensitivity changes from the other determinants, we measured and modeled reward seeking as a function of both stimulation strength and opportunity cost. The principal effect of cocaine was a two-fourfold increase in willingness to pay for the electrical reward, an effect consistent with increased gain or decreased subjective cost. This finding challenges the long-standing view that cocaine increases the sensitivity of brain reward circuitry. We discuss the implications of the results and the analytic approach for theories of how dopaminergic neurons and other diffuse modulatory brain systems contribute to reward pursuit, and we explore the implications of the conceptual framework for the study of natural rewards, drug reward, and mood.     

Positive Reinforcement Mediated by Midbrain Dopamine Neurons Requires D1 and D2 Receptor Activation in the Nucleus Accumbens

The neural basis of positive reinforcement is often studied in the laboratory using intracranial self-stimulation (ICSS), a simple behavioral model in which subjects perform an action in order to obtain exogenous stimulation of a specific brain area. Recently we showed that activation of ventral tegmental area (VTA) dopamine neurons supports ICSS behavior, consistent with proposed roles of this neural population in reinforcement learning. However, VTA dopamine neurons make connections with diverse brain regions, and the specific efferent target(s) that mediate the ability of dopamine neuron activation to support ICSS have not been definitively demonstrated. Here, we examine in transgenic rats whether dopamine neuron-specific ICSS relies on the connection between the VTA and the nucleus accumbens (NAc), a brain region also implicated in positive reinforcement. We find that optogenetic activation of dopaminergic terminals innervating the NAc is sufficient to drive ICSS, and that ICSS driven by optical activation of dopamine neuron somata in the VTA is significantly attenuated by intra-NAc injections of D1 or D2 receptor antagonists. These data demonstrate that the NAc is a critical efferent target sustaining dopamine neuron-specific ICSS, identify receptor subtypes through which dopamine acts to promote this behavior, and ultimately help to refine our understanding of the neural circuitry mediating positive reinforcement.     

Participation of brain stimulation reward substrates in memory: anatomical and biochemical evidence.

On the basis of the pioneering leads provided by James Olds, brain stimulation reward has been shown to be a) derived from specific anatomical locations, b) influenced by psychotropic drugs, and c) functionally related to feeding behavior and sexual activity. These results recommend the view that it is worthwhile to understand, not necessarily the curious intracranial self-stimulation behavior itself, but the physiological function of the substrate revealed by the intracranial self-stimulation (ICSS) technique. I have suggested that certain components of the brain stimulation reward system may function as a memory consolidation system. In view of the biological specificity of brain reinforcement pathways, I suggest the hypothesis that activity in the mesocortical dopaminergic brain stimulation reward pathways participates in the memory consolidation process. Consequent to activity in such anatomical systems, phosphorylation of band F in the frontal cortex is altered. Thus, intracranial self-stimulation pathways are considered to play a role in memory formation by providing a biochemical residual following learning.     

Effects of antianxiety and antipsychotic drugs on DRL responding for brain stimulation

Satiated rats could be trained to give stable rates of responding for rewarding stimulation of the lateral hypothalamus delivered on differential reinforcement of low rate (DRL) schedule requiring 2 to 8 sec interresponse intervals for reinforcement (DRL-2 to 8). The performance on a DRL-8 schedule was tested 30 min after the oral administration of benzodiazepines. Diazepam (5 and 10 mg/kg) and meprobamate (200 mg/kg) caused significant increases in response rates during the first 5 min of a session, but not thereafter. Bromazepam (1 and 5 mg/kg) also caused a significant increase in the rates during the first and second 5 min. On the other hand, chlorpromazine (20 mg/kg) caused no effect in the first 5 min but decrease in second and third 5 min. These results indicate that DRL schedules with a brain stimulation reward provided a useful tool for evaluation of antianxiety drugs. The advantage of the brain stimulation reward over food reward is that the possible effects of the drugs on hunger motivation need not be considered.     

下丘脑食欲素在药物成瘾中的作用

下丘脑是调控自然奖赏的重要脑区,它能特异性地表达一种神经肽——食欲素(orexin),这种神经肽在药物奖赏中的作用受到广泛关注。在成瘾研究中,发现不同脑区中的食欲素神经元对奖赏和动机行为的调节作用是不相同的:围穹窿区(PFA)和背内侧下丘脑区(DMH)的食欲素神经元主要参与激活应激系统,而外侧下丘脑(LH)的食欲素神经元主要通过激活与奖赏学习相关的大脑环路参与奖赏行为的调控。提示食欲素系统可在延长戒断防止复吸发生中成为新的研究目标,食欲素受体可以作为治疗药物成瘾的一种新的治疗靶标。     

Optogenetic mimicry of the transient activation of dopamine neurons by natural reward is sufficient for operant reinforcement

Activation of dopamine receptors in forebrain regions, for minutes or longer, is known to be sufficient for positive reinforcement of stimuli and actions. However, the firing rate of dopamine neurons is increased for only about 200 milliseconds following natural reward events that are better than expected, a response which has been described as a "reward prediction error" (RPE). Although RPE drives reinforcement learning (RL) in computational models, it has not been possible to directly test whether the transient dopamine signal actually drives RL. Here we have performed optical stimulation of genetically targeted ventral tegmental area (VTA) dopamine neurons expressing Channelrhodopsin-2 (ChR2) in mice. We mimicked the transient activation of dopamine neurons that occurs in response to natural reward by applying a light pulse of 200 ms in VTA. When a single light pulse followed each self-initiated nose poke, it was sufficient in itself to cause operant reinforcement. Furthermore, when optical stimulation was delivered in separate sessions according to a predetermined pattern, it increased locomotion and contralateral rotations, behaviors that are known to result from activation of dopamine neurons. All three of the optically induced operant and locomotor behaviors were tightly correlated with the number of VTA dopamine neurons that expressed ChR2, providing additional evidence that the behavioral responses were caused by activation of dopamine neurons. These results provide strong evidence that the transient activation of dopamine neurons provides a functional reward signal that drives learning, in support of RL theories of dopamine function.     

Food reward and cocaine increase extracellular dopamine in the nucleus accumbens as measured by microdialysis

Dopamine was measured by microdialysis in the nucleus accumbens of freely moving rats while they experienced rewarding food, brain stimulation and drugs. Extracellular dopamine increased 37% when the animals pressed a lever for food reward. Electrical stimulation of a lateral hypothalamic feeding-reward (self-stimulation) site caused a similar increase in dopamine, with or without food. At the site in the nucleus accumbens where rats will administer amphetamine to themselves, injections of amphetamine or cocaine increased extracellular dopamine five-fold. Thus amphetamine and cocaine increase dopamine in a behavior reinforcement system which is normally activated by eating. Conversely, the release of dopamine by eating could be a factor in addiction to food.     

A role for brain stress systems in addiction

Drug addiction is a chronically relapsing disorder characterized by compulsion to seek and take drugs and has been linked to dysregulation of brain regions that mediate reward and stress. Activation of brain stress systems is hypothesized to be key to the negative emotional state produced by dependence that drives drug seeking through negative reinforcement mechanisms. This review explores the role of brain stress systems (corticotropin-releasing factor, norepinephrine, orexin [hypocretin], vasopressin, dynorphin) and brain antistress systems (neuropeptide Y, nociceptin [orphanin FQ]) in drug dependence, with emphasis on the neuropharmacological function of extrahypothalamic systems in the extended amygdala. The brain stress and antistress systems may play a key role in the transition to and maintenance of drug dependence once initiated. Understanding the role of brain stress and antistress systems in addiction provides novel targets for treatment and prevention of addiction and insights into the organization and function of basic brain emotional circuitry.     

New perspectives on cocaine addiction: recent findings from animal research

Research with laboratory animals has provided several insights into the nature of cocaine abuse and addiction. First, the nature of drug addiction has been reevaluated and the emphasis has shifted from physical dependence to compulsive drug-taking behavior. Second, animal studies suggest that cocaine is at least as addictive as heroin and possibly even more addictive. Third, cocaine is potentially more dangerous than heroin as evidenced by the higher fatality rate seen in laboratory animals given unlimited access to these drugs. Fourth, the neural basis of cocaine reinforcement has been identified and involves an enhancement of dopaminergic neurotransmission in the ventral tegmental dopamine system. Other addictive drugs (e.g., opiates) may also derive at least part of their reinforcing impact by pharmacologically activating this reward system. Fifth, although the biological consequences of repeated cocaine self-administration on central nervous system functioning are poorly understood, preliminary findings suggest that intravenous cocaine self-administration may decrease neural functioning in this brain reward system. This has important clinical implications because diminished functioning of an important brain reward system may significantly contribute to relapse into cocaine addiction. These and other findings from experimentation with laboratory animals suggest new considerations for the etiology and treatment of drug addiction.     

Neural and psychological mechanisms underlying appetitive learning: links to drug addiction

The complexity of drug addiction mirrors the complexity of the psychological processes that motivate animals to work for any reinforcer, be it a natural reward or a drug. Here, we review the role of the nucleus accumbens, together with its dopaminergic and cortical innervation, in responding to reinforcement. One important contribution made by the nucleus accumbens is to the process through which neutral stimuli, once paired with a reinforcer such as a drug, have the capacity to motivate behaviour. This process may be one of several contributing to addiction, and it may be amenable to pharmacological intervention.     

Conditioned place preference for cocaine and methylphenidate in female mice from lines selectively bred for high voluntary wheel-running behavior

Behavioral addictions can come in many forms, including overeating, gambling and overexercising. All addictions share a common mechanism involving activation of the natural reward circuit and reinforcement learning, but the extent to which motivation for natural and drug rewards share similar neurogenetic mechanisms remains unknown. A unique mouse genetic model in which four replicate lines of female mice were selectively bred (>76 generations) for high voluntary wheel running (High Runner or HR lines) alongside four non-selected control (C) lines were used to test the hypothesis that high motivation for exercise is associated with greater reward for cocaine (20 mg/kg) and methylphenidate (10 mg/kg) using the conditioned place preference (CPP) test. HR mice run ~three times as many revolutions/day as C mice, but the extent to which they have increased motivation for other rewards is unknown. Both HR and C mice displayed significant CPP for cocaine and methylphenidate, but with no statistical difference between linetypes for either drug. Taken together, results suggest that selective breeding for increased voluntary running has modified the reward circuit in the brain in a way that increases motivation for running without affecting cocaine or methylphenidate reward.     

Dissociation of food and opiate preference by a genetic mutation in zebrafish

Both natural rewards and addictive substances have the ability to reinforce behaviors. It has been unclear whether identical neural pathways mediate the actions of both. In addition, little is known about these behaviors and the underlying neural mechanisms in a genetically tractable vertebrate, the zebrafish Danio rerio. Using a conditioned place preference paradigm, we demonstrate that wildtype zebrafish exhibit a robust preference for food as well as the opiate drug morphine that can be blocked by the opioid receptor antagonist naloxone. Moreover, we show that the too few mutant, which disrupts a conserved zinc finger-containing gene and exhibits a reduction of selective groups of dopaminergic and serotonergic neurons in the basal diencephalon, displays normal food preference but shows no preference for morphine. Pretreatment with dopamine receptor antagonists abolishes morphine preference in the wildtype. These studies demonstrate that zebrafish display measurable preference behavior for reward and show that the preference for natural reward and addictive drug is dissociable by a single-gene mutation that alters subregions of brain monoamine neurotransmitter systems. Future genetic analysis in zebrafish shall uncover further molecular and cellular mechanisms underlying the formation and function of neural circuitry that regulate opiate and food preference behavior.     

Monetary Reward Modulates Task-Irrelevant Perceptual Learning for Invisible Stimuli

Task Irrelevant Perceptual Learning (TIPL) shows that the brain’s discriminative capacity can improve also for invisible and unattended visual stimuli. It has been hypothesized that this form of “unconscious” neural plasticity is mediated by an endogenous reward mechanism triggered by the correct task performance. Although this result has challenged the mandatory role of attention in perceptual learning, no direct evidence exists of the hypothesized link between target recognition, reward and TIPL. Here, we manipulated the reward value associated with a target to demonstrate the involvement of reinforcement mechanisms in sensory plasticity for invisible inputs. Participants were trained in a central task associated with either high or low monetary incentives, provided only at the end of the experiment, while subliminal stimuli were presented peripherally. Our results showed that high incentive-value targets induced a greater degree of perceptual improvement for the subliminal stimuli, supporting the role of reinforcement mechanisms in TIPL.     

Beautiful faces have variable reward value: fMRI and behavioral evidence.

The brain circuitry processing rewarding and aversive stimuli is hypothesized to be at the core of motivated behavior. In this study, discrete categories of beautiful faces are shown to have differing reward values and to differentially activate reward circuitry in human subjects. In particular, young heterosexual males rate pictures of beautiful males and females as attractive, but exert effort via a keypress procedure only to view pictures of attractive females. Functional magnetic resonance imaging at 3 T shows that passive viewing of beautiful female faces activates reward circuitry, in particular the nucleus accumbens. An extended set of subcortical and paralimbic reward regions also appear to follow aspects of the keypress rather than the rating procedures, suggesting that reward circuitry function does not include aesthetic assessment.     

Review. Neurobiological mechanisms for opponent motivational processes in addiction

The conceptualization of drug addiction as a compulsive disorder with excessive drug intake and loss of control over intake requires motivational mechanisms. Opponent process as a motivational theory for the negative reinforcement of drug dependence has long required a neurobiological explanation. Key neurochemical elements involved in reward and stress within basal forebrain structures involving the ventral striatum and extended amygdala are hypothesized to be dysregulated in addiction to convey the opponent motivational processes that drive dependence. Specific neurochemical elements in these structures include not only decreases in reward neurotransmission such as dopamine and opioid peptides in the ventral striatum, but also recruitment of brain stress systems such as corticotropin-releasing factor (CRF), noradrenaline and dynorphin in the extended amygdala. Acute withdrawal from all major drugs of abuse produces increases in reward thresholds, anxiety-like responses and extracellular levels of CRF in the central nucleus of the amygdala. CRF receptor antagonists block excessive drug intake produced by dependence. A brain stress response system is hypothesized to be activated by acute excessive drug intake, to be sensitized during repeated withdrawal, to persist into protracted abstinence and to contribute to stress-induced relapse. The combination of loss of reward function and recruitment of brain stress systems provides a powerful neurochemical basis for the long hypothesized opponent motivational processes responsible for the negative reinforcement driving addiction.     

Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain

Avoidance learning poses a challenge for reinforcement-based theories of instrumental conditioning, because once an aversive outcome is successfully avoided an individual may no longer experience extrinsic reinforcement for their behavior. One possible account for this is to propose that avoiding an aversive outcome is in itself a reward, and thus avoidance behavior is positively reinforced on each trial when the aversive outcome is successfully avoided. In the present study we aimed to test this possibility by determining whether avoidance of an aversive outcome recruits the same neural circuitry as that elicited by a reward itself. We scanned 16 human participants with functional MRI while they performed an instrumental choice task, in which on each trial they chose from one of two actions in order to either win money or else avoid losing money. Neural activity in a region previously implicated in encoding stimulus reward value, the medial orbitofrontal cortex, was found to increase, not only following receipt of reward, but also following successful avoidance of an aversive outcome. This neural signal may itself act as an intrinsic reward, thereby serving to reinforce actions during instrumental avoidance.     

Brain endorphins: possible role in reward and memory formation.

A role for enkephalin in the mediation of behavioral reinforcement is supported by several lines of evidence: i) central injections of enkephalin serve as reinforcement for self-administration behavior, ii) electrical stimulation of many enkephalin-rich regions serves as reinforcement for self-stimulation behavior, which is blocked by moderate doses of naloxone, and iii) long-term retention of a passive avoidance response is facilitated by immediate post-learning injections of methionine-enkephalin and morphine.     

Progressive-ratio responding for palatable high-fat and high-sugar food in mice

Foods that are rich in fat and sugar significantly contribute to over-eating and escalating rates of obesity. The consumption of palatable foods can produce a rewarding effect that strengthens action-outcome associations and reinforces future behavior directed at obtaining these foods. Increasing evidence that the rewarding effects of energy-dense foods play a profound role in overeating and the development of obesity has heightened interest in studying the genes, molecules and neural circuitry that modulate food reward. The rewarding impact of different stimuli can be studied by measuring the willingness to work to obtain them, such as in operant conditioning tasks. Operant models of food reward measure acquired and voluntary behavioral responses that are directed at obtaining food. A commonly used measure of reward strength is an operant procedure known as the progressive ratio (PR) schedule of reinforcement. In the PR task, the subject is required to make an increasing number of operant responses for each successive reward. The pioneering study of Hodos (1961) demonstrated that the number of responses made to obtain the last reward, termed the breakpoint, serves as an index of reward strength. While operant procedures that measure changes in response rate alone cannot separate changes in reward strength from alterations in performance capacity, the breakpoint derived from the PR schedule is a well-validated measure of the rewarding effects of food. The PR task has been used extensively to assess the rewarding impact of drugs of abuse and food in rats (e.g., 6-8), but to a lesser extent in mice. The increased use of genetically engineered mice and diet-induced obese mouse models has heightened demands for behavioral measures of food reward in mice. In the present article we detail the materials and procedures used to train mice to respond (lever-press) for a high-fat and high-sugar food pellets on a PR schedule of reinforcement. We show that breakpoint response thresholds increase following acute food deprivation and decrease with peripheral administration of the anorectic hormone leptin and thereby validate the use of this food-operant paradigm in mice.     

Experimental genetic approaches to addiction

Drugs of abuse are able to elicit compulsive drug-seeking behaviors upon repeated administration, which ultimately leads to the phenomenon of addiction. Evidence indicates that the susceptibility to develop addiction is influenced by sources of reinforcement, variable neuroadaptive mechanisms, and neurochemical changes that together lead to altered homeostasis of the brain reward system. Addiction is hypothesized to be a cycle of progressive dysregulation of the brain reward system that results in the compulsive use and loss of control over drug taking and the initiation of behaviors associated with drug seeking. The view that addiction represents a pathological state of reward provides an approach to identifying the factors that contribute to vulnerability, addiction, and relapse in genetic animal models.