Pentagastrin modulation of the neuronal sensitivity of the lateral hypothalamus to noradrenaline and dopamine

Experimental study is dedicated to mechanisms of interaction of pentagastrin and monoamines (noradrenaline and dopamine) at the level of single neurones of the rabbits lateral hypothalamus under alimentary motivation and under saturation. It is shown that pentagastrin can modulate the effects of noradrenaline and dopamine on neuronal impulse activity in hungry and fed up animals, and the character of its action depends on the rabbits initial state. It is suggested that pentagastrin is a factor initiating alimentary motivational excitation, while noradrenaline maintains the latter at the definite level up to obtaining useful result by the animal, when dopaminergic mechanisms participating in the process of reinforcement join the noradrenergic ones.     

Food reward and cocaine increase extracellular dopamine in the nucleus accumbens as measured by microdialysis

Dopamine was measured by microdialysis in the nucleus accumbens of freely moving rats while they experienced rewarding food, brain stimulation and drugs. Extracellular dopamine increased 37% when the animals pressed a lever for food reward. Electrical stimulation of a lateral hypothalamic feeding-reward (self-stimulation) site caused a similar increase in dopamine, with or without food. At the site in the nucleus accumbens where rats will administer amphetamine to themselves, injections of amphetamine or cocaine increased extracellular dopamine five-fold. Thus amphetamine and cocaine increase dopamine in a behavior reinforcement system which is normally activated by eating. Conversely, the release of dopamine by eating could be a factor in addiction to food.     

Optogenetic mimicry of the transient activation of dopamine neurons by natural reward is sufficient for operant reinforcement

Activation of dopamine receptors in forebrain regions, for minutes or longer, is known to be sufficient for positive reinforcement of stimuli and actions. However, the firing rate of dopamine neurons is increased for only about 200 milliseconds following natural reward events that are better than expected, a response which has been described as a "reward prediction error" (RPE). Although RPE drives reinforcement learning (RL) in computational models, it has not been possible to directly test whether the transient dopamine signal actually drives RL. Here we have performed optical stimulation of genetically targeted ventral tegmental area (VTA) dopamine neurons expressing Channelrhodopsin-2 (ChR2) in mice. We mimicked the transient activation of dopamine neurons that occurs in response to natural reward by applying a light pulse of 200 ms in VTA. When a single light pulse followed each self-initiated nose poke, it was sufficient in itself to cause operant reinforcement. Furthermore, when optical stimulation was delivered in separate sessions according to a predetermined pattern, it increased locomotion and contralateral rotations, behaviors that are known to result from activation of dopamine neurons. All three of the optically induced operant and locomotor behaviors were tightly correlated with the number of VTA dopamine neurons that expressed ChR2, providing additional evidence that the behavioral responses were caused by activation of dopamine neurons. These results provide strong evidence that the transient activation of dopamine neurons provides a functional reward signal that drives learning, in support of RL theories of dopamine function.     

The model of the reward choice basing on the theory of reinforcement learning

We developed the model of alimentary instrumental conditioned bar-pressing reflex for cats making a choice between either immediate small reinforcement ("impulsive behavior") or delayed more valuable reinforcement ("self-control behavior"). Our model is based on the reinforcement learning theory. We emulated dopamine contribution by discount coefficient of this theory (a subjective decrease in the value of a delayed reinforcement). The results of computer simulation showed that "cats" with large discount coefficient demonstrated "self-control behavior"; small discount coefficient was associated with "impulsive behavior". This data are in agreement with the experimental data indicating that the impulsive behavior is due to a decreased amount of dopamine in striatum.     

Learning to differentiate microintervals of time using verbal feedback

Reaction time (RT) and the number of correct estimations of time microintervals (10 and 180 ms) between two visual stimuli were recorded in healthy subjects. It has been shown that 10 ms interval is better estimated when the stimuli are presented in the right visual field, i.e. when they are addressed directly to the left hemisphere. At the same time the number of correct estimations of 180 ms interval is greater and their RT is less when the stimuli are addressed directly to the right hemisphere. This points to different hemispheric mechanisms of time microintervals estimation. Study of the influence of different forms of verbal reinforcement on this learning has shown that after positive reinforcement (the word "good") the number of correct estimations is on average by 10% greater than after negative reinforcement (the word "error"). This may be connected with such processes as isolation and identification of erroneous reaction.     

Positive Reinforcement Mediated by Midbrain Dopamine Neurons Requires D1 and D2 Receptor Activation in the Nucleus Accumbens

The neural basis of positive reinforcement is often studied in the laboratory using intracranial self-stimulation (ICSS), a simple behavioral model in which subjects perform an action in order to obtain exogenous stimulation of a specific brain area. Recently we showed that activation of ventral tegmental area (VTA) dopamine neurons supports ICSS behavior, consistent with proposed roles of this neural population in reinforcement learning. However, VTA dopamine neurons make connections with diverse brain regions, and the specific efferent target(s) that mediate the ability of dopamine neuron activation to support ICSS have not been definitively demonstrated. Here, we examine in transgenic rats whether dopamine neuron-specific ICSS relies on the connection between the VTA and the nucleus accumbens (NAc), a brain region also implicated in positive reinforcement. We find that optogenetic activation of dopaminergic terminals innervating the NAc is sufficient to drive ICSS, and that ICSS driven by optical activation of dopamine neuron somata in the VTA is significantly attenuated by intra-NAc injections of D1 or D2 receptor antagonists. These data demonstrate that the NAc is a critical efferent target sustaining dopamine neuron-specific ICSS, identify receptor subtypes through which dopamine acts to promote this behavior, and ultimately help to refine our understanding of the neural circuitry mediating positive reinforcement.     

Dopamine, affordance and active inference

The role of dopamine in behaviour and decision-making is often cast in terms of reinforcement learning and optimal decision theory. Here, we present an alternative view that frames the physiology of dopamine in terms of Bayes-optimal behaviour. In this account, dopamine controls the precision or salience of (external or internal) cues that engender action. In other words, dopamine balances bottom-up sensory information and top-down prior beliefs when making hierarchical inferences (predictions) about cues that have affordance. In this paper, we focus on the consequences of changing tonic levels of dopamine firing using simulations of cued sequential movements. Crucially, the predictions driving movements are based upon a hierarchical generative model that infers the context in which movements are made. This means that we can confuse agents by changing the context (order) in which cues are presented. These simulations provide a (Bayes-optimal) model of contextual uncertainty and set switching that can be quantified in terms of behavioural and electrophysiological responses. Furthermore, one can simulate dopaminergic lesions (by changing the precision of prediction errors) to produce pathological behaviours that are reminiscent of those seen in neurological disorders such as Parkinson's disease. We use these simulations to demonstrate how a single functional role for dopamine at the synaptic level can manifest in different ways at the behavioural level.     

Trafficking of dopamine transporters in psychostimulant actions

Brain dopamine (DA) plays a pivotal role in drug addiction. Since the plasma membrane DA transporter (DAT) is critical for terminating DA neurotransmission, it is important to understand how DATs are regulated and this regulation impacts drug addiction. The number of cell surface DATs is controlled by constitutive and regulated endocytic trafficking. Psychostimulants impact this trafficking. Amphetamines, DAT substrates, cause rapid up-regulation and slower down-regulation of DAT whereas cocaine, a DAT inhibitor, increases surface DATs. Recent reports have begun to elucidate the molecular mechanisms of these psychostimulant effects and link changes in DAT trafficking to psychostimulant-induced reward/reinforcement in animal models.     

Different roles of dopamine and serotonin in conditioned passive avoidance response of rats

Deamination of dopamine and serotonin by monoamine oxidase was studied in the prefrontal cortex, striatum, hippocampus and amygdaloid complex of the brain of rats during retrieval of conditioned passive avoidance response. Changes in the dopamine and serotonin metabolism were observed in different brain structures. A decrease in dopamine-deaminating activity of monoamine oxidase was found in the hippocampus, striatum and prefrontal cortex. At the same time, serotonin-deaminating activity of the enzyme was decreased in the striatum and increased in the amygdaloid complex, whereas it did not change in the prefrontal cortex and hippocampus. The observed changes in dopamine metabolism in the prefrontal cortex and hippocampus and serotonin metabolism in the amygdaloid complex indicate that dopamine and serotonin are involved in the regulation of two different processes mediating the memory trace retrieval. Dopamine is involved in neuronal mechanisms of information processes providing the strategy of behavior, whereas serotonin is related to emotional mechanisms of memory.     

Catecholamine reward pathways and schizophrenia: the mechanism of the antipsychotic effect and the site of the primary disturbance.

Two catecholamine-containing pathways, the locus ceruleus system and the dopamine neurons arising from the ventral mid-brain, may be involved in reward. Dopamine neurons function as a system for energizing the organism's responses and directing them toward significant environmental stimuli, but the functions of the locus ceruleus system remain obscure. It appears increasingly likely that neuroleptic drugs exert their anti-psychotic effects in acute schizophrenia by blocking dopamine receptors, although the time course of the effect suggests that the mechanism is more complex than a simple reversal of a neurohumoral imbalance. Evidence from postmortem studies suggests that, at least in the chronic state, dopamine turnover is not increased, but that there may be an increase in postsynaptic receptor density in some cases, including some patients who apparently had not received medication in the year before death. The evidence is consistent with Olds and Travis' conjecture that "counteraction of positive feedback processes subserving positive reinforcement mechanisms may be a key to control of certain psychotic episodes".     

Recent physiological and pathophysiological aspects of Parkinsonian movement disorders

Deficits in the neural control of limb movements constitute a major part of Parkinsonian symptoms and are linked to a decay of dopaminergic neurotransmission. In animal models, Parkinsonian-like hypokinesia is consistently reproduced with large nigrostriatal dopamine depletions, while tremor and rigidity are less readily obtained. Lesions leading to a less than 70% striatal dopamine depletion are largely compensated by an increased activity of dopamine terminals. With more important lesions, supersensitivity of striatal non-adenylate cyclase-linked dopamine receptors occurs. Electrophysiological studies in Parkinsonian patients demonstrate increased reaction times and a reduced build-up of movement-related muscular activity underlying hypokinesia and provide circumstantial evidence for a central origin of tremor and rigidity. Single cell activity in unlesioned, behaving monkeys shows an increasingly direct relationship to movements when following the neural connections from mid-brain dopamine cells via striatum, globus pallidus, thalamus to pyramidal tract neurons of motor cortex. These data corroborate experimentally the concept that Parkinsonian hypokinesia is due to a failure of basic behavioral activating mechanisms.     

A New Framework for Cortico-Striatal Plasticity: Behavioural Theory Meets In Vitro Data at the Reinforcement-Action Interface

Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s) coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem—action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and behaviour, our model shows how striatum acts as the action-reinforcement interface.     

The effects of 5-hydroxytryptophan and 5-hydroxytryptamine on dopamine synthesis and release in rat brain striatal synaptosomes.

The effects of 5-hydroxytryptophan (5-HTP) and serotonin (5-HT) on dopamine synthesis and release in rat brain striatal synaptosomes have been examined and compared to the effects of tyramine and dopamine. Serotonin inhibited dopamine synthesis from tyrosine, with 25% inhibition occurring at 3 μM-5-HT and 60% inhibition at 200 μM. Dopamine synthesis from DOPA was also inhibited by 5-HT, with 30% inhibition occurring at 200 μ. At 200 μM-5-HTP, dopamine synthesis from both tyrosine and DOPA was inhibited about 70%. When just the tyrosine hydroxylation step was measured in the intact synaptosome, 5-HT, 5-HTP, tyramine and dopamine all caused significant inhibition, but only dopamine inhibited soluble tyrosine hydroxylase [L-tyrosine 3-monooxygenase; L-tyrosine, tetrahydropteridine oxygen oxidoreductase (3-hydroxylating); EC 1.14.16.2] prepared from lysed synaptosomes. Particulate tyrosine hydroxylase was not inhibited by 10 μM-5-HT, but was about 20% inhibited by 200 μM-5-HT and 5-HTP. At 200 μM both 5-HT and 5-HTP stimulated endogenous dopamine release. These experiments suggest that exposure of dopaminergic neurons to 5-HT or 5-HTP leads to an inhibition of dopamine synthesis, mediated in part by an intraneuronal displacement of dopamine from vesicle storage sites, leading to an increase in dopamine-induced feedback inhibition of tyrosine hydroxylase, and in part by a direct inhibition of DOPA decarboxylation.     

Three dopamine pathways induce aversive odor memories with different stability

Animals acquire predictive values of sensory stimuli through reinforcement. In the brain of Drosophila melanogaster, activation of two types of dopamine neurons in the PAM and PPL1 clusters has been shown to induce aversive odor memory. Here, we identified the third cell type and characterized aversive memories induced by these dopamine neurons. These three dopamine pathways all project to the mushroom body but terminate in the spatially segregated subdomains. To understand the functional difference of these dopamine pathways in electric shock reinforcement, we blocked each one of them during memory acquisition. We found that all three pathways partially contribute to electric shock memory. Notably, the memories mediated by these neurons differed in temporal stability. Furthermore, combinatorial activation of two of these pathways revealed significant interaction of individual memory components rather than their simple summation. These results cast light on a cellular mechanism by which a noxious event induces different dopamine signals to a single brain structure to synthesize an aversive memory.     

Dopamine transporters are involved in the onset of hypoxia-induced dopamine efflux in striatum as revealed by in vivo microdialysis

Although many studies have revealed alterations in neurotransmission during ischaemia, few works have been devoted to the neurochemical effects of mild hypoxia, a situation encountered during life in altitude or in several pathologies. In that context, the present work was undertaken to determine the in vivo mechanisms underlying the striatal dopamine efflux induced by mild hypoxaemic hypoxia. For that purpose, the extracellular concentrations of dopamine and its metabolite 3,4-dihydroxyphenyl acetic acid were simultaneously measured using brain microdialysis during acute hypoxic exposure (10% O₂, 1 h) in awake rats. Hypoxia induced a +80% increase in dopamine. Application of the dopamine transporters inhibitor, nomifensine (10 μM), just before the hypoxia prevented the rise in dopamine during the early part of hypoxia; in contrast the application of nomifensine after the beginning of hypoxia, failed to alter the increase in dopamine. Application of the voltage-dependent Na⁺ channel blocker tetrodotoxin abolished the increase in dopamine, whether administered just before or after the beginning of hypoxia. These data show that the neurochemical mechanisms of the dopamine efflux may change over the course of the hypoxic exposure, dopamine transporters being involved only at the beginning of hypoxia.     

Lesions of the Fasciculus Retroflexus Alter Footshock-Induced cFos Expression in the Mesopontine Rostromedial Tegmental Area of Rats

Midbrain dopamine neurons are an essential part of the circuitry underlying motivation and reinforcement. They are activated by rewards or reward-predicting cues and inhibited by reward omission. The lateral habenula (lHb), an epithalamic structure that forms reciprocal connections with midbrain dopamine neurons, shows the opposite response being activated by reward omission or aversive stimuli and inhibited by reward-predicting cues. It has been hypothesized that habenular input to midbrain dopamine neurons is conveyed via a feedforward inhibitory pathway involving the GABAergic mesopontine rostromedial tegmental area. Here, we show that exposing rats to low-intensity footshock (four, 0.5 mA shocks over 20 min) induces cFos expression in the rostromedial tegmental area and that this effect is prevented by lesions of the fasciculus retroflexus, the principal output pathway of the habenula. cFos expression is also observed in the medial portion of the lateral habenula, an area that receives dense DA innervation via the fr and the paraventricular nucleus of the thalamus, a stress sensitive area that also receives dopaminergic input. High-intensity footshock (120, 0.8 mA shocks over 40 min) also elevates cFos expression in the rostromedial tegmental area, medial and lateral aspects of the lateral habenula and the paraventricular thalamus. In contrast to low-intensity footshock, increases in cFos expression within the rostromedial tegmental area are not altered by fr lesions suggesting a role for non-habenular inputs during exposure to highly aversive stimuli. These data confirm the involvement of the lateral habenula in modulating the activity of rostromedial tegmental area neurons in response to mild aversive stimuli and suggest that dopamine input may contribute to footshock- induced activation of cFos expression in the lateral habenula.     

Operant Sensation Seeking in the Mouse

Operant methods are powerful behavioral tools for the study of motivated behavior. These ''self-administration'' methods have been used extensively in drug addiction research due to their high construct validity. Operant studies provide researchers a tool for preclinical investigation of several aspects of the addiction process. For example, mechanisms of acute reinforcement (both drug and non-drug) can be tested using pharmacological or genetic tools to determine the ability of a molecular target to influence self-administration behavior^1-6. Additionally, drug or food seeking behaviors can be studied in the absence of the primary reinforcer, and the ability of pharmacological compounds to disrupt this process is a preclinical model for discovery of molecular targets and compounds that may be useful for the treatment of addiction^3,7-9. One problem with performing intravenous drug self-administration studies in the mouse is the technical difficulty of maintaining catheter patency. Attrition rates in these experiments are high and can reach 40% or higher^10-15. Another general problem with drug self-administration is discerning which pharmacologically-induced effects of the reinforcer produce specific behaviors. For example, measurement of the reinforcing and neurological effects of psychostimulants can be confounded by their psychomotor effects. Operant methods using food reinforcement can avoid these pitfalls, although their utility in studying drug addiction is limited by the fact that some manipulations that alter drug self-administration have a minimal impact on food self-administration. For example, mesolimbic dopamine lesion or knockout of the D1 dopamine receptor reduce cocaine self-administration without having a significant impact on food self-administration ^12,16.Sensory stimuli have been described for their ability to support operant responding as primary reinforcers (i.e. not conditioned reinforcers)^17-22. Auditory and visual stimuli are self-administered by several species^18,21,23, although surprisingly little is known about the neural mechanisms underlying this reinforcement. The operant sensation seeking (OSS) model is a robust model for obtaining sensory self-administration in the mouse, allowing the study of neural mechanisms important in sensory reinforcement²⁴. An additional advantage of OSS is the ability to screen mutant mice for differences in operant behavior that may be relevant to addiction. We have reported that dopamine D1 receptor knockout mice, previously shown to be deficient in psychostimulant self-administration, also fail to acquire OSS²⁴. This is a unique finding in that these mice are capable of learning an operant task when food is used as a reinforcer. While operant studies using food reinforcement can be useful in the study of general motivated behavior and the mechanisms underlying food reinforcement, as mentioned above, these studies are limited in their application to studying molecular mechanisms of drug addiction. Thus, there may be similar neural substrates mediating sensory and psychostimulant reinforcement that are distinct from food reinforcement, which would make OSS a particularly attractive model for the study of drug addiction processes. The degree of overlap between other molecular targets of OSS and drug reinforcers is unclear, but is a topic that we are currently pursuing. While some aspects of addiction such as resistance to extinction may be observed with OSS, we have found that escalation ²⁵ is not observed in this model²⁴. Interestingly, escalation of intake and some other aspects of addiction are observed with self-administration of sucrose²⁶. Thus, when non-drug operant procedures are desired to study addiction-related processes, food or sensory reinforcers can be chosen to best fit the particular question being asked.In conclusion, both food self-administration and OSS in the mouse have the advantage of not requiring an intravenous catheter, which allows a higher throughput means to study the effects of pharmacological or genetic manipulation of neural targets involved in motivation. While operant testing using food as a reinforcer is particularly useful in the study of the regulation of food intake, OSS is particularly apt for studying reinforcement mechanisms of sensory stimuli and may have broad applicability to novelty seeking and addiction.Download video file.^{(54M, mov)}