One-trial instrumental conditioning in rats

A new method for a single-trial operant conditioning task with food reward in rats is described. The non-contingent reward of the first random lever press resulted in a higher response rate in both 24 and 48-hour retention tests as compared to contingently rewarded individuals. The latency to press the lever was shorter in non-contingently rewarded subjects than in contingently rewarded rats. The response rate in the first 5-min interval of the 24-hour retention test was higher in non-contingently rewarded subjects. The neuroleptic haloperidol (500 micrograms/kg., i.p.) given immediately following the "event to be remembered" significantly depresses the response rate compared to that for saline-treated controls. The advantages of the new method are discussed with respect to its applicability for memory research in animals and especially in neurophysiology and psychopharmacology.     

Reward-related behavioral paradigms for addiction research in the mouse: performance of common inbred strains

The mouse has emerged as a uniquely valuable species for studying the molecular and genetic basis of complex behaviors and modeling neuropsychiatric disease states. While valid and reliable preclinical assays for reward-related behaviors are critical to understanding addiction-related processes, and various behavioral procedures have been developed and characterized in rats and primates, there have been relatively few studies using operant-based addiction-relevant behavioral paradigms in the mouse. Here we describe the performance of the C57BL/6J inbred mouse strain on three major reward-related paradigms, and replicate the same procedures in two other commonly used inbred strains (DBA/2J, BALB/cJ). We examined Pavlovian-instrumental transfer (PIT) by measuring the ability of an auditory cue associated with food reward to promote an instrumental (lever press) response. In a separate experiment, we assessed the acquisition and extinction of a simple stimulus-reward instrumental behavior on a touch screen based task. Reinstatement of this behavior was then examined following either continuous exposure to cues (conditioned reinforcers, CRs) associated with reward, brief reward and CR exposure, or brief reward exposure followed by continuous CR exposure. The third paradigm examined sensitivity of an instrumental (lever press) response to devaluation of food reward (a probe for outcome insensitive, habitual behavior) by repeated pairing with malaise. Results showed that C57BL/6J mice displayed robust PIT, as well as clear extinction and reinstatement, but were insensitive to reinforcer devaluation. DBA/2J mice showed good PIT and (rewarded) reinstatement, but were slow to extinguish and did not show reinforcer devaluation or significant CR-reinstatement. BALB/cJ mice also displayed good PIT, extinction and reinstatement, and retained instrumental responding following devaluation, but, unlike the other strains, demonstrated reduced Pavlovian approach behavior (food magazine head entries). Overall, these assays provide robust paradigms for future studies using the mouse to elucidate the neural, molecular and genetic factors underpinning reward-related behaviors relevant to addiction research.     

Rats smell: odour-mediated local enhancement, in a vertical movement two-action test

In two experiments, hungry rats, Rattus norvegicus, were present in one side of an operant chamber while a conspecific demonstrator in the adjacent compartment moved a single lever either up or down for a food reward. During a subsequent test session, in which these rats were allowed access to the lever for the first time, all responses were rewarded regardless of their direction. In experiment 1, rats that were prevented from observing the direction of lever movement by means of a screen showed a reliable demonstrator-consistent response bias, while rats that had observed the direction of lever movement and in addition had access to any odour cues deposited on the lever did not. In experiment 2, each rat observed another rat (the ‘viewed’ demonstrator) moving a lever either up or down. They were then transferred into the test compartment of a different operant chamber in which another rat (the ‘box’ demonstrator) had moved the lever in the same direction as the viewed demonstrator or in the opposite direction. These observer rats showed a reliable preference for their box demonstrator's direction, but responded in the opposite direction to their viewed demonstrator. Taken together, the results of these experiments suggest that directional responding by rats in a vertical movement two-action test is influenced by demonstrator-deposited odour cues in addition to visual experience of a demonstrator's behaviour. Furthermore, while odour-mediated local enhancement gave rise to demonstrator-consistent responding, visual observation of a conspecific appeared to have the reverse effect. Copyright 2002 The Association for the Study of Animal Behaviour. Published by Elsevier Science Ltd. All rights reserved.     

Quantifying Social Motivation in Mice Using Operant Conditioning

In this protocol, social motivation is measured in mice through a pair of operant conditioning paradigms. To conduct the experiments, two-chambered shuttle boxes were equipped with two operant levers (left and right) and a food receptacle in one chamber, which was then divided from the second chamber by an automated guillotine door covered by a wire grid. Different stimulus mice, rotated across testing days, served as a social stimulus behind the wire grid, and were only visible following the opening of the guillotine door. Test mice were trained to lever press in order to open the door and gain access to the stimulus partner for 15 sec. The number of lever presses required to obtain the social reward progressively increased on a fixed schedule of 3. Testing sessions ended after test mice stopped lever pressing for 5 consecutive minutes. The last reinforced ratio or breakpoint can be used as a quantitative measure of social motivation. For the second paradigm, test mice were trained to discriminate between left and right lever presses in order to obtain either a food reward or the social reward. Mice were rewarded for every 3 presses of each respective lever. The number of food and social rewards can be compared as a measurement of the value placed upon each reward. The ratio of each reward type can also be compared between mouse strains and the change in this ratio can be monitored within testing sessions to measure satiation with a given reward type. Both of these operant conditioning paradigms are highly useful for the quantification of social motivation in mouse models of autism and other disorders of social behavior.     

Prelimbic and Infralimbic Neurons Signal Distinct Aspects of Appetitive Instrumental Behavior

It is thought that discrete subregions of the medial prefrontal cortex (mPFC) regulate different aspects of appetitive behavior, however, physiological support for this hypothesis has been lacking. In the present study, we used multichannel single-unit recording to compare the response of neurons in the prelimbic (PL) and infralimbic (IL) subregions of the mPFC, in rats pressing a lever to obtain sucrose pellets on a variable interval schedule of reinforcement (VI-60). Approximately 25% of neurons in both structures exhibited prominent excitatory responses during rewarded, but not unrewarded, lever presses. The time courses of reward responses in PL and IL, however, were markedly different. Most PL neurons exhibited fast and transient responses at the delivery of sucrose pellets, whereas most IL neurons exhibited delayed and prolonged responses associated with the collection of earned sucrose pellets. We further examined the functional significance of reward responses in IL and PL with local pharmacological inactivation. IL inactivation significantly delayed the collection of earned sucrose pellets, whereas PL inactivation produced no discernible effects. These findings support the hypothesis that PL and IL signal distinct aspects of appetitive behavior, and suggest that IL signaling facilitates reward collection.     

Extinction of an appetitive operant response after administration of MSH

Hungry rats were trained to press a lever in order to obtain food on a fixed ratio (FR) or variable ratio (VR) of reinforcement. Rats trained on the FR schedule and injected with synthetic α-MSH had delayed extinction of the task as compared with control rats injected with diluent. The results show that MSH affects the behavior of rats in another type of behavioral situation involving an appetitive operant response.     

A general method for evaluating incubation of sucrose craving in rats

For someone on a food-restricted diet, food craving in response to food-paired cues may serve as a key behavioral transition point between abstinence and relapse to food taking. Food craving conceptualized in this way is akin to drug craving in response to drug-paired cues. A rich literature has been developed around understanding the behavioral and neurobiological determinants of drug craving; we and others have been focusing recently on translating techniques from basic addiction research to better understand addiction-like behaviors related to food. As done in previous studies of drug craving, we examine sucrose craving behavior by utilizing a rat model of relapse. In this model, rats self-administer either drug or food in sessions over several days. In a session, lever responding delivers the reward along with a tone+light stimulus. Craving behavior is then operationally defined as responding in a subsequent session where the reward is not available. Rats will reliably respond for the tone+light stimulus, likely due to its acquired conditioned reinforcing properties. This behavior is sometimes referred to as sucrose seeking or cue reactivity. In the present discussion we will use the term "sucrose craving" to subsume both of these constructs. In the past decade, we have focused on how the length of time following reward self-administration influences reward craving. Interestingly, rats increase responding for the reward-paired cue over the course of several weeks of a period of forced-abstinence. This "incubation of craving" is observed in rats that have self-administered either food or drugs of abuse. This time-dependent increase in craving we have identified in the animal model may have great potential relevance to human drug and food addiction behaviors. Here we present a protocol for assessing incubation of sucrose craving in rats. Variants of the procedure will be indicated where craving is assessed as responding for a discrete sucrose-paired cue following extinction of lever pressing within the sucrose self-administration context (Extinction without cues) or as responding for sucrose-paired cues in a general extinction context (Extinction with cues).     

Dopaminergic Modulation of Effort-Related Choice Behavior as Assessed by a Progressive Ratio Chow Feeding Choice Task: Pharmacological Studies and the Role of Individual Differences

Mesolimbic dopamine (DA) is involved in behavioral activation and effort-related processes. Rats with impaired DA transmission reallocate their instrumental behavior away from food-reinforced tasks with high response requirements, and instead select less effortful food-seeking behaviors. In the present study, the effects of several drug treatments were assessed using a progressive ratio (PROG)/chow feeding concurrent choice task. With this task, rats can lever press on a PROG schedule reinforced by a preferred high-carbohydrate food pellet, or alternatively approach and consume the less-preferred but concurrently available laboratory chow. Rats pass through each ratio level 15 times, after which the ratio requirement is incremented by one additional response. The DA D₂ antagonist haloperidol (0.025–0.1 mg/kg) reduced number of lever presses and highest ratio achieved but did not reduce chow intake. In contrast, the adenosine A_2A antagonist MSX-3 increased lever presses and highest ratio achieved, but decreased chow consumption. The cannabinoid CB1 inverse agonist and putative appetite suppressant AM251 decreased lever presses, highest ratio achieved, and chow intake; this effect was similar to that produced by pre-feeding. Furthermore, DA-related signal transduction activity (pDARPP-32(Thr34) expression) was greater in nucleus accumbens core of high responders (rats with high lever pressing output) compared to low responders. Thus, the effects of DA antagonism differed greatly from those produced by pre-feeding or reduced CB1 transmission, and it appears unlikely that haloperidol reduces PROG responding because of a general reduction in primary food motivation or the unconditioned reinforcing properties of food. Furthermore, accumbens core signal transduction activity is related to individual differences in work output.     

Das Erlernen eines Anwendungszeitraumes mit Hilfe eines Signals1

It was examined if animals can learn to perform bar-pressing only within a determined period of time by the aid of light and tone signals. 6 male BDE-rats (70 days old at the beginning of the experiments) were trained in a modified test chamber to press a bar only in 2h/day (experimental subjects 1 and 2: 13.00–15.00 h; 3 and 4: 15.00–17.00 h; 5 and 6: 24.00–2.00 h). During this time light and tone signals were given and only during this time bar-pressing was rewarded. At the end of the training period all 6 rats showed a change of behavior according to the task, that means: at least 90% of the whole bar pressing activity fell into the determined period of time. Under test conditions the maintenance of this behavior was less stable.     

Multisensory Control of Multimodal Behavior: Do the Legs Know What the Tongue Is Doing?

Understanding of adaptive behavior requires the precisely controlled presentation of multisensory stimuli combined with simultaneous measurement of multiple behavioral modalities. Hence, we developed a virtual reality apparatus that allows for simultaneous measurement of reward checking, a commonly used measure in associative learning paradigms, and navigational behavior, along with precisely controlled presentation of visual, auditory and reward stimuli. Rats performed a virtual spatial navigation task analogous to the Morris maze where only distal visual or auditory cues provided spatial information. Spatial navigation and reward checking maps showed experience-dependent learning and were in register for distal visual cues. However, they showed a dissociation, whereby distal auditory cues failed to support spatial navigation but did support spatially localized reward checking. These findings indicate that rats can navigate in virtual space with only distal visual cues, without significant vestibular or other sensory inputs. Furthermore, they reveal the simultaneous dissociation between two reward-driven behaviors.     

Promising high monetary rewards for future task performance increases intermediate task performance

In everyday life contexts and work settings, monetary rewards are often contingent on future performance. Based on research showing that the anticipation of rewards causes improved task performance through enhanced task preparation, the present study tested the hypothesis that the promise of monetary rewards for future performance would not only increase future performance, but also performance on an unrewarded intermediate task. Participants performed an auditory Simon task in which they responded to two consecutive tones. While participants could earn high vs. low monetary rewards for fast responses to every second tone, their responses to the first tone were not rewarded. Moreover, we compared performance under conditions in which reward information could prompt strategic performance adjustments (i.e., when reward information was presented for a relatively long duration) to conditions preventing strategic performance adjustments (i.e., when reward information was presented very briefly). Results showed that high (vs. low) rewards sped up both rewarded and intermediate, unrewarded responses, and the effect was independent of the duration of reward presentation. Moreover, long presentation led to a speed-accuracy trade-off for both rewarded and unrewarded tones, whereas short presentation sped up responses to rewarded and unrewarded tones without this trade-off. These results suggest that high rewards for future performance boost intermediate performance due to enhanced task preparation, and they do so regardless whether people respond to rewards in a strategic or non-strategic manner.     

Hippocampal Lesions Impair Rapid Learning of a Continuous Spatial Alternation Task

The hippocampus is essential for the formation of memories for events, but the specific features of hippocampal neural activity that support memory formation are not yet understood. The ideal experiment to explore this issue would be to monitor changes in hippocampal neural coding throughout the entire learning process, as subjects acquire and use new episodic memories to guide behavior. Unfortunately, it is not clear whether established hippocampally-dependent learning paradigms are suitable for this kind of experiment. The goal of this study was to determine whether learning of the W-track continuous alternation task depends on the hippocampal formation. We tested six rats with NMDA lesions of the hippocampal formation and four sham-operated controls. Compared to controls, rats with hippocampal lesions made a significantly higher proportion of errors and took significantly longer to reach learning criterion. The effect of hippocampal lesion was not due to a deficit in locomotion or motivation, because rats with hippocampal lesions ran well on a linear track for food reward. Rats with hippocampal lesions also exhibited a pattern of perseverative errors during early task experience suggestive of an inability to suppress behaviors learned during pretraining on a linear track. Our findings establish the W-track continuous alternation task as a hippocampally-dependent learning paradigm which may be useful for identifying changes in the neural representation of spatial sequences and reward contingencies as rats learn and apply new task rules.     

Functional polymorphism of the mu-opioid receptor gene (OPRM1) influences reinforcement learning in humans

Previous reports on the functional effects (i.e., gain or loss of function), and phenotypic outcomes (e.g., changes in addiction vulnerability and stress response) of a commonly occurring functional single nucleotide polymorphism (SNP) of the mu-opioid receptor (OPRM1 A118G) have been inconsistent. Here we examine the effect of this polymorphism on implicit reward learning. We used a probabilistic signal detection task to determine whether this polymorphism impacts response bias to monetary reward in 63 healthy adult subjects: 51 AA homozygotes and 12 G allele carriers. OPRM1 AA homozygotes exhibited typical responding to the rewarded response--that is, their bias to the rewarded stimulus increased over time. However, OPRM1 G allele carriers exhibited a decline in response to the rewarded stimulus compared to the AA homozygotes. These results extend previous reports on the heritability of performance on this task by implicating a specific polymorphism. Through comparison with other studies using this task, we suggest a possible mechanism by which the OPRM1 polymorphism may confer reduced response to natural reward through a dopamine-mediated decrease during positive reinforcement learning.     

Motivational state and reward content determine choice behavior under risk in mice

Risk is a ubiquitous feature of the environment for most organisms, who must often choose between a small and certain reward and a larger but less certain reward. To study choice behavior under risk in a genetically well characterized species, we trained mice (C57BL/6) on a discrete trial, concurrent-choice task in which they must choose between two levers. Pressing one lever (safe choice) is always followed by a small reward. Pressing the other lever (risky choice) is followed by a larger reward, but only on some of the trials. The overall payoff is the same on both levers. When mice were not food deprived, they were indifferent to risk, choosing both levers with equal probability regardless of the level of risk. In contrast, following food or water deprivation, mice earning 10% sucrose solution were risk-averse, though the addition of alcohol to the sucrose solution dose-dependently reduced risk aversion, even before the mice became intoxicated. Our results falsify the budget rule in optimal foraging theory often used to explain behavior under risk. Instead, they suggest that the overall demand or desired amount for a particular reward determines risk preference. Changes in motivational state or reward identity affect risk preference by changing demand. Any manipulation that increases the demand for a reward also increases risk aversion, by selectively increasing the frequency of safe choices without affecting frequency of risky choices.     

Selective interaction of drugs with a discriminable stimulus associated with narcotic actions

Rats were trained to bar press on either one of two levers depending on whether they received an injection of morphine (10 mg/kg) or saline. The rats responded on the morphine-correct lever when injected with another narcotic, fentanyl, but responded on the saline-correct lever when injected with a narcotic antagonist or another CNS active, but non-narcotic, drug (e.g., amphetamine, apomorphine). The narcotic antagonist, naloxone, prevented the occurrence of the narcotic discriminable stimulus, but the rats responded on the morphine-correct lever when injected with morphine plus any of a number of potent CNS active, but non-narcotic compounds. These results are discussed with reference to the specificity of this procedure for detecting drugs with narcotic agonist or antagonist properties.     

Preference for fixed vs self-selected durations of brain stimulation in rats administered d-amphetamine sulphate

Rats implanted with stimulating electrodes in the region of the lateral hypothalamus were allowed the opportunity to respond on (a) a fixed duration lever (b) a variable duration lever or (c) were given a choice between responding on either the lever that delivered the fixed duration of electrical brain stimulation or the lever that allowed the animal to control the duration of electrical brain stimulation. Amphetamine increased rates of responding in both the fixed and variable duration conditions but total duration of brain stimulation was comparable in the two conditions. When given a choice of levers, rats did not prefer either the fixed or variable duration lever. After treatment with d-amphetamine, however, a marked preference for the variable duration lever was observed, although total duration of brain stimulation received in the choice procedure was comparable to that received when either the fixed or variable levers were presented alone. These findings were discussed in terms of brain stimulation preference following amphetamine and its relationship to reward value in the rat.     

Reward improves long-term retention of a motor memory through induction of offline memory gains

In humans, training in which good performance is rewarded or bad performance punished results in transient behavioral improvements. The relative effects of reward and punishment on consolidation and long-term retention, critical behavioral stages for successful learning, are not known. Here, we investigated the effects of reward and punishment on these different stages of human motor skill learning. We studied healthy subjects who trained on a motor task under rewarded, punished, or neutral control conditions. Performance was tested before and immediately, 6 hr, 24 hr, and 30 days after training in the absence of reward or punishment. Performance improvements immediately after training were comparable in the three groups. At 6 hr, the rewarded group maintained performance gains, whereas the other two groups experienced significant forgetting. At 24 hr, the reward group showed significant offline (posttraining) improvements, whereas the other two groups did not. At 30 days, the rewarded group retained the gains identified at 24 hr, whereas the other two groups experienced significant forgetting. We conclude that training under rewarded conditions is more effective than training under punished or neutral conditions in eliciting lasting motor learning, an advantage driven by offline memory gains that persist over time.     

Effects of anticholinergic and cholinesterase blocking drugs on appetitive behavior under different deprivation conditions

The effects of scopolamine HBr (0.125?1.0 mg/kg) methscopolamine bromide (0.125?1.0 mg/kg), physostigmine sulphate (0.05?0.4 mg/kg), and neostigmine bromide (0.025?0.2 mg/kg) were studied under four different states of deprivation. The dependent measures were differentiated into two appetitive behaviors: lever pressing for food reward and for water reward. The dosages of both anticholinergic agents were effective in reducing appetitive behavior for food reward. The central acting anticholinergic (scopolamine) alone suppressed appetitive behavior for water reward. The effect of cholinesterase blockade was significantly effected by the locus of action with physostigmine suppressing both forms of appetitive behavior while the effect of peripheral cholinesterase blockade on appetitive behavior was specific to suppressing lever pressing for food reward. There were significant interactions of these agents with the deprivation conditions which were particularly evident in the effects of the less extensive acting drugs.     

The pentylenetetrazol model of anxiety detects withdrawal from diazepam in rats

This experiment tested whether benzodiazepine withdrawal could be detected in an animal model of anxiety. Rats were trained in operant chambers using food reward to press one lever after pentylenetetrazol (PTZ), 20 mg/kg, injection and the other lever after saline injection. Previously, the PTZ cue has been shown to be simulated by anxiogenic drugs and blocked by anxiolytic drugs. After rats reliably performed this discrimination, they were injected with diazepam, 20 mg/kg, from 1 to 4 times a day for six days. For one group of subjects, on the third, fourth and sixth days, they were also injected with 40 mg/kg of RO 15-1788, a benzodiazepine receptor antagonist, and tested for lever selection: 50–80% of the subjects selected the PTZ lever; these results are in contrast to those obtained prior to chronic diazepam treatment in which RO 15-1788 did not generalize to PTZ. A second group of subjects was also injected for six days with diazepam and then allowed to withdraw spontaneously for eight days: PTZ lever selection over this period varied from 20 to 60% of rats. These data indicate that animals trained to discriminate a PTZ cue: 1) generalize the benzodiazepine withdrawal state to the PTZ cue, and 2) discriminate the withdrawal state for long periods of time, agreeing with clinical observations of long-lasting anxiety signs during benzodiazepine withdrawal.