Second-order schedules of drug injection.

Key-press responding of squirrel monkeys produced intravenous injections of cocaine under two simple types of schedule. Under a fixed ratio schedule, every 30th response produced an injection; steady responding at high rates of over one per second were maintained during each fixed-ratio component. Under a fixed-interval schedule, the first response occurring after a fixed time of 5 min produced an injection; there was a pause at the start of each interval and then progressively increasing responding until cocaine was injected at the end of the interval. Both squirrel monkeys and rhesus monkeys also were studied under second-order schedules of drug injection. Under one type of second-order schedule, studies only in squirrel monkeys, completion of each fixed-interval component produced only a 2 sec light; completion of the 10th fixed-interval component produced the brief light and an intravenous injection of cocaine. Under a second type of second-order schedule, each fixed-ratio component completed during a fixed time interval (5 or 60 min) produced only a 2-sec light; the first fixed-ratio component completed after the interval of time elapsed produced the brief light and an intravenous (squirrel monkeys) or intramuscular (rhesus monkeys) injection of cocaine. Under both types of second-order schedules, repeated sequences of responding were maintained during each session and characteristic fixed-interval or fixed-ratio patterns of responding were controlled by the brief visual stimuli.     

Obese OLETF rats exhibit increased operant performance for palatable sucrose solutions and differential sensitivity to D2 receptor antagonism

CCK-1-receptor-deficient Otsuka Long-Evans Tokushima fatty (OLETF) rats are hyperphagic and exhibit a greater preference for sucrose compared with lean controls [Long-Evans Tokushima Otsuka (LETO)]. To directly assess motivation to work for sucrose reward in this model of obesity and type 2 diabetes, we examined the operant performance of OLETF rats at nondiabetic and prediabetic stages (14 and 24 wk of age, respectively) on fixed-ratio (FR) and progressive-ratio (PR) schedules of reinforcement. To evaluate the involvement of dopamine systems, the effects of the D1 receptor antagonist SCH23390 (100 and 200 nmol/kg ip) and the D2 receptor antagonist raclopride (200 and 400 nmol/kg ip), were also tested on PR responding for sucrose. Compared with age-matched LETO rats, 14-wk-old OLETF rats emitted more licks on the "active" empty spout operant on the FR-10 schedule of reinforcement to obtain 0.01 M and 0.3 M sucrose and completed higher ratio requirements on the PR schedule to gain access to 0.3 M and 1.0 M sucrose. At 24 wk, this effect was limited to 1.0 M sucrose. Both antagonists were potent in reducing operant responding to 0.3 M sucrose in both strains at both ages, and there was no strain effect to SCH23390 at either age. OLETF rats, on the other hand, showed an increased sensitivity to the higher dose of raclopride, resulting in reduced responding to sucrose reinforcement at 24 wk. Taken together, these findings provide the first direct evidence for an increased motivation for sucrose reward in the OLETF rats and suggest altered D2 receptor regulation with the progression of obesity and prediabetes.     

Effects of ethanol on punished and nonpunished responding under conditions of equated reinforcement rates and similar response rates

Lever pressing by pairs of rats was maintained under random-ratio (first subject) and yoked-interval (second subject) schedules of food presentation. The inter-reinforcement intervals generated under the ratio schedule comprised the interval values for the second (yoked) subject. This arrangement yielded nearly equivalent rates of food presentation for each subject pair. For the first rat of each pair a random-ratio schedule of shock presentation was added to the ratio schedule of food presentation. This manipulation resulted in similar rates of punished (first rat) and nonpunished (second rat) responding within subject pairs. Ethanol administration (0.25–1.5 g/kg) generally resulted in dose-related decreases in both punished and nonpunished responding. In general, punishment-specific effects were not obtained. These results suggest that ethanol may not be as effective as chlordiazepoxide or pentobarbital in increasing punished responding even when the effects of baseline response and reinforcement rates are controlled.     

EMG Biofeedback: The Effects of CRF, FR, VR, FI, and VI Schedules of Reinforcement on the Acquisition and Extinction of Increases in Forearm Muscle Tension

Biofeedback was used to increase forearm-muscle tension. Feedback was delivered under continuous reinforcement (CRF), variable interval (VI), fixed interval (FI), variable ratio (VR), and fixed ratio (FR) schedules of reinforcement when college students increased their muscle tension (electromyograph, EMG) above a high threshold. There were three daily sessions of feedback, and Session 3 was immediately followed by a session without feedback (extinction). The CRF schedule resulted in the highest EMG, closely followed by the FR and VR schedules, and the lowest EMG scores were produced by the FI and VI schedules. Similarly, the CRF schedule resulted in the greatest amount of time-above-threshold and the VI and FI schedules produced the lowest time-above-threshold. The highest response rates were generated by the FR schedule, followed by the VR schedule. The CRF schedule produced relatively low response rates, comparable to the rates under the VI and FI schedules. Some of the data are consistent with the partial-reinforcement-extinction effect. The present data suggest that different schedules of feedback should be considered in muscle-strengthening contexts such as during the rehabilitation of muscles following brain damage or peripheral nervous-system injury.     

Response-cost punishment via token loss with pigeons

Two experiments were conducted to investigate punishment via response-contingent removal of conditioned token reinforcers (response cost) with pigeons. In Experiment 1, key pecking was maintained on a two-component multiple second-order schedule of token delivery, with light emitting diodes (LEDs) serving as token reinforcers. In both components, responding produced tokens according to a random-interval 20-s schedule and exchange periods according to a variable-ratio schedule. During exchange periods, each token was exchangeable for 2.5-s access to grain. In one component, responses were conjointly punished according to fixed-ratio schedules of token removal. Response rates in this punishment component decreased to low levels while response rates in the alternate (no-punishment) component were unaffected. Responding was eliminated when it produced neither tokens nor exchange periods (Extinction), but was maintained at moderate levels when it produced tokens in the signaled absence of food reinforcement, suggesting that tokens served as effective conditioned reinforcers. In Experiment 2, the effect of the response-cost punishment contingency was separated from changes in the density of food reinforcement. This was accomplished by yoking either the number of food deliveries per component (Yoked Food) or the temporal placement of all stimulus events (tokens, exchanges, food deliveries) (Yoked Complete), from the punishment to the no-punishment component. Response rates decreased in both components, but decreased more rapidly and were generally maintained at lower levels in the punishment component than in the yoked component. In showing that the response-cost contingency had a suppressive effect on responding in addition to that produced by reductions in reinforcement density, the present results suggest that response-cost punishment shares important features with other forms of punishment.     

Effects of thyrotropin-releasing hormone (TRH) and MK-771 on behavior of squirrel monkeys controlled by noxious stimuli

The effects of TRH (0.1-30 mg/kg) and an enzyme-resistant analogue, MK-771 (0.1-10 mg/kg), were characterized in squirrel monkeys on responding maintained in the presence of different visual stimuli by a multiple 3-min fixed-interval (FI), 30-response fixed-ratio (FR) schedule of stimulus-shock termination or by a multiple 5-min FI schedule of food or shock presentation. Under the termination schedule, the first response at the end of 3 min in the FI component or the completion of the 30-response requirement in the FR component terminated the visual stimulus in the presence of which shocks occurred (escape schedule). Under the schedule of food or shock presentation, the first response at the end of the 5-min FI produced food in the presence of red stimulus lights or shock in the presence of white lights. TRH and MK-771 produced large, dose-related increases in responding maintained under the FR stimulus-shock termination schedule whereas these peptides produced smaller increases or did not affect responding under the FI schedule. TRH and MK-771 also produced marked increases in responding maintained by shock presentation at doses that did not alter or decreased food-maintained responding in the same subject. Thus, performances maintained by noxious stimuli are uniquely sensitive to the rate-increasing effects of TRH and MK-771. These findings suggest that the behavioral effects of the neuropeptides, TRH and MK-771, can depend on the specific consequences of behavior and, as such, the effects of these substances are determined by many of the same variables that determine the effects of other behaviorally-active drugs.     

Extending mathematical principles of reinforcement into the domain of behavioral pharmacology

Mathematical principles of reinforcement (MPR) attempts to integrate the empirical laws of reinforcement schedules that have accumulated over the decades. MPR is based on three principles: incentives excite behavior; there are temporal constraints on responding; and coupling of responses to reinforcers strengthens behavior [Behav. Brain Sci. 17 (1994) 105]. In the present paper MPR is extended into the domain of behavioral pharmacology, specifically to model the effects of D-amphetamine on operant behavior. In Experiment 1a, a five-component multiple fixed-ratio schedule was designed to generate behavioral baselines that were subsequently used to assess drug effect. In Experiments 1b and 1c, the quality and quantity of reinforcer were manipulated. The data generated by the three experiments were consistent with MPR. In Experiment 2, MPR was used to model the effects of D-amphetamine on rats responding under the five-component multiple fixed-ratio schedule. According to the model, the rate-decreasing effects of D-amphetamine were due primarily to motor disruption and secondarily to increased impulsivity; at the highest dosages, D-amphetamine also may have decreased the incentive value of food.     

Interactions between the benzodiazepine receptor antagonist Ro 15-1788 (flumazepil) and the inverse agonist beta-CCE: behavioral studies with squirrel monkeys

The effects of Ro 15-1788 and ethyl-beta-carboline-3-carboxylate (beta-CCE) were studied alone and in combination on the behavioral performances of squirrel monkeys. Under one procedure, performances maintained by food were suppressed by electric shock presentation (punishment or "conflict" procedure). Under a second procedure, responding was maintained either by food or electric shock delivery under a 5-min fixed-interval schedule. Doses of beta-CCE between 0.1 and 3.0 mg/kg, i.m., produced graded decreases in punished responding which were reversed by pretreatment with Ro 15-1788 (1.0 - 10.0 mg/kg, i.m.). Low doses of beta-CCE (0.03 - 0.3 mg/kg, i.m.) increased responding of monkeys maintained by shock presentation, but did not affect food-maintained responding; higher doses of beta-CCE decreased responding under both schedules. These effects of beta-CCE are opposite those produced by the benzodiazepines under this procedure. Ro 15-1788 (1.0 mg/kg i.m.) antagonized the effects of beta-CCE, producing a shift to the right in the dose-response curves. These findings provide further support for the view that beta-CCE and Ro 15-1788 produce effects mediated by the same benzodiazepine receptor recognition site.     

High reinforcing efficacy of nicotine in non-human primates

Although tobacco appears highly addictive in humans, there has been persistent controversy about the ability of its psychoactive ingredient nicotine to induce self-administration behavior in laboratory animals, bringing into question nicotine's role in reinforcing tobacco smoking. Because of ethical difficulties in inducing nicotine dependence in na?ve human subjects, we explored reinforcing effects of nicotine in experimentally-naive non-human primates given access to nicotine for periods of time up to two years. Five squirrel monkeys with no experimental history were allowed to intravenously self-administer nicotine by pressing one of two levers. The number of presses on the active lever needed to obtain each injection was fixed (fixed-ratio schedule) or increased progressively with successive injections during the session (progressive-ratio schedule), allowing evaluation of both reinforcing and motivational effects of nicotine under conditions of increasing response cost. Over time, a progressive shift toward high rates of responding on the active lever, but not the inactive lever, developed. The monkeys' behavior was clearly directed toward nicotine self-administration, rather than presentation of environmental stimuli associated with nicotine injection. Both schedules of reinforcement revealed a high motivation to self-administer nicotine, with monkeys continuing to press the lever when up to 600 lever-presses were needed for each injection of nicotine. Thus, nicotine, by itself, in the absence of behavioral or drug-exposure history, is a robust and highly effective reinforcer of drug-taking behavior in a non-human primate model predictive of human behavior. This supports the use of nicotinic ligands for the treatment of smokers, and this novel preclinical model offers opportunities to test future medications for the treatment of nicotine dependence.     

Effects of chronic fluvoxamine on ethanol- and food-maintained behaviors

Acute treatment with fluvoxamine reduces responding for ethanol more than responding for food. However, pharmacotherapy for alcoholism is likely to require chronic treatment. These experiments were performed to assess the effects of chronic fluvoxamine on ethanol- and food-maintained behaviors. Effects of chronic fluvoxamine (10 and 17.8 mg/kg/dayx30 days) on ethanol- and food-maintained responding were compared to responding during saline treatment in four Sprague-Dawley rats responding for ethanol and food under a multiple fixed-ratio 5, fixed-ratio 5 schedule. In two subjects, chronic fluvoxamine reduced ethanol-maintained responding more than food-maintained responding; however this effect was transient. In another subject, treatment persistently decreased food-maintained responding relative to ethanol-maintained responding. Finally, in one subject, fluvoxamine nonspecifically disrupted responding for food and ethanol. Similar to results in humans, outbred Sprague-Dawley rats had differential responses to chronic fluvoxamine. The effect was transient in rats that responded favorably (greater reduction of ethanol relative to food responding), while response reductions persisted throughout treatment in rats that responded unfavorably (greater reduction of food relative to ethanol or nonspecific reductions).     

Interactions among unit-price, fixed-ratio value, and dosing regimen in determining effects of repeated cocaine administration

Previous research has shown tolerance to cocaine that was dependent on fixed-ratio (FR) parameter size in the context of a multiple FR schedule of food reinforcement. Completion of the FR requirement in these studies resulted in the same magnitude of reinforcement, regardless of ratio size. The cost-to-benefits ratio (unit-price) was therefore not equated across the different FR components. The current study examined the effects of repeated administration of cocaine to pigeons when unit-price under FR schedules was either the same or different. Additionally, the role of a chronic variable-dosing versus chronic fixed-dosing procedure was examined when unit-price was equated across different ratio values. Pigeons were trained to key peck in daily sessions under a three-component multiple FR schedule of food presentation, according to which either 10, 30, or 100 pecks were required for each delivery of food. In Experiment 1, completion of the FR 10, FR 30, and FR 100 resulted in 1.5, 4.5, and 15.0 s access to food, respectively. That is, the response requirement was correlated with access to food time so that unit-price (pecks per second of access to food) was equated across components. After assessing acute effects of a range of doses of cocaine, drug administration occurred daily before each session, with the dose varying from day to day. Tolerance, the magnitude of which was unrelated to the peck requirement, developed under the repeated-dosing regimen. In order to assess whether having equal unit-price was responsible for producing similar levels of tolerance across components, daily drug administration continued in Experiment 2 using the variable-dose regimen, but the amount of food presented each time was fixed at 4.5 s access to food, yielding different unit-prices under the three pecking requirements. Subsequently, the conditions of Experiment 1 were reinstated, i.e., unit-price was equated. Making unit-price different or the same had little influence on effects of cocaine. In Experiment 3, a fixed dose of cocaine was administered before each session while programmed unit-price remained the same across components. Under these conditions, tolerance became peck-requirement related. Specifically, tolerance was most prevalent under the smaller requirements and less robust or absent when the largest requirement was in effect. Differences in unit-price, therefore, were not related to degree of tolerance, but work requirement was. Differences in effects of cocaine across responses requirements, however, were observed only when each session was preceded by the same dose, not when dose varied from session to session.     

Habituation and the reinforcing effectiveness of visual stimuli

The term "sensory reinforcer" has been used to refer to sensory stimuli (e.g. light onset) that are primary reinforcers in order to differentiate them from other more biologically important primary reinforcers (e.g. food and water). Acquisition of snout poke responding for a visual stimulus (5s light onset) with fixed ratio 1 (FR 1), variable-interval 1min (VI 1min), or variable-interval 6min (VI 6min) schedules of reinforcement was tested in three groups of rats (n=8/group). The VI 6min schedule of reinforcement produced a higher response rate than the FR 1 or VI 1min schedules of visual stimulus reinforcement. One explanation for greater responding on the VI 6min schedule relative to the FR 1 and VI 1min schedules is that the reinforcing effectiveness of light onset habituated more rapidly in the FR 1 and VI 1min groups as compared to the VI 6min group. The inverse relationship between response rate and the rate of visual stimulus reinforcement is opposite to results from studies with biologically important reinforcers which indicate a positive relationship between response and reinforcement rate. Rapid habituation of reinforcing effectiveness may be a fundamental characteristic of sensory reinforcers that differentiates them from biologically important reinforcers, which are required to maintain homeostatic balance.     

A further study of choice and percentage reinforcement

In Experiment I four pigeons were trained in a concurrent chains procedure with fixed-ratio schedules (FR1) in the initial components and fixed-time schedules in the terminal components. Pecking one of the keys when both keys were white initiated a fixed time schedule on that key. A peck to the left key produced three stripes on the key. At the termination of the fixed-time component food always occurred. Pecking the other key produced either a circle or a triangle. If a circle appeared, reinforcement occurred. If a triangle appeared a brief timeout was given. Initially the stripes appeared on the left key and the circle and triangle on the left. This was reversed during the course of the experiment. In addition, sessions were conducted in which both circle and triangle sometimes preceded reinforcement and sometimes timeout. For most birds under most conditions there was a preference for the key that produced the circle and triangle. When these were uncorrelated with reinforcement and time out three of the birds preferred the key producing 100% reinforcement.

In Experiment II three factors were varied and VI 20 sec schedules were used in the initial links instead of FR1. The results showed that pigeons preferred the 50% condition more 1) the greater the duration of the terminal links, 2) the smaller the value on the initial link VI schedules and 3) the less the probability of food in the terminal link with stripes on the key.     

Effect of behavioral history on subsequent responding: experiments using rotary fixed-ratio schedules

The present study proposes a novel experimental approach to examine the effect of previous schedule history on subsequent responses. College students used a three-button device to indicate their choice of response. We performed two experiments. Each experiment consisted of three phases. In experiment I, fixed-ratio 10 (FR 10) reinforcement schedule was employed in phases 1 and 3. Phase 2 employed "Rotary-FR 10" schedule: after obtaining a point produced by pressing one of three specified buttons in a triangular array 10 times, the effective button changed to another in a clockwise direction. Optimal response pattern for Rotary-FR 10 was analyzed. Two types of behavior, referred to as "consecutive" and "scatter" were observed in phase 1. The mean number of optimal responses in consecutive type was significantly higher than scatter type in phases 2 and 3. In experiment II, two schedules were used in phase 1, and the same protocol used in experiment I was followed. We found that the reinforcement schedule used in phase 1 affected acquisition of the optimal response in phase 2. These results suggest that the earlier behavioral tendencies play an important role in determining subsequent behavior and that such behavioral tendency can be modified by specifically designed protocols.     

Effect of soman on schedule-controlled behavior and brain acetylcholinesterase in rats

Rats were trained to press a lever under a multiple fixed-ratio 25 fixed-interval 50-second (FR25 FI50-sec) schedule of food reinforcement. Soman, 70-90 micrograms/kg, s.c., suppressed response rates in both components, with a slightly greater effect in the FI schedule. The pattern of responding under the FI schedule, however, was maintained until lever-pressing was nearly completely suppressed. At the highest doses, soman occasionally caused tremors or mild tonic seizures with hindlimb abduction. The suppression of response rate was correlated with inhibition of acetylcholinesterase (AChE) in all brain regions examined: cortex, striatum, hippocampus, hypothalamus and brainstem. Cortical AChE was inhibited to the highest degree, while striatal AChE was most resistant to inhibition by soman.     

Small and large fixed ratios with the same unit price

Key pressing of rats was maintained under multiple and discrete-trial choice schedules with reinforcer units of 45 mg food pellets or 3.5 s dips of sucrose solution. Both smaller and larger fixed ratio (FR) schedules were associated with the same unit price in a manner, for example, that each of eight iterations of FR120 was associated with delivery of a single reinforcer unit and one instance of FR960 was associated with eight reinforcer units. FR requirement varied between 20 and 1560 per aggregate reinforcer and unit price varied between 20 and 240 per reinforcer unit. During multiple schedules with food reinforcers, rates and patterns of responding were comparable over nearly a 50-fold range of FR requirements (20-1380) when unit price was 20; over nearly a six-fold range of FR requirements (120-720) when unit price was 120; and was only marginally maintained when unit price was 240. Demand for food pellets was comparatively inelastic at FRs between 20 and 120, during which subjects did not receive supplemental feeding outside experimental sessions, but was elastic at FRs greater than 240, when subjects sometimes did receive supplemental feeding. In a discrete-trial choice procedure with a constant unit price of 120 for sucrose solution, subjects were indifferent between smaller FRs and alternative FRs as large as 480, but began switching away from larger FRs that were 600 or greater. Because responding had been comparably maintained under both FR120 and FRs as large as 960 in the multiple schedule, results from the choice procedure indicated that choice performance was influenced by variables other than FR requirement and unit price. Because aggregate reinforcers were the same for smaller and larger FRs, the most likely reason for preferring smaller FRs was the nearness in time to some reinforcer.     

The Development of a Preference for Cocaine over Food Identifies Individual Rats with Addiction-Like Behaviors

Rationale

Cocaine dependence is characterized by compulsive drug taking that supercedes other recreational, occupational or social pursuits. We hypothesized that rats vulnerable to addiction could be identified within the larger population based on their preference for cocaine over palatable food rewards.

Objectives

To validate the choice self-administration paradigm as a preclinical model of addiction, we examined changes in motivation for cocaine and recidivism to drug seeking in cocaine-preferring and pellet-preferring rats. We also examined behavior in males and females to identify sex differences in this “addicted” phenotype.

Methods

Preferences were identified during self-administration on a fixed-ratio schedule with cocaine-only, pellet-only and choice sessions. Motivation for each reward was probed early and late during self-administration using a progressive-ratio schedule. Reinstatement of cocaine- and pellet-seeking was examined following exposure to their cues and non-contingent delivery of each reward.

Results

Cocaine preferring rats increased their drug intake at the expense of pellets, displayed increased motivation for cocaine, attenuated motivation for pellets and greater cocaine and cue-induced reinstatement of drug seeking. Females were more likely to develop cocaine preferences and recidivism of cocaine- and pellet-seeking was sexually dimorphic.

Conclusions

The choice self-administration paradigm is a valid preclinical model of addiction. The unbiased selection criteria also revealed sex-specific vulnerability factors that could be differentiated from generalized sex differences in behavior, which has implications for the neurobiology of addiction and effective treatments in each sex.     

Concurrent schedule performance in domestic goats: persistent undermatching

Performance of nine domestic goats responding under concurrent variable-interval variable-interval schedules of food delivery was examined, with results analyzed in terms of the generalized matching equation. Substantial undermatching of response and time allocation ratios to obtained reinforcement ratios was evident. Post-reinforcement pause time ratios approximately matched obtained reinforcement ratios. Subtracting these times from total time allocation values yielded net time allocation ratios, which undermatched obtained reinforcement ratios to a greater degree than whole-session time allocation ratios. Slopes of regression lines relating behavioral outputs to environmental inputs characteristically were below 0.6, which is similar to previous findings in dairy cows tested under comparable conditions.     

60 Hz electric fields and incandescent light as aversive stimuli controlling the behavior of rats responding under concurrent schedules of reinforcement

Several reports have shown that animals will sometimes engage in behaviors that reduce their exposure to a 60 Hz electric field (E-field). The field, therefore, can function as an aversive stimulus. In other studies, the E-field at equivalent strengths failed to function as an aversive stimulus. The present experiment, using rats, demonstrates how factors other than field strength can influence whether a subject engages in behavior that reduces field exposure. The general design consisted of giving the rat a choice between two alternatives, one of which sometimes included an added stimulus. Each subject was trained to press each of two levers to obtain food. Pressing one lever was reinforced intermittently under a variable interval 2 min schedule (VI 2); pressing the other lever was reinforced by a second VI 2 schedule operating independently of the first. Under this concurrent schedule the rat spent 50% of the daily 50 min session responding to each of the levers, indicating that they were equally “valued.” Next, while the schedules remained in effect, the first response to one of the levers turned on a 100 kV/m E-field which remained on until the rat pressed the other lever. The time spent responding under the schedule associated with the field was reduced by about 5–10%. When the procedure was changed so that no lever presses produced food, i.e., extinction, but the added stimulus contingency remained, the rats spent even less time in the presence of the field. Similar outcomes were observed during both the concurrent food or extinction schedules when incandescent light was used. Thus, both an E-field and incandescent light functioned as aversive stimuli, but the magnitude of the aversiveness was small. Aversiveness depended not only on stimulus intensity, but also on behavioral factors. Bioelectromagnetics 19:210–221, 1998. © 1998 Wiley-Liss, Inc.