Habituation and the reinforcing effectiveness of visual stimuli

The term "sensory reinforcer" has been used to refer to sensory stimuli (e.g. light onset) that are primary reinforcers in order to differentiate them from other more biologically important primary reinforcers (e.g. food and water). Acquisition of snout poke responding for a visual stimulus (5s light onset) with fixed ratio 1 (FR 1), variable-interval 1min (VI 1min), or variable-interval 6min (VI 6min) schedules of reinforcement was tested in three groups of rats (n=8/group). The VI 6min schedule of reinforcement produced a higher response rate than the FR 1 or VI 1min schedules of visual stimulus reinforcement. One explanation for greater responding on the VI 6min schedule relative to the FR 1 and VI 1min schedules is that the reinforcing effectiveness of light onset habituated more rapidly in the FR 1 and VI 1min groups as compared to the VI 6min group. The inverse relationship between response rate and the rate of visual stimulus reinforcement is opposite to results from studies with biologically important reinforcers which indicate a positive relationship between response and reinforcement rate. Rapid habituation of reinforcing effectiveness may be a fundamental characteristic of sensory reinforcers that differentiates them from biologically important reinforcers, which are required to maintain homeostatic balance.     

Oscillations following periodic reinforcement

Three experiments examined behavior in extinction following periodic reinforcement. During the first phase of Experiment 1, four groups of pigeons were exposed to fixed interval (FI 16 s or FI 48 s) or variable interval (VI 16 s or VI 48 s) reinforcement schedules. Next, during the second phase, each session started with reinforcement trials and ended with an extinction segment. Experiment 2 was similar except that the extinction segment was considerably longer. Experiment 3 replaced the FI schedules with a peak procedure, with FI trials interspersed with non-food peak interval (PI) trials that were four times longer. One group of pigeons was exposed to FI 20 s PI 80 s trials, and another to FI 40 s PI 160 s trials. Results showed that, during the extinction segment, most pigeons trained with FI schedules, but not with VI schedules, displayed pause-peck oscillations with a period close to, but slightly greater than the FI parameter. These oscillations did not start immediately after the onset of extinction. Comparing the oscillations from Experiments 1 and 2 suggested that the alternation of reconditioning and re-extinction increases the reliability and earlier onset of the oscillations. In Experiment 3 the pigeons exhibited well-defined pause-peck cycles since the onset of extinction. These cycles had periods close to twice the value of the FI and lasted for long intervals of time. We discuss some hypotheses concerning the processes underlying behavioral oscillations following periodic reinforcement.     

EMG Biofeedback: The Effects of CRF, FR, VR, FI, and VI Schedules of Reinforcement on the Acquisition and Extinction of Increases in Forearm Muscle Tension

Biofeedback was used to increase forearm-muscle tension. Feedback was delivered under continuous reinforcement (CRF), variable interval (VI), fixed interval (FI), variable ratio (VR), and fixed ratio (FR) schedules of reinforcement when college students increased their muscle tension (electromyograph, EMG) above a high threshold. There were three daily sessions of feedback, and Session 3 was immediately followed by a session without feedback (extinction). The CRF schedule resulted in the highest EMG, closely followed by the FR and VR schedules, and the lowest EMG scores were produced by the FI and VI schedules. Similarly, the CRF schedule resulted in the greatest amount of time-above-threshold and the VI and FI schedules produced the lowest time-above-threshold. The highest response rates were generated by the FR schedule, followed by the VR schedule. The CRF schedule produced relatively low response rates, comparable to the rates under the VI and FI schedules. Some of the data are consistent with the partial-reinforcement-extinction effect. The present data suggest that different schedules of feedback should be considered in muscle-strengthening contexts such as during the rehabilitation of muscles following brain damage or peripheral nervous-system injury.     

Nicotine and the behavioral mechanisms of intertemporal choice

Nicotine has been found to produce dose-dependent increases in impulsive choice (preference for smaller, sooner reinforcers relative to larger, later reinforcers) in rats. Such increases could be produced by either of two behavioral mechanisms: (1) an increase in delay discounting (i.e., exacerbating the impact of differences in reinforcer delays) which would increase the value of a sooner reinforcer relative to a later one, or (2) a decrease in magnitude sensitivity (i.e., diminishing the impact of differences in reinforcer magnitudes) which would increase the value of a smaller reinforcer relative to a larger one. To isolate which of these two behavioral mechanisms was likely responsible for nicotine's effect on impulsive choice, we manipulated reinforcer delay and magnitude using a concurrent, variable interval (VI 30 s, VI 30 s) schedule of reinforcement with 2 groups of Long-Evans rats (n = 6 per group). For one group, choices were made between a 1-s delay and a 9-s delay to 2 food pellets. For a second group, choices were made between 1 pellet and 3 pellets. Nicotine (vehicle, 0.03, 0.1, 0.3, 0.56 and 0.74 mg/kg) produced dose-dependent decreases in preference for large versus small magnitude reinforcers and had no consistent effect on preference for short versus long delays. This suggests that nicotine decreases sensitivity to reinforcer magnitude.     

Effects of d-amphetamine on behavior maintained by signaled and unsignaled delays to reinforcement

Four pigeons responded under a progressive-delay procedure. In a signaled-delay condition, a chained variable interval (VI) 30-s progressive time (PT) 4-s schedule was arranged; in an unsignaled-delay condition, a tandem VI 30-s PT 4-s schedule was arranged. Two pigeons experienced a signaled-unsignaled-signaled sequence; whereas, two pigeons experienced an unsignaled-signaled-unsignaled sequence. Effects of saline and d-amphetamine were determined under each condition. At intermediate doses (1.0 and 1.78 m/kg) delay functions were shallower, area under the curve was increased, and, when possible, break points were increased compared to saline; these effects were not systematically related to signaling conditions. These effects on control by delay often were accompanied by decreased response rates at 0 s. These results suggest that stimulus conditions associated with the delay may not play a crucial role in effects of d-amphetamine and other stimulants on behavior controlled by reinforcement delay.     

Effects of unpredictable changes in initial-link duration on choice and timing

Four pigeons responded in a concurrent-chains procedure in which terminal-link schedules were fixed-interval (FI) 10 s and FI 20 s. Across sessions, the location of the shorter terminal-link changed according to a pseudorandom binary sequence. Each session, the variable-interval initial-link schedule value was sampled from a uniform distribution that ranged from 0.01 to 30 s. On some terminal links, food was withheld to obtain measures of temporal control. Terminal-link delays determined choice (log initial-link response ratios) and timing (start and stop times on no-food trials) measures, which stabilized within the 1st half of each session. Preference for the shorter terminal-link delay was a monotonically decreasing function of initial-link duration. There was no evidence of control by initial-link durations from previous sessions.     

Conditioned suppression as a method of detecting taste thresholds in the rat

Taste thresholds of seven male Sprague—Dawley rats (meanage 10 weeks, mean weight 250 g) were determined for four basictaste qualities: sweet, sour, salty and bitter. The method ofconditioned suppression was employed. An apparatus capable ofpresenting any one of eight separate drinking tubes during atesting session was designed. Animals were reduced to 80–85%ad lib. body weight. They were then trained to lick a sippertube through a slot in the back of an experimental chamber forpellet reinforcements. Animals progressed through a series ofreinforcement schedules starting with a fixed ratio (FR) scheduleof five licks for each reinforcement. They advanced to a variableratio (VR) schedule of reinforcement and finally a variableinterval (VI) schedule with a mean of 17.5 s was used. Whileon the VI schedule animals were trained to suppress lickingwhen any tastant other than water was presented. The first lickon any tastant was followed 10 s later by a mild electric shockif a rat made more than 20 licks on the tube in the 10-s period.Less than 20 licks on a tastant tube resulted in no shock anda 5-s time out before proceeding to the next tube. Thresholdwas determined using a suppression ratio formula. Thresholdwas defined as the 0.33 suppression ratio. Results from thisexperiment reveal mean thresholds for the seven animals as:sucrose = 2.3 mM, NaCl = 0.63 mM, quinine HCl = 0.005 mM andcitric acid = 0.085 mM.     

Effects of unsignaled delays of reinforcement on fixed-interval schedule performance

Key pecking of pigeons was maintained by a fixed-interval (FI) 61-s schedule. The effects of resetting and nonresetting unsignaled delays of reinforcement then were examined. The resetting delay was programmed as a differential-reinforcement-of-other-behavior schedule, and the nonresetting delay as a fixed-time schedule. Three delay durations (0.5, 1 and 10 s) were examined. Overall response rates were decreased by one and 10-s delays and increased by 0.5-s delays. Response patterns changed from positively accelerated to more linear when resetting or nonresetting 10-s delays were imposed, but remained predominantly positively accelerated when resetting and nonresetting 0.5- and 1-s delays were in effect. In general, temporal control, as measured by quarter-life values, changed less than overall response rates when delays of reinforcement were in effect. The response patterns controlled by FI schedules are more resilient to the nominally disruptive effects of delays of reinforcement than are corresponding overall response rates.     

Rapid acquisition in concurrent chains: effects of initial-link duration

Pigeons were trained in a concurrent chains procedure in which the terminal-link schedules in each session were either fixed-interval (FI) 10s FI 20s or FI 20s FI 10s, as determined by a pseudorandom binary series. The initial-link was a variable-interval (VI) 10-s schedule. Training continued until initial-link response allocation stabilized about midway through each session and was sensitive to the terminal-link immediacy ratio in that session. The initial-link schedule was then varied across sessions between VI 0.01 s and VI 30s according to an ascending and descending sequence. Initial-link response allocation was a bitonic function over the full range of durations. Preference for the FI 10-s terminal-link at first increased as programmed initial-link duration varied from 0.01 to 7.5s, and then decreased as initial-link duration increased to 30s. The bitonic function poses a potential challenge for existing models for steady-state choice, such as delay-reduction theory (DRT) [Fantino, E., 1969. Choice and rate of reinforcement. J. Exp. Anal. Behav. 12, 723-730], which predict a monotonic function. However, an extension of Grace and McLean's [Grace, R.C., McLean, A.P., 2006. Rapid acquisition in concurrent chains: evidence for a decision model. J. Exp. Anal. Behav. 85, 181-202] decision model predicted the bitonic function, and may ultimately provide an integrated account of choice in concurrent chains under both steady-state and dynamic conditions.     

Interactions among unit-price, fixed-ratio value, and dosing regimen in determining effects of repeated cocaine administration

Previous research has shown tolerance to cocaine that was dependent on fixed-ratio (FR) parameter size in the context of a multiple FR schedule of food reinforcement. Completion of the FR requirement in these studies resulted in the same magnitude of reinforcement, regardless of ratio size. The cost-to-benefits ratio (unit-price) was therefore not equated across the different FR components. The current study examined the effects of repeated administration of cocaine to pigeons when unit-price under FR schedules was either the same or different. Additionally, the role of a chronic variable-dosing versus chronic fixed-dosing procedure was examined when unit-price was equated across different ratio values. Pigeons were trained to key peck in daily sessions under a three-component multiple FR schedule of food presentation, according to which either 10, 30, or 100 pecks were required for each delivery of food. In Experiment 1, completion of the FR 10, FR 30, and FR 100 resulted in 1.5, 4.5, and 15.0 s access to food, respectively. That is, the response requirement was correlated with access to food time so that unit-price (pecks per second of access to food) was equated across components. After assessing acute effects of a range of doses of cocaine, drug administration occurred daily before each session, with the dose varying from day to day. Tolerance, the magnitude of which was unrelated to the peck requirement, developed under the repeated-dosing regimen. In order to assess whether having equal unit-price was responsible for producing similar levels of tolerance across components, daily drug administration continued in Experiment 2 using the variable-dose regimen, but the amount of food presented each time was fixed at 4.5 s access to food, yielding different unit-prices under the three pecking requirements. Subsequently, the conditions of Experiment 1 were reinstated, i.e., unit-price was equated. Making unit-price different or the same had little influence on effects of cocaine. In Experiment 3, a fixed dose of cocaine was administered before each session while programmed unit-price remained the same across components. Under these conditions, tolerance became peck-requirement related. Specifically, tolerance was most prevalent under the smaller requirements and less robust or absent when the largest requirement was in effect. Differences in unit-price, therefore, were not related to degree of tolerance, but work requirement was. Differences in effects of cocaine across responses requirements, however, were observed only when each session was preceded by the same dose, not when dose varied from session to session.     

Resistance to change varies inversely with reinforcement context

We report two experiments which test whether resistance to prefeeding and satiation for a variable-interval (VI) schedule that delivers a constant rate of reinforcement varies inversely with the reinforcement rate for an alternative schedule. In Experiment 1, eight pigeons responded in a multiple schedule in which the red key was always associated with a VI 90-s schedule and the green key with either a richer (VI 18s) or leaner (VI 540s) schedule in different conditions. After baseline training in each condition, prefeeding test sessions were conducted in which 10g, 20g, 30g, 40g, and 50g food were provided one-hour prior to test. Additional baseline training was given between each test session. In Experiment 2, two groups of pigeons responded in a multiple schedule similar to Experiment 1. After baseline training, pigeons were exposed to a 5-h satiation test session in which the VI 90-s schedule was available continuously. Test sessions were conducted when pigeons were maintained at 85%, 95%, and 85% of their body weights in an ABA design. Results of both experiments showed that responding in the VI 90-s schedule that alternated with a leaner schedule during baseline was more resistant to prefeeding and satiation. These data rule out alternative explanations for results of previous studies, and confirm that resistance to change varies inversely with reinforcement context.     

A further study of choice and percentage reinforcement

In Experiment I four pigeons were trained in a concurrent chains procedure with fixed-ratio schedules (FR1) in the initial components and fixed-time schedules in the terminal components. Pecking one of the keys when both keys were white initiated a fixed time schedule on that key. A peck to the left key produced three stripes on the key. At the termination of the fixed-time component food always occurred. Pecking the other key produced either a circle or a triangle. If a circle appeared, reinforcement occurred. If a triangle appeared a brief timeout was given. Initially the stripes appeared on the left key and the circle and triangle on the left. This was reversed during the course of the experiment. In addition, sessions were conducted in which both circle and triangle sometimes preceded reinforcement and sometimes timeout. For most birds under most conditions there was a preference for the key that produced the circle and triangle. When these were uncorrelated with reinforcement and time out three of the birds preferred the key producing 100% reinforcement.

In Experiment II three factors were varied and VI 20 sec schedules were used in the initial links instead of FR1. The results showed that pigeons preferred the 50% condition more 1) the greater the duration of the terminal links, 2) the smaller the value on the initial link VI schedules and 3) the less the probability of food in the terminal link with stripes on the key.     

Choice in a variable environment: Effects of d-amphetamine on sensitivity to reinforcement

Four pigeons responded under a 7-component mixed schedule in which each component arranged a different left:right reinforcer ratio (27:1, 9:1, 3:1, 1:1, 1:3, 1:9, 1:27). Components were unsignaled, and the order within each session was randomly determined. After extensive exposure to these contingencies, effects of a range of doses of d-amphetamine (0.3-5.6 mg/kg) on estimates of sensitivity to reinforcement at several levels of analysis were assessed. Under non-drug conditions, the structure of choice was similar to that previously reported under this procedure. That is, responding adjusted within components to the reinforcer ratio in effect (i.e., sensitivity estimates were higher in the 2nd than in the 1st half of components), and individual reinforcers produced “preference pulses” (i.e., each food presentation produced an immediate, local, shift in preference toward the response that just produced food). Although there was a general tendency for d-amphetamine to reduce overall sensitivity to reinforcement, the size of this effect and its reliability varied across pigeons. Further analysis, however, revealed that intermediate d-amphetamine doses consistently reduced sensitivity immediately following reinforcer presentations; that is, these doses consistently attenuated preference pulses.     

Choice behavior in spontaneously hypertensive rats: Variable vs. fixed schedules of reinforcement

Spontaneously hypertensive rats (SHR) and Wistar rats were evaluated in the successive-encounters procedure (an operant simulation of natural foraging) with the idea of assessing differences between them in their preference for variable schedules of reinforcement. In this procedure, after satisfying a schedule of reinforcement associated with search time, the subjects could “accept” or “reject” another schedule of reinforcement associated with handling time. Two schedules of reinforcement were available: a fixed interval (FI), and a variable interval (VI) with the same mean value. The results indicated preference for the variable schedule in both strains, as suggested by the observation that the VI was always accepted while the FI was often rejected. The difference in FI acceptability between strains was not statistically significant, a result which is relevant for the current debate of SHR as an adequate animal model of Attention Deficit Hyperactivity Disorder.     

Effect of contingent auditory stimuli on concurrent schedule performance: an alternative punisher to electric shock

This study explored whether load auditory stimuli could be used as functional punishing stimuli in place of electric shock. Three experiments examined the effect of a loud auditory stimulus on rats’ responding maintained by a concurrent reinforcement schedule. In Experiment 1, overall response rate decreased when a concurrent 1.5 s tone presentation schedule was superimposed on the concurrent variable interval (VI) 180-s, VI 180-s reinforcement schedule. On the contrary, response rate increased when a click presentation schedule was added. In Experiment 2, the extent of the response suppression with a 1.5 s tone presentation varied as a function of the frequency of the reinforcement schedule maintaining responses; the leaner the schedule employed, the greater the response suppression. In Experiment 3, response suppression was observed to be inversely related to the duration of the tone; response facilitation was observed when a 3.0-s tone was used. In Experiments 1 and 2, a preference shift towards the alternative with richer reinforcement was observed when the tone schedule was added. In contrast, the preference shifted towards the leaner alternative when the click or longer duration stimulus was used. These results imply that both the type and duration of a loud auditory stimulus, as well as the reinforcement schedule maintaining responses, have a critical role in determining the effect of the stimuli on responding. They also suggest that a loud auditory stimulus can be used as a positive punisher in a choice situation for rats, when the duration of the tone is brief, and the reinforcement schedule maintaining responses is lean.     

Impact of pubertal and adult estradiol treatments on cocaine self-administration

Estradiol is thought to play a critical role in the increased vulnerability to psychostimulant abuse in women. Sex differences in the ability of estradiol to influence cocaine self-administration in adult rats have been hypothesized to depend upon pubertal estradiol exposure. The current study investigated whether the presence of gonadal hormones during puberty affected cocaine self-administration behavior and its sensitivity to adult estradiol treatment in male and female Sprague–Dawley rats. Subjects were gonadectomized or SHAM-operated at postnatal day (PD) 22, and received either OIL or estradiol benzoate (EB) during the approximate time of puberty (PD27 to PD37). Adult rats were subsequently treated with either EB or OIL 30 min before cocaine self-administration (0.3 mg/kg/inf) in order to examine the effects of pubertal manipulations on the estradiol sensitivity of acquisition on a fixed ratio (FR) 1 schedule, total intake on a FR5 schedule and motivation on a progressive ratio schedule. Adult EB treatment only affected cocaine self-administration in females, which is consistent with previous research. Adult EB treatment enhanced acquisition in all females irrespective of puberty manipulations. All females, except those treated with EB during puberty, displayed increased cocaine intake following adult EB treatment. Adult EB treatment only enhanced motivation in females that were intact during puberty, whereas those treated with EB during puberty showed reduced motivation. Therefore, the sensitivities of different self-administration behaviors to adult estradiol treatment are organized independently in females, with pubertal estradiol exerting a greater influence over motivational processes, and negligible effects on learning/acquisition.     

Effects of a parasitic nematode on male mate choice in a livebearing fish with a coercive mating system (western mosquitofish, Gambusia affinis)

Studies of choice in steady state have shown that sensitivity to reinforcement increases with increasing fixed-ratio changeover (FR CO) requirements. We assessed the generality of this finding with choice in transition. Food deliveries were programmed according to concurrent variable-interval (VI) schedules. Seven different VI pairs arranged ratios of food deliveries (left/right) of 27:1, 9:1, 3:1, 1:1, 1:3, 1:9, and 1:27 at a constant overall rate across components. Within sessions, all seven ratios were presented in random order. Each component lasted for 10 food deliveries; components were separated by 60-s blackouts. A changeover lever required 1, 2, 4, 8, 16, 32, and 64 responses to alternate between two main levers. Redeterminations to all FR COs, but 64 responses, were obtained in descending order. Choice adjusted rapidly to rapid changes in the reinforcer ratio, tracking the lever associated with the highest probability of reinforcer. Sensitivity to reinforcement increased with increasing FR CO, replicating the negatively accelerated function found in our earlier study. With successive reinforcers in components, however, sensitivity reached asymptote values sooner with the largest (8, 16, and 32 responses), than with the smallest (1, 2, and 4 responses), FR CO requirements.     

Psychological distance to reward: equating the number of stimulus and response segments

Psychological distance to reward, or the segmentation effect, refers to the preference for a terminal link of a concurrent-chains schedule consisting of a simple reinforcement schedule (e.g. fixed interval [FI] 30s) relative to its chained-schedule counterpart (e.g. chained FI 15s FI 15s). This experiment was conducted to examine whether the segmentation effect is due to the number of terminal-link stimulus and response segments per se. Three pigeons pecked under a concurrent-chains schedule in which identical variable-interval (VI) schedules operated in the initial links. In each session, half the terminal-link entries followed one initial-link key and the other half followed the other initial-link key. The initial-link keys correlated with the different terminal links were manipulated across conditions. In the first three conditions, each terminal link contained a chained fixed-time (FT) FT schedule, and in the final three conditions, each terminal link contained a chained FI FI schedule. In each condition, in one terminal link (alternating), the order of two key colors correlated with the different schedule segments alternated across terminal-link entries, whereas in the other terminal link (constant), the order of two other key colors was identical for each entry. With the chained FT FT schedule terminal links, there was indifference between the alternating and constant terminal links within and across pigeons, as indexed by initial-link choice proportions. In addition, terminal-link response rates were relatively low. With the chained FI FI schedule terminal links, for each pigeon, there was relatively more preference for the alternating terminal link and terminal-link response rates increased relative to conditions with the chained FT FT schedule terminal links. These data suggest that the segmentation effect is not due simply to the number of terminal-link stimulus or response segments per se, but rather to a required period of responding during a stimulus segment that never is paired with reinforcement.