Dishabituation with component transitions may contribute to the interactions observed during multiple schedules

Rates of responding by rats were usually higher during the variable interval (VI) 30-s component of a multiple VI 30-s fixed interval (FI) 30-s schedule than during the same component of a multiple VI 30-s VI 30-s schedule (Experiment 1). Response rates were also usually higher during the FI 30-s component of a multiple VI 30-s FI 30-s schedule than during the same component of a multiple FI 30-s FI 30-s schedule (Experiment 2). The differences in response rates were not observed when the components provided VI or FI 120-s schedules. These results were predicted by the idea that differences in habituation to the reinforcer between multiple schedules contribute to behavioral interactions, such as behavioral contrast. However, differences in habituation were not apparent in the within-session patterns of responding. Finding differences in response rates in both experiments violates widely-held assumptions about behavioral interactions, including that behavioral contrast does not occur for rats and that improving the conditions of reinforcement decreases, rather than increases, response rate in the alternative component.     

Effects of partial reinforcement and time between reinforced trials on terminal response rate in pigeon autoshaping

Partial reinforcement often leads to asymptotically higher rates of responding and number of trials with a response than does continuous reinforcement in pigeon autoshaping. However, comparisons typically involve a partial reinforcement schedule that differs from the continuous reinforcement schedule in both time between reinforced trials and probability of reinforcement. Two experiments examined the relative contributions of these two manipulations to asymptotic response rate. Results suggest that the greater responding previously seen with partial reinforcement is primarily due to differential probability of reinforcement and not differential time between reinforced trials. Further, once established, differences in responding are resistant to a change in stimulus and contingency. Secondary response theories of autoshaped responding (theories that posit additional response-augmenting or response-attenuating mechanisms specific to partial or continuous reinforcement) cannot fully accommodate the current body of data. It is suggested that researchers who study pigeon autoshaping train animals on a common task prior to training them under different conditions.     

Amygdala-prefrontal synchronization underlies resistance to extinction of aversive memories

Emotional memories can persist for a lifetime but can also undergo extinction. Although we know about the mechanisms involved in expression and extinction, we know very little about the mechanisms that determine whether a specific memory would persist or not. Here, we use partial reinforcement extinction effect (PREE) to explore the neural mechanisms that differentiate persistent from labile memories. We recorded the simultaneous activity of neurons in the amygdala and the dorsal anterior cingulate cortex (dACC) while monkeys engaged in tone-odor aversive conditioning. We report that under continuous reinforcement schedule (ConS), activity in the amygdala precedes behavioral response, whereas under partial schedule (ParS), dACC activity precedes it. Moreover, we find that ParS induced cross-regional pairwise correlations throughout the entire acquisition session, and their magnitude and precision predicted the later resistance to extinction. Our results suggest that memory persistence depends on distributed representations and, specifically, resistance to extinction of aversive memories is maintained by correlated amygdala-dACC activity.     

Reinforcement omission in concurrent fixed-interval and random-interval schedules

The present experiment examined overall and local effects of omission of reinforcers in a choice situation. Pigeons' key-pecking responses were reinforced under concurrent fixed-interval and random-interval schedules of food presentation. After some weeks of baseline sessions in which the probability of reinforcement was 1.00, approximately 25% of food presentations from the fixed-interval schedule were omitted and replaced by timeout periods. In such omission sessions, the overall relative rates of responding to the fixed-interval schedule became lower than those in the baseline sessions. On the other hand, when relative rates of responding to the fixed-interval schedule in the omission sessions were calculated separately for fixed-interval cycles preceded by timeout periods and those preceded by food presentations, the relative rates in the former type of fixed-interval cycles were higher than those in the latter type for three out of four pigeons. These results mean that relative rates of responding cannot always be regarded as reflecting a relative value of an alternative, and that the overall effect of the omission of fixed-interval reinforcers is not reducible to the local effect of omission.     

Performance correlates of social behavior and organization: Effects of group formation on operant performance in rhesus monkeys (M. mulatta)

The performance of six adult rhesus monkeys on a fixed interval 1-min reinforcement schedule was examined under conditions where the reinforcement probabilities were either 1.00 or .80. The animals were tested with a reinforcement probability of 1.00 immediately after removal from their social group, tested again several months later with reinforcement probabilities of 1.00 and .80, and then retested with a reinforcement probability of .80 before, during, and after the formation of a new social group with the six monkeys. The results supported an earlier report in which it was found that high ranking animals responded at a lower rate under a reinforcement probability of 1.00 than did low ranking animals and that the former also had a higher ratio of nonreinforced to reinforced responses than the lower ranked animals. These relationships were present in five of the six animals immediately after removal of the animals from their original group and during and after formation of the new group. They did not appear on tests conducted after the animals had been out of a social group for several months. A third result was a suppression of operant performance which appeared in all animals following group formation. The magnitude and duration of this effect was inversely related to social rank and the time the animal had been a member of the group. The data are discussed in terms of a carry-over of the effects of recent social experience on two factors: a more deliberate rate of response by higher ranking animals under 100% reinforcement and an inhibition of response bursting following omission of reinforcement by the lower ranking monkeys. Supported by USAMRDC Contract No. DADA 17-73-C-3007.     

Simulation of rat behavior by a reinforcement learning algorithm in consideration of appearance probabilities of reinforcement signals

Brown and Wanger [Brown, R.T., Wanger, A.R., 1964. Resistance to punishment and extinction following training with shock or nonreinforcement. J. Exp. Psychol. 68, 503-507] investigated rat behaviors with the following features: (1) rats were exposed to reward and punishment at the same time, (2) environment changed and rats relearned, and (3) rats were stochastically exposed to reward and punishment. The results are that exposure to nonreinforcement produces resistance to the decremental effects of behavior after stochastic reward schedule and that exposure to both punishment and reinforcement produces resistance to the decremental effects of behavior after stochastic punishment schedule. This paper aims to simulate the rat behaviors by a reinforcement learning algorithm in consideration of appearance probabilities of reinforcement signals. The former algorithms of reinforcement learning were unable to simulate the behavior of the feature (3). We improve the former reinforcement learning algorithms by controlling learning parameters in consideration of the acquisition probabilities of reinforcement signals. The proposed algorithm qualitatively simulates the result of the animal experiment of Brown and Wanger.     

Simple and multiple schedule responding and behavioral contrast when pigeons press treadles

Positive behavioral contrast has been observed when pigeons press treadles on multiple schedules for high rates of reinforcement, but not for low rates. Negative treadle-press contrast has been observed for low rates of reinforcement. Two experiments showed that differences between response rates emitted during simple and multiple schedules appear and fail to appear under similar conditions. The experiments showed that the rate of pressing during the less favorable component of a multiple schedule was less than the rate of pressing during a comparable simple schedule (negative contrast). The rate of treadle–pressing during the more favorable component was not greater than the rate of pressing during a comparable simple schedule, when the schedules provided a low rate of reinforcement (absence of positive contrast), but it was when the schedules provided a high rate of reinforcement (positive contrast). These results help to clarify the definition of behavioral contrast by showing that simple schedules may be appropriate baselines from which to define and measure contrast.     

Effect of behavioral history on subsequent responding: experiments using rotary fixed-ratio schedules

The present study proposes a novel experimental approach to examine the effect of previous schedule history on subsequent responses. College students used a three-button device to indicate their choice of response. We performed two experiments. Each experiment consisted of three phases. In experiment I, fixed-ratio 10 (FR 10) reinforcement schedule was employed in phases 1 and 3. Phase 2 employed "Rotary-FR 10" schedule: after obtaining a point produced by pressing one of three specified buttons in a triangular array 10 times, the effective button changed to another in a clockwise direction. Optimal response pattern for Rotary-FR 10 was analyzed. Two types of behavior, referred to as "consecutive" and "scatter" were observed in phase 1. The mean number of optimal responses in consecutive type was significantly higher than scatter type in phases 2 and 3. In experiment II, two schedules were used in phase 1, and the same protocol used in experiment I was followed. We found that the reinforcement schedule used in phase 1 affected acquisition of the optimal response in phase 2. These results suggest that the earlier behavioral tendencies play an important role in determining subsequent behavior and that such behavioral tendency can be modified by specifically designed protocols.     

Probability analysis of elements of rat behavior after sudden and signalled interruption of food reinforcement

Probability analysis was carried out of the appearance of single elements of rats behaviour in the process of extinction of a conditioned alimentary motor reflex. The dynamics of effector behavioural components at a sudden cessation of reinforcement (usual schedule of extinction) was compared with cessation of reinforcement signalled by a previously differentiated signal and with reinforcement cessation preceded by a stimulus initially unknown to the animal. If the reinforcement cessation is signalled by a previously differentiated (negative) stimulus, in response to its action the animals "loose the aim", what is revealed in a rapid complete reduction of all elements of the goal-directed alimentary behaviour. Obviously differentiation signal actualises the memory trace of "nonreinforcement" which was formed in the previous negative experience of the animal; this is revealed in accelerated inhibition of the alimentary motor reflex under extinction.     

Mathematical principles of reinforcement and resistance to change

Although Killeen's mathematical principles of reinforcement (MPR) apply to the asymptotic rate of a free operant after extended exposure to a single schedule of reinforcement, they can be extended to resistance to change in multiple schedules via alterations in the parameter representing the activating effects of reinforcers. MPR's predictions of resistance to change in relation to reinforcer rate on variable-interval (VI) schedules are empirically correct and agree with behavioral momentum theory (BMT). However, both MPR and BMT encounter problems in accounting for the effects of delayed reinforcement on resistance to change, relative to immediate reinforcement at the same rate. Further problems are raised by differences in resistance to change between variable-ratio (VR) and variable-interval performances maintained by the same reinforcer rate. With both delayed versus immediate reinforcement and variable-ratio versus variable-interval reinforcement, differential resistance to change is negatively correlated with the log ratios of baseline response rates when reinforcer rates are equated. Cases where resistance to change varies despite equated reinforcer rates, and the correlations among behavioral measures, provide challenges and opportunities for both MPR and BMT.     

EMG Biofeedback: The Effects of CRF, FR, VR, FI, and VI Schedules of Reinforcement on the Acquisition and Extinction of Increases in Forearm Muscle Tension

Biofeedback was used to increase forearm-muscle tension. Feedback was delivered under continuous reinforcement (CRF), variable interval (VI), fixed interval (FI), variable ratio (VR), and fixed ratio (FR) schedules of reinforcement when college students increased their muscle tension (electromyograph, EMG) above a high threshold. There were three daily sessions of feedback, and Session 3 was immediately followed by a session without feedback (extinction). The CRF schedule resulted in the highest EMG, closely followed by the FR and VR schedules, and the lowest EMG scores were produced by the FI and VI schedules. Similarly, the CRF schedule resulted in the greatest amount of time-above-threshold and the VI and FI schedules produced the lowest time-above-threshold. The highest response rates were generated by the FR schedule, followed by the VR schedule. The CRF schedule produced relatively low response rates, comparable to the rates under the VI and FI schedules. Some of the data are consistent with the partial-reinforcement-extinction effect. The present data suggest that different schedules of feedback should be considered in muscle-strengthening contexts such as during the rehabilitation of muscles following brain damage or peripheral nervous-system injury.     

Effects of unsignaled delays of reinforcement on fixed-interval schedule performance

Key pecking of pigeons was maintained by a fixed-interval (FI) 61-s schedule. The effects of resetting and nonresetting unsignaled delays of reinforcement then were examined. The resetting delay was programmed as a differential-reinforcement-of-other-behavior schedule, and the nonresetting delay as a fixed-time schedule. Three delay durations (0.5, 1 and 10 s) were examined. Overall response rates were decreased by one and 10-s delays and increased by 0.5-s delays. Response patterns changed from positively accelerated to more linear when resetting or nonresetting 10-s delays were imposed, but remained predominantly positively accelerated when resetting and nonresetting 0.5- and 1-s delays were in effect. In general, temporal control, as measured by quarter-life values, changed less than overall response rates when delays of reinforcement were in effect. The response patterns controlled by FI schedules are more resilient to the nominally disruptive effects of delays of reinforcement than are corresponding overall response rates.     

Effect of contingent auditory stimuli on concurrent schedule performance: an alternative punisher to electric shock

This study explored whether load auditory stimuli could be used as functional punishing stimuli in place of electric shock. Three experiments examined the effect of a loud auditory stimulus on rats’ responding maintained by a concurrent reinforcement schedule. In Experiment 1, overall response rate decreased when a concurrent 1.5 s tone presentation schedule was superimposed on the concurrent variable interval (VI) 180-s, VI 180-s reinforcement schedule. On the contrary, response rate increased when a click presentation schedule was added. In Experiment 2, the extent of the response suppression with a 1.5 s tone presentation varied as a function of the frequency of the reinforcement schedule maintaining responses; the leaner the schedule employed, the greater the response suppression. In Experiment 3, response suppression was observed to be inversely related to the duration of the tone; response facilitation was observed when a 3.0-s tone was used. In Experiments 1 and 2, a preference shift towards the alternative with richer reinforcement was observed when the tone schedule was added. In contrast, the preference shifted towards the leaner alternative when the click or longer duration stimulus was used. These results imply that both the type and duration of a loud auditory stimulus, as well as the reinforcement schedule maintaining responses, have a critical role in determining the effect of the stimuli on responding. They also suggest that a loud auditory stimulus can be used as a positive punisher in a choice situation for rats, when the duration of the tone is brief, and the reinforcement schedule maintaining responses is lean.     

Effects of reinforcer magnitude on response acquisition with unsignaled delayed reinforcement

Sixteen rats received eight 1-h sessions of a tandem fixed-ratio 1 differential-reinforcement-of-other-behavior 30-s schedule with reinforcer magnitude at one or six food pellet(s) across groups of eight rats. The larger reinforcer magnitude established the lever press more effectively. First, mean schedule completions differed across groups, in terms of both their overall difference and in their increase across sessions. Second, whereas the larger reinforcer magnitude established reliable response acquisition in all eight rats by the end of the experiment, the smaller reinforcer magnitude only established reliable response acquisition in four of eight rats. The present results systematically replicate earlier findings obtaining more reliable response acquisition with unsignaled delayed reinforcement with greater food deprivation. Taken together, this work demonstrates the influence of motivational variables on response acquisition with unsignaled delayed reinforcement.     

"The stone which the builders rejected...": Delay of reinforcement and response rate on fixed-interval and related schedules

The article deals with response rates (mainly running and peak or terminal rates) on simple and on some mixed-FI schedules and explores the idea that these rates are determined by the average delay of reinforcement for responses occurring during the response periods that the schedules generate. The effects of reinforcement delay are assumed to be mediated by a hyperbolic delay of reinforcement gradient. The account predicts that (a) running rates on simple FI schedules should increase with increasing rate of reinforcement, in a manner close to that required by Herrnstein's equation, (b) improving temporal control during acquisition should be associated with increasing running rates, (c) two-valued mixed-FI schedules with equiprobable components should produce complex results, with peak rates sometimes being higher on the longer component schedule, and (d) that effects of reinforcement probability on mixed-FI should affect the response rate at the time of the shorter component only. All these predictions were confirmed by data, although effects in some experiments remain outside the scope of the model. In general, delay of reinforcement as a determinant of response rate on FI and related schedules (rather than temporal control on such schedules) seems a useful starting point for a more thorough analysis of some neglected questions about performance on FI and related schedules.     

Conditioning the pecking motions of pigeons

The spatio-temporal courses of head and neck motions of pigeons while pecking at small grains are described. Single and serial pecks are distinguished but the inter- and intraindividual variability of the peck kinetics is stressed. Pigeons were then trained with instrumental conditioning procedures to speed-up their pecking. A partial reinforcement schedule where pigeons had to peck repeatedly before receiving reward led to a mild shortening of inter-peck intervals at lower reinforcement rates but surprisingly, a lengthening at higher rates. A schedule where short inter-peck intervals were differentially rewarded yielded a pronounced abbreviation of the inter-peck intervals, but this was achieved by a reduction of the movement path rather than an increase in motion velocity. A schedule whereby increased approach velocities were differentially rewarded yielded marked movement accelerations. When pigeons were rewarded for diminished approach speeds they also showed significant movement decelerations. Finally, it is shown that pigeons could learn to reliably abort their peck approach movement when a visual stimulus signalling a penalty was occasionally presented during the approach movement. The proportion of successful peck interruptions decreased as these interruption signals occurred later during the approach phase. It is concluded that the pecking of pigeons is neither an innately fixed nor a visually ballistic movement. It is instead a multiply controlled and flexibly adaptable response pattern.     

Contextual determinants of temporal control: Behavioral contrast in a free-operant psychophysical procedure

The question of how temporal control of responding might be influenced by contingency changes in other contexts was investigated. Each of three pigeons first was exposed to a two-component multiple schedule in which a two-key free-operant psychophysical procedure operated in one component and a variable-interval schedule operated in the other component. The variable-interval schedule then was changed to extinction while the free-operant psychophysical procedure remained unchanged. Finally, the variable-interval schedule was reintroduced. Response rates on the left key and the estimated temporal threshold under the free-operant psychophysical procedure increased for each pigeon when the alternate component schedule was changed to extinction and then decreased again when the variable-interval schedule was reintroduced. The results suggest one way that temporal control is affected by its context, and may be interpreted through the direct effects of overall reinforcement rate on temporal control mechanisms or the disruptive effects of alternative sources of reinforcement on temporally controlled behavior.     

Performance correlates of social behavior and organization: Social rank and omission of reinforcement in rhesus monkeys (M. mulatta)

The performance of 22 adult male rhesus monkeys on a Fixed Interval 1-min reinforcement schedule was examined under conditions where the reinforcement probabilities were either 1.00 or .80. The results were then related to the social rank of animals at the time that they were taken from their social groups. Both high and low ranked animals reached criterion performance in the same number of trials. In general, high ranking animals responded at lower rates than low ranking animals when the reinforcement probability was 1.00. When the reinforcement probability was shifted to .80, all animals showed an increase in responding after nonreinforced intervals as compared with responses after reinforced intervals. The higher ranked animals tended to have a higher ratio of nonreinforced to reinforced responses than lower ranked animals. Supported by USAMRDC Contract No. DADA 17-73-C-3007.     

Pentobarbital-like effects of N-methyl-D-aspartate antagonists in mice

The effects of the competitive N-methyl-D-aspartate (NMDA) antagonist, 3-[(+/-)-2-carboxypiperazin-4-yl]propyl-1-phosphonic acid (CPP), and of the noncompetitive NMDA antagonist, dizocilpine (MK-801), were determined in mice trained to discriminate pentobarbital (20 mg/kg i.p.) from saline under a standard two-lever fixed-ratio 20 schedule of sweetened milk reinforcement. CPP substituted for pentobarbital; however, pentobarbital-lever responding was usually associated with decreases in response rates. Dizocilpine produced a maximum average of only 62% pentobarbital-lever responding, accompanied by a 50% decrease in response rates. These results suggest that pentobarbital-like discriminative stimulus effects are more likely to be produced by competitive than by noncompetitive NMDA antagonists. This extends previous observation in rats and provides further evidence for differences in the behavioral effects of competitive and noncompetitive NMDA antagonists and for an overlap in the behavioral pharmacology of NMDA antagonists and classical CNS depressants.     

Lack of behavioral effects in the rhesus monkey: high peak microwave pulses at 1.3 GHz

The current safety standards for radiofrequency and microwave exposure do not limit the peak power of microwave pulses for general or occupational exposures. While some biological effects, primarily the auditory effect, depend on pulsed microwaves, hazards associated with very high peak-power microwave pulses in the absence of whole-body heating are unknown. Five rhesus monkeys, Macaca mulatta, were exposed to peak-power densities of 131.8 W/cm2 (RMS) while performing a time-related behavioral task. The task was composed of a multiple schedule of reinforcement consisting of three distinct behavioral components: inter-response time, time discrimination, and fixed interval. Trained monkeys performed the multiple schedule during exposure to 1.3-GHz pulses at low pulse-repetition rates (2-32 Hz). No significant change was observed in any behavior during irradiation as compared to sham-irradiation sessions. Generalization of these findings to experimental results with higher peak-power densities, other pulse rates, different carrier frequencies, or other behaviors is limited.