The effect of non-contingent negative and positive reinforcement operations on the acquisition of superstitious behaviors

The differential effects of non-contingent positive and negative reinforcement operations on the acquisition of superstitious behaviors and rules were investigated in two experiments. College students were instructed to try to produce and/or keep the word "GOOD" on a computer screen (positive reinforcement), or to try to prevent and/or remove the word "BAD" from the screen (negative reinforcement) using response keys. Data from both experiments indicated that participants exposed to lean schedules of negative reinforcers believed that they had greater control over non-contingent stimulus events than participants exposed to either rich or lean schedules of positive reinforcers. These findings and results from research investigating everyday superstitious activities suggest that, relative to positive reinforcement, negative reinforcement operations may provide a more fertile condition for the development and maintenance of superstitious behaviors.     

Superstitious responding and reinforcement rate under concurrent variable-interval extinction schedules

To examine superstitious responding, four pigeons key pecked under multiple concurrent variable-interval 45 s variable-interval 90 s concurrent variable-interval 90 s variable-interval 180 s schedules in the absence of a changeover delay. The two variable-interval 90 s schedules then were replaced by extinction, and key-peck responding during extinction was examined as a function of the prevailing reinforcement rate. During the first several sessions, extinction-key responding was maintained closer to baseline levels in the presence of the higher reinforcement rate, and this effect dissipated or even reversed with continued exposure to extinction. Although extinction-key responding generally decreased to near-zero levels after several sessions, in a few instances, it continued for 30 and 45 sessions. These results demonstrate how concurrent variable-interval extinction schedules can be used to investigate what often has been labeled superstitious responding.     

An Assessment of Fixed Interval Timing in Free-Flying Honey Bees (Apis mellifera ligustica): An Analysis of Individual Performance

Interval timing is a key element of foraging theory, models of predator avoidance, and competitive interactions. Although interval timing is well documented in vertebrate species, it is virtually unstudied in invertebrates. In the present experiment, we used free-flying honey bees (Apis mellifera ligustica) as a model for timing behaviors. Subjects were trained to enter a hole in an automated artificial flower to receive a nectar reinforcer (i.e. reward). Responses were continuously reinforced prior to exposure to either a fixed interval (FI) 15-sec, FI 30-sec, FI 60-sec, or FI 120-sec reinforcement schedule. We measured response rate and post-reinforcement pause within each fixed interval trial between reinforcers. Honey bees responded at higher frequencies earlier in the fixed interval suggesting subject responding did not come under traditional forms of temporal control. Response rates were lower during FI conditions compared to performance on continuous reinforcement schedules, and responding was more resistant to extinction when previously reinforced on FI schedules. However, no “scalloped” or “break-and-run” patterns of group or individual responses reinforced on FI schedules were observed; no traditional evidence of temporal control was found. Finally, longer FI schedules eventually caused all subjects to cease returning to the operant chamber indicating subjects did not tolerate the longer FI schedules.     

"The stone which the builders rejected...": Delay of reinforcement and response rate on fixed-interval and related schedules

The article deals with response rates (mainly running and peak or terminal rates) on simple and on some mixed-FI schedules and explores the idea that these rates are determined by the average delay of reinforcement for responses occurring during the response periods that the schedules generate. The effects of reinforcement delay are assumed to be mediated by a hyperbolic delay of reinforcement gradient. The account predicts that (a) running rates on simple FI schedules should increase with increasing rate of reinforcement, in a manner close to that required by Herrnstein's equation, (b) improving temporal control during acquisition should be associated with increasing running rates, (c) two-valued mixed-FI schedules with equiprobable components should produce complex results, with peak rates sometimes being higher on the longer component schedule, and (d) that effects of reinforcement probability on mixed-FI should affect the response rate at the time of the shorter component only. All these predictions were confirmed by data, although effects in some experiments remain outside the scope of the model. In general, delay of reinforcement as a determinant of response rate on FI and related schedules (rather than temporal control on such schedules) seems a useful starting point for a more thorough analysis of some neglected questions about performance on FI and related schedules.     

Choice in humans: Functional properties of reinforcers established by instruction

Adult human subjects chose between schedules containing stimuli (indicator lights) that the subjects were instructed to consider pleasurable. The schedules differed in amount of reinforcement (period of illumination) or delay (interval between a choice response and light onset). Although subjects preferred large to small amounts of reinforcement, they were essentially indifferent between immediate and delayed reinforcement. In contrast, previous data on video game reinforcement demonstrated preferences for both immediate and large amounts of reinforcement. The instructed reinforcer was thus partially effective in controlling choice but was not equivalent to a reinforcer that presumably had intrinsic value.     

Does the terminal-link effect depend on duration or reinforcement rate?

Preference in concurrent chains for the richer terminal-link schedule becomes more extreme as the schedule values increase with their ratio held constant, a result known as the terminal-link effect. We report two experiments that attempt to determine whether this effect is related to terminal-link duration or the overall rate of reinforcement. These variables have been confounded in prior studies, but can be separated by comparing variable-duration schedules that end after a single reinforcer has been earned, with constant-duration schedules during which a variable number of reinforcers may be earned. In Experiment 1, the terminal-link effect was obtained with variable-duration schedules when duration and overall reinforcement rate were manipulated, but not with constant-duration schedules when overall reinforcement rate was changed with duration held constant. In Experiment 2, the terminal-link effect was obtained with constant-duration schedules when duration was manipulated with overall reinforcement rate held constant. Taken together, these results show that the terminal-link effect depends on changes in terminal-link duration, not overall reinforcement rate (or equivalently, average time to reinforcement). This accords with the account of the terminal-link effect provided by the contextual choice model [J. Exp. Anal. Behav. 61 (1994) 113] but not delay-reduction theory [J. Exp. Anal. Behav. 12 (1969) 723].     

Reinforcement schedules and extrapolations to humans from animals in behavioral pharmacology.

Behavior controlled by various schedules of reinforcement is useful for characterizing drugs as well as for analyzing the mechanisms of action of their effects on behavior. Conditioned avoidance techniques have been useful for studying neuroleptics and for predicting their clinical antipsychotic acitivity; the possible involvement of dopaminergic mechanisms in the effect of neurolpetics on avoidance behavior is discussed. Tricyclic antidepressant agents have been studied in assays involving interactions with other agents, such as cocaine, amphetamine and tetrabenazine. One type of operant behavior, Sidman avoidance, has been used as particularly sensitive assay for such drug interactions. Another schedule, in which "observing" responses in pigeons are measured. seems to provide a method for studying antidepressants without involving drug interaction phenomena. For tricyclic compounds, facilitation of observing responses and weak potency of conditioned avoidance inhibition constitute a pharmacological profile that seems to have some predictive value for clinical imipramine-like antidepressant activity. "Conflict (punishment) schedules have been useful for predicting antianxiety activity in man. Although the degree of anticonflict effect observed is consistent with Dew's rate dependency hypothesis, this principle does not fully account for the observed drug effects. In the conflict model, the actions of benzodiazepines differ in drug-naive versus drug-experienced animals. Experiments with parachlorophenylalnine have not yet provided clear support for the postulated role of serotonin in related phenomena.     

Energy budget versus temporal discounting as determinants of preference in risky choice

Four pigeons and three ringneck doves responded on an operant simulation of natural foraging. After satisfying a schedule of reinforcement associated with search time, subjects could "accept" or "reject" another schedule of reinforcement associated with handling time. Two schedules of reinforcement were available, a variable interval, and a fixed interval with the same mean value. Food available in the session (a variable related to the energy budget) was manipulated in the different conditions either by increases of the value of the search state schedule of reinforcement, or by increases in the mean value of the handling state schedules. The results indicate that the amount of food available in the session did not affect the preference for variable schedules of reinforcement, as would be predicted by an influential theory of risk sensitive foraging. Instead, the preference for variability depended on the relationship between the time spent in the search and the handling states, as is predicted by a family of models of choice that are based on the temporal proximity to the reinforcer.     

Habituation and the reinforcing effectiveness of visual stimuli

The term "sensory reinforcer" has been used to refer to sensory stimuli (e.g. light onset) that are primary reinforcers in order to differentiate them from other more biologically important primary reinforcers (e.g. food and water). Acquisition of snout poke responding for a visual stimulus (5s light onset) with fixed ratio 1 (FR 1), variable-interval 1min (VI 1min), or variable-interval 6min (VI 6min) schedules of reinforcement was tested in three groups of rats (n=8/group). The VI 6min schedule of reinforcement produced a higher response rate than the FR 1 or VI 1min schedules of visual stimulus reinforcement. One explanation for greater responding on the VI 6min schedule relative to the FR 1 and VI 1min schedules is that the reinforcing effectiveness of light onset habituated more rapidly in the FR 1 and VI 1min groups as compared to the VI 6min group. The inverse relationship between response rate and the rate of visual stimulus reinforcement is opposite to results from studies with biologically important reinforcers which indicate a positive relationship between response and reinforcement rate. Rapid habituation of reinforcing effectiveness may be a fundamental characteristic of sensory reinforcers that differentiates them from biologically important reinforcers, which are required to maintain homeostatic balance.     

EMG Biofeedback: The Effects of CRF, FR, VR, FI, and VI Schedules of Reinforcement on the Acquisition and Extinction of Increases in Forearm Muscle Tension

Biofeedback was used to increase forearm-muscle tension. Feedback was delivered under continuous reinforcement (CRF), variable interval (VI), fixed interval (FI), variable ratio (VR), and fixed ratio (FR) schedules of reinforcement when college students increased their muscle tension (electromyograph, EMG) above a high threshold. There were three daily sessions of feedback, and Session 3 was immediately followed by a session without feedback (extinction). The CRF schedule resulted in the highest EMG, closely followed by the FR and VR schedules, and the lowest EMG scores were produced by the FI and VI schedules. Similarly, the CRF schedule resulted in the greatest amount of time-above-threshold and the VI and FI schedules produced the lowest time-above-threshold. The highest response rates were generated by the FR schedule, followed by the VR schedule. The CRF schedule produced relatively low response rates, comparable to the rates under the VI and FI schedules. Some of the data are consistent with the partial-reinforcement-extinction effect. The present data suggest that different schedules of feedback should be considered in muscle-strengthening contexts such as during the rehabilitation of muscles following brain damage or peripheral nervous-system injury.     

Oscillations following periodic reinforcement

Three experiments examined behavior in extinction following periodic reinforcement. During the first phase of Experiment 1, four groups of pigeons were exposed to fixed interval (FI 16 s or FI 48 s) or variable interval (VI 16 s or VI 48 s) reinforcement schedules. Next, during the second phase, each session started with reinforcement trials and ended with an extinction segment. Experiment 2 was similar except that the extinction segment was considerably longer. Experiment 3 replaced the FI schedules with a peak procedure, with FI trials interspersed with non-food peak interval (PI) trials that were four times longer. One group of pigeons was exposed to FI 20 s PI 80 s trials, and another to FI 40 s PI 160 s trials. Results showed that, during the extinction segment, most pigeons trained with FI schedules, but not with VI schedules, displayed pause-peck oscillations with a period close to, but slightly greater than the FI parameter. These oscillations did not start immediately after the onset of extinction. Comparing the oscillations from Experiments 1 and 2 suggested that the alternation of reconditioning and re-extinction increases the reliability and earlier onset of the oscillations. In Experiment 3 the pigeons exhibited well-defined pause-peck cycles since the onset of extinction. These cycles had periods close to twice the value of the FI and lasted for long intervals of time. We discuss some hypotheses concerning the processes underlying behavioral oscillations following periodic reinforcement.     

Avoidance and escape responses of the gastropod mollusc Olivella biplicata (Sowerby) to predatory asteroids

The sand-dwelling gastropod Olivella biplicata (Sowerby) gives two chemically-mediated defensive responses to predatory asteroids — an avoidance and an escape response. An avoidance response is triggered by water-borne chemicals diffusing from distant starfish. When snails on the surface of the sand sense a distant predatory starfish, they avoid the predator by burying themselves in the sand. Strong avoidance responses are given to the predatory asteroids Pisaster brevispinus (Stimpson), P. ochraceus (Brandt), and Pycnopodia helianthoides (Brandt). Little or no response is given to the omnivorous asteroid Patiria miniata (Brandt) or to the predatory gastropod Polinices lewisii (Gould). An escape response is given when a snail on the surface is contacted by the starfish Pisaster brevispinus. Upon contact, the snail immediately retracts the propodium, sometimes throwing the metapodium forward at the same time; the snail then turns sharply away from the point of contact, and buries itself. Both avoidance and escape responses can be observed in the field and laboratory.     

Simple and multiple schedule responding and behavioral contrast when pigeons press treadles

Positive behavioral contrast has been observed when pigeons press treadles on multiple schedules for high rates of reinforcement, but not for low rates. Negative treadle-press contrast has been observed for low rates of reinforcement. Two experiments showed that differences between response rates emitted during simple and multiple schedules appear and fail to appear under similar conditions. The experiments showed that the rate of pressing during the less favorable component of a multiple schedule was less than the rate of pressing during a comparable simple schedule (negative contrast). The rate of treadle–pressing during the more favorable component was not greater than the rate of pressing during a comparable simple schedule, when the schedules provided a low rate of reinforcement (absence of positive contrast), but it was when the schedules provided a high rate of reinforcement (positive contrast). These results help to clarify the definition of behavioral contrast by showing that simple schedules may be appropriate baselines from which to define and measure contrast.     

Choice behavior in spontaneously hypertensive rats: Variable vs. fixed schedules of reinforcement

Spontaneously hypertensive rats (SHR) and Wistar rats were evaluated in the successive-encounters procedure (an operant simulation of natural foraging) with the idea of assessing differences between them in their preference for variable schedules of reinforcement. In this procedure, after satisfying a schedule of reinforcement associated with search time, the subjects could “accept” or “reject” another schedule of reinforcement associated with handling time. Two schedules of reinforcement were available: a fixed interval (FI), and a variable interval (VI) with the same mean value. The results indicated preference for the variable schedule in both strains, as suggested by the observation that the VI was always accepted while the FI was often rejected. The difference in FI acceptability between strains was not statistically significant, a result which is relevant for the current debate of SHR as an adequate animal model of Attention Deficit Hyperactivity Disorder.     

A further study of choice and percentage reinforcement

In Experiment I four pigeons were trained in a concurrent chains procedure with fixed-ratio schedules (FR1) in the initial components and fixed-time schedules in the terminal components. Pecking one of the keys when both keys were white initiated a fixed time schedule on that key. A peck to the left key produced three stripes on the key. At the termination of the fixed-time component food always occurred. Pecking the other key produced either a circle or a triangle. If a circle appeared, reinforcement occurred. If a triangle appeared a brief timeout was given. Initially the stripes appeared on the left key and the circle and triangle on the left. This was reversed during the course of the experiment. In addition, sessions were conducted in which both circle and triangle sometimes preceded reinforcement and sometimes timeout. For most birds under most conditions there was a preference for the key that produced the circle and triangle. When these were uncorrelated with reinforcement and time out three of the birds preferred the key producing 100% reinforcement.

In Experiment II three factors were varied and VI 20 sec schedules were used in the initial links instead of FR1. The results showed that pigeons preferred the 50% condition more 1) the greater the duration of the terminal links, 2) the smaller the value on the initial link VI schedules and 3) the less the probability of food in the terminal link with stripes on the key.     

Study on the optimum number of trials and intensity of unconditioned stimulants and strain difference in the two-way shuttle-box avoidance test using rats

In order to determine the optimun conditions suitable for a number of trials and the intensity of unconditioned stimulant (US) in the two-way shuttle-box avoidance test in Sprague-Dawley strain rats, which are used most frequently in reproduction studies, conditioned avoidance response was observed under various conditions for 30 and 60 trials and the low and high US levels. Investigation was also conducted in Wistar rats under a high US level with 30 and 60 trials. Latency time of the escape response in Sprague-Dawley rats was shortened with increasing trials. Body weight gains of both strains of rats in the high US level with the 60-trial group decreased during the observation period. These findings suggest that the high US level with the 60-trial group is not suitable for the two-way shuttle-box avoidance test. The rate and latency time of the avoidance response were lower in Wistar rats than in Sprague-Dawley rats, although those of the escape response were higher. Significant changes in the following were observed, mainly from first to third sessions: the avoidance rate of all groups in strains of rats, escape rate of 60-trial group in Sprague-Dawley rats, avoidance and escape latency time of the 60-trial groups in both strains of rats and escape latency time of the 30-trial group in Sprague-Dawley strain rats.     

Postnatal development of conditioned reflex behavior: comparison of the periods related to the onset of the different forms of hippocampal plasticity in rats

Fear conditioning, escape and active avoidance reactions in two-way avoidance paradigm were compared in rats of different ages. Fear conditioning, but not escape and active avoidance reactions could be acquired on the 16-17th postnatal days, and the acquisition was more effective than in adults. Escape behavior matured beginning from the 18th postnatal day reaching the adult level within the 3d-4th postnatal weeks. Maturation of the mechanisms of Pavlovian (fear reaction) and instrumental (escape reaction) conditioning did not facilitate the acquisition of two-way avoidance until the 4th postnatal week, young animals displayed low acquisition in this period. The maturation of these memory processes is proposed to be related to developmental stages of different mechanisms of hippocampal plasticity.     

Mathematical principles of reinforcement and resistance to change

Although Killeen's mathematical principles of reinforcement (MPR) apply to the asymptotic rate of a free operant after extended exposure to a single schedule of reinforcement, they can be extended to resistance to change in multiple schedules via alterations in the parameter representing the activating effects of reinforcers. MPR's predictions of resistance to change in relation to reinforcer rate on variable-interval (VI) schedules are empirically correct and agree with behavioral momentum theory (BMT). However, both MPR and BMT encounter problems in accounting for the effects of delayed reinforcement on resistance to change, relative to immediate reinforcement at the same rate. Further problems are raised by differences in resistance to change between variable-ratio (VR) and variable-interval performances maintained by the same reinforcer rate. With both delayed versus immediate reinforcement and variable-ratio versus variable-interval reinforcement, differential resistance to change is negatively correlated with the log ratios of baseline response rates when reinforcer rates are equated. Cases where resistance to change varies despite equated reinforcer rates, and the correlations among behavioral measures, provide challenges and opportunities for both MPR and BMT.     

The effect of training conditions on passive avoidance performance in rats

Male outbred rats were trained for step-through avoidance in different experimental conditions: 1) in a two-compartment chamber with a small dark compartment and a chamber with illuminated suspended platform; 2) with or without acquaintance with experimental chamber 24 hours prior to learning; 3) with or without a possibility of the chamber exploration 3 min immediately before training procedure; 4) with inescapable or escapable pain reinforcement (5 electric footshocks, 0.45 mA, 1 s, with 2-s between-shock intervals); 5) with 2 or 5 inescapable footshocks of the same rate, duration, and intensity. The acquired performance was tested 24 h after training. It was shown that the procedure of exploration of the experimental chamber 24 prior to training improved the reproduction and the same procedure conducted immediately before training impaired the reproduction of the acquired behavior. Different training conditions improved the reproduction of the acquired behavior in the chamber with a suspended platform to a greater extent than in the two-compartment chamber. The decrease in the number of pain stimuli from 5 to 2 or the possibility to escape the punishment during training did not significantly change the reproduction.     

The Parasite Moniliformis moniliformis Alters the Escape Response of its Cockroach Host Periplaneta americana

The immature stages of some parasites live in prey animals (intermediate hosts) and only reach reproductive maturity when they are eaten by final host predators. Some of these parasites alter intermediate host behavior in ways that increase the likelihood of predation (parasite transmission). Using the acanthocephalan (Moniliformis moniliformis) in the cockroach (Periplaneta americana), we show that infected cockroaches experienced a decrease in wind-evoked escape responses, a predator avoidance behavior, that consisted of fewer escape responses, longer latency and higher threshold for escape behavior. We found no correlates of the impairment of the escape behavior in the abdominal portion of the escape neuronal circuitry. This study demonstrates a parasite-induced alteration of a behavior dedicated to predator avoidance.