Non-contingent positive and negative reinforcement schedules of superstitious behaviors

The role of schedules of reinforcement on the development of superstitious conditioning was investigated in a college age population. Participants were randomly assigned to one of eight operant schedules and instructed to remove (escape), prevent and/or remove (avoidance and escape) or produce (positive) the appearance of a computer generated stimulus using a response pad. Results from the experiment indicate that concomitant (escape and avoidance) schedules of reinforcement are most effective in facilitating acquisition of superstitious behavior as measured by self-reports of participants.     

Habituation and the reinforcing effectiveness of visual stimuli

The term "sensory reinforcer" has been used to refer to sensory stimuli (e.g. light onset) that are primary reinforcers in order to differentiate them from other more biologically important primary reinforcers (e.g. food and water). Acquisition of snout poke responding for a visual stimulus (5s light onset) with fixed ratio 1 (FR 1), variable-interval 1min (VI 1min), or variable-interval 6min (VI 6min) schedules of reinforcement was tested in three groups of rats (n=8/group). The VI 6min schedule of reinforcement produced a higher response rate than the FR 1 or VI 1min schedules of visual stimulus reinforcement. One explanation for greater responding on the VI 6min schedule relative to the FR 1 and VI 1min schedules is that the reinforcing effectiveness of light onset habituated more rapidly in the FR 1 and VI 1min groups as compared to the VI 6min group. The inverse relationship between response rate and the rate of visual stimulus reinforcement is opposite to results from studies with biologically important reinforcers which indicate a positive relationship between response and reinforcement rate. Rapid habituation of reinforcing effectiveness may be a fundamental characteristic of sensory reinforcers that differentiates them from biologically important reinforcers, which are required to maintain homeostatic balance.     

Momentary and global maximizing

Four adult females responded at a computer console, on three constant probability concurrent variable-interval reinforcement schedules. The subjects were instructed to try to obtain as many reinforcers as possible, but were not given any instructions on how to accomplish this task. Three of the four subjects typically allocated responses to the schedule offering the higher reinforcement probability. These results show that some subjects trying to maximize overall reinforcement may respond in accordance with a momentary maximizing strategy.     

Effects of intertrial reinforcers on rats' timing behavior

Two experiments were conducted to assess the effects of non-contingent intertrial interval (ITI) reinforcers on rats' discrimination of duration. In the first experiment, rats' discrimination of a 2 vs. 8 s of light was significantly disrupted when reinforcers were presented in the ITI. Disruption was not different on short (2 s) and long (8 s) trials. The second experiment showed that this disruptive effect was not specific to trials preceded by ITI reinforcers; responding on empty ITI trials run in the same session as ITI-reinforcer trials was also disrupted. This disruption however was not as great as on the ITI-reinforcer trials. The results of these experiments show that ITI reinforcers affect timing discriminations in much the same way they affect classical conditioning and delayed matching to sample. However, detailed examination of the results suggests that the deleterious effects of ITI reinforcers in these different paradigms might be produced by different rather than the same mechanism. The results also support the conclusion that pre-trial reinforcement “priming” produces disruption rather than facilitation in complex tasks.     

Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement

Across two experiments, a peak procedure was used to assess the timing of the onset and offset of an opportunity to run as a reinforcer. The first experiment investigated the effect of reinforcer duration on temporal discrimination of the onset of the reinforcement interval. Three male Wistar rats were exposed to fixed-interval (FI) 30-s schedules of wheel-running reinforcement and the duration of the opportunity to run was varied across values of 15, 30, and 60s. Each session consisted of 50 reinforcers and 10 probe trials. Results showed that as reinforcer duration increased, the percentage of postreinforcement pauses longer than the 30-s schedule interval increased. On probe trials, peak response rates occurred near the time of reinforcer delivery and peak times varied with reinforcer duration. In a second experiment, seven female Long-Evans rats were exposed to FI 30-s schedules leading to 30-s opportunities to run. Timing of the onset and offset of the reinforcement period was assessed by probe trials during the schedule interval and during the reinforcement interval in separate conditions. The results provided evidence of timing of the onset, but not the offset of the wheel-running reinforcement period. Further research is required to assess if timing occurs during a wheel-running reinforcement period.     

Modification of instinctive herding dog behavior using reinforcement and punishment

Abstract

“Instinctive” behavior may be modified using operant techniques. We report here on a field study of training herding dogs in which reinforcers and punishers were used by owners, who were themselves being trained to control their dogs. Access to sheep was assumed to be a primary reinforcer for herding dogs. While blocking their access was aversive to them. Over several months, the number of blocking and access actions by the human were scored during the training of seven naïve herding dogs. We found that rates of punishment by blocking the dog's access to sheep or by stopping the dog occurred at higher levels than positive reinforcement from access or verbal praise. While positive reinforcement can be used exclusively for the training of certain behaviors, it is suggested that in the context of instinctive motor patterns, negative reinforcement and punishment may be desirable and necessary additions to positive reinforcement techniques.     

Response-reinforcer relations and resistance to change

Behavioral momentum theory suggests that the relation between a response and a reinforcer (i.e., response-reinforcer relation) governs response rates and the relation between a stimulus and a reinforcer (i.e., stimulus-reinforcer relation) governs resistance to change. The present experiments compared the effects degrading response-reinforcer relations with response-independent or delayed reinforcers on resistance to change in conditions with equal stimulus-reinforcer relations. In Experiment 1, pigeons responded on equal variable-interval schedules of immediate reinforcement in three components of a multiple schedule. Additional response-independent reinforcers were available in one component and additional delayed reinforcers were available in another component. The results showed that resistance to disruption was greater in the components with added reinforcers than without them (i.e., better stimulus-reinforcer relations), but did not differ for the components with added response-independent and delayed reinforcement. In Experiment 2, a component presenting immediate reinforcement alternated with either a component that arranged equal rates of reinforcement with a proportion of those reinforcers being response independent or a component with a proportion of the reinforcers being delayed. Results showed that resistance to disruption tended to be either similar across components or slightly lower when response-reinforcer relations were degraded with either response-independent or delayed reinforcers. These findings suggest that degrading response-reinforcer relations can impact resistance to change, but that impact does not depend on the specific method and is small relative to the effects of the stimulus-reinforcer relation.     

Operant behavior in dwarf hamsters (Phodopus campbelli): Effects of rate of reinforcement and reinforcer flavor variety

We investigated operant behavior in a novel species, the dwarf hamster (Phodopus campbelli). In two experiments, hamsters were trained to lever-press for food reinforcement. In Experiment 1, rate of reinforcement was manipulated across conditions using four variable-interval schedules of reinforcement (delivering one to eight reinforcers per min). As predicted, within-session decreases in responding were steepest on the richest schedule. In Experiment 2, lever-pressing was reinforced by either a constant or a variety of flavored food pellets. Within-session decreases in responding were steeper when the reinforcer flavor remained constant than when it was varied within the session. In both experiments, subjects hoarded most reinforcers in their cheek pouches rather than consuming them in the operant chambers. These results are incompatible with post-ingestive satiety variables as explanations for within-session decreases in operant responding and suggest that habituation to repeatedly presented reinforcers best accounts for subjects’ response patterns. Additionally, a mathematical model that describes behavior undergoing habituation also described the present results, thus strengthening the conclusion that habituation mediates the reinforcing efficacy of food.     

Superstitious responding and reinforcement rate under concurrent variable-interval extinction schedules

To examine superstitious responding, four pigeons key pecked under multiple concurrent variable-interval 45 s variable-interval 90 s concurrent variable-interval 90 s variable-interval 180 s schedules in the absence of a changeover delay. The two variable-interval 90 s schedules then were replaced by extinction, and key-peck responding during extinction was examined as a function of the prevailing reinforcement rate. During the first several sessions, extinction-key responding was maintained closer to baseline levels in the presence of the higher reinforcement rate, and this effect dissipated or even reversed with continued exposure to extinction. Although extinction-key responding generally decreased to near-zero levels after several sessions, in a few instances, it continued for 30 and 45 sessions. These results demonstrate how concurrent variable-interval extinction schedules can be used to investigate what often has been labeled superstitious responding.     

Simple and multiple schedule responding and behavioral contrast when pigeons press treadles

Positive behavioral contrast has been observed when pigeons press treadles on multiple schedules for high rates of reinforcement, but not for low rates. Negative treadle-press contrast has been observed for low rates of reinforcement. Two experiments showed that differences between response rates emitted during simple and multiple schedules appear and fail to appear under similar conditions. The experiments showed that the rate of pressing during the less favorable component of a multiple schedule was less than the rate of pressing during a comparable simple schedule (negative contrast). The rate of treadle–pressing during the more favorable component was not greater than the rate of pressing during a comparable simple schedule, when the schedules provided a low rate of reinforcement (absence of positive contrast), but it was when the schedules provided a high rate of reinforcement (positive contrast). These results help to clarify the definition of behavioral contrast by showing that simple schedules may be appropriate baselines from which to define and measure contrast.     

Does the terminal-link effect depend on duration or reinforcement rate?

Preference in concurrent chains for the richer terminal-link schedule becomes more extreme as the schedule values increase with their ratio held constant, a result known as the terminal-link effect. We report two experiments that attempt to determine whether this effect is related to terminal-link duration or the overall rate of reinforcement. These variables have been confounded in prior studies, but can be separated by comparing variable-duration schedules that end after a single reinforcer has been earned, with constant-duration schedules during which a variable number of reinforcers may be earned. In Experiment 1, the terminal-link effect was obtained with variable-duration schedules when duration and overall reinforcement rate were manipulated, but not with constant-duration schedules when overall reinforcement rate was changed with duration held constant. In Experiment 2, the terminal-link effect was obtained with constant-duration schedules when duration was manipulated with overall reinforcement rate held constant. Taken together, these results show that the terminal-link effect depends on changes in terminal-link duration, not overall reinforcement rate (or equivalently, average time to reinforcement). This accords with the account of the terminal-link effect provided by the contextual choice model [J. Exp. Anal. Behav. 61 (1994) 113] but not delay-reduction theory [J. Exp. Anal. Behav. 12 (1969) 723].     

Some effects of overall rate of earning reinforcers on run lengths and visit durations

In a concurrent schedule, responding at each alternative is controlled by a pair of schedules that arrange reinforcers for staying at that alternative and reinforcers for switching to the other alternative. Each pair of schedules operates only while at the associated alternative. When only one pair of stay and switch schedules is presented, the rates of earning reinforcers for staying divided by the rates of earning reinforcers for switching controls the mean number responses in a visit and the mean duration of visits. The purpose of the present experiment was to see whether the sum of the rates of earning stay and switch reinforcers changed the way that run length and visit duration were affected by the ratio of the rates of stay to switch reinforcers. Rats were exposed to pairs of stay and switch schedules that varied both the ratio of the rates of earning stay and switch reinforcers and the sum of the rates of earning stay and switch reinforcers. Run lengths and visit durations were joint functions of the ratio of the rates of earning stay and switch reinforcers and the sum of the rates of earning stay and switch reinforcers. These results shows that the effect of the ratio of the sum of the rates of earning stay and switch reinforcers results from processes operating at the alternative, rather than from processes operating at both alternatives.     

Is extinction the hallmark of operant discrimination?: Reinforcement and S effects

Using a successive discrimination procedure with rats, three experiments investigated the contribution of reinforcement rate and amount of S(Delta) exposure on the acquisition of an operant discrimination. S(D) components and were always 2 min in length, while S(Delta) (extinction) components were either 1 min or 4 min in length; responses in S(D) were reinforced on one of four schedules. In Experiment 1, each of eight groups were exposed to one possible combination of rate of reinforcement and S(Delta) component length. At every level of reinforcement, the 4 min S(Delta) groups acquired the discrimination more quickly. However, within each level of reinforcement, the proportions of responding in S(D) as a function cumulative S(Delta) exposure were equivalent, regardless of the number of reinforcers earned in S(D), suggesting that extinction is the "hallmark" of discrimination. Experiment 2 sought to replicate these results in a within-subjects design, and although the 4 min S(Delta) conditions always produced superior discriminations, the lack of discriminated responding in some conditions suggested that stimulus disparity was reduced. Experiment 3 clarified those results and extended the finding that the acquisition of operant discrimination closely parallels extinction of responding in S(Delta). In sum, it appears that higher reinforcement rates and longer S(Delta) exposure facilitate the acquisition of discriminated operant responding.     

Maximizing the effectiveness of environmental enrichment: Suggestions from the experimental analysis of behavior

Environmental enrichment programs provide benefits to both captive animals and the facilities that house them, but cost time and resources to design, implement, and maintain. As yet, there have been few theoretically based guidelines to assist animal care staff in establishing cost-efficient enrichment methods that both elicit the desired behavioral changes and maintain their success over time. We describe several well-studied principles from the field of experimental analysis of behavior, including intrinsic reinforcement, extrinsic reinforcement, habituation, extinction, and schedules of reinforcement that could be very useful for evaluating the short- and long-term effectiveness of enrichment. We use this theoretical framework to generate testable hypotheses and provide examples of enrichment studies relevant to our predictions. In particular, we suggest that enrichment devices that offer extrinsic reinforcement (food, social access, etc. as a result of performing behaviors) should produce greater and more prolonged changes in behavior than devices that rely on the behavior itself being reinforcing to the animal. For techniques that provide no extrinsic reinforcement, using stimuli that are novel, are more different from the environment, have been withheld or altered in some way, or are presented less frequently may help reduce habituation. For techniques that provide extrinsic reinforcement, making reinforcement more difficult to obtain and providing more or higher quality reinforcers may increase the long-term success of the enrichment program. In addition, enrichment may be more effective if animal care staff avoid continuously reinforcing behaviors after they are established, enriching immediately after feeding, and exposing animals to enrichment when reinforcement is no longer available. While the current enrichment literature supports the application of behavior analytic theory, empirical evaluation of many of our predictions is still needed.     

Varying reinforcer duration produces behavioral interactions during multiple schedules

The experiments tested the idea that changes in habituation to the reinforcer contribute to behavioral interactions during multiple schedules. This idea predicts that changing an aspect of the reinforcer should disrupt habituation and produce an interaction. Pigeons and rats responded on multiple variable interval variable interval schedules. Introducing variability into the duration of reinforcers in one component increased response rates in both components when the schedules provided high, but not low, rates of reinforcement. The increases in constant-component response rates grew larger as the session progressed. Within-session decreases in responding were smaller when the other component provided variable-, rather than fixed-, duration reinforcers. These results are consistent with the idea that changes in habituation to the reinforcer contribute to behavioral interactions. They help to explain why interactions do not occur for some subjects under conditions that produce them for others. Finally, the results question the assumption that induction and behavioral contrast are always produced by different theoretical mechanisms.     

Influences of rewarding and aversive outcomes on activity in macaque lateral prefrontal cortex

Both appetitive and aversive outcomes can reinforce animal behavior. It is not clear, however, whether the opposing kinds of reinforcers are processed by specific or common neural mechanisms. To investigate this issue, we studied macaque monkeys that performed a memory-guided saccade task for three different outcomes, namely delivery of liquid reward, avoidance of air puff, and feedback sound only. Animals performed the task best in rewarded trials, intermediately in aversive trials, and worst in sound-only trials. Most task-related activity in lateral prefrontal cortex was differentially influenced by the reinforcers. Aversive avoidance had clear effects on some prefrontal neurons, although the effects of rewards were more common. We also observed neurons modulated by both positive and negative reinforcers, reflecting reinforcement or attentional processes. Our results demonstrate that information about positive and negative reinforcers is processed differentially in prefrontal cortex, which could contribute to the role of this structure in goal-directed behavior.     

The significance of unrecognized feedback stimuli in cognitive activity

In adult emotionally excitable persons the role was studied of unrecognized reinforcement forms in the function of differentiation of time microintervals. As feedback stimuli the word "good" (positive reinforcement) and a word connected with negative emotions of the subject (negative reinforcement) were applied. Experimental confirmation was obtained of the hypothesis that unrecognized phenomena of environment can influence conscious psychic activity, the process of man learning. Unrecognized semantic stimuli can function as a reinforcing factor and in this way participate in the process of learning of cognitive activity realized at conscious level.     

Re-Examining the Automaticity and Directionality of the Activation of the Spatial-Valence "Good is Up" Metaphoric Association

According to the Conceptual Metaphor Theory, people understand abstract concepts depending on the activation of more concrete concepts, but not vice versa. The present research aims to investigate the role of directionality and automaticity regarding the activation of the conceptual metaphor “good is up”. Experiment 1 tested the automaticity of the spatial-to-valence metaphoric congruency effect by having participants judge the valence of a positive or negative word that appeared either at the top or at the bottom of the screen. They performed the task concurrently with a 6-digit verbal rehearsal task in the working-memory-load (WML) blocks and without this task in the non-WML blocks. The spatial-to-valence metaphoric congruency effect occurred for the positive words in the non-WML blocks (i.e., positive words are judged more quickly when they appeared at the top than at the bottom of the screen), but not in the WML blocks, suggesting that this metaphoric association might not be activated automatically. Experiments 2-6 investigated the valence-to-spatial metaphoric association and its automaticity. Participants processed a positive or negative prime, which appeared at the center of the screen, and then identified a letter (p/q) that subsequently appeared at the top or bottom of the screen. The valence-to-spatial metaphoric congruency effect did not occur in the WML (6-digit verbal rehearsal) or non-WML blocks, whether response modality to the prime was key-press or vocal, or whether the prime was a word or a picture. The effect only unexpectedly occurred when the task was simultaneously performed with a 4-dot-position visuospatial rehearsal task. Nevertheless, the data collapsed across multiple experiments showed a null valence-to-spatial metaphoric congruency effect, suggesting the absence of the valence-to-spatial metaphoric association in general. The implications of the current findings for the Conceptual Metaphor Theory and its alternatives are discussed.     

Response-cost punishment via token loss with pigeons

Two experiments were conducted to investigate punishment via response-contingent removal of conditioned token reinforcers (response cost) with pigeons. In Experiment 1, key pecking was maintained on a two-component multiple second-order schedule of token delivery, with light emitting diodes (LEDs) serving as token reinforcers. In both components, responding produced tokens according to a random-interval 20-s schedule and exchange periods according to a variable-ratio schedule. During exchange periods, each token was exchangeable for 2.5-s access to grain. In one component, responses were conjointly punished according to fixed-ratio schedules of token removal. Response rates in this punishment component decreased to low levels while response rates in the alternate (no-punishment) component were unaffected. Responding was eliminated when it produced neither tokens nor exchange periods (Extinction), but was maintained at moderate levels when it produced tokens in the signaled absence of food reinforcement, suggesting that tokens served as effective conditioned reinforcers. In Experiment 2, the effect of the response-cost punishment contingency was separated from changes in the density of food reinforcement. This was accomplished by yoking either the number of food deliveries per component (Yoked Food) or the temporal placement of all stimulus events (tokens, exchanges, food deliveries) (Yoked Complete), from the punishment to the no-punishment component. Response rates decreased in both components, but decreased more rapidly and were generally maintained at lower levels in the punishment component than in the yoked component. In showing that the response-cost contingency had a suppressive effect on responding in addition to that produced by reductions in reinforcement density, the present results suggest that response-cost punishment shares important features with other forms of punishment.     

An Assessment of Fixed Interval Timing in Free-Flying Honey Bees (Apis mellifera ligustica): An Analysis of Individual Performance

Interval timing is a key element of foraging theory, models of predator avoidance, and competitive interactions. Although interval timing is well documented in vertebrate species, it is virtually unstudied in invertebrates. In the present experiment, we used free-flying honey bees (Apis mellifera ligustica) as a model for timing behaviors. Subjects were trained to enter a hole in an automated artificial flower to receive a nectar reinforcer (i.e. reward). Responses were continuously reinforced prior to exposure to either a fixed interval (FI) 15-sec, FI 30-sec, FI 60-sec, or FI 120-sec reinforcement schedule. We measured response rate and post-reinforcement pause within each fixed interval trial between reinforcers. Honey bees responded at higher frequencies earlier in the fixed interval suggesting subject responding did not come under traditional forms of temporal control. Response rates were lower during FI conditions compared to performance on continuous reinforcement schedules, and responding was more resistant to extinction when previously reinforced on FI schedules. However, no “scalloped” or “break-and-run” patterns of group or individual responses reinforced on FI schedules were observed; no traditional evidence of temporal control was found. Finally, longer FI schedules eventually caused all subjects to cease returning to the operant chamber indicating subjects did not tolerate the longer FI schedules.