Psychological distance to reward in monkeys

A concurrent-chains schedule was employed to examine crab-eating macaques' choice between segmented and unsegmented fixed-interval 30-s schedules. A stimulus change occurred during one of the fixed-interval schedules (segmented schedule), while no stimulus change occurred during the other (unsegmented schedule). The subjects were exposed to three consecutive stimulus-change conditions, in an attempt to explore whether correlation of the stimulus changes with a fixed interval of nonreinforcement affects the preference between the alternatives. When the stimulus change always occurred at a half point of the segmented schedule (the first and third conditions), all subjects preferred the fixed-interval schedule without stimulus changes. This replicated previous findings with pigeons and humans. On the other hand, when the intervals before and after the stimulus change varied on every entry to the segmented schedule with the sum of them being constant (the second condition), the subjects were indifferent to the alternatives. These results are consistent with the view that the preference for unsegmented schedules over segmented ones might be due to aversiveness which the initial component in the segmented schedule possesses as a result of its correlation with a fixed interval of nonreinforcement.     

Effect of signaling reinforcement on resistance to change in a multiple schedule

This study evaluated the effect of a signal on resistance to change using a multiple schedule of reinforcement. Experiment 1 presented pigeons with three schedules: a signaled delay to reinforcement schedule (a two-link chain schedule with a variable-interval 120-s initial link followed by a 5-s fixed-time schedule), an unsignaled delay schedule (a comparable two-link tandem schedule), and an immediate, zero-delay variable-interval 125-s schedule. Two separate disruption procedures assessed resistance to change: extinction and adding a variable-time 20-s schedule of reinforcement to the inter-component interval. Resistance to change tests were conducted twice, once with the signal stimulus (the terminal link of the chain schedule) present and once with it absent. Results from both disruption procedures showed that signal absence reduced resistance to change for the pre-signal stimulus. In probe choice tests subjects strongly preferred the signal stimulus over the unsignaled stimulus and exhibited no reliable preference when given a choice between the signal stimulus and immediate stimulus. Experiment 2 presented two equal signaled schedules where, during resistance to change tests, the signal remained for one schedule and was removed for the second. Resistance to change was consistently lower when the signal was absent.     

Segmentation and the pairing hypothesis

The effect of stimulus contiguity and response contingency on responding in chain schedules was examined in two experiments. In Experiment 1, four pigeons were trained on two simple three-link chain schedules that alternated within sessions. Initial links were correlated with a variable-interval 30s schedule, and middle and terminal links were correlated with interdependent variable-interval 30s variable-interval 30s schedules. The combined duration of the interdependent schedules summed to 60s. The two chains differed with respect to signaling of the schedule components: a two-stimulus chain had one stimulus paired with the initial link and one stimulus paired with both the middle and the terminal link, while a three-stimulus chain had a different stimulus paired with the each of the three links. The results showed that the two-stimulus chain maintained lower initial-link responding than the three-stimulus chain. In Experiment 2, four pigeons were exposed to three separate conditions, the two- and three-stimulus chains of Experiment 1 and a three-stimulus chain that had a 3s delay to terminal-link entry from the middle-link response that produced it. The two-stimulus chain maintained lower initial-link responding than the three-stimulus chain, as in Experiment 1, and a similar initial-link responding was maintained by the two-stimulus chain and the three-stimulus chain with the delay contingency. The results demonstrate that a stimulus noncontiguous with food can maintain responding that is sometimes greater than a stimulus contiguous with food, depending on the response contingency for terminal-link entry. The results are contrary to the pairing hypothesis of conditioned reinforcement.     

Habituation and the reinforcing effectiveness of visual stimuli

The term "sensory reinforcer" has been used to refer to sensory stimuli (e.g. light onset) that are primary reinforcers in order to differentiate them from other more biologically important primary reinforcers (e.g. food and water). Acquisition of snout poke responding for a visual stimulus (5s light onset) with fixed ratio 1 (FR 1), variable-interval 1min (VI 1min), or variable-interval 6min (VI 6min) schedules of reinforcement was tested in three groups of rats (n=8/group). The VI 6min schedule of reinforcement produced a higher response rate than the FR 1 or VI 1min schedules of visual stimulus reinforcement. One explanation for greater responding on the VI 6min schedule relative to the FR 1 and VI 1min schedules is that the reinforcing effectiveness of light onset habituated more rapidly in the FR 1 and VI 1min groups as compared to the VI 6min group. The inverse relationship between response rate and the rate of visual stimulus reinforcement is opposite to results from studies with biologically important reinforcers which indicate a positive relationship between response and reinforcement rate. Rapid habituation of reinforcing effectiveness may be a fundamental characteristic of sensory reinforcers that differentiates them from biologically important reinforcers, which are required to maintain homeostatic balance.     

Non-contingent positive and negative reinforcement schedules of superstitious behaviors

The role of schedules of reinforcement on the development of superstitious conditioning was investigated in a college age population. Participants were randomly assigned to one of eight operant schedules and instructed to remove (escape), prevent and/or remove (avoidance and escape) or produce (positive) the appearance of a computer generated stimulus using a response pad. Results from the experiment indicate that concomitant (escape and avoidance) schedules of reinforcement are most effective in facilitating acquisition of superstitious behavior as measured by self-reports of participants.     

The role of terminal-link stimuli in concurrent-chain schedules: revisited using a behavioral-history procedure

A behavioral-history procedure was used to study the function of terminal-link stimuli as conditioned reinforcers in multiple concurrent-chain schedules of reinforcement. First, three pigeons were exposed to multiple concurrent-chain schedules in which the two multiple-schedule components were correlated with a blue and a white stimulus, respectively. In each component the initial links were equal independent variable-interval (VI) 15 s schedules. A fixed-interval (FI) 10 s schedule operated on the red key in one terminal link while extinction operated on the green key in the alternative terminal link. When large preferences for the red stimulus had been established, two tests were conducted. In the terminal-link test, under new initial-link stimuli--purple and brown--an FI 10 s schedule operated for both the red and green terminal-link stimuli. In the subsequent initial-link test, the blue and white initial-link stimuli were reintroduced, and, as in the terminal-link test, FI 10s operated for both the red and the green terminal-link stimuli. In the terminal-link test, the three pigeons showed no preference for the terminal links with the red stimulus, but showed clear and consistent preferences for the red stimulus when blue and white stimuli were reintroduced as initial-link stimuli in the initial-link test. This suggests that there are multiple sources of control over initial-link response allocation in concurrent-chains, including control by both terminal- and initial-link stimuli.     

60 Hz electric fields and incandescent light as aversive stimuli controlling the behavior of rats responding under concurrent schedules of reinforcement

Several reports have shown that animals will sometimes engage in behaviors that reduce their exposure to a 60 Hz electric field (E-field). The field, therefore, can function as an aversive stimulus. In other studies, the E-field at equivalent strengths failed to function as an aversive stimulus. The present experiment, using rats, demonstrates how factors other than field strength can influence whether a subject engages in behavior that reduces field exposure. The general design consisted of giving the rat a choice between two alternatives, one of which sometimes included an added stimulus. Each subject was trained to press each of two levers to obtain food. Pressing one lever was reinforced intermittently under a variable interval 2 min schedule (VI 2); pressing the other lever was reinforced by a second VI 2 schedule operating independently of the first. Under this concurrent schedule the rat spent 50% of the daily 50 min session responding to each of the levers, indicating that they were equally “valued.” Next, while the schedules remained in effect, the first response to one of the levers turned on a 100 kV/m E-field which remained on until the rat pressed the other lever. The time spent responding under the schedule associated with the field was reduced by about 5–10%. When the procedure was changed so that no lever presses produced food, i.e., extinction, but the added stimulus contingency remained, the rats spent even less time in the presence of the field. Similar outcomes were observed during both the concurrent food or extinction schedules when incandescent light was used. Thus, both an E-field and incandescent light functioned as aversive stimuli, but the magnitude of the aversiveness was small. Aversiveness depended not only on stimulus intensity, but also on behavioral factors. Bioelectromagnetics 19:210–221, 1998. © 1998 Wiley-Liss, Inc.     

Transformation of a waiting schedule into a temporal regulation schedule (DRRD) by addition of external stimuli in the dog

Waiting schedules do not impose temporal regulation but condition the animal to give the operant response during a given time. At the end of the required delay, a positive discriminative stimulus is presented. The suspension of the response while the discriminative stimulus is being given suspension of the response while the discriminative stimulus is being given is accompanied by reinforcement. The transformation of a waiting schedule into a temporal regulation schedule is generally achieved by suppressing the external facilitating factors or by physically modifying them. Our study reveals that a similar transformation can be achieved in the dog by the addition of a further stimulus. This stimulus, which is physically exactly the same as the excitatory stimulus and which punctuates the waiting period, is randomly introduced into the temporal delay. The absence of reinforcement in response to the added stimulus should force the animal to regulate its behavior in time and the additional negative discriminative stimulus favours the expression of the active nature of the inhibation. The results show that subjects can differentiate their response durations according to stimuli that only differ according to temporal location. Thus this pattern resembles a DRRD schedule. The peak of responses at the time of the inhibition stimulus reveals considerable behavioral conflict : either the response must be maintained or the inhibition suppressed. The positive or negative resolution of this conflict reveals noteworthy aspects of the behavioural inhibition process.     

A pitfall for the expectancy theory of human eyelid conditioning

Two simple eyeblink conditioning experiments with random intermittent reinforcement schedules were performed. In Experiment 1, subjects had to rate their expectancy for an unconditioned stimulus (US) on a seven-level scale prior to each trial. As anticipated, expectancy for US increased with a successive conditioned stimulus (CS) alone, and decreased with successive CS-US pairings. However, Experiments 1 and 2 showed that the frequency of eyeblink conditioned responses (CRs) evolved in a direction opposite to that of expectancy changes: CRs increased, whereas expectancy for US decreased, and vice versa. The possible effect of sensitization on eyeblink response was ruled out by the lack of a run effect in an unpaired control group in Experiment 2. These results tend to disconfirm the expectancy theory of conditioning. Although they were explicitly predicted by the conventional "strength" theory of conditioning, an alternative interpretation is proposed within a cognitive framework.     

The effect of non-contingent negative and positive reinforcement operations on the acquisition of superstitious behaviors

The differential effects of non-contingent positive and negative reinforcement operations on the acquisition of superstitious behaviors and rules were investigated in two experiments. College students were instructed to try to produce and/or keep the word "GOOD" on a computer screen (positive reinforcement), or to try to prevent and/or remove the word "BAD" from the screen (negative reinforcement) using response keys. Data from both experiments indicated that participants exposed to lean schedules of negative reinforcers believed that they had greater control over non-contingent stimulus events than participants exposed to either rich or lean schedules of positive reinforcers. These findings and results from research investigating everyday superstitious activities suggest that, relative to positive reinforcement, negative reinforcement operations may provide a more fertile condition for the development and maintenance of superstitious behaviors.     

Reinforcer satiation and resistance to change of responding maintained by qualitatively different reinforcers

In previous research on resistance to change, differential disruption of operant behavior by satiation has been used to assess the relative strength of responding maintained by different rates or magnitudes of the same reinforcer in different stimulus contexts. The present experiment examined resistance to disruption by satiation of one reinforcer type when qualitatively different reinforcers were arranged in different contexts. Rats earned either food pellets or a 15% sucrose solution on variable-interval 60-s schedules of reinforcement in the two components of a multiple schedule. Resistance to satiation was assessed by providing free access either to food pellets or the sucrose solution prior to or during sessions. Responding systematically decreased more relative to baseline in the component associated with the satiated reinforcer. These findings suggest that when qualitatively different reinforcers maintain responding, relative resistance to change depends upon the relations between reinforcers and disrupter types.     

Neurochemical changes associated with schedule-controlled behavior.

Studies of monoamine metabolites within the cerebrospinal fluid compartment have indicated that this approach may be useful in examining central metabolic changes in vivo. By combining the technologies of radioisotope chemistry, operant behavior control and modification, and brain perfusion with push-pull cannulas, we have been able to examine minute to minute changes in the disposition of radiolabeled monoamine transmitter candidates and their metabolites. These substances appear to co-vary with changes in complex behavior maintained by operant schedules of reinforcement and affected by changes in schedules or administration of psychotropic drugs. In agreement with other perfusion studies, we have observed changes in fractional distribution of radiolabeled urea, a so-called extracellular marker, along with shifts in monoamines; but the former appear more transient. These observations nevertheless support the concept of dynamic changes within the extracellular environment of the CNS that may be part of a hormone-like communicating system with functional significance. Furthermore, the presence of peaks and/or troughs, in perfusates of [14C]urea of similar substances should not be taken as a priori evidence for nonspecificity of the technic, since selective release or inhibition of release of monoamines can be shown with appropriate drugs that are thought to act through these aminergic systems. Destruction of catecholamine nerve terminals with 6-hydroxydopamine likewise attenuates the signal-locked release of radiolabeled norepinephrine by a conditioned stimulus after conditioning occurs. No such release is seen on presentation of the to-be-conditioned neutral stimulus in control or 6-hydroxydopamine treated rats. These initial studies indicate the availability of a powerful tool for the study of drug-neurochemical-behavioral interactions using subjects as their own controls for extended periods of time so that phenomena of plasticity, tolerance and dependence may likewise be examined.     

Total control - pollen presentation and floral longevity in loasaceae (blazing star family) are modulated by light, temperature and pollinator visitation rates

Stamen movements can be understood as a mechanism influencing pollen presentation and increasing outbreeding success of hermaphroditic flowers via optimized male function. In this study we experimentally analyzed the factors regulating autonomous and thigmonastic (triggered by flower visitors) stamen movements in eight species of Loasaceae. Both types of stamen movements are positively influenced by light and temperature and come to a virtual standstill in the dark and at low temperatures (12°C). Pollen presentation is thus discontinued during periods where pollinators are not active. Overall stamen presentation increases with increasing flower age. Contrary to expectation, no geometrical correlation between the floral scale stimulated and the stamen fascicle reacting exists, indicating that the stimulus is transmitted over the receptacle and stamen maturation dictates which and how many stamens react. Thigmonastic stamen presentation is dramatically accelerated compared to autonomous movement (3-37 times), indicating that the rate of stamen maturation can be adjusted to different visitation schedules. Flowers can react relatively uniformly down to stimulation intervals of 10-15 min., consistently presenting comparable numbers of stamens in the flower c. 5 min. after the stimulus and can thus keep the amount of pollen presented relatively constant even under very high visitation frequencies of 4-6 visits/h. Thigmonastic pollen presentation dramatically reduces the overall duration of the staminate phase (to 1/3(rd) in Nasa macrothyrsa). Similarly, the carpellate phase is dramatically reduced after pollination, down to 1 d from 4 d. Overall flower longevity is reduced by more than 2/3(rds) under high visitation rates (<3 d versus 10 d under visitor exclusion) and depleted and pollinated flowers are rapidly removed from the pool. Complex floral behaviour in Loasaceae thus permits a near-total control over pollen dispensation schedules and floral longevity of the individual flower by an extraordinary fine-tuning to both biotic and abiotic factors.     

Extended exposure to a discriminated,limited-hold temporal schedule does not produce stimulus control

Four rats were exposed to two different stimuli (either lights or tones), each stimulus being correlated with independent probabilities of water delivery in a temporally defined schedule. The schedule consisted of a 60 s T cycle with 30 s t(D) and t(-) successive subcycles; t(D) was correlated with a probability of water delivery of 1.0 and t(-) was correlated with a probability of water delivery of 0.0. The schedule was maintained during 180 sessions and extended for 25 extra sessions omitting the stimulus in t(-). The four rats showed low frequencies of responding, response frequency being slightly higher in t(-) than in t(D). The percentage of lost reinforcers was independent of response frequency. The rats which lost less reinforcers were those which obtained more water deliveries during the first 15 cycles of each session. These results show that stimulus control does not develop in limited-hold temporal schedules, and that response-reinforcer effectiveness may depend on the initial contact with reinforcers in the first cycles of the session.     

Signals, resistance to change, and conditioned reinforcement in a multiple schedule

The effect of signals on resistance to change was evaluated using pigeons responding on a three-component multiple schedule. Each component contained a variable-interval initial link followed by a fixed-time terminal link. One component was an unsignaled-delay schedule, and two were equivalent signaled-delay schedules. After baseline training, resistance to change was assessed through (a) extinction and (b) adding free food to the intercomponent interval. During these tests, the signal stimulus from one of the signaled-delay components (SIG-T) was replaced with the initial-link stimulus from that component, converting it to an unsignaled-delay schedule. That signal stimulus was added to the delay period of the unsignaled-delay component (UNS), converting it to a signaled-delay schedule. The remaining signaled component remained unchanged (SIG-C). Resistance-to-change tests showed removing the signal had a minimal effect on resistance to change in the SIG-T component compared to the unchanged SIG-C component except for one block during free-food testing. Adding the signal to the UNS component significantly increased response rates suggesting that component had low response strength. Interestingly, the direction of the effect was in the opposite direction from what is typically observed. Results are consistent with the conclusion that the signal functioned as a conditioned reinforcer and inconsistent with a generalization-decrement explanation.     

The operant reserve: a computer simulation in (accelerated) real time

In Skinner's Reflex Reserve theory, reinforced responses added to a reserve depleted by responding. It could not handle the finding that partial reinforcement generated more responding than continuous reinforcement, but it would have worked if its growth had depended not just on the last response but also on earlier responses preceding a reinforcer, each weighted by delay. In that case, partial reinforcement generates steady states in which reserve decrements produced by responding balance increments produced when reinforcers follow responding. A computer simulation arranged schedules for responses produced with probabilities proportional to reserve size. Each response subtracted a fixed amount from the reserve and added an amount weighted by the reciprocal of the time to the next reinforcer. Simulated cumulative records and quantitative data for extinction, random-ratio, random-interval, and other schedules were consistent with those of real performances, including some effects of history. The model also simulated rapid performance transitions with changed contingencies that did not depend on molar variables or on differential reinforcement of inter-response times. The simulation can be extended to inhomogeneous contingencies by way of continua of reserves arrayed along response and time dimensions, and to concurrent performances and stimulus control by way of different reserves created for different response classes.     

A further application of the active time model to multiple concurrent variable-interval schedules

In this experiment we show that the active time model (ATM) accurately predicts probe data from multiple concurrent VI VI schedules. Subjects were trained under a concurrent VI 30-s VI 60-s and a concurrent VI 60-s VI 120-s schedule. Two types of unreinforced probes were then conducted. The first paired the two VI 60-s stimuli. These stimuli, while equivalent in their associated absolute rates of reinforcement, differed in their relative rates of reinforcement. The second probe paired the VI 30-s stimulus with the relatively rich VI 60-s stimulus. In contrast with the first probe, these stimuli differed in their absolute rates of reinforcement, while being similar in their relative rates. During the first set of probes, birds preferred the VI 60-s stimulus trained with the VI 120-s schedule. During the second set of probes, birds were indifferent to the two stimuli. These results are less extreme than others reported in the literature. Nonetheless, we found that ATM accurately fit individual subject data in both sets of probes. In contrast a variant of scalar expectancy theory did not fit the data at either the individual or group level.     

Sensitivity to reinforcement in successive and delayed discriminations

Pigeons' responses were recorded in successive 15-s subintervals of 60-s components of several multiple variable-interval schedules of food reinforcement. In the standard multiple schedule or successive discrimination, discriminative stimuli were present throughout each component. In the delayed discrimination or memory procedure, red or green stimuli were present in the first 15 s of components and were followed by a white stimulus for the remainder of both components. Ratios of responses in the first 15 s of the two components, where discriminative stimuli were present, were sensitive to ratios of reinforcers obtained in the two components, to the same extent in both multiple and memory procedures. In both procedures, sensitivity to reinforcement decreased systematically over component subintervals, but to a greater extent in the memory procedure where discriminative stimuli were absent. The reduction in sensitivity with time since presentation of prior discriminative stimuli in the memory procedure was therefore influenced by two main factors: delayed stimulus control by the discriminative stimuli presented earlier in the component, and a decrease in sensitivity to reinforcement with increasing time since component alternation.     

Perceptual learning in an appetitive Pavlovian procedure: Analysis of the effectiveness of the common element

Non-reinforced preexposure to two stimuli often enhances discrimination between them. Analyses of this perceptual learning phenomenon have mainly focused on the role played by the distinctive stimulus features; this study examined the contribution of the non-distinctive common elements. A standard appetitive Pavlovian procedure was used. Rats received two different schedules of exposure - alternated or blocked - to two compound auditory stimuli, AX and BX. In Experiment 1 a generalization test to BX that followed conditioning to AX showed that animals responded less, and hence discriminated better, following alternated exposure, thus extending the generality of this perceptual learning effect to standard appetitive Pavlovian procedures. The degree to which the common element X was mediating this effect was explored in the next three experiments. Experiment 2 assessed the effectiveness of X following conditioning to AX. Experiment 3 explored X's effectiveness throughout extensive conditioning to X. Experiment 4 tested the ability of X to overshadow a novel stimulus Y. The results were consistent with the suggestion that alternated preexposure can reduce the relative effectiveness of the common element.     

Quantifying and understanding reproductive allocation schedules in plants

A plant's reproductive allocation (RA) schedule describes the fraction of surplus energy allocated to reproduction as it increases in size. While theorists use RA schedules as the connection between life history and energy allocation, little is known about RA schedules in real vegetation. Here we review what is known about RA schedules for perennial plants using studies either directly quantifying RA or that collected data from which the shape of an RA schedule can be inferred. We also briefly review theoretical models describing factors by which variation in RA may arise. We identified 34 studies from which aspects of an RA schedule could be inferred. Within those, RA schedules varied considerably across species: some species abruptly shift all resources from growth to reproduction; most others gradually shift resources into reproduction, but under a variety of graded schedules. Available data indicate the maximum fraction of energy allocated to production ranges from 0.1 to 1 and that shorter lived species tend to have higher initial RA and increase their RA more quickly than do longer‐lived species. Overall, our findings indicate, little data exist about RA schedules in perennial plants. Available data suggest a wide range of schedules across species. Collection of more data on RA schedules would enable a tighter integration between observation and a variety of models predicting optimal energy allocation, plant growth rates, and biogeochemical cycles.