Effects of prefrontal ablations on the reaction of the active choice of feeder under different probability and value of the reinforcement on dog

The role of the prefrontal cortex was investigated on the reaction of the active choice of the two feeders under changes value and probability reinforcement. The experiments were performed on 2 dogs with prefrontal ablation (g. proreus). Before the lesions the dogs were taught to receive food in two different feeders to conditioned stimuli with equally probable alimentary reinforcement. After ablation in the inter-trial intervals the dogs were running from the one feeder to another. In the answer to conditioned stimuli for many times the dogs choose the same feeder. The disturbance of the behavior after some times completely restored. In the experiments with competition of probability events and values of reinforcement the dogs chose the feeder with low-probability but better quality of reinforcement. In the experiments with equal value but different probability the intact dogs chose the feeder with higher probability. In our experiments the dogs with prefrontal lesions chose the each feeder equiprobably. Thus in condition of free behavior one of different functions of the prefrontal cortex is the reactions choose with more probability of reinforcement.     

When to respond? And how much? : Temporal control and response output on mixed-fixed-interval schedules with unequally probable components

Rats were trained on mixed-fixed-interval (FI) schedules, with component FIs of 30 and 60s. The probability of reinforcement according to FI 30s varied between conditions, across values of 0.1, 0.3, 0.5, 0.7 and 0.9. When response rate in the 60s intervals was measured, separate response peaks, one close to 30s, the other at 60s, could be identified when the probability of reinforcement at 30s was 0.3 or greater. Nonlinear regression found that the location of the earlier peak was always close to 30s, that the coefficient of variation of the response functions at 30 and 60s were unaffected by reinforcement probability, but that the 30s component appeared to be timed slightly more precisely than the 60s one. Response rate at around 30s increased with increasing probability of reinforcement according to FI 30s, but responding at 60s was unaffected by reinforcement probability. The data are discussed with respect to a number of contemporary models of animal timing (scalar expectancy theory, the Behavioural Theory of Timing and the Learning to Time model), and a recent account of response output on FI-like schedules.     

Performance correlates of social behavior and organization: Social rank and omission of reinforcement in rhesus monkeys (M. mulatta)

The performance of 22 adult male rhesus monkeys on a Fixed Interval 1-min reinforcement schedule was examined under conditions where the reinforcement probabilities were either 1.00 or .80. The results were then related to the social rank of animals at the time that they were taken from their social groups. Both high and low ranked animals reached criterion performance in the same number of trials. In general, high ranking animals responded at lower rates than low ranking animals when the reinforcement probability was 1.00. When the reinforcement probability was shifted to .80, all animals showed an increase in responding after nonreinforced intervals as compared with responses after reinforced intervals. The higher ranked animals tended to have a higher ratio of nonreinforced to reinforced responses than lower ranked animals. Supported by USAMRDC Contract No. DADA 17-73-C-3007.     

Choice between probability and value of alimentary reinforcement as means for revealing individual typological features of dog behavior

Individual typological features of behavior of dogs were investigated by the method of choice between the low-valuable food available constantly and food of high quality presented with low probability. Animals were subjected to instrumental conditioning with the same conditioned stimuli but different types of reinforcement. Depression of a white pedal was always reinforced with meat-bread-crumb mixture, depression of a black pedal was reinforced with two pieces of liver (with probabilities of 100, 40, 33, 20, or 0%). The choice of reinforcement depended on probability of valuable food and individual typological features of the nervous system of a dog. Decreasing the probability of the reinforcement value to 40-20% revealed differences in behavior of dogs. Dogs of the first group, presumably with the weak type of the nervous system, more frequently pressed the white pedal (always reinforced) than the black pedal thus "avoiding a situation of risk" to receive an empty cup. They displayed symptoms of neurosis: whimper, refusals of food or of the choice of reinforcement, and obtrusive movements. Dogs of the second group, presumably with the strong type of the nervous system, more frequently pressed the black pedal (more valuable food) for the low-probability reward until they obtained the valuable food. They did not show neurosis symptoms and were not afraid of "situation of risk". A decrease in probability of the valuable reinforcement increased a percentage of long-latency depressions of pedals. It can be probably suggested that this phenomenon was associated with increasing involvement of cognitive processes, when contributions of the assessments of probability and value of the reinforcement to decision making became approximately equal. Choice between the probability and value of alimentary reinforcement is a good method for revealing individual typological features of dogs.     

Choice in humans: Functional properties of reinforcers established by instruction

Adult human subjects chose between schedules containing stimuli (indicator lights) that the subjects were instructed to consider pleasurable. The schedules differed in amount of reinforcement (period of illumination) or delay (interval between a choice response and light onset). Although subjects preferred large to small amounts of reinforcement, they were essentially indifferent between immediate and delayed reinforcement. In contrast, previous data on video game reinforcement demonstrated preferences for both immediate and large amounts of reinforcement. The instructed reinforcer was thus partially effective in controlling choice but was not equivalent to a reinforcer that presumably had intrinsic value.     

Probabilistic reinforcement and neurotic breakdown in unrestrained dogs]

The substitution of a constant reinforcement for a random one with a probability of 0.5 in experiments on two dogs with a simple motor stereotype was attended with nervous breakdowns with motor excitation, inadequate orienting reactions or passive-defensive behaviour. The change in the probability of reinforcement from 0.5 to 0.3 had a positive effect in experiments on one dog, while in the other it developed drowsiness. In two other dogs with a complicated stereotype, the change in the mode of reinforcement was attended with a peculiar preventive effect of probabilistic, but ordered reinforcement in experimental surroundings, including signals with a probabilistic random reinforcement as well. In this case the dogs displayed primarily signs of emotional stress only. Behaviour was somewhat disturbed in one animal only when testing signals with a probabilistic random reinforcement.     

Performance correlates of social behavior and organization: Effects of group formation on operant performance in rhesus monkeys (M. mulatta)

The performance of six adult rhesus monkeys on a fixed interval 1-min reinforcement schedule was examined under conditions where the reinforcement probabilities were either 1.00 or .80. The animals were tested with a reinforcement probability of 1.00 immediately after removal from their social group, tested again several months later with reinforcement probabilities of 1.00 and .80, and then retested with a reinforcement probability of .80 before, during, and after the formation of a new social group with the six monkeys. The results supported an earlier report in which it was found that high ranking animals responded at a lower rate under a reinforcement probability of 1.00 than did low ranking animals and that the former also had a higher ratio of nonreinforced to reinforced responses than the lower ranked animals. These relationships were present in five of the six animals immediately after removal of the animals from their original group and during and after formation of the new group. They did not appear on tests conducted after the animals had been out of a social group for several months. A third result was a suppression of operant performance which appeared in all animals following group formation. The magnitude and duration of this effect was inversely related to social rank and the time the animal had been a member of the group. The data are discussed in terms of a carry-over of the effects of recent social experience on two factors: a more deliberate rate of response by higher ranking animals under 100% reinforcement and an inhibition of response bursting following omission of reinforcement by the lower ranking monkeys. Supported by USAMRDC Contract No. DADA 17-73-C-3007.     

Effects of partial reinforcement and time between reinforced trials on terminal response rate in pigeon autoshaping

Partial reinforcement often leads to asymptotically higher rates of responding and number of trials with a response than does continuous reinforcement in pigeon autoshaping. However, comparisons typically involve a partial reinforcement schedule that differs from the continuous reinforcement schedule in both time between reinforced trials and probability of reinforcement. Two experiments examined the relative contributions of these two manipulations to asymptotic response rate. Results suggest that the greater responding previously seen with partial reinforcement is primarily due to differential probability of reinforcement and not differential time between reinforced trials. Further, once established, differences in responding are resistant to a change in stimulus and contingency. Secondary response theories of autoshaped responding (theories that posit additional response-augmenting or response-attenuating mechanisms specific to partial or continuous reinforcement) cannot fully accommodate the current body of data. It is suggested that researchers who study pigeon autoshaping train animals on a common task prior to training them under different conditions.     

Maladaptive "gambling" by pigeons

When humans buy a lottery ticket or gamble at a casino they are engaging in an activity that on average leads to a loss of money. Although animals are purported to engage in optimal foraging behavior, similar sub-optimal behavior can be found in pigeons. They show a preference for an alternative that is associated with a low probability of reinforcement (e.g., one that is followed by a red hue on 20% of the trials and then reinforcement or by a green hue on 80% of the trials and then the absence of reinforcement) over an alternative that is associated with a higher probability of reinforcement (e.g., blue or yellow each of which is followed by reinforcement 50% of the time). This effect appears to result from the strong conditioned reinforcement associated with the stimulus that is always followed by reinforcement. Surprisingly, although it is experienced four times as much, the stimulus that is never followed by reinforcement does not appear to result in significant conditioned inhibition (perhaps due to the absence of observing behavior). Similarly, human gamblers tend to overvalue wins and undervalue losses. Thus, this animal model may provide a useful analog to human gambling behavior, one that is free from the influence of human culture, language, social reinforcement, and other experiential biases that may influence human gambling behavior.     

The dependence of hippocampal electrical activity on the probability of the reinforcement of a conditioned food stimulus

In three dogs the dynamics was studied of changes in the number of instrumental motor reactions, the heart rate and the hippocampus theta rhythm was studied at the change of constant reinforcement to various regimes of probable reinforcement of alimentary conditioned stimulus. At 70 and 50% levels of the reinforcement the instrumental reflex appeared in response to all presentations of the signal stimulus. The greatest decrease in the number of reactions took place at 5% probability. The heart rate both in the intersignal period and at getting of empty feeder depended on the used probability of reinforcement and individual characteristics of animals. The frequency of the theta rhythm in all dogs was the highest at the 30% level and the lowest values were found at the 5% level of reinforcement. The obtained facts testify once more to complex genesis of emotional excitation, the value of which depends both on informational (uncertainty) and motivational (significance of reinforcement) factors.     

Simulation of patch use by monkeys using operant conditioning

Food patch use by Japanese monkeys was examined using an operant conditioning procedure. Modified progressive ratio schedules, in which the probability of reinforcement decreases exponentially with the number of bar presses, were presented to 2 Japanese monkeys. Two types of schedule were used in each experimental session. One represented high quality food patches, where the probability of reinforcement was twice as high as in the other, which represented low quality food patches. The number of bar presses in each food patch was counted. Monkeys responded more frequently in high quality patches. The probability of reinforcement for the last response in each patch was the same in both types of schedule. The number of responses increased with a decrease in the occurrence of high quality patches, and with an increase in the inter-patch time interval. These results are in agreement with the predictions of Charnov’s marginal value theorem (Charnov, 1976). The pattern of patch use by monkeys observed in this study is discussed in terms of optimal foraging strategy.     

Within-trial contrast: The effect of probability of reinforcement in training

There is evidence that pigeons prefer conditioned reinforcers that are preceded by greater effort over those that are preceded by less effort (an effect that has been attributed to within-trial contrast). In past research the probability of reinforcement for correct choice of the conditioned reinforcer has been 100%, however, the high level of reinforcement for both alternatives in training may result in a performance ceiling when choice between those alternatives is provided on test trials. In the present study we tested this hypothesis by including a group for which the probability of reinforcement in training was only 50%. Pigeons were trained on two simultaneous discriminations, one that was preceded by a 30 peck requirement the other by a single peck requirement. On test trials, we found a significant preference for the S+ that required the greater effort in training for pigeons trained with 100% and a small but nonsignificant effect for pigeons trained with 50% reinforcement. Although the hypothesis that the within-trial contrast effect was constrained by a performance ceiling was not confirmed, we did find a reliable within-trial contrast effect with 100% reinforcement.     

The production and generalization of large-magnitude heart rate deceleration by contingently faded biofeedback

Eight subjects were taught to decrease their heart rates via biofeedback training. Four of these received contingently faded, beat-by-beat analogue feedback and contingent reinforcement each time their performance met a specified and adjusting criterion. The other four received continuous, beat-by-beat analogue feedback, but not the contingent reinforcement. Subjects in the two groups were yoked to ensure equal densities of reinforcement. Subjects in the first group were asked to decrease heart rates 15% from baseline and were then trained using only 75%, 50% and 25% of beat-by-beat feedback. It was hypothesized that the immediate reinforcement of appropriate behavior and the contingent fading(following mastery) of feedback would aid in the generalization of the response. Following completion of all criterion steps or 10 training sessions, whichever came first, all subjects were tested with no feedback and no contingent reinforcement. The group receiving contingently faded feedback training showed a significantly greater heart rate decrease in the training sessions and also the test session. These results were interpreted as indicating that biofeedback can be conceptualized as an operant conditioning paradigm, and that the use of operant techniques may help subjects produce clinically significant changes.This research was supported in part by a grant to Robert J. Gatchel from the National Heart, Lung, and Blood Institute (Grant No. NIH HL 21426-01).     

"The stone which the builders rejected...": Delay of reinforcement and response rate on fixed-interval and related schedules

The article deals with response rates (mainly running and peak or terminal rates) on simple and on some mixed-FI schedules and explores the idea that these rates are determined by the average delay of reinforcement for responses occurring during the response periods that the schedules generate. The effects of reinforcement delay are assumed to be mediated by a hyperbolic delay of reinforcement gradient. The account predicts that (a) running rates on simple FI schedules should increase with increasing rate of reinforcement, in a manner close to that required by Herrnstein's equation, (b) improving temporal control during acquisition should be associated with increasing running rates, (c) two-valued mixed-FI schedules with equiprobable components should produce complex results, with peak rates sometimes being higher on the longer component schedule, and (d) that effects of reinforcement probability on mixed-FI should affect the response rate at the time of the shorter component only. All these predictions were confirmed by data, although effects in some experiments remain outside the scope of the model. In general, delay of reinforcement as a determinant of response rate on FI and related schedules (rather than temporal control on such schedules) seems a useful starting point for a more thorough analysis of some neglected questions about performance on FI and related schedules.     

Energy budget versus temporal discounting as determinants of preference in risky choice

Four pigeons and three ringneck doves responded on an operant simulation of natural foraging. After satisfying a schedule of reinforcement associated with search time, subjects could "accept" or "reject" another schedule of reinforcement associated with handling time. Two schedules of reinforcement were available, a variable interval, and a fixed interval with the same mean value. Food available in the session (a variable related to the energy budget) was manipulated in the different conditions either by increases of the value of the search state schedule of reinforcement, or by increases in the mean value of the handling state schedules. The results indicate that the amount of food available in the session did not affect the preference for variable schedules of reinforcement, as would be predicted by an influential theory of risk sensitive foraging. Instead, the preference for variability depended on the relationship between the time spent in the search and the handling states, as is predicted by a family of models of choice that are based on the temporal proximity to the reinforcer.     

Swift action increases the success of population reinforcement for a declining prairie grouse

Translocations have become an increasingly valuable tool for conservation in recent years, but assessing the successfulness of translocations and identifying factors that contribute to their success continue to challenge biologists. As a unique class of translocation, population reinforcements have received relatively little attention despite representing a substantial portion of translocation programs. Here, we conducted population viability analyses to quantify the effects of 216 reinforcement scenarios on the long‐term viability of four populations of Greater Prairie‐Chickens (Tympanuchus cupido pinnatus) in Wisconsin, USA, and used multiple linear regression to identify factors that had the greatest relative influence on population viability. We considered reinforcements from outside of the study area in addition to translocations among Wisconsin populations. We observed the largest decreases in site‐specific extinction probability and the largest increases in the number of sites persisting for 50 years when more vulnerable populations were targeted for reinforcement. Conversely, reinforcing the most stable sites caused the greatest reduction in regional extinction probability. We found that the number of translocated hens was a comparatively poor predictor of changes in long‐term population viability, whereas the earlier onset of reinforcement was consistently associated with the greatest increases in viability. Our results highlight the value of evaluating alternative reinforcement strategies a priori and considering the effects of reinforcement on metrics of long‐term population persistence.     

Resistance to extinction, generalization decrement, and conditioned reinforcement

This study investigated generalization decrement during an extinction resistance-to-change test for pigeon key pecking using a two-component multiple schedule with equal variable-interval 3-min schedules and different reinforcer amounts (one component presented 2-s access to reinforcement and the other 8s). After establishing baseline responding, subjects were assigned to one of the two extinction conditions: hopper stimuli (hopper and hopper light were activated but no food was available) or Control (inactive hopper and hopper light). Responding in the 8-s component was more resistant to extinction than responding in the 2-s component, the hopper stimuli group was more resistant to extinction compared to the Control group, and an interaction between amount of reinforcement, extinction condition, and session block was present. This finding supports generalization decrement as a factor that influences resistance to extinction. Hopper-time data (the amount of time subjects spent with their heads in the hopper) were compared to resistance-to-change data in an investigation of the role of conditioned reinforcement on resistance to change.     

Undermatching in humans to amount of reinforcement

Human subjects were exposed to a concurrent-chain procedure in which amount of reinforcement in the terminal links was varied. The experimental procedure was designed to resemble as closely as possible animal operant procedures: verbal instructions were eliminated, the key-press operant response was shaped, and a “consummatory” response was required to receive reward. In addition to varying amount of reward, three different pairs of initial-link values in the concurrent chain were studied. The human subjects showed undermatching to amount of reinforcement (as do animal subjects). Moreover, the degree of undermatching tended to increase as the values of the initial links increased, consistent with Fantino's delay reduction hypothesis (1977) that choice for a larger reward decreases as the length of the initial link increases.     

Significance of motivation balance for a choice of dog's behavior under conditions of environmental uncertainty

Two experimental models with a choice between two reinforcements were used for assessment of individual typological features of dogs. In the first model dogs were given the choice of homogeneous food reinforcements: between less valuable constantly delivered reinforcement and more valuable reinforcement but delivered with low probabilities. In the second model the dogs had the choice of heterogeneous reinforcements: between performing alimentary and defensive reactions. Under conditions of rise of uncertainty owing to a decrease in probability of getting the valuable food, two dogs continued to prefer the valuable reinforcement, while the third animal gradually shifted its behavior from the choice of a highly valuable but infrequent reward to a less valuable but easily achieved reinforcement. Under condition of choice between the valuable food reinforcement and avoidance of electrocutaneous stimulation, the first two dogs preferred food, whereas the third animal which had been previously oriented to the choice of the low-valuable constant reinforcement, steadily preferred the avoidance behavior. The data obtained are consistent with the hypothesis according to which the individual typological characteristics of animals's (human's) behavior substantially depend on two parameters: extent of environmental uncertainty and subjective features of reinforcement assessment.