Free-choice preference under uncertainty

Preferences in pigeons for free choice over forced choice under uncertain contingencies were compared with the one under certain contingencies in multiple concurrent-chain schedules of reinforcement. The uncertain condition examined the preference for two alternatives over one alternative when reinforcement probability at the end of the terminal link equalled 0.5, and with all keys in each terminal link lit green (two mixed fixed-interval extinction keys versus one mixed fixed-interval extinction key). Key and schedule arrangement in the certain condition was the same except that a peck on any terminal-link key after the FI interval always produced food. When naive pigeons were first exposed to uncertain contingencies, preference for two lit keys over one lit key was observed, and the preference was confirmed by sequential reversals of the terminal-link contingencies. However, no consistent preference was observed when uncertain condition followed the certain condition. Under certain contingencies, unlike the earlier experiments, very small and inconsistent preferences for free choice were demonstrated. A possible reason for the different preferences in the uncertain conditions was that pigeons may lessen their sensitivity to the circumstances with uncertainty by any history or carry-over effect of the prior contingency.     

Reaction times identify a Pavlovian component in a two-choice discrimination

Six pigeons discriminated on discrete trials between two colors. In Experiment 1, two luminous spots were both either blue or green and the reinforced responses were “peck left” for blue and “peck right” for green. In Experiment 2, the hue of a center spot controlled subsequent choice pecks to left or right. In both experiments response bias was manipulated in two ways. During stimulus frequency (“SF”) sessions correct responses brought food on 40% of trials; in “imbalanced” blocks of sessions one hue appeared on 80% of trials and the other on 20%. During reinforcement probability (“RNF”) sessions the hues appeared equally often, but in imbalanced blocks the hues signaled different reinforcement probabilities, either 64% or 16%. In “balanced” control blocks the hues appeared equally often and were both reinforced at 40%. The experiments gave similar results. When bias was computed from choice percentages the imbalanced conditions yielded substantial response bias, and the amount of bias was about the same under RNF and SF treatments. However, reaction times (RTs) gave a different outcome. RNF imbalance slowed responses directed at the less reinforced stimulus, but SF imbalance had little RT effect (Experiment 1) or no effect (Experiment 2). These results suggest that choice was controlled by an instrumental stimulus-response-reinforcement association, whereas RTs were controlled by a Pavlovian stimulus-reinforcement association.     

A further study of choice and percentage reinforcement

In Experiment I four pigeons were trained in a concurrent chains procedure with fixed-ratio schedules (FR1) in the initial components and fixed-time schedules in the terminal components. Pecking one of the keys when both keys were white initiated a fixed time schedule on that key. A peck to the left key produced three stripes on the key. At the termination of the fixed-time component food always occurred. Pecking the other key produced either a circle or a triangle. If a circle appeared, reinforcement occurred. If a triangle appeared a brief timeout was given. Initially the stripes appeared on the left key and the circle and triangle on the left. This was reversed during the course of the experiment. In addition, sessions were conducted in which both circle and triangle sometimes preceded reinforcement and sometimes timeout. For most birds under most conditions there was a preference for the key that produced the circle and triangle. When these were uncorrelated with reinforcement and time out three of the birds preferred the key producing 100% reinforcement.

In Experiment II three factors were varied and VI 20 sec schedules were used in the initial links instead of FR1. The results showed that pigeons preferred the 50% condition more 1) the greater the duration of the terminal links, 2) the smaller the value on the initial link VI schedules and 3) the less the probability of food in the terminal link with stripes on the key.     

Behavioral contrast of a complex operant

Two experiments were conducted in which pigeons were trained to perform a complex operant consisting of a peck to one key followed by a peck to another key. In the first experment this performance was reinforced on a variable-interval schedule and the birds were then subjected to a multiple schedule in which the variable-interval was alternated with extinction. The first two pigeons trained showed some deterioration of the cohesiveness of the response sequence. After measures were taken to correct this the pigeon gave evidence of behavioral contrast in a condition where sessions with discrimination were alternated rapidly with sessions with no discrimination (variable-interval only). Another two birds were trained on variable-interval followed by discrimination. One of these birds showed evidence of contrast but the results of the other were not clear.Experiment II was a three phase experiment in which animals were first trained to make the complex response. In the second phase they were extinguished by providing noncontingent reinforcement in each of two components. In the third phase reinforcement was removed from one component, providing for a stimulus-reinforcer contingency. In the third phase the complex response reappeared temporarily.The results of these two studies indicate that a complex response may show behavioral contrast and that it may be enhanced by stimulus-reinforcer contingencies if it has already been trained.     

Species differences between rats and pigeons in choices with probabilistic and delayed reinforcers

An adjusting-delay procedure was used to study rats' choices with probabilistic and delayed reinforcers, and to compare them with previous results from pigeons. A left lever press led to a 5-s delay signaled by a light and a tone, followed by a food pellet on 50% of the trials. A right lever press led to an adjusting delay signaled by a light followed by a food pellet on 100% of the trials. In some conditions, the light and tone for the probabilistic reinforcer were present only on trials that delivered food. In other conditions, the light and tone were present on all trials that the left lever was chosen. Similar studies with pigeons [Mazur, J.E., 1989. Theories of probabilistic reinforcement. J. Exp. Anal. Behav. 51, 87-99; Mazur, J.E., 1991. Conditioned reinforcement and choice with delayed and uncertain primary reinforcers. J. Exp. Anal. Behav. 63, 139-150] found that choice of the probabilistic reinforcer increased dramatically when the delay-interval stimuli were omitted on no-food trials, but this study found no such effect with the rats. In other conditions, the probability of food was varied, and comparisons to previous studies with pigeons indicated that rats showed greater sensitivity to decreasing reinforcer probabilities. The results support the hypothesis that rats' choices in these situations depend on the total time between a choice response and a reinforcer, whereas pigeons' choices are strongly influenced by the presence of delay-interval stimuli.     

Pigeons shift their preference toward locations of food that take more effort to obtain

Although animals typically prefer to exert less effort rather than more effort to obtain food, the present research shows that requiring greater effort to obtain food at a particular location appears to increase the value of that location. In Experiment 1, pigeons' initial preference for one feeder was significantly reduced by requiring 1 peck to obtain food from that feeder and requiring 30 pecks to obtain food from the other feeder. In Experiment 2, a similar decrease in preference was not found when pigeons received reinforcement from both feeders independently of the amount of effort required. These results are consistent with the within-trial contrast effect proposed by in which the relative hedonic value of a reward depends on the state of the animal immediately prior to the reward. The greater the improvement from that prior state the greater the value of the reinforcer.     

An instance of viewpoint consistency in pigeon object recognition

Pigeons were trained on a visual discrimination, a task using a TV monitor. Two different types of stimuli appeared as pictures on the TV screen, one was a feeder used in their home cages and the other a coffee mug. One group of the pigeons was trained to peck the screen when the feeder appeared on it, while the other group was trained to peck the screen when the mug appeared. The feeder was considered to be a `familiar object' but the mug an `unfamiliar object' for the subjects. After training, to peck the familiar object, the subjects showed generalization to unusual view pictures of the object, but they did not show such generalization after training to peck the unfamiliar object. These results suggest that view point consistency is limited to familiar objects in pigeons.     

Conditioning the pecking motions of pigeons

The spatio-temporal courses of head and neck motions of pigeons while pecking at small grains are described. Single and serial pecks are distinguished but the inter- and intraindividual variability of the peck kinetics is stressed. Pigeons were then trained with instrumental conditioning procedures to speed-up their pecking. A partial reinforcement schedule where pigeons had to peck repeatedly before receiving reward led to a mild shortening of inter-peck intervals at lower reinforcement rates but surprisingly, a lengthening at higher rates. A schedule where short inter-peck intervals were differentially rewarded yielded a pronounced abbreviation of the inter-peck intervals, but this was achieved by a reduction of the movement path rather than an increase in motion velocity. A schedule whereby increased approach velocities were differentially rewarded yielded marked movement accelerations. When pigeons were rewarded for diminished approach speeds they also showed significant movement decelerations. Finally, it is shown that pigeons could learn to reliably abort their peck approach movement when a visual stimulus signalling a penalty was occasionally presented during the approach movement. The proportion of successful peck interruptions decreased as these interruption signals occurred later during the approach phase. It is concluded that the pecking of pigeons is neither an innately fixed nor a visually ballistic movement. It is instead a multiply controlled and flexibly adaptable response pattern.     

Maladaptive "gambling" by pigeons

When humans buy a lottery ticket or gamble at a casino they are engaging in an activity that on average leads to a loss of money. Although animals are purported to engage in optimal foraging behavior, similar sub-optimal behavior can be found in pigeons. They show a preference for an alternative that is associated with a low probability of reinforcement (e.g., one that is followed by a red hue on 20% of the trials and then reinforcement or by a green hue on 80% of the trials and then the absence of reinforcement) over an alternative that is associated with a higher probability of reinforcement (e.g., blue or yellow each of which is followed by reinforcement 50% of the time). This effect appears to result from the strong conditioned reinforcement associated with the stimulus that is always followed by reinforcement. Surprisingly, although it is experienced four times as much, the stimulus that is never followed by reinforcement does not appear to result in significant conditioned inhibition (perhaps due to the absence of observing behavior). Similarly, human gamblers tend to overvalue wins and undervalue losses. Thus, this animal model may provide a useful analog to human gambling behavior, one that is free from the influence of human culture, language, social reinforcement, and other experiential biases that may influence human gambling behavior.     

Interactions among unit-price, fixed-ratio value, and dosing regimen in determining effects of repeated cocaine administration

Previous research has shown tolerance to cocaine that was dependent on fixed-ratio (FR) parameter size in the context of a multiple FR schedule of food reinforcement. Completion of the FR requirement in these studies resulted in the same magnitude of reinforcement, regardless of ratio size. The cost-to-benefits ratio (unit-price) was therefore not equated across the different FR components. The current study examined the effects of repeated administration of cocaine to pigeons when unit-price under FR schedules was either the same or different. Additionally, the role of a chronic variable-dosing versus chronic fixed-dosing procedure was examined when unit-price was equated across different ratio values. Pigeons were trained to key peck in daily sessions under a three-component multiple FR schedule of food presentation, according to which either 10, 30, or 100 pecks were required for each delivery of food. In Experiment 1, completion of the FR 10, FR 30, and FR 100 resulted in 1.5, 4.5, and 15.0 s access to food, respectively. That is, the response requirement was correlated with access to food time so that unit-price (pecks per second of access to food) was equated across components. After assessing acute effects of a range of doses of cocaine, drug administration occurred daily before each session, with the dose varying from day to day. Tolerance, the magnitude of which was unrelated to the peck requirement, developed under the repeated-dosing regimen. In order to assess whether having equal unit-price was responsible for producing similar levels of tolerance across components, daily drug administration continued in Experiment 2 using the variable-dose regimen, but the amount of food presented each time was fixed at 4.5 s access to food, yielding different unit-prices under the three pecking requirements. Subsequently, the conditions of Experiment 1 were reinstated, i.e., unit-price was equated. Making unit-price different or the same had little influence on effects of cocaine. In Experiment 3, a fixed dose of cocaine was administered before each session while programmed unit-price remained the same across components. Under these conditions, tolerance became peck-requirement related. Specifically, tolerance was most prevalent under the smaller requirements and less robust or absent when the largest requirement was in effect. Differences in unit-price, therefore, were not related to degree of tolerance, but work requirement was. Differences in effects of cocaine across responses requirements, however, were observed only when each session was preceded by the same dose, not when dose varied from session to session.     

Categorical counting

Pigeons pecked on three keys, responses to one of which could be reinforced after a few pecks, to a second key after a somewhat larger number of pecks, and to a third key after the maximum pecking requirement. The values of the pecking requirements and the proportion of trials ending with reinforcement were varied. Transits among the keys were an orderly function of peck number, and showed approximately proportional changes with changes in the pecking requirements, consistent with Weber's law. Standard deviations of the switch points between successive keys increased more slowly within a condition than across conditions. Changes in reinforcement probability produced changes in the location of the psychometric functions that were consistent with models of timing. Analyses of the number of pecks emitted and the duration of the pecking sequences demonstrated that peck number was the primary determinant of choice, but that passage of time also played some role. We capture the basic results with a standard model of counting, which we qualify to account for the secondary experiments.     

The effects of concurrent task and gap events on peak time in the peak procedure

The effect of a concurrent task on timing performance of pigeons was investigated with the peak interval procedure. Birds were trained to peck a side key on a discrete-trial schedule that included reinforced fixed-interval (FI) 30-s trials and nonreinforced extended probe trials. Then, in separate sessions, birds were trained to peck a 6-s center key for food. In a subsequent test phase, the FI procedure was in effect along with dual-task probe test trials. On those test trials, the 6-s center key (task cue) was presented at 3, 9, or 15s after probe trial onset. During another test phase, a 6-s gap (the FI keylight was extinguished) was presented at 3, 9, or 15s after probe trial onset. Peak time increased with center key time of onset, and was greater under task than gap conditions. Moreover, peak time under task conditions exceeded values predicted by stop and reset clock mechanisms. These results are at variance with current attentional accounts of timing behavior in dual-task conditions, and suggest a role of nontemporal factors in the control of timing behavior.     

Discrete-trial vs. continuous free-operant procedures in assessing whether reinforcement context affects reinforcement value

Pigeons were trained to respond to two alternating concurrent reinforcement schedules. The reinforcement probabilities were .05 and .10 in one component, and .10 and .20 in the other. In one condition, the pigeons received training on a discrete-trial procedure in which the keylights remained illuminated for 5s or until a response occurred. In another condition, pigeons received training on a procedure in which the reinforcement contingencies were the same as in the discrete-trial procedure, but the stimuli were not turned off after 5s or after a response. Following training in each condition, probe tests were presented. In both conditions, the .20 alternative was, overall, preferred to the .05 alternative during probe tests. Following discrete-trial training, there was no reliable preference between the two .10 alternatives. However, when the stimuli remained illuminated during the intertrial interval periods during training, probe tests results showed preference for the .10 alternative that had been presented in the leaner context during training. The pattern of results is consistent with the notion that probe preference can be influenced both by the absolute reinforcement schedules associated with each alternative, as well as changeover behavior developed during training.     

Attention, the presolution period, and choice accuracy in pigeons

Six pigeons were trained first on a color then on a form discrimination; four other pigeons were trained first on form and then on color. One of two colors or one of two forms (sample stimuli) appeared in the center of a touch sensitive monitor for 5 pigeons and in the center and in 16 other locations for 5 other pigeons. A peck anywhere within the region in which the sample stimuli appeared produced two white disks (comparison stimuli), one on the left and one on the lower right corners of the screen. Correct left-right choices provided food. Although of no consequence, the location of pecks in presence of the sample was predictive of the pigeon's subsequent choice. Accuracy, choice of the correct comparison stimulus, was greater when the sample stimuli appeared in the center as well as 16 other locations than when it appeared only in the center. The presolution period, the period of chance accuracy prior to evidence of discrimination learning, was decreased on each task following training on the other task. This evidence of facilitation following an extra-dimensional shift was attributed to continued relevance of the conditions under which the first task was learned. The duration of the presolution period was inversely related to asymptotic accuracy-data accounted for by Heinemann's (1983) theory of information processing during the presolution period.     

Feature-short and feature-long discrimination learning in the pigeon: conditional control of a two-event temporal map

Eight pigeons were trained to peck an illuminated target key on discrete-trial fixed-interval schedules of reinforcement by food. Four birds were exposed to a feature-short (FS) task where a feature light signaled shortening of the forthcoming target-outcome interval from 30 to 15s, while the other four birds were exposed to a feature-long (FL) task where a feature light signaled extension of the forthcoming target-outcome interval from 15 to 30s. The discrimination performance measured by differential temporal distributions of pecks between featured and non-featured target trials suggested that the target-food temporal map was under conditional control of the feature light in both groups. The FS discrimination was more difficult to learn than the FL discrimination. This FS inferiority implies that our birds did not resort on the simple temporal discrimination by timing from the trial onset. The simple temporal discrimination account was also negated by the finding that increasing the feature-target gap did not have a predicted effect on the response distribution.     

Maladaptive choice behaviour by pigeons: an animal analogue and possible mechanism for gambling (sub-optimal human decision-making behaviour)

Consistent with human gambling behaviour but contrary to optimal foraging theory, pigeons showed maladaptive choice behaviour in experiment 1 by choosing an alternative that provided on average two food pellets over an alternative that provided a certain three food pellets. On 20 per cent of the trials, choice of the two-pellet alternative resulted in a stimulus that always predicted ten food pellets; on the remaining 80 per cent of the trials, the two-pellet alternative resulted in a different stimulus that always predicted zero food pellets. Choice of the three-pellet alternative always resulted in three food pellets. This choice behaviour mimics human monetary gambling in which the infrequent occurrence of a stimulus signalling the winning event (10 pellets) is overemphasized and the more frequent occurrence of a stimulus signalling the losing event (zero pellets) is underemphasized, compared with the certain outcome associated with not gambling (the signal for three pellets). In experiment 2, choice of the two-pellet alternative resulted in ten pellets with a probability of 20 per cent following presentation of either stimulus. Choice of the three-pellet alternative continued to result in three food pellets. In this case, the pigeons reliably chose the alternative that provided a certain three pellets over the alternative that provided an average of two pellets. Thus, in experiment 1, the pigeons were responding to obtain the discriminative stimuli signalling reinforcement and the absence of reinforcement, rather than to obtain the variability in reinforcement.     

Reinforcement omission in concurrent fixed-interval and random-interval schedules

The present experiment examined overall and local effects of omission of reinforcers in a choice situation. Pigeons' key-pecking responses were reinforced under concurrent fixed-interval and random-interval schedules of food presentation. After some weeks of baseline sessions in which the probability of reinforcement was 1.00, approximately 25% of food presentations from the fixed-interval schedule were omitted and replaced by timeout periods. In such omission sessions, the overall relative rates of responding to the fixed-interval schedule became lower than those in the baseline sessions. On the other hand, when relative rates of responding to the fixed-interval schedule in the omission sessions were calculated separately for fixed-interval cycles preceded by timeout periods and those preceded by food presentations, the relative rates in the former type of fixed-interval cycles were higher than those in the latter type for three out of four pigeons. These results mean that relative rates of responding cannot always be regarded as reflecting a relative value of an alternative, and that the overall effect of the omission of fixed-interval reinforcers is not reducible to the local effect of omission.     

Matching-to-sample by pigeons: the dissociation of comparison choice frequency from the probability of reinforcement

It has been proposed that comparison choice in matching-to-sample should depend on two factors, the relative probability of reinforcement associated with each of the comparison stimuli and the conditional probability of each comparison stimulus being correct given presentation of one of the samples. DiGian and Zentall [DiGian, K.A., Zentall, T.R., 2007. Matching-to-sample in pigeons: in the absence of sample memory, sample frequency is a better predictor of comparison choice than the probability of reinforcement for comparison choice. Learn Behav. 35, 242-261] have shown that sample frequency together with the probability of choosing each of the comparison stimuli in training can influence comparison choice when delays are introduced, when the number of reinforcements associated with each of the comparison stimuli is equated. Furthermore, Zentall and Clement [Zentall, T.R., Clement, T.S., 2002. Memory mechanisms in pigeons: Evidence of base-rate neglect. J. Exp. Psych.: Anim. Behav. Proc. 28, 111-115] have found that sample frequency can affect comparison choice when delays are introduced independently of the number of choices of each of the comparison stimuli in training and the number of reinforcements associated with each of the comparison stimuli is equated. In the present experiment we found that the probability of choosing each of the comparison stimuli in training can affect comparison choice when delays are introduced, independently of sample frequency and when the number of reinforcements associated with each of the comparison stimuli is equated. Together, these experiments suggest that when the sample is not available, there is a partial dissociation between comparison choice and the probability of reinforcement associated with each of the comparison stimuli.     

Stimulus magnitude effects on the extinction of conditioned consummatory responses to flavored sucrose solution in mice

Paradoxical extinction effects in the conditioned consummatory behavior of rodents have remained largely elusive. Here, appetitive flavor conditioning was studied to determine if a paradoxical magnitude of reinforcement extinction effect (MREE) can occur in the consummatory behavior of mice. During acquisition training of two experiments with factorial design, animals received daily access to either 32% or 4% sucrose solution, and goal tracking time was measured in one-minute bins. In Experiment 1 the solutions were flavored with either 5% or 0.5% almond essence and in Experiment 2 with 2% almond essence, but combined with continuous or partial schedules of reinforcement. During extinction tests of Experiment 1 and 2, water flavored with 0.5% or 2% almond was presented, respectively. Consummatory performance decreased more abruptly during the initial portion of the extinction sessions after training with 32% as compared to 4% sucrose solution. Furthermore, when given a choice test after extinction training (Experiment 2), animals trained with 32% sucrose, preferred the flavored solution, but animals trained with 4% preferred the unflavored solution. These results are interpreted as indicative of the occurrence of a paradoxical MREE in conditioned consummatory behaviors.     

Toward a framework for the evaluation of feature binding in pigeons

Pigeons were trained in a new procedure to test for visual binding errors between the dimensions of color and shape. In Experiment 1, pigeons learned to discriminate a target compound from 15 non-target compounds (constructed from four colors and shapes) by choosing one of two hoppers in a two-hopper choice task. The similarity of the target to non-target stimuli influenced choice responding. In Experiment 2, pigeons learned to detect a target compound when presented with a non-target compound within the same trial under conditions of simultaneity and sequentiality. Non-target trials were arranged to allow for the testing of binding errors (i.e., false identifications of the target on certain non-target trials). Transient evidence for binding errors in two of the birds occurred at the start of two-item training, but decreased with training. The experiments represent an important step toward developing a framework for the evaluation of visual feature binding in nonhumans.