Effects of reinforcer magnitude on response acquisition with unsignaled delayed reinforcement

Sixteen rats received eight 1-h sessions of a tandem fixed-ratio 1 differential-reinforcement-of-other-behavior 30-s schedule with reinforcer magnitude at one or six food pellet(s) across groups of eight rats. The larger reinforcer magnitude established the lever press more effectively. First, mean schedule completions differed across groups, in terms of both their overall difference and in their increase across sessions. Second, whereas the larger reinforcer magnitude established reliable response acquisition in all eight rats by the end of the experiment, the smaller reinforcer magnitude only established reliable response acquisition in four of eight rats. The present results systematically replicate earlier findings obtaining more reliable response acquisition with unsignaled delayed reinforcement with greater food deprivation. Taken together, this work demonstrates the influence of motivational variables on response acquisition with unsignaled delayed reinforcement.     

Small and large fixed ratios with the same unit price

Key pressing of rats was maintained under multiple and discrete-trial choice schedules with reinforcer units of 45 mg food pellets or 3.5 s dips of sucrose solution. Both smaller and larger fixed ratio (FR) schedules were associated with the same unit price in a manner, for example, that each of eight iterations of FR120 was associated with delivery of a single reinforcer unit and one instance of FR960 was associated with eight reinforcer units. FR requirement varied between 20 and 1560 per aggregate reinforcer and unit price varied between 20 and 240 per reinforcer unit. During multiple schedules with food reinforcers, rates and patterns of responding were comparable over nearly a 50-fold range of FR requirements (20-1380) when unit price was 20; over nearly a six-fold range of FR requirements (120-720) when unit price was 120; and was only marginally maintained when unit price was 240. Demand for food pellets was comparatively inelastic at FRs between 20 and 120, during which subjects did not receive supplemental feeding outside experimental sessions, but was elastic at FRs greater than 240, when subjects sometimes did receive supplemental feeding. In a discrete-trial choice procedure with a constant unit price of 120 for sucrose solution, subjects were indifferent between smaller FRs and alternative FRs as large as 480, but began switching away from larger FRs that were 600 or greater. Because responding had been comparably maintained under both FR120 and FRs as large as 960 in the multiple schedule, results from the choice procedure indicated that choice performance was influenced by variables other than FR requirement and unit price. Because aggregate reinforcers were the same for smaller and larger FRs, the most likely reason for preferring smaller FRs was the nearness in time to some reinforcer.     

Species differences between rats and pigeons in choices with probabilistic and delayed reinforcers

An adjusting-delay procedure was used to study rats' choices with probabilistic and delayed reinforcers, and to compare them with previous results from pigeons. A left lever press led to a 5-s delay signaled by a light and a tone, followed by a food pellet on 50% of the trials. A right lever press led to an adjusting delay signaled by a light followed by a food pellet on 100% of the trials. In some conditions, the light and tone for the probabilistic reinforcer were present only on trials that delivered food. In other conditions, the light and tone were present on all trials that the left lever was chosen. Similar studies with pigeons [Mazur, J.E., 1989. Theories of probabilistic reinforcement. J. Exp. Anal. Behav. 51, 87-99; Mazur, J.E., 1991. Conditioned reinforcement and choice with delayed and uncertain primary reinforcers. J. Exp. Anal. Behav. 63, 139-150] found that choice of the probabilistic reinforcer increased dramatically when the delay-interval stimuli were omitted on no-food trials, but this study found no such effect with the rats. In other conditions, the probability of food was varied, and comparisons to previous studies with pigeons indicated that rats showed greater sensitivity to decreasing reinforcer probabilities. The results support the hypothesis that rats' choices in these situations depend on the total time between a choice response and a reinforcer, whereas pigeons' choices are strongly influenced by the presence of delay-interval stimuli.     

The matching law and effects of reinforcer rate and magnitude on choice in transition

Four pigeons responded in a concurrent-schedule procedure in which reinforcer rates and magnitudes changed unpredictably across sessions according to independent random series. Programmed relative reinforcement rates and magnitudes were always either 2:1 or 1:2. Pigeons' response allocation tended to stabilize within sessions and multiple regression analyses showed that it was determined by rates and magnitudes from the current session. Sensitivity coefficients were positive and statistically significant for current-session reinforcement and magnitude ratios. Although there were individual differences in sensitivity to rate and magnitude, their interaction was not significant across subjects. Rate and magnitude both controlled responding in single sessions and individual interreinforcer intervals. Analyses of responding within sessions showed that preference was more extreme when the richer rate and larger magnitude were associated with the same alternative than when they were associated with different alternatives. Overall, results support the concatenated generalized matching law's assumptions of additivity and independence as applied to choice in transition.     

Demand for sucrose in the genetically obese Zucker (fa/fa) rat

Obese Zucker rats (fa/fa) eat more food than lean controls in free-feeding conditions, which strongly influences their phenotypic expression. Few studies, however, characterize their food consumption in environments that are more representative of foraging conditions, e.g., how effort plays a role in food procurement. This study examined the reinforcing efficacy of sucrose in obese Zucker rats by varying the responses required to obtain single sucrose pellets. Male Zucker rats (15 lean, 14 obese) lever-pressed under eight fixed ratio (FR) schedules of sucrose reinforcement, in which the number of lever-presses required to gain access to a single sucrose pellet varied from 1 to 300. Linear and exponential demand equations, which characterize the value of a reinforcer by its sensitivity to price (FR), were fit to the number of food reinforcers and responses made. Free food consumption was also examined. Obese Zuckers, compared to leans, consumed more food under free-feeding conditions. Moreover, they had higher levels of consumption and response output, but only at low FR values. Both groups were equally sensitive to price increases at higher FR values. This suggests that environmental conditions may interact with genes in the expression of food reinforcer efficacy.     

Food preference of young rhesus monkeys fed marginally zinc deficient diets

Zinc deprivation has been shown to produce hypogeusia in rhesus monkeys but the possible consequences of altered taste sensitivity for food preference and selection is not known. We studied 13 rhesus monkeys (2.5 to 3 years of age) for food preference in a structured choice situation. These animals were fed two levels of dietary zinc: marginally zinc deprived (fed 4 μg zinc/g diet from conception and throughout development;N=6), or control (fed a 100 μg zinc/g diet throughout the study;N=7). Preference for familiar vs unfamiliar food items, order of food preferences, persistence in retrieval of preferred and nonpreferred foods, and preference under four deprivation periods were examined. Animals were offered a choice of two food items, both containing minimal zinc levels, and food choice and latency of choice were measured. Results indicate that animals fed the marginal zinc diets had reduced preference for unfamiliar foods relative to controls and different patterns of food preference. These results may be relevant to maintenance of appropriate food selection in marginally zinc deprived populations.     

Recovery growth following periods of restricted rations and starvation in rainbow trout Salmo gairdneri Richardson

Fingerling rainbow trout (6.9 to 11.6 cm fork length) maintained at 12° C were either: fed restricted rations (3% of dry body weight/day) for 16 weeks, or starved for 3 weeks (14.5% weight loss), or for 13 weeks (32.5% weight loss). Whole body wet weights and dry weights (as % of wet weight), and wet and dry weights of tissues (heart, spleen, liver, skin, gonad, gut, visceral fat, and 'carcase') resulting from these treatments were compared to corresponding weights of an 'initial sample'. Visceral fat was completely utilized during both short and long-term starvation and gut was significantly reduced in the extended starvation group.
During subsequent recovery growth on full rations, following food deprivation, fish grew at approximately the same rate as fully-fed controls, which had not suffered food deprivation, as regards body wet weight and condition ( K ), but tended to surpass controls in % dry weight.
Heart, liver, gonad, gut and visceral fat also increased during recovery growth in a manner suggestive of overcompensation in response to former food deprivation. Gut, skin and carcase dry weights were less in the control, and in the 14.5% weight loss groups following recovery growth than in those in 3% rations and in the severely starved group, indicating that the effects of slow growth resulting from limited rations resemble those of severe starvation more than those of limited starvation.     

The effect of chronic food and water restriction on open-field behaviour and serum corticosterone levels in rats

In operant conditioning experiments, two methods are commonly used to motivate laboratory rats to perform designated tasks. The first is restricting food so that rats are forced to lose 20% of body weight within one week, followed by maintenance at 80% of the baseline weight for the remainder of the experiment. The second is restricting access to water to 15 min in each 24 h period. These methods are effective in motivating the animals. There is, however, little information available on the effects on performance in tests of behaviour that are not related to operant conditioning. In addition, it is not clear if these commonly used methods of food and water restriction will lead to physiological stress as indicated by an elevation of serum corticosterone. Male rats were either food-restricted to reduce and maintain their weight at 80% of baseline weight, or were restricted to 15 min access to water every 24 h. Activity in the open field was significantly greater in food-restricted rats than in water-restricted or control rats, but freezing behaviour was similar in all experimental groups. Food-restricted rats had a higher mean serum corticosterone level than water-restricted and control rats 37 days after the start of the experimental period. These data suggested that chronically restricting food and maintenance of body weight at 80% of baseline body weight led to significant behavioural changes and physiological stress. In contrast, water restriction did not lead to changes in behaviour or corticosterone levels. A second experiment was conducted to compare the effects of food restriction to 80% of baseline body weight, as described above, with a less stringent protocol in which test rats were initially reduced to 80% of baseline weight, but were then maintained at 80% of an ad libitum fed control rat's weight. Serum corticosterone levels and adrenal gland weights were measured after the initial week of forced weight loss and after maintenance for 21 days. Forced loss of 20% of body weight in the first week led to significantly increased serum corticosterone levels and adrenal gland weights compared to ad libitum fed controls. Serum corticosterone levels and adrenal gland weights in rats maintained at 80% of their initial body weight for 21 days remained higher than ad libitum fed control rats. However, rats maintained at 80% of an ad libitum fed control rat's weight did not differ from control rats in serum corticosterone levels or adrenal gland weights at the end of the 21-day study period. Adjustment of the feeding regimen in this manner eliminated physiological evidence of chronic stress.     

Comparative effects of food and water deprivation on movement patterns in the pigeon (Columba Livia)

Groups of naive pigeons were either food or water deprived and exposed to response-independent fixed time schedules of reinforcement. Reinforcers were always appropriate to the deprivation state. After 15 Phase One sessions under the original deprivation state, an additional 15 Phase Two sessions were run under the alternate deprivation regimen. The Phase One results indicated that the frequency of movement over time, as measured by floorboard panels, had a characteristic distribution depending on deprivation state. After state-switch in Phase Two, both groups exhibited changes in frequency distribution over time, but the new distributions were different from those produced in subjects without any deprivation history. There appeared to be sustained effects on movement patterns due to prior exposure to alternate deprivation and periodic feeding regiments, results which confirm earlier experimental work. The findings are discussed in light of research on the effect of prior response-independent schedule training, the matching law, and the effects of contextual conditioning.     

Reinforcement omission in concurrent fixed-interval and random-interval schedules

The present experiment examined overall and local effects of omission of reinforcers in a choice situation. Pigeons' key-pecking responses were reinforced under concurrent fixed-interval and random-interval schedules of food presentation. After some weeks of baseline sessions in which the probability of reinforcement was 1.00, approximately 25% of food presentations from the fixed-interval schedule were omitted and replaced by timeout periods. In such omission sessions, the overall relative rates of responding to the fixed-interval schedule became lower than those in the baseline sessions. On the other hand, when relative rates of responding to the fixed-interval schedule in the omission sessions were calculated separately for fixed-interval cycles preceded by timeout periods and those preceded by food presentations, the relative rates in the former type of fixed-interval cycles were higher than those in the latter type for three out of four pigeons. These results mean that relative rates of responding cannot always be regarded as reflecting a relative value of an alternative, and that the overall effect of the omission of fixed-interval reinforcers is not reducible to the local effect of omission.     

Transitional choice behavior in concurrent-chain schedules

The choice responses of four pigeons were examined in 20 periods of transition in a concurrent-chain procedure with variable-interval schedules as initial links and fixed delays to reinforcement as terminal links. In some conditions, the delays to reinforcement were different for the two terminal links, and changes in preference were recorded after the delays for the two response keys were switched. In other conditions, the reinforcer delays were equal for the two keys, but which key delivered 80% of the reinforcers was periodically switched. Choice proportions changed more quickly after a switch in reinforcement percentages than after a switch in the delays, thereby contradicting the hypothesis that faster changes would occur when the switch in conditions was easier to discriminate. Analyses of response sequences showed that the effects of individual reinforcers were larger and lasted longer in conditions with changing reinforcement percentages than in conditions with changing terminal-link delays. Rates of change in choice behavior do not appear to be limited by the unpredictability of variable reinforcement schedules, because the changes in behavior were slow and gradual even when there was a large and sudden change in reinforcer delays.     

Human performance on a two-alternative rapid-acquisition choice task

Davison and Baum [Davison, M., Baum, W. M., 2000. Choice in a variable environment: every reinforcer counts. Journal of the Experimental Analysis of Behavior 74, 1-24.] developed a concurrent-schedule procedure where, within each session, different reinforcer ratios were arranged across components separated by brief black-outs. Behaviour adapted quickly to the reinforcer ratios and reinforcers also had local effects on responding. This procedure has been used with pigeons and rats. In the present experiment, we adapted the Davison and Baum procedure to study the effects of reinforcement on human choice behaviour. Eighteen participants were presented with four different reinforcer ratios within a single 50-minute session. Mean sensitivity to the reinforcer ratios increased within components, and preference was greater for the just-reinforced response alternative immediately following reinforcer delivery, similar to the results from non-human experiments. Although there were limitations to the current procedure, the local time scale analyses are a novel way of examining human operant behaviour.     

Postnatal protein deprivation in rhesus monkeys

We studied the growth of 97 rhesus monkeys (M. mulatta) that from birth until 120 days of age were fed a diet containing 13.4%, 6.7%, 3.35% protein or a commercial simulated human-milk formula (SMA) affording 9% protein. After 120 days, all animals were fed SMA. Females fed the diet lowest in protein, but not the other diets, were moderately affected, gaining less weight than their well-fed counterparts did, but they recovered their deficit during the repletion period, so that at 240 days no group differences remained. Males fed the two lowest-protein diets were severely and profoundly affected, in keeping with the depth of their protein deficiency. Moreover, unlike the females, they recovered none of their weight deficit during the repletion period. Food intake during deprivation was lower in the animals eating the low-protein diets. During SMA repletion, intakes followed no simple rule, but they did not converge. Except for the males fed the 3.35% protein diet, relative food intake (ml formula/kg body weight) did not differ substantially between diets or sexes at any time during the deprivation and diminished as the animals got older. Those males' relative intakes did not decline. During repletion, the SMA females ate the most in proportion to weight and the 13.4% group the least. The lowest-protein males, in contrast, ate the most in proportion to their weight during both deprivation and repletion. Males fed the lowest-protein diet gained little for what they ate; those fed the highest-protein diet gained much. Females were more efficient than males were when eating the low-protein diet and less efficient when eating the high-protein diet. When fed SMA during repletion, males' food efficiencies (grams gained/liter of diet) were nearly equal; females still differed: the SMA group was the least efficient and the low-protein group most efficient. Its counterpart, protein efficiency, was greatest (during deprivation) for females eating the low-protein diet and least (among females) for those eating the high-protein diet. Males were least protein efficient if eating the low-protein diet. Evidently, a 4 month bout of protein deprivation had prolonged effects on the amount of food the animals needed to produce a given gain in weight.     

Tradeoffs among delay, rate, and amount of reinforcement

In Experiment 1, an adjusting-delay procedure was used to measure pigeons' choices between a single delayed reinforcer and a range of different variable-time schedules. Indifference points showed an inverse relation between rate of reinforcement and delay that was well described by a hyperbolic equation. An adjusting-amount procedure was used in Experiment 2, in which pigeons chose between an adjusting amount of food delivered after a 0.5-s delay and 3 s of food delivered after a range of different delays, and the effects of delay were similar to those found in Experiment 1. The results from both experiments indicated that, for pigeons, the strength of a reinforcer decreased rapidly with increasing delay. Estimates of a decay rate parameter in the hyperbolic equation were similar to those found in other studies with pigeons, but the rates of temporal discounting were three or four times faster than those found in studies with rats, suggesting a possible species difference.     

A comparison of dietary selection behaviour in larval Locusta migratoria and Spodoptera littoralis

ABSTRACT. . Final instar nymphs of the oligophagous acridid Locusta migratoria (L.) and larvae of the polyphagous noctuid Spodoptera littoralis (Boisduval) were fed for 4, 8 or 12 h, the conditioning period, on one of four artificial diets. Of these, diet PC contained 20% protein and 10% digestible carbohydrate; another, P, contained 20% protein but with the digestible carbohydrate component replaced by cellulose; a third, C, had the protein component substituted by cellulose, and the fourth, O, had both protein and digestible carbohydrate replaced. After this conditioning period, insects were given a choice of two diets, P and C, and hence an opportunity to select for the nutrients, if any, which were lacking in their previous food. Amounts eaten and selection behaviour were then recorded in detail for a total of 9 h. This paper deals with total amounts of diet eaten during the conditioning and choice periods. Spodoptera larvae were more sensitive than the locusts to being fed a nutritionally inadequate conditioning diet, and ate only small quantities of the P, C and O diets as compared with the PC diet, irrespective of the duration of conditioning. Locusts, on the other hand, when restricted to the P diet continued to eat relatively large amounts of it throughout a 12 h conditioning period. Those nymphs fed the C diet ingested large quantities (more than of the PC diet) up until 8 h, after which intake fell. When offered a choice, both species selected for the nutrients missing from the conditioning diet, even if the conditioning period had been as short as 4 h. During the first hour of choice locusts selected the P diet if they had been previously fed C and the C diet if previously fed P. Those deprived of both nutrients increased consumption of both P and C diets. Spodoptera larvae were more sensitive to prior deprivation of digestible carbohydrate than of protein. During the first hour of choice they selected the C diet if previously fed P or O but did not choose the P diet if previously fed C. In the subsequent 8 h of choice, however, a strong selection for the P diet after previous deprivation became apparent. In the locust, the selection for nutrients missing from the conditioning diet continued for the following 8 h of choice but became masked by a tendency, shown by all nymphs, to select C over P. The functional significance and possible physiological basis of all these responses is discussed.     

Choice in a variable environment: Effects of d-amphetamine on sensitivity to reinforcement

Four pigeons responded under a 7-component mixed schedule in which each component arranged a different left:right reinforcer ratio (27:1, 9:1, 3:1, 1:1, 1:3, 1:9, 1:27). Components were unsignaled, and the order within each session was randomly determined. After extensive exposure to these contingencies, effects of a range of doses of d-amphetamine (0.3-5.6 mg/kg) on estimates of sensitivity to reinforcement at several levels of analysis were assessed. Under non-drug conditions, the structure of choice was similar to that previously reported under this procedure. That is, responding adjusted within components to the reinforcer ratio in effect (i.e., sensitivity estimates were higher in the 2nd than in the 1st half of components), and individual reinforcers produced “preference pulses” (i.e., each food presentation produced an immediate, local, shift in preference toward the response that just produced food). Although there was a general tendency for d-amphetamine to reduce overall sensitivity to reinforcement, the size of this effect and its reliability varied across pigeons. Further analysis, however, revealed that intermediate d-amphetamine doses consistently reduced sensitivity immediately following reinforcer presentations; that is, these doses consistently attenuated preference pulses.     

Different Levels of Food Restriction Reveal Genotype-Specific Differences in Learning a Visual Discrimination Task

In behavioural experiments, motivation to learn can be achieved using food rewards as positive reinforcement in food-restricted animals. Previous studies reduce animal weights to 80–90% of free-feeding body weight as the criterion for food restriction. However, effects of different degrees of food restriction on task performance have not been assessed. We compared learning task performance in mice food-restricted to 80 or 90% body weight (BW). We used adult wildtype (WT; C57Bl/6j) and knockout (ephrin-A2^−/−) mice, previously shown to have a reverse learning deficit. Mice were trained in a two-choice visual discrimination task with food reward as positive reinforcement. When mice reached criterion for one visual stimulus (80% correct in three consecutive 10 trial sets) they began the reverse learning phase, where the rewarded stimulus was switched to the previously incorrect stimulus. For the initial learning and reverse phase of the task, mice at 90%BW took almost twice as many trials to reach criterion as mice at 80%BW. Furthermore, WT 80 and 90%BW groups significantly differed in percentage correct responses and learning strategy in the reverse learning phase, whereas no differences between weight restriction groups were observed in ephrin-A2^−/− mice. Most importantly, genotype-specific differences in reverse learning strategy were only detected in the 80%BW groups. Our results indicate that increased food restriction not only results in better performance and a shorter training period, but may also be necessary for revealing behavioural differences between experimental groups. This has important ethical and animal welfare implications when deciding extent of diet restriction in behavioural studies.     

Reward context determines risky choice in pigeons and humans

Whereas humans are risk averse for monetary gains, other animals can be risk seeking for food rewards, especially when faced with variable delays or under significant deprivation. A key difference between these findings is that humans are often explicitly told about the risky options, whereas non-human animals must learn about them from their own experience. We tested pigeons (Columba livia) and humans in formally identical choice tasks where all outcomes were learned from experience. Both species were more risk seeking for larger rewards than for smaller ones. The data suggest that the largest and smallest rewards experienced are overweighted in risky choice. This observed bias towards extreme outcomes represents a key step towards a consilience of these two disparate literatures, identifying common features that drive risky choice across phyla.     

Recruitment in starved nests: the role of direct and indirect interactions between scouts and nestmates in the ant <Emphasis Type="Italic">Lasius niger</Emphasis>

In social insects, the foraging activity usually increases with the length of food deprivation. In Lasius niger, a mass-recruiting ant species, the foraging adjustment to the level of food deprivation is regulated by the scout that fed at the food source and by the response of the nestmates to signals performed by the scout inside the nest. In this study, we look at the role of these direct interactions (antennations or trophallaxes) and indirect interactions (pheromonal emission) and how they are influenced by the level of food deprivation. At the beginning of recruitment, the relative number of nestmates leaving the nest to forage increases with the level of deprivation. The nestmates’ propensity to exit the nest is not influenced by a previous trophallactic and/or antennal contact with a scout. Our results strongly suggest that the exit of nestmates is triggered by a chemical signal emitted by a scout. Deprivation lowers the response threshold of nestmates to this chemical signal resulting in a more important exit from the nest. Surprisingly, 27% of starved nestmates that receive food from the scout relay the information by depositing a chemical signal before having discovered and drunk the food source. Both phenomena boost the recruitment process. Though successful foragers returning to the nest have a significant role in the recruitment to the food source, we observed that the response of the nestmates inside the nest also greatly influence regulation of the foraging activity.