Macaque Monkeys Can Learn Token Values from Human Models through Vicarious Reward

Monkeys can learn the symbolic meaning of tokens, and exchange them to get a reward. Monkeys can also learn the symbolic value of a token by observing conspecifics but it is not clear if they can learn passively by observing other actors, e.g., humans. To answer this question, we tested two monkeys in a token exchange paradigm in three experiments. Monkeys learned token values through observation of human models exchanging them. We used, after a phase of object familiarization, different sets of tokens. One token of each set was rewarded with a bit of apple. Other tokens had zero value (neutral tokens). Each token was presented only in one set. During the observation phase, monkeys watched the human model exchange tokens and watched them consume rewards (vicarious rewards). In the test phase, the monkeys were asked to exchange one of the tokens for food reward. Sets of three tokens were used in the first experiment and sets of two tokens were used in the second and third experiments. The valuable token was presented with different probabilities in the observation phase during the first and second experiments in which the monkeys exchanged the valuable token more frequently than any of the neutral tokens. The third experiments examined the effect of unequal probabilities. Our results support the view that monkeys can learn from non-conspecific actors through vicarious reward, even a symbolic task like the token-exchange task.     

Reinforcer satiation and resistance to change of responding maintained by qualitatively different reinforcers

In previous research on resistance to change, differential disruption of operant behavior by satiation has been used to assess the relative strength of responding maintained by different rates or magnitudes of the same reinforcer in different stimulus contexts. The present experiment examined resistance to disruption by satiation of one reinforcer type when qualitatively different reinforcers were arranged in different contexts. Rats earned either food pellets or a 15% sucrose solution on variable-interval 60-s schedules of reinforcement in the two components of a multiple schedule. Resistance to satiation was assessed by providing free access either to food pellets or the sucrose solution prior to or during sessions. Responding systematically decreased more relative to baseline in the component associated with the satiated reinforcer. These findings suggest that when qualitatively different reinforcers maintain responding, relative resistance to change depends upon the relations between reinforcers and disrupter types.     

Language-trained chimpanzees (Pan troglodytes) delay gratification by choosing token exchange over immediate reward consumption

Token exchange inherently introduces an element of delay between behavior and reward and so token studies may help us better understand delay of gratification and self-control. To examine this possibility, we presented three language-trained chimpanzees with repeated choices involving different foods that could be eaten immediately or lexigram (graphic symbol) tokens that represented (and could be traded for) foods later. When both options were foods, chimpanzees always chose more preferred foods over less preferred foods. When both options were lexigram tokens representing those same foods, performance remained the same as chimpanzees selected the higher value token and then traded it for food. Then, when faced with choosing a token that could be traded later or choosing a food item that could be eaten immediately, most chimpanzees learned to make whatever response led to the more preferred food. They did this even when that meant selecting a high value lexigram token that could be traded only 2 to 3 min later instead of a medium value, but immediately available, food item. Thus, chimpanzees flexibly selected tokens even though such selections necessarily delayed gratification and required forgoing immediately available food. This finding illustrates the utility of symbolic token exchange for assessing self-control in nonhuman animals.     

Habituation and the reinforcing effectiveness of visual stimuli

The term "sensory reinforcer" has been used to refer to sensory stimuli (e.g. light onset) that are primary reinforcers in order to differentiate them from other more biologically important primary reinforcers (e.g. food and water). Acquisition of snout poke responding for a visual stimulus (5s light onset) with fixed ratio 1 (FR 1), variable-interval 1min (VI 1min), or variable-interval 6min (VI 6min) schedules of reinforcement was tested in three groups of rats (n=8/group). The VI 6min schedule of reinforcement produced a higher response rate than the FR 1 or VI 1min schedules of visual stimulus reinforcement. One explanation for greater responding on the VI 6min schedule relative to the FR 1 and VI 1min schedules is that the reinforcing effectiveness of light onset habituated more rapidly in the FR 1 and VI 1min groups as compared to the VI 6min group. The inverse relationship between response rate and the rate of visual stimulus reinforcement is opposite to results from studies with biologically important reinforcers which indicate a positive relationship between response and reinforcement rate. Rapid habituation of reinforcing effectiveness may be a fundamental characteristic of sensory reinforcers that differentiates them from biologically important reinforcers, which are required to maintain homeostatic balance.     

Contextual determinants of temporal control: Behavioral contrast in a free-operant psychophysical procedure

The question of how temporal control of responding might be influenced by contingency changes in other contexts was investigated. Each of three pigeons first was exposed to a two-component multiple schedule in which a two-key free-operant psychophysical procedure operated in one component and a variable-interval schedule operated in the other component. The variable-interval schedule then was changed to extinction while the free-operant psychophysical procedure remained unchanged. Finally, the variable-interval schedule was reintroduced. Response rates on the left key and the estimated temporal threshold under the free-operant psychophysical procedure increased for each pigeon when the alternate component schedule was changed to extinction and then decreased again when the variable-interval schedule was reintroduced. The results suggest one way that temporal control is affected by its context, and may be interpreted through the direct effects of overall reinforcement rate on temporal control mechanisms or the disruptive effects of alternative sources of reinforcement on temporally controlled behavior.     

Reinforcement omission in concurrent fixed-interval and random-interval schedules

The present experiment examined overall and local effects of omission of reinforcers in a choice situation. Pigeons' key-pecking responses were reinforced under concurrent fixed-interval and random-interval schedules of food presentation. After some weeks of baseline sessions in which the probability of reinforcement was 1.00, approximately 25% of food presentations from the fixed-interval schedule were omitted and replaced by timeout periods. In such omission sessions, the overall relative rates of responding to the fixed-interval schedule became lower than those in the baseline sessions. On the other hand, when relative rates of responding to the fixed-interval schedule in the omission sessions were calculated separately for fixed-interval cycles preceded by timeout periods and those preceded by food presentations, the relative rates in the former type of fixed-interval cycles were higher than those in the latter type for three out of four pigeons. These results mean that relative rates of responding cannot always be regarded as reflecting a relative value of an alternative, and that the overall effect of the omission of fixed-interval reinforcers is not reducible to the local effect of omission.     

Crows and common ravens do not reciprocally exchange tokens with a conspecific to gain food rewards

Human economic transactions are based on complex forms of reciprocity, which involve the capacities to share and to keep track of what was given and received over time. Animals too engage in reciprocal interactions, but mechanisms such as calculated reciprocity have only been shown experimentally in few species. Various forms of cooperation, for example food and information sharing, are frequently observed in corvids, and they can engage in exchange interactions with human experimenters and accept delayed rewards. Here, we tested whether crows and common ravens would reciprocally exchange tokens with a conspecific in an exchange task. Birds received a set of three different types of tokens, some valuable for themselves, that is, they could exchange them for a food reward with a human experimenter, some valuable for their partner and some without value. The valuable tokens differed between the birds, which means that each bird could obtain more self-value tokens from their partner's compartment. We did not observe any active transfers, that is one individual giving a token to the experimental partner by placing it in its beak. We only observed 6 indirect transfers, that is one individual transferring a token into the compartment of the partner (3 no-value, 1 partner-value and 2 self-value tokens) and 67 “passive” transfers, that is one subject taking the token lying in reach in the compartment of the partner. Individuals took significantly more self-value tokens compared with no-value and partner-value tokens. This indicates a preference for tokens valuable to focal individuals. Significantly more no-value tokens compared with partner-value tokens were taken, likely to be caused by experimental partners exchanging self-value tokens with the human experimenter, and therefore, more no-value tokens being available in the compartment. Our results presently do not provide empirical support for reciprocity in crows and ravens, most likely caused by them not understanding the potential roles of receiver and donor. We therefore suggest further empirical tests of calculated reciprocity to be necessary in corvids.     

Naloxone enhancement of punishment in the rat

Rats were trained to lever press on a variable interval one minute schedule of reinforcement for food pellets. Subsequently in punishment sessions, lever presses also produced electric grid shocks on a fixed ratio 20 schedule. With punishment, lever pressing levels were reduced by about 15%. When naloxone was administered prior to punishment sessions, shocks reduced lever pressing levels by about 50% following the 3 mg/kg dose and by aboaut 80% following the 10 mg/kg dose. This naloxone-induced enhancement of the effect of punishment was probably mediated through an antagonism of naturally-occurring brain peptides with opiate-like activities.     

A concept of value during experimental exchange in brown capuchin monkeys, Cebus apella

We evaluated the response of brown capuchin monkeys to two differentially valued tokens in an experimental exchange situation akin to a simple barter. Monkeys were given a series of three tests to evaluate their ability to associate tokens with food, then their responses were examined in a barter situation in which tokens were either limited or unlimited. Capuchins did not perform barter in the typical sense, returning the tokens which were associated with the reward. However, females, but not males, showed a different response, preferring the higher-value token. This may indicate that they learned to prefer one token over the other rather than to associate the tokens with their specific rewards. This sex difference parallels previous findings of greater reciprocity in female brown capuchins than in males.     

How Social Context,Token Value,and Time Course Affect Token Exchange in Capuchin Monkeys (<Emphasis Type="Italic">Cebus apella</Emphasis>)

Although numerous studies have examined token-directed behaviors in primates, few have done so in a social context despite the fact that most primate species live in complex groups. Here, we provided capuchin monkeys with a relatively limited budget of tokens, likely to elicit intragroup competition, and, after an overnight delay, we allowed them to exchange tokens while in a group setting. We aimed to 1) evaluate whether social context affects token-directed behaviors of knowledgeable subjects, i.e., subjects already proficient in token exchange before the present study, as well as of naïve subjects, i.e., subjects that never showed exchange behavior before this study; 2) appraise whether capuchins indeed value tokens; and 3) assess whether capuchins can refrain from throwing tokens outside their enclosure when the experimenter is not present. Overall, the social context positively affected high-ranking individuals and negatively affected low-ranking ones. All 6 high-ranking naïve subjects, but none of the 4 low-ranking ones, quickly acquired token exchange behavior, whereas 9 of 12 low-ranking knowledgeable subjects, but only 1 high-ranking knowledgeable subject, never displayed token exchange in social contexts. Thus, competition constrained token exchange in low-ranking subjects and prompted exchange behavior in high-ranking naïve subjects. Capuchins were unable to inhibit the exchange of valueless items when the experimenter was soliciting them and, at the group level, knowledgeable subjects did not exchange more valuable tokens than less valuable (or valueless) ones. However, the 3 high-ranking knowledgeable subjects that exchanged most of the tokens first preferentially exchanged more valuable tokens over less valuable or valueless ones. Finally, capuchins inhibited exchange behavior in the absence of the experimenter, thus recognizing the appropriate conditions in which a successful exchange could occur.     

Segmentation and the pairing hypothesis

The effect of stimulus contiguity and response contingency on responding in chain schedules was examined in two experiments. In Experiment 1, four pigeons were trained on two simple three-link chain schedules that alternated within sessions. Initial links were correlated with a variable-interval 30s schedule, and middle and terminal links were correlated with interdependent variable-interval 30s variable-interval 30s schedules. The combined duration of the interdependent schedules summed to 60s. The two chains differed with respect to signaling of the schedule components: a two-stimulus chain had one stimulus paired with the initial link and one stimulus paired with both the middle and the terminal link, while a three-stimulus chain had a different stimulus paired with the each of the three links. The results showed that the two-stimulus chain maintained lower initial-link responding than the three-stimulus chain. In Experiment 2, four pigeons were exposed to three separate conditions, the two- and three-stimulus chains of Experiment 1 and a three-stimulus chain that had a 3s delay to terminal-link entry from the middle-link response that produced it. The two-stimulus chain maintained lower initial-link responding than the three-stimulus chain, as in Experiment 1, and a similar initial-link responding was maintained by the two-stimulus chain and the three-stimulus chain with the delay contingency. The results demonstrate that a stimulus noncontiguous with food can maintain responding that is sometimes greater than a stimulus contiguous with food, depending on the response contingency for terminal-link entry. The results are contrary to the pairing hypothesis of conditioned reinforcement.     

Response-reinforcer relations and resistance to change

Behavioral momentum theory suggests that the relation between a response and a reinforcer (i.e., response-reinforcer relation) governs response rates and the relation between a stimulus and a reinforcer (i.e., stimulus-reinforcer relation) governs resistance to change. The present experiments compared the effects degrading response-reinforcer relations with response-independent or delayed reinforcers on resistance to change in conditions with equal stimulus-reinforcer relations. In Experiment 1, pigeons responded on equal variable-interval schedules of immediate reinforcement in three components of a multiple schedule. Additional response-independent reinforcers were available in one component and additional delayed reinforcers were available in another component. The results showed that resistance to disruption was greater in the components with added reinforcers than without them (i.e., better stimulus-reinforcer relations), but did not differ for the components with added response-independent and delayed reinforcement. In Experiment 2, a component presenting immediate reinforcement alternated with either a component that arranged equal rates of reinforcement with a proportion of those reinforcers being response independent or a component with a proportion of the reinforcers being delayed. Results showed that resistance to disruption tended to be either similar across components or slightly lower when response-reinforcer relations were degraded with either response-independent or delayed reinforcers. These findings suggest that degrading response-reinforcer relations can impact resistance to change, but that impact does not depend on the specific method and is small relative to the effects of the stimulus-reinforcer relation.     

Acquisition of object-robbing and object/food-bartering behaviours: a culturally maintained token economy in free-ranging long-tailed macaques

The token exchange paradigm shows that monkeys and great apes are able to use objects as symbolic tools to request specific food rewards. Such studies provide insights into the cognitive underpinnings of economic behaviour in non-human primates. However, the ecological validity of these laboratory-based experimental situations tends to be limited. Our field research aims to address the need for a more ecologically valid primate model of trading systems in humans. Around the Uluwatu Temple in Bali, Indonesia, a large free-ranging population of long-tailed macaques spontaneously and routinely engage in token-mediated bartering interactions with humans. These interactions occur in two phases: after stealing inedible and more or less valuable objects from humans, the macaques appear to use them as tokens, by returning them to humans in exchange for food. Our field observational and experimental data showed (i) age differences in robbing/bartering success, indicative of experiential learning, and (ii) clear behavioural associations between value-based token possession and quantity or quality of food rewards rejected and accepted by subadult and adult monkeys, suggestive of robbing/bartering payoff maximization and economic decision-making. This population-specific, prevalent, cross-generational, learned and socially influenced practice may be the first example of a culturally maintained token economy in free-ranging animals.This article is part of the theme issue ‘Existence and prevalence of economic behaviours among non-human primates''.     

Tokens improve capuchin performance in the reverse–reward contingency task

In humans and apes, one of the most adaptive functions of symbols is to inhibit strong behavioural predispositions. However, to our knowledge, no study has yet investigated whether using symbols provides some advantage to non-ape primates. We aimed to trace the evolutionary roots of symbolic competence by examining whether tokens improve performance in the reverse–reward contingency task in capuchin monkeys, which diverged from the human lineage approximately 35 Ma. Eight capuchins chose between: (i) two food quantities, (ii) two quantities of ‘low-symbolic distance tokens’ (each corresponding to one unit of food), and (iii) two ‘high-symbolic distance tokens’ (each corresponding to a different amount of food). In all conditions, subjects had to select the smaller quantity to obtain the larger reward. No procedural modifications were employed. Tokens did improve performance: five subjects succeeded with high-symbolic distance tokens, though only one succeeded with food, and none succeeded with low-symbolic distance tokens. Moreover, two of the five subjects transferred the rule to novel token combinations. Learning effects or preference reversals could not account for the successful performance with high-symbolic distance tokens. This is, to our knowledge, the first demonstration that tokens do allow monkeys to inhibit strong behavioural predispositions, as occurs in chimpanzees and children.     

Reinforcer devaluation by extinction depends on the food restriction protocol

A common feature of reinforcer devaluation studies is that new learning induces the devaluation. The present study used extinction to induce new learning about the conditioned reinforcer in a heterogeneous chain schedule. Rats pressed a lever in a heterogeneous chain schedule to produce a conditioned reinforcer (light) associated with the opportunity to obtain an unconditioned reinforcer (food) by pulling a chain. The density of food reinforcement correlated with the conditioned reinforcer was varied in a comparison of continuous and variable-ratio reinforcement schedules of chain pulling; this had no noticeable effect on conditioned reinforcer devaluation produced by extinction of chain pulling. In contrast, how rats were deprived appeared to matter very much. Restricting meal duration to 1h daily produced more lever pressing during baseline training and a greater reductive effect of devaluation on lever pressing than restricting body weight to 80% of a control rat's weight, which eliminated the devaluation effect. Further analysis suggested that meal-duration restriction may have produced devaluation effects because it was more effective than weight restriction in reducing rats' body weights. Our results exposed an important limitation on the devaluation of conditioned reinforcers: slight differences in food restriction, using two commonly employed food-restriction procedures, can produce completely different interpretations of reinforcer devaluation while leaving reinforcer-based learning intact.     

Effects of a parasitic nematode on male mate choice in a livebearing fish with a coercive mating system (western mosquitofish, Gambusia affinis)

Studies of choice in steady state have shown that sensitivity to reinforcement increases with increasing fixed-ratio changeover (FR CO) requirements. We assessed the generality of this finding with choice in transition. Food deliveries were programmed according to concurrent variable-interval (VI) schedules. Seven different VI pairs arranged ratios of food deliveries (left/right) of 27:1, 9:1, 3:1, 1:1, 1:3, 1:9, and 1:27 at a constant overall rate across components. Within sessions, all seven ratios were presented in random order. Each component lasted for 10 food deliveries; components were separated by 60-s blackouts. A changeover lever required 1, 2, 4, 8, 16, 32, and 64 responses to alternate between two main levers. Redeterminations to all FR COs, but 64 responses, were obtained in descending order. Choice adjusted rapidly to rapid changes in the reinforcer ratio, tracking the lever associated with the highest probability of reinforcer. Sensitivity to reinforcement increased with increasing FR CO, replicating the negatively accelerated function found in our earlier study. With successive reinforcers in components, however, sensitivity reached asymptote values sooner with the largest (8, 16, and 32 responses), than with the smallest (1, 2, and 4 responses), FR CO requirements.     

Operant behavior in dwarf hamsters (Phodopus campbelli): Effects of rate of reinforcement and reinforcer flavor variety

We investigated operant behavior in a novel species, the dwarf hamster (Phodopus campbelli). In two experiments, hamsters were trained to lever-press for food reinforcement. In Experiment 1, rate of reinforcement was manipulated across conditions using four variable-interval schedules of reinforcement (delivering one to eight reinforcers per min). As predicted, within-session decreases in responding were steepest on the richest schedule. In Experiment 2, lever-pressing was reinforced by either a constant or a variety of flavored food pellets. Within-session decreases in responding were steeper when the reinforcer flavor remained constant than when it was varied within the session. In both experiments, subjects hoarded most reinforcers in their cheek pouches rather than consuming them in the operant chambers. These results are incompatible with post-ingestive satiety variables as explanations for within-session decreases in operant responding and suggest that habituation to repeatedly presented reinforcers best accounts for subjects’ response patterns. Additionally, a mathematical model that describes behavior undergoing habituation also described the present results, thus strengthening the conclusion that habituation mediates the reinforcing efficacy of food.     

Responses to a simple barter task in chimpanzees, Pan troglodytes

Chimpanzees (Pan troglodytes) frequently participate in social exchange involving multiple goods and services of variable value, yet they have not been tested in a formalized situation to see whether they can barter using multiple tokens and rewards. We set up a simple barter economy with two tokens and two associated rewards and tested chimpanzees on their ability to obtain rewards by returning the matching token in situations in which their access to tokens was unlimited or limited. Chimpanzees easily learned to associate value with the tokens, as expected, and did barter, but followed a simple strategy of favoring the higher-value token, regardless of the reward proffered, instead of a more complex but more effective strategy of returning the token that matched the reward. This response is similar to that shown by capuchin monkeys in our previous study. We speculate that this response, while not ideal, may be sufficient to allow for stability of the social exchange system in these primates, and that the importance of social barter to both species may have led to this convergence of strategies.     

Long-term effects of suppressing the preratio pause

The preratio pause is a characteristic feature of performances under fixed-ratio schedules of reinforcement, even though the pause is not required by the schedule and it reduces the reinforcement rate. To investigate the reduction of pausing, five rats trained on fixed-ratio schedules were exposed to timeout punishment of pauses that exceeded a specified duration. After a series of 30 punishment sessions, most of the longest pauses were eliminated. For some subjects punishment was withdrawn abruptly, whereas for others a fading procedure was employed. Postpunishment observations then were continued for an additional 60 sessions. The reduced pausing was accompanied by reductions in the positive skew of the baseline distribution of pause durations, and by substantial increases in reinforcement rates. However, the results did not indicate differences as a function of the method of withdrawing the punishment contingency. Although postpunishment performances indicated some degree of recovery in the number of long pauses, performances had stabilized below prepunishment levels when the experiment ended. The results suggest the possibility that reduced pause durations can be self-maintained by the resulting increase in reinforcement rates.     

Choice in a variable environment: Effects of d-amphetamine on sensitivity to reinforcement

Four pigeons responded under a 7-component mixed schedule in which each component arranged a different left:right reinforcer ratio (27:1, 9:1, 3:1, 1:1, 1:3, 1:9, 1:27). Components were unsignaled, and the order within each session was randomly determined. After extensive exposure to these contingencies, effects of a range of doses of d-amphetamine (0.3-5.6 mg/kg) on estimates of sensitivity to reinforcement at several levels of analysis were assessed. Under non-drug conditions, the structure of choice was similar to that previously reported under this procedure. That is, responding adjusted within components to the reinforcer ratio in effect (i.e., sensitivity estimates were higher in the 2nd than in the 1st half of components), and individual reinforcers produced “preference pulses” (i.e., each food presentation produced an immediate, local, shift in preference toward the response that just produced food). Although there was a general tendency for d-amphetamine to reduce overall sensitivity to reinforcement, the size of this effect and its reliability varied across pigeons. Further analysis, however, revealed that intermediate d-amphetamine doses consistently reduced sensitivity immediately following reinforcer presentations; that is, these doses consistently attenuated preference pulses.