Get It While It’s Hot: A Peak-First Bias in Self-Generated Choice Order in Rhesus Macaques

Animals typically must make a number of successive choices to achieve a goal: e.g., eating multiple food items before becoming satiated. However, it is unclear whether choosing the best first or saving the best for last represents the best choice strategy to maximize overall reward. Specifically, since outcomes can be evaluated prospectively (with future rewards discounted and more immediate rewards preferred) or retrospectively (with prior rewards discounted and more recent rewards preferred), the conditions under which each are used remains unclear. On the one hand, humans and non-human animals clearly discount future reward, preferring immediate rewards to delayed ones, suggesting prospective evaluation; on the other hand, it has also been shown that a sequence that ends well, i.e., with the best event or item last, is often preferred, suggesting retrospective evaluation. Here we hypothesized that when individuals are allowed to build the sequence themselves they are more likely to evaluate each item individually and therefore build a sequence using prospective evaluation. We examined the relationship between self-generated choice order and preference in rhesus monkeys in two experiments in which the distinctiveness of options were relatively high and low, respectively. We observed a positive linear relationship between choice order and preference among highly distinct options, indicating that the rhesus monkeys chose their preferred food first: i.e., a peak-first order preference. Overall, choice order depended on the degree of relative preference among alternatives and a peak-first bias, providing evidence for prospective evaluation when choice order is self-generated.     

A general framework for understanding the effects of variability and interruptions on foraging behaviour

A general framework for analysing the effects of variability and the effects of interruptions on foraging is presented. The animal is characterised by its level of energetic reserves, x. We consider behaviour over a period of time [0,T]. A terminal reward function R(x) determines the expected future reproductive success of an animal with reserves x at time T. For any state x at a time in the period, we give the animal a choice between various options and then constrain it to follow a background strategy. The best option is the one that maximizes expected future reproductive success. Using this framework, we show that sensitivity to variability in amount of energy gained is logically distinct from sensitivity to variability in the time at which food is obtained. We also show that incorporating interruptions results in both a preference for variability in time and a preference for a reward followed by a delay as opposed to the same delay before the reward.     

Context-dependent foraging decisions in rufous hummingbirds

A core assumption implicit in economic models of animal choice is that subjects assign absolute utilities to options that are independent of the type and number of alternatives available. Humans sometimes appear to violate this assumption and employ relative, as opposed to absolute, currencies when making choices. Recent evidence suggests that animals too might sometimes employ relative choice mechanisms. We tested this idea by measuring the foraging preferences of rufous hummingbirds (Selasphorus rufus) faced with choices analogous to those in which human use of relative currencies is evident. The birds experienced three treatments: a binary choice between two artificial flower types designated concentration (20 microl, 40% sucrose solution) and volume (40 microl, 20%), and two trinary treatments in which a third decoy option (either concentration decoy: 10 microl, 30% or volume decoy: 30 microl, 10%) was added to the set. The birds' preferences differed significantly across the three treatments. In the trinary treatments, the effect of the decoy options was to increase the preference for the option that dominated the decoy. These results are similar to those reported in the human choice literature, and are compatible with the hummingbirds using a relative evaluation mechanism in decision making.     

Quorum responses and consensus decision making

Animal groups are said to make consensus decisions when group members come to agree on the same option. Consensus decisions are taxonomically widespread and potentially offer three key benefits: maintenance of group cohesion, enhancement of decision accuracy compared with lone individuals and improvement in decision speed. In the absence of centralized control, arriving at a consensus depends on local interactions in which each individual''s likelihood of choosing an option increases with the number of others already committed to that option. The resulting positive feedback can effectively direct most or all group members to the best available choice. In this paper, we examine the functional form of the individual response to others'' behaviour that lies at the heart of this process. We review recent theoretical and empirical work on consensus decisions, and we develop a simple mathematical model to show the central importance to speedy and accurate decisions of quorum responses, in which an animal''s probability of exhibiting a behaviour is a sharply nonlinear function of the number of other individuals already performing this behaviour. We argue that systems relying on such quorum rules can achieve cohesive choice of the best option while also permitting adaptive tuning of the trade-off between decision speed and accuracy.     

The mating game: do opposites really attract?

When selecting a mate, females of many species face a complicated decision: choosing a very closely related mate will lead to inbreeding, while choosing a mate who is too genetically dissimilar risks breaking up beneficial gene complexes or local genetic adaptations. To ensure the best genetic quality of their offspring, the perfect compromise lies somewhere in between: an optimally genetically dissimilar partner. Empirical evidence demonstrating female preference for genetically dissimilar mates is proof of the adage 'opposites attract'. In stark contrast, Chandler & Zamudio (2008) show in this issue of Molecular Ecology that female spotted salamanders often choose males that are genetically more similar to themselves (although not if the males are small). Along with other recent work, these field studies highlight the broad spectrum of options available to females with respect to relatedness in their choice of mate that belies this rule of thumb.     

Gaze-dependent evidence accumulation predicts multi-alternative risky choice behaviour

Choices are influenced by gaze allocation during deliberation, so that fixating an alternative longer leads to increased probability of choosing it. Gaze-dependent evidence accumulation provides a parsimonious account of choices, response times and gaze-behaviour in many simple decision scenarios. Here, we test whether this framework can also predict more complex context-dependent patterns of choice in a three-alternative risky choice task, where choices and eye movements were subject to attraction and compromise effects. Choices were best described by a gaze-dependent evidence accumulation model, where subjective values of alternatives are discounted while not fixated. Finally, we performed a systematic search over a large model space, allowing us to evaluate the relative contribution of different forms of gaze-dependence and additional mechanisms previously not considered by gaze-dependent accumulation models. Gaze-dependence remained the most important mechanism, but participants with strong attraction effects employed an additional similarity-dependent inhibition mechanism found in other models of multi-alternative multi-attribute choice.     

Rational mate choice decisions vary with female age and multidimensional male signals in swordtails

Biologists have long been interested in intransitive preferences: circular preferences in which options cannot be ranked and no single option dominates, similar to a game of rock‐paper‐scissors. Intransitive preferences violate rational decision‐making, an assumption made by models of evolution by mate choice. Despite its potential importance in the study of sexual selection, few studies have tested for intransitive preferences. Even fewer have asked whether females differ in whether they choose mates transitively or intransitively and what factors might predict (in)transitive choice. Though intransitive choice is thought to be more common as options become more complex, this prediction is untested in animals. To fill this gap, we tested whether female Xiphophorus nigrensis swordtails can rank digitally animated males differing in size, courtship intensity, or both size and courtship intensity, and whether female responses were predicted by a female's age. Females choosing among males that varied only in size showed higher than expected levels of intransitivity, whereas females choosing among males that varied in their courtship or both properties did not. Older females were more likely to be irrational than younger females when evaluating male size, suggesting that experience modifies transitive decision‐making processes. These results show that mate choice irrationality may vary by a female's experience and the signal characteristics during decision‐making.     

Assessing the robustness and optimality of alternative decision rules with varying assumptions

Several alternative decision rules have been proposed for how individuals assess and choose options, such as mates and territories. Three of these rules are the threshold rule, where individuals choose the first option that exceeds a preset level of quality, the best-of- n rule, where individuals assess a fixed number of options and then choose the best of those options, and the comparative Bayes rule, where individuals use estimates of options to selectively assess and choose options. It has been previously concluded that the threshold rule produces higher average fitness than the best-of- n rule when assessment costs are not trivial. However, previous comparisons assumed that time and options are infinite, individuals can estimate the distribution of option quality without uncertainty or mistakes, and individuals receive perfect information about the quality of assessed options. I found that the best-of- n rule produces higher average fitness than the threshold rule despite significant assessment costs, when time for choosing an option is limited, when individuals are choosing from a small pool of options, when estimates of the distribution of option quality are error-prone, and when there is uncertainty about the distribution of option quality. I also found that the comparative Bayes rule produces higher average fitness than the threshold and best-of-n rules when time or options are limited and when individuals receive imperfect information about the quality of assessed options. Therefore, the optimality of alternative decision rules depends on more than the size of assessment costs and the previous conclusions of empirical studies that have assumed such need to be re-examined. Copyright 2002 The Association for the Study of Animal Behaviour. Published by Elsevier Science Ltd. All rights reserved.     

Behavioural and Physiological Effects of Finely Balanced Decision-Making in Chickens

In humans, more difficult decisions result in behavioural and physiological changes suggestive of increased arousal, but little is known about the effect of decision difficulty in other species. A difficult decision can have a number of characteristics; we aimed to monitor how finely balanced decisions, compared to unbalanced ones, affected the behaviour and physiology of chickens. An unbalanced decision was one in which the two options were of unequal net value (1 (Q1) vs. 6 (Q6) pieces of sweetcorn with no cost associated with either option); a finely balanced decision was one in which the options were of equal net value (i.e. hens were "indifferent" to both options). To identify hens'' indifference, a titration procedure was used in which a cost (electromagnetic weight on an access door) was applied to the Q6 option, to find the individual point at which hens chose this option approximately equally to Q1 via a non-weighted door. We then compared behavioural and physiological indicators of arousal (head movements, latency to choose, heart-rate variability and surface body temperature) when chickens made decisions that were unbalanced or finely balanced. Significant physiological (heart-rate variability) and behavioural (latency to pen) differences were found between the finely balanced and balanced conditions, but these were likely to be artefacts of the greater time and effort required to push through the weighted doors. No other behavioural and physiological measures were significantly different between the decision categories. We suggest that more information is needed on when best to monitor likely changes in arousal during decision-making and that future studies should consider decisions defined as difficult in other ways.     

Counterfactual choice and learning in a neural network centered on human lateral frontopolar cortex

Decision making and learning in a real-world context require organisms to track not only the choices they make and the outcomes that follow but also other untaken, or counterfactual, choices and their outcomes. Although the neural system responsible for tracking the value of choices actually taken is increasingly well understood, whether a neural system tracks counterfactual information is currently unclear. Using a three-alternative decision-making task, a Bayesian reinforcement-learning algorithm, and fMRI, we investigated the coding of counterfactual choices and prediction errors in the human brain. Rather than representing evidence favoring multiple counterfactual choices, lateral frontal polar cortex (lFPC), dorsomedial frontal cortex (DMFC), and posteromedial cortex (PMC) encode the reward-based evidence favoring the best counterfactual option at future decisions. In addition to encoding counterfactual reward expectations, the network carries a signal for learning about counterfactual options when feedback is available-a counterfactual prediction error. Unlike other brain regions that have been associated with the processing of counterfactual outcomes, counterfactual prediction errors within the identified network cannot be related to regret theory. Furthermore, individual variation in counterfactual choice-related activity and prediction error-related activity, respectively, predicts variation in the propensity to switch to profitable choices in the future and the ability to learn from hypothetical feedback. Taken together, these data provide both neural and behavioral evidence to support the existence of a previously unidentified neural system responsible for tracking both counterfactual choice options and their outcomes.     

Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice

In choosing between different rewards expected after unequal delays, humans and animals often prefer the smaller but more immediate reward, indicating that the subjective value or utility of reward is depreciated according to its delay. Here, we show that neurons in the primate caudate nucleus and ventral striatum modulate their activity according to temporally discounted values of rewards with a similar time course. However, neurons in the caudate nucleus encoded the difference in the temporally discounted values of the two alternative targets more reliably than neurons in the ventral striatum. In contrast, neurons in the ventral striatum largely encoded the sum of the temporally discounted values, and therefore, the overall goodness of available options. These results suggest a more pivotal role for the dorsal striatum in action selection during intertemporal choice.     

Starlings uphold principles of economic rationality for delay and probability of reward

Rationality principles are the bedrock of normative theories of decision-making in biology and microeconomics, but whereas in microeconomics, consistent choice underlies the notion of utility; in biology, the assumption of consistent selective pressures justifies modelling decision mechanisms as if they were designed to maximize fitness. In either case, violations of consistency contradict expectations and attract theoretical interest. Reported violations of rationality in non-humans include intransitivity (i.e. circular preferences) and lack of independence of irrelevant alternatives (changes in relative preference between options when embedded in different choice sets), but the extent to which these observations truly represent breaches of rationality is debatable. We tested both principles with starlings (Sturnus vulgaris), training subjects either with five options differing in food delay (exp. 1) or with six options differing in reward probability (exp. 2), before letting them choose repeatedly one option out of several binary and trinary sets of options. The starlings conformed to economic rationality on both tests, showing strong stochastic transitivity and no violation of the independence principle. These results endorse the rational choice and optimality approaches used in behavioural ecology, and highlight the need for functional and mechanistic enquiring when apparent violations of such principles are observed.     

Chimpanzees and bonobos distinguish between risk and ambiguity

Although recent research has investigated animal decision-making under risk, little is known about how animals choose under conditions of ambiguity when they lack information about the available alternatives. Many models of choice behaviour assume that ambiguity does not impact decision-makers, but studies of humans suggest that people tend to be more averse to choosing ambiguous options than risky options with known probabilities. To illuminate the evolutionary roots of human economic behaviour, we examined whether our closest living relatives, chimpanzees (Pan troglodytes) and bonobos (Pan paniscus), share this bias against ambiguity. Apes chose between a certain option that reliably provided an intermediately preferred food type, and a variable option that could vary in the probability that it provided a highly preferred food type. To examine the impact of ambiguity on ape decision-making, we interspersed trials in which chimpanzees and bonobos had no knowledge about the probabilities. Both species avoided the ambiguous option compared with their choices for a risky option, indicating that ambiguity aversion is shared by humans, bonobos and chimpanzees.     

Context-dependent decisions among options varying in a single dimension

Contrary to theories of rational choice, adding alternatives to a choice set can change the choices made by both humans and animals. This is usually done by adding an inferior decoy to a choice set of two favoured options that are characterized on two distinct dimensions. We presented wild, free-living rufous hummingbirds (Selasphorus rufus) with choices between two or three options that varied in a single dimension only. The options varied in concentration, in volume or in corolla length. When the options varied in concentration, the addition of a medium option to a choice set of a low and a high concentration caused birds to increase their preference for the high option. However, they decreased their preference for the high concentration option when a low option was added to a choice set of high and medium concentrations. When the options varied only in volume, the addition of a high volume option to a choice set of low and medium options decreased the birds’ preference for the medium option. We saw no effects of adding a third option when the options varied in corolla length alone. Hummingbirds, then, make context-dependent decisions even when the options vary in only a single dimension although which effect occurs seems to depend on the dimension being manipulated. None of the current theories alone adequately explain these results.     

A computer-aided approach to compare the production economics of fed-batch and perfusion culture under uncertainty

Fed-batch and perfusion culture dominate mammalian cell culture production processes. In this paper, a decision-support tool was employed to evaluate the economic feasibility of both culture modes via a case study based upon the large-scale production of monoclonal antibodies. The trade-offs between the relative simplicity but higher start-up costs of fed-batch processes and the high productivity but higher chances of equipment failure of perfusion processes were analysed. Deterministic analysis showed that whilst there was an insignificant difference (3%) between the cost of goods per gram (COG/g) values, the perfusion option benefited from a 42% reduction in capital investment and a 12% higher projected net present value (NPV). When Monte Carlo simulations were used to account for uncertainties in titre and yield, as well as the risks of contamination and filter fouling, the frequency distributions for the output metrics revealed that neither process route offered the best of both NPV or product output. A product output criterion was formulated and the options that met the criterion were compared based on their reward/risk ratio. The perfusion option was no longer feasible as it failed to meet the product output criterion and the fed-batch option had a 100% higher reward/risk ratio. The tool indicated that in this particular case, the probabilities of contamination and fouling in the perfusion option need to be reduced from 10% to 3% for this option to have the higher reward/risk ratio. The case study highlighted the limitations of relying on deterministic analysis alone.     

The Effect of Including an Opt-Out Option in Discrete Choice Experiments

Objective

to determine to what extent the inclusion of an opt-out option in a DCE may have an effect on choice behaviour and therefore might influence the attribute level estimates, the relative importance of the attributes and calculated trade-offs.

Methods

781 Dutch Type 2 Diabetes Mellitus patients completed a questionnaire containing nine choice tasks with an opt-out option and nice forced choice tasks. Mixed-logit models were used to estimate the relative importance of the five lifestyle program related attributes that were included. Willingness to pay (WTP) values were calculated and it was tested whether results differed between respondents who answered the choice tasks with an opt-out option in the first or second part of the questionnaire.

Results

21.4% of the respondents always opted out. Respondents who were given the opt-out option in the first part of the questionnaire as well as lower educated respondents significantly more often opted out. For both the forced and unforced choice model, different attributes showed significant estimates, the relative importance of the attributes was equal. However, due to differences in relative importance weights, the WTP values for the PA schedule differed significantly between both datasets.

Conclusions

Results show differences in opting out based on the location of the opt-out option and respondents'' educational level; this resulted in small differences between the forced and unforced choice model. Since respondents seem to learn from answering forced choice tasks, a dual response design might result in higher data quality compared to offering a direct opt-out option. Future research should empirically explore how choice sets should be presented to make them as easy and less complex as possible in order to reduce the proportion of respondents that opts-out due to choice task complexity. Moreover, future research should debrief respondents to examine the reasons for choosing the opt-out alternative.     

Humans copy rapidly increasing choices in a multiarmed bandit problem

Conformist social learning, the tendency to acquire the most common trait in a group, allows individuals to rapidly acquire established beneficial traits from a multitude of options. However, conformist strategies hinder acquisition of novel advantageous behavior patterns, because such innovations are by definition uncommon. This raises the possibility that proxy cues of the success of novel traits may be utilized to identify and acquire advantageous innovations and disregard failing options. We show that humans use changes in trait frequency over time as such a cue in an economic game. Participants played a three-alternative forced choice game (i.e., a multi-armed bandit), using social information to attempt to locate a high reward that could change location. Participants viewed temporal changes in how many players chose each option in two successive rounds. Participants supplemented conformist strategies with a “copy-increasing-traits” strategy. That is, regardless of the traits absolute population frequencies, participants' choices were guided by changes in trait frequencies. Thus, humans can detect advantageous innovations by monitoring how many individuals adopt these over time, adopting traits increasing in frequency, and abandoning traits decreasing in frequency. Copying rapidly increasing traits allows identification and acquisition of advantageous innovations, and is thus potentially key in facilitating their early diffusion and cultural evolution.     

Comparative performance of orangutans (Pongo spp.), gorillas (Gorilla gorilla gorilla), and drills (Mandrillus leucophaeus), in an ephemeral foraging task

A goal of the comparative approach is to test a variety of species on the same task. Here, we examined whether the factors that helped capuchin monkeys improve their performance in a dichotomous choice task would generalize to three other primate species: orangutans, gorillas, and drill monkeys. In this task, subjects have access to two options, each resulting in an identical food, but one (the ephemeral option) is only available if it is chosen first, whereas the other one (the permanent option) is always available. Therefore, the food‐maximizing solution is to choose the ephemeral option first, followed by the permanent option for an additional reward. On the original version (plate task), the options were discriminated by the color and pattern of the plates holding the food, while on two subsequent versions we used altered cues that we predicted would improve performance: (1) the color of the foods themselves (color task), which we hypothesized was relevant to primates, who choose foods rather than substrates on which foods are found when foraging, and (2) patterned cups covering the foods (cup task), which we hypothesized would help primates avoid the prepotent response associated with visible food. Like capuchins, all three species initially failed to solve the plate task. However, while orangutans improved their performance from the plate to the color task, they did not for the cup task, and only a few gorillas and no drills succeeded in either task. Unfortunately, our ability to interpret these data was obscured by differences in the subjects' level of experience with cognitive testing and practical constraints that precluded the use of completely identical procedures across species. Nonetheless, we consider what these results can tell us, and discuss the value of conducting studies across multiple sites despite unavoidable differences.     

Honeybee (Apis mellifera ligustica) Response to Differences in Handling Time, Rewards and Flower Colours

Free flying honeybees were tested outdoors on blue–white and blue–yellow dimorphic artificial flower patches to examine the influence of reward difference, flower handling‐time difference and flower colour choice on foraging decisions. We employed different flower‐well depths to vary handling times (costs), and differences in sucrose molarity to vary reward quality. Tests were performed with 2 and 6 μl rewards to vary quantity. We show that when handling time is correlated with flower‐colour morphs on a pedicellate artificial flower patch, a honeybee's foraging behaviour is dependent on the flower colours used in the choice tests. This supports a honeybee foraging model where constraints are a significant factor in decision making. Bees visiting blue–yellow flower patches exhibited flower constancy to colour, where they restricted most visits to a single flower colour, some bees to blue and others to yellow, irrespective of handing time differences. When offered a choice of equally rewarding blue or white flowers, bees were not constrained by flower colour and chose to visit flowers with a lower handling time. When reward molarity varied with well depth between blue and white flowers, foragers chose shallow‐well flowers (short‐handling time) with a smaller net harvest rate over deep‐well flowers (long‐handling time) with a greater net harvest rate. Results using the blue–white dimorphic flower patch suggest that when foraging options simultaneously involve reward and handling‐time choices, honeybee forager behaviour is inconsistent with an absolute method of evaluating profit.