Neural correlates of economic game playing

The theory of games provides a mathematical formalization of strategic choices, which have been studied in both economics and neuroscience, and more recently has become the focus of neuroeconomics experiments with human and non-human actors. This paper reviews the results from a number of game experiments that establish a unitary system for forming subjective expected utility maps in the brain, and acting on these maps to produce choices. Social situations require the brain to build an understanding of the other person using neuronal mechanisms that share affective and intentional mental states. These systems allow subjects to better predict other players' choices, and allow them to modify their subjective utility maps to value pro-social strategies. New results for a trust game are presented, which show that the trust relationship includes systems common to both trusting and trustworthy behaviour, but they also show that the relative temporal positions of first and second players require computations unique to that role.     

Timing in reward and decision processes

Sensitivity to time, including the time of reward, guides the behaviour of all organisms. Recent research suggests that all major reward structures of the brain process the time of reward occurrence, including midbrain dopamine neurons, striatum, frontal cortex and amygdala. Neuronal reward responses in dopamine neurons, striatum and frontal cortex show temporal discounting of reward value. The prediction error signal of dopamine neurons includes the predicted time of rewards. Neurons in the striatum, frontal cortex and amygdala show responses to reward delivery and activities anticipating rewards that are sensitive to the predicted time of reward and the instantaneous reward probability. Together these data suggest that internal timing processes have several well characterized effects on neuronal reward processing.     

Modeling hunter-gatherer decision making: complementing optimal foraging theory

While optimal foraging theory has been of considerable value for understanding hunter-gatherer subsistence patterns, there is a need for a complementary approach to human foraging behavior which focuses on decision-making processes. Having made this argument, the paper proposes the type of modeling approach that should be developed, using decision making during encounter foraging as an example. This model concerns the individual decision maker attempting to improve his foraging efficiency, rather than maximize it, under the constraint of limited information and with conflicting goals. This is illustrated by applying it to the Valley Bisa hunters using computer simulation.     

Akt1 deficiency modulates reward learning and reward prediction error in mice

In contemporary reinforcement learning models, reward prediction error (RPE), the difference between the expected and actual reward, is thought to guide action value learning through the firing activity of dopaminergic neurons. Given the importance of dopamine in reward learning and the involvement of Akt1 in dopamine-dependent behaviors, the aim of this study was to investigate whether Akt1 deficiency modulates reward learning and the magnitude of RPE using Akt1 mutant mice as a model. In comparison to wild-type littermate controls, the expression of Akt1 proteins in mouse brains occurred in a gene-dosage-dependent manner and Akt1 heterozygous (HET) mice exhibited impaired striatal Akt1 activity under methamphetamine challenge. No genotypic difference was found in the basal levels of dopamine and its metabolites. In a series of reward-related learning tasks, HET mice displayed a relatively efficient method of updating reward information from the environment during the acquisition phase of the two natural reward tasks and in the reverse section of the dynamic foraging T-maze but not in methamphetamine-induced or aversive-related reward learning. The implementation of a standard reinforcement learning model and the Bayesian hierarchical parameter estimation show that HET mice have higher RPE magnitudes and that their action values are updated more rapidly among all three test sections in T-maze. These results indicate that Akt1 deficiency modulates natural reward learning and RPE. This study showed a promising avenue for investigating RPE in mutant mice and provided evidence for the potential link from genetic deficiency, to neurobiological abnormalities, to impairment in higher-order cognitive functioning.     

Conflicting constraints on male mating success shape reward size in pollen-rewarding plants

Premise

Pollen-rewarding plants face two conflicting constraints: They must prevent consumptive emasculation while remaining attractive to pollen-collecting visitors. Small pollen packages (the quantity of pollen available in a single visit) may discourage visitors from grooming (reducing consumptive loss) but may also decrease a plant's attractiveness to pollen-collecting visitors. What package size best balances these two constraints?

Methods

We modeled the joint effects of pollinators' grooming behaviors and package size preferences on the optimal package size (i.e., the size that maximizes pollen donation). We then used this model to examine Darwin's conjecture that selection should favor increased pollen production in pollen-rewarding plants.

Results

When package size preferences are weak, minimizing package size reduces grooming losses and should be favored (as in previous theoretical studies). Stronger preferences select for larger packages despite the associated increase to grooming loss because loss associated with nonremoval of smaller packages is even greater. Total pollen donation increases with production (as Darwin suggested). However, if floral visitation declines or packages size preference increases with overall pollen availability, the fraction of pollen donated may decline as per-plant pollen production increases. Hence, increasing production may result in diminishing returns.

Conclusions

Pollen-rewarding plants can balance conflicting constraints on pollen donation by producing intermediate-sized pollen packages. Strictly pollen-rewarding plants may have responded to past selection to produce more pollen in total, but diminishing returns may limit the strength of that selection.

     

The role of social cognition in decision making

Successful decision making in a social setting depends on our ability to understand the intentions, emotions and beliefs of others. The mirror system allows us to understand other people's motor actions and action intentions. 'Empathy' allows us to understand and share emotions and sensations with others. 'Theory of mind' allows us to understand more abstract concepts such as beliefs or wishes in others. In all these cases, evidence has accumulated that we use the specific neural networks engaged in processing mental states in ourselves to understand the same mental states in others. However, the magnitude of the brain activity in these shared networks is modulated by contextual appraisal of the situation or the other person. An important feature of decision making in a social setting concerns the interaction of reason and emotion. We consider four domains where such interactions occur: our sense of fairness, altruistic punishment, trust and framing effects. In these cases, social motivations and emotions compete with each other, while higher-level control processes modulate the interactions of these low-level biases.     

Risk-sensitivity and the mean-variance trade-off: decision making in sensorimotor control

Numerous psychophysical studies suggest that the sensorimotor system chooses actions that optimize the average cost associated with a movement. Recently, however, violations of this hypothesis have been reported in line with economic theories of decision-making that not only consider the mean payoff, but are also sensitive to risk, that is the variability of the payoff. Here, we examine the hypothesis that risk-sensitivity in sensorimotor control arises as a mean-variance trade-off in movement costs. We designed a motor task in which participants could choose between a sure motor action that resulted in a fixed amount of effort and a risky motor action that resulted in a variable amount of effort that could be either lower or higher than the fixed effort. By changing the mean effort of the risky action while experimentally fixing its variance, we determined indifference points at which participants chose equiprobably between the sure, fixed amount of effort option and the risky, variable effort option. Depending on whether participants accepted a variable effort with a mean that was higher, lower or equal to the fixed effort, they could be classified as risk-seeking, risk-averse or risk-neutral. Most subjects were risk-sensitive in our task consistent with a mean-variance trade-off in effort, thereby, underlining the importance of risk-sensitivity in computational models of sensorimotor control.     

Time-based reward maximization

Humans and animals time intervals from seconds to minutes with high accuracy but limited precision. Consequently, time-based decisions are inevitably subjected to our endogenous timing uncertainty, and thus require temporal risk assessment. In this study, we tested temporal risk assessment ability of humans when participants had to withhold each subsequent response for a minimum duration to earn reward and each response reset the trial time. Premature responses were not penalized in Experiment 1 but were penalized in Experiment 2. Participants tried to maximize reward within a fixed session time (over eight sessions) by pressing a key. No instructions were provided regarding the task rules/parameters. We evaluated empirical performance within the framework of optimality that was based on the level of endogenous timing uncertainty and the payoff structure. Participants nearly tracked the optimal target inter-response times (IRTs) that changed as a function of the level of timing uncertainty and maximized the reward rate in both experiments. Acquisition of optimal target IRT was rapid and abrupt without any further improvement or worsening. These results constitute an example of optimal temporal risk assessment performance in a task that required finding the optimal trade-off between the ‘speed’ (timing) and ‘accuracy’ (reward probability) of timed responses for reward maximization.     

An agent-based model of group decision making in baboons

We present an agent-based model of the key activities of a troop of chacma baboons (Papio hamadryas ursinus) based on the data collected at De Hoop Nature Reserve in South Africa. We analyse the predictions of the model in terms of how well it is able to duplicate the observed activity patterns of the animals and the relationship between the parameters that control the agent's decision procedure and the model's predictions. At the current stage of model development, we are able to show that across a wide range of decision parameter values, the baboons are able to achieve their energetic and social time requirements. The simulation results also show that decisions concerning movement (group action selection) have the greatest influence on the outcomes. Those cases where the model's predictions fail to agree with the observed activity patterns have highlighted key elements that were missing from the field data, and that would need to be collected in subsequent fieldwork. Based on our experience, we believe group decision making is a fertile field for future research, and agent-based modelling offers considerable scope for understanding group action selection.     

Risky decisions: a test of risk sensitivity in socially foraging flocks of Lonchura punctulata

Group foraging allows for individuals to exploit the food discoveriesof other group members. If searching for food and searchingfor exploitation opportunities within a group are mutually exclusivealternatives, the decision to use one or the other is modeledas a producer-scrounger game because the value of each alternativeis frequency dependent. Stochastic producer-scrounger modelsgenerally assume that producer provides a more variable anduncertain reward than does the scrounger and hence is a riskierforaging alternative. Socially foraging animals that are attemptingto reduce their risk of starvation should therefore alter theiruse of producer and scrounger alternatives in response to changesin energy budget. We observed flocks of nutmeg mannikins (L.punctulata) foraging in an indoor aviary to determine whethertheir use of producer and scrounger alternatives were risk sensitive.Analyses of the foraging rewards of three flocks of seven birdsconfirm that producer is a riskier foraging strategy than isscrounger, although the difference in risk is rather small.We then submitted two other flocks to two different energy budgetsand observed the foraging decision of four focal birds in eachflock. All but one bird increased their relative use of theriskier producer strategy in the low food reserve treatment,but the overall use of producer did not differ significantlybetween treatments, providing evidence for a small but consistenteffect.     

Emergence and resilience of cooperation in the spatial prisoner's dilemma via a reward mechanism

We study the problem of the emergence of cooperation in the spatial Prisoner's Dilemma. The pioneering work by Nowak and May [1992. Evolutionary games and spatial chaos. Nature 415, 424-426] showed that large initial populations of cooperators can survive and sustain cooperation in a square lattice with imitate-the-best evolutionary dynamics. We revisit this problem in a cost-benefit formulation suitable for a number of biological applications. We show that if a fixed-amount reward is established for cooperators to share, a single cooperator can invade a population of defectors and form structures that are resilient to re-invasion even if the reward mechanism is turned off. We discuss analytically the case of the invasion by a single cooperator and present agent-based simulations for small initial fractions of cooperators. Large cooperation levels, in the sustainability range, are found. In the conclusions we discuss possible applications of this model as well as its connections with other mechanisms proposed to promote the emergence of cooperation.     

Game theory and the evolution of fearfulness in wild birds

When animals in a group live under predation threat, the fate of each individual depends on the way it reacts to danger, but also on the behaviour of its companions. Game theory should then help to understand the evolution of fearful behaviour in gregarious animals. To illustrate this approach, a model determines evolutionarily stable levels of fearfulness in bird flocks, assuming that flocks are the object of both predatory attacks and nonlethal disturbance. In the model, high levels of flightiness limit the risk of being killed by predators, but increase the amount of energy lost in flights during the season. The predicted levels of fearfulness are extremely variable. They depend on the respective frequencies of predatory attacks and simple disturbing events, and on the capacity of birds to detect and escape predators. These results may help to explain the variability of flightiness reported in birds.     

Quorum responses and consensus decision making

Animal groups are said to make consensus decisions when group members come to agree on the same option. Consensus decisions are taxonomically widespread and potentially offer three key benefits: maintenance of group cohesion, enhancement of decision accuracy compared with lone individuals and improvement in decision speed. In the absence of centralized control, arriving at a consensus depends on local interactions in which each individual''s likelihood of choosing an option increases with the number of others already committed to that option. The resulting positive feedback can effectively direct most or all group members to the best available choice. In this paper, we examine the functional form of the individual response to others'' behaviour that lies at the heart of this process. We review recent theoretical and empirical work on consensus decisions, and we develop a simple mathematical model to show the central importance to speedy and accurate decisions of quorum responses, in which an animal''s probability of exhibiting a behaviour is a sharply nonlinear function of the number of other individuals already performing this behaviour. We argue that systems relying on such quorum rules can achieve cohesive choice of the best option while also permitting adaptive tuning of the trade-off between decision speed and accuracy.