The Effect of Incentives and Meta-incentives on the Evolution of Cooperation

Although positive incentives for cooperators and/or negative incentives for free-riders in social dilemmas play an important role in maintaining cooperation, there is still the outstanding issue of who should pay the cost of incentives. The second-order free-rider problem, in which players who do not provide the incentives dominate in a game, is a well-known academic challenge. In order to meet this challenge, we devise and analyze a meta-incentive game that integrates positive incentives (rewards) and negative incentives (punishments) with second-order incentives, which are incentives for other players’ incentives. The critical assumption of our model is that players who tend to provide incentives to other players for their cooperative or non-cooperative behavior also tend to provide incentives to their incentive behaviors. In this paper, we solve the replicator dynamics for a simple version of the game and analytically categorize the game types into four groups. We find that the second-order free-rider problem is completely resolved without any third-order or higher (meta) incentive under the assumption. To do so, a second-order costly incentive, which is given individually (peer-to-peer) after playing donation games, is needed. The paper concludes that (1) second-order incentives for first-order reward are necessary for cooperative regimes, (2) a system without first-order rewards cannot maintain a cooperative regime, (3) a system with first-order rewards and no incentives for rewards is the worst because it never reaches cooperation, and (4) a system with rewards for incentives is more likely to be a cooperative regime than a system with punishments for incentives when the cost-effect ratio of incentives is sufficiently large. This solution is general and strong in the sense that the game does not need any centralized institution or proactive system for incentives.     

Exit,Punishment and Rewards in Commons Dilemmas: An Experimental Study

Commons dilemmas are interaction situations where a common good is provided or exploited by a group of individuals so that optimal collective outcomes clash with private interests. Although in these situations, social norms and institutions exist that might help individuals to cooperate, little is known about the interaction effects between positive and negative incentives and exit options by individuals. We performed a modified public good game experiment to examine the effect of exit, rewards and punishment, as well as the interplay between exit and rewards and punishment. We found that punishment had a stronger effect than rewards on cooperation if considered by itself, whereas rewards had a stronger effect when combined with voluntary participation. This can be explained in terms of the ‘framing effect’, i.e., as the combination of exit and rewards might induce people to attach higher expected payoffs to cooperative strategies and expect better behaviour from others.     

Unintended Consequences of Incentive Provision for Behaviour Change and Maintenance around Childbirth

Financial (positive or negative) and non-financial incentives or rewards are increasingly used in attempts to influence health behaviours. While unintended consequences of incentive provision are discussed in the literature, evidence syntheses did not identify any primary research with the aim of investigating unintended consequences of incentive interventions for lifestyle behaviour change. Our objective was to investigate perceived positive and negative unintended consequences of incentive provision for a shortlist of seven promising incentive strategies for smoking cessation in pregnancy and breastfeeding. A multi-disciplinary, mixed-methods approach included involving two service-user mother and baby groups from disadvantaged areas with experience of the target behaviours as study co-investigators. Systematic reviews informed the shortlist of incentive strategies. Qualitative semi-structured interviews and a web-based survey of health professionals asked open questions on positive and negative consequences of incentives. The participants from three UK regions were a diverse sample with and without direct experience of incentive interventions: 88 pregnant women/recent mothers/partners/family members; 53 service providers; 24 experts/decision makers and interactive discussions with 63 conference attendees. Maternity and early years health professionals (n = 497) including doctors, midwives, health visitors, public health and related staff participated in the survey. Qualitative analysis identified ethical, political, cultural, social and psychological implications of incentive delivery at population and individual levels. Four key themes emerged: how incentives can address or create inequalities; enhance or diminish intrinsic motivation and wellbeing; have a positive or negative effect on relationships with others within personal networks or health providers; and can impact on health systems and resources by raising awareness and directing service delivery, but may be detrimental to other health care areas. Financial incentives are controversial and generated emotive and oppositional responses. The planning, design and delivery of future incentive interventions should evaluate unexpected consequences to inform the evidence for effectiveness, cost-effectiveness and future implementation.     

The Typical Dreams of Canadian University Students

To investigate the dimensional structure of dreams, the Typical Dreams Questionnaire (TDQ) was administered to 1181 first-year University students in three Canadian cities. A profile of themes was found that varied little by age, gender or region; however, differences that were identified could be interpreted as due to developmental milestones, personality attributes or sociocultural factors. Factor analysis produced a solution consisting of 16 coherent factors that were differentially associated with demographic variables and that accounted for 51% of the variance. Women loaded primarily on negative factors (failure, loss of control, snakes-insects), men primarily on positive factors (magic-myth, alien life). Results support the concept of typical dream themes as consistent over time, region and gender and as reflecting the influence of fundamental dream dimensions that may be influenced by sociocultural, personality, cognitive or physiological factors.     

The Effect of Positive and Negative Feedback on Risk-Taking across Different Contexts

Preferences for risky choices have often been shown to be unstable and context-dependent. Though people generally avoid gambles with mixed outcomes, a phenomenon often attributed to loss aversion, contextual factors can impact this dramatically. For example, people typically prefer risky options after a financial loss, while generally choosing safer options after a monetary gain. However, it is unclear what exactly contributes to these preference shifts as a function of prior outcomes, as these gain/loss outcomes are usually confounded with participant performance, and therefore it is unclear whether these effects are driven purely by the monetary gains or losses, or rather by success or failure at the actual task. Here, we experimentally separated the effects of monetary gains/losses from performance success/failure prior to a standard risky choice. Participants performed a task in which they experienced contextual effects: 1) monetary gain or loss based directly on performance, 2) monetary gain or loss that was randomly awarded and was, crucially, independent from performance, and 3) success or failure feedback based on performance, but without any monetary incentive. Immediately following these positive/negative contexts, participants were presented with a gain-loss gamble that they had to decide to either play or pass. We found that risk preferences for identical sets of gambles were biased by positive and negative contexts containing monetary gains and losses, but not by contexts containing performance feedback. This data suggests that the observed framing effects are driven by aversion for monetary losses and not simply by the positive or negative valence of the context, or by potential moods resulting from positive or negative contexts. These results highlight the specific context dependence of risk preferences.     

The neural basis of following advice

Learning by following explicit advice is fundamental for human cultural evolution, yet the neurobiology of adaptive social learning is largely unknown. Here, we used simulations to analyze the adaptive value of social learning mechanisms, computational modeling of behavioral data to describe cognitive mechanisms involved in social learning, and model-based functional magnetic resonance imaging (fMRI) to identify the neurobiological basis of following advice. One-time advice received before learning had a sustained influence on people's learning processes. This was best explained by social learning mechanisms implementing a more positive evaluation of the outcomes from recommended options. Computer simulations showed that this "outcome-bonus" accumulates more rewards than an alternative mechanism implementing higher initial reward expectation for recommended options. fMRI results revealed a neural outcome-bonus signal in the septal area and the left caudate. This neural signal coded rewards in the absence of advice, and crucially, it signaled greater positive rewards for positive and negative feedback after recommended rather than after non-recommended choices. Hence, our results indicate that following advice is intrinsically rewarding. A positive correlation between the model's outcome-bonus parameter and amygdala activity after positive feedback directly relates the computational model to brain activity. These results advance the understanding of social learning by providing a neurobiological account for adaptive learning from advice.     

Money Affects Theory of Mind Differently by Gender

Theory of Mind (ToM) ─ the ability to understand other’s thoughts, intentions, and emotions ─ is important for navigating interpersonal relationships, avoiding conflict, and empathizing. Prior research has identified many factors that affect one’s ToM ability, but little work has examined how different kinds of monetary incentives affect ToM ability. We ask: Does money affect ToM ability? If so, how does the effect depend on the structure of monetary incentives? How do the differences depend on gender? We hypothesize that money will affect ToM ability differently by gender: monetary rewards increase males’ motivation to express ToM ability while simultaneously crowding out females’ motivation. This prediction is confirmed in an experiment that varies the structure of monetary rewards for correct answers in the Reading the Mind in the Eyes Test (RMET). RMET scores decrease for females and increase for males with individual payments, and this effect is stronger with competitively-structured payments. RMET scores do not significantly change when monetary earnings go to a charity. Whether money improves or hinders ToM ability, and, hence, success in social interactions, thus depends on the interaction of gender and monetary incentive structure.     

Why Don't You Try Harder? An Investigation of Effort Production in Major Depression

Depression is mainly characterized as an emotional disorder, associated with reduced approach behavior. It remains unclear whether the difficulty in energising behavior relates to abnormal emotional states or to a flattened response to potential rewards, as suggested by several neuroimaging studies. Here, we aimed to demonstrate a specific incentive motivation deficit in major depression, independent of patients' emotional state. We employed a behavioral paradigm designed to measure physical effort in response to both emotional modulation and incentive motivation. Patients did exert more effort following emotionally arousing pictures (whether positive or negative) but not for higher monetary incentives, contrary to healthy controls. These results show that emotional and motivational sources of effort production are dissociable in pathological conditions. In addition, patients' ratings of perceived effort increased for high incentives, whereas controls' ratings were decreased. Thus, depressed patients objectively behave as if they do not want to gain larger rewards, but subjectively feel that they try harder. We suggest that incentive motivation impairment is a core deficit of major depression, which may render everyday tasks abnormally effortful for patients.     

Gebruiksvriendelijkheid van computerondersteunde cognitieve training bij psychogeriatrische patiënten met lichte tot matige cognitieve functiestoornissen

Cognitive impairment associated with dementia is characterized by a continuous decline. Cognitive training is a method to train specific brain functions such as memory and attention to prevent or slow down cognitive decline. A small number of studies has shown that cognitive training on a computer has a positive effect on both cognition and mood in people with cognitive impairment. This pilot study tested if serious games could be integrated in a psychogeriatric rehabilitation center. Fourteen psychogeriatric patients participated twice weekly in cognitive training sessions on a computer. Both the participants and the facilitator reported positive interactions and outcomes. However, after five weeks only half of the sample still participated in the training. This was partly because of patient turn-over as well as incorporating this new task in the facilitators’ daily work. Fear of failure, physical limitations and rapidly decreasing cognitive function led to drop out according to the facilitator. The engagement of patients in the games and the role of the facilitator seemed essential for success, especially monitoring (and adjusting) the difficulty level of the program for every individual participant.     

Reward type influences performance and search structure of an omnivorous bird in an open-field maze

Open-field mazes are routinely used to study the spatial cognitive abilities of birds and are often implicitly assumed to be suitable tests of generic spatial memory ability. In recent years there has been extensive research motivated by considerations of an animals’ ecology, demonstrating potential examples of specialisations of spatial cognition, as a result of foraging niche. The study reported here demonstrates differences in maze performance as a function of reward type (nectar and invertebrates) that can be predicted from the natural distributions of these rewards. As well as specific implications for the nature of spatial memory specialisation in this species, the results hold more general implications for the use of open-field mazes as tools for measuring and comparing spatial memory ability between species.     

Neurocomputational mechanism of controllability inference under a multi-agent setting

Controllability perception significantly influences motivated behavior and emotion and requires an estimation of one’s influence on an environment. Previous studies have shown that an agent can infer controllability by observing contingency between one’s own action and outcome if there are no other outcome-relevant agents in an environment. However, if there are multiple agents who can influence the outcome, estimation of one’s genuine controllability requires exclusion of other agents’ possible influence. Here, we first investigated a computational and neural mechanism of controllability inference in a multi-agent setting. Our novel multi-agent Bayesian controllability inference model showed that other people’s action-outcome contingency information is integrated with one’s own action-outcome contingency to infer controllability, which can be explained as a Bayesian inference. Model-based functional MRI analyses showed that multi-agent Bayesian controllability inference recruits the temporoparietal junction (TPJ) and striatum. Then, this inferred controllability information was leveraged to increase motivated behavior in the vmPFC. These results generalize the previously known role of the striatum and vmPFC in single-agent controllability to multi-agent controllability, and this generalized role requires the TPJ in addition to the striatum of single-agent controllability to integrate both self- and other-related information. Finally, we identified an innate positive bias toward the self during the multi-agent controllability inference, which facilitated behavioral adaptation under volatile controllability. Furthermore, low positive bias and high negative bias were associated with increased daily feelings of guilt. Our results provide a mechanism of how our sense of controllability fluctuates due to other people in our lives, which might be related to social learned helplessness and depression.     

An imperfect dopaminergic error signal can drive temporal-difference learning

An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards.     

Programa de reminiscencia apoyado en inteligencia artificial y basado en el patrimonio cultural inmaterial dirigido a personas mayores: un estudio piloto

IntroductionTechnological applications are an innovative way of providing reminiscence therapy and must meet the users’ needs. Intangible cultural heritage as a basis for such therapy has not been explored yet. We evaluated the applicability of a new technological application supported by artificial intelligence for reminiscence therapy based on intangible cultural heritage aimed at older people.Material and methodsA prospective observational study was carried out with people aged 65 or over, without cognitive impairment and with mild and moderate cognitive impairment who attended six centers for older people in Spain and Portugal. Participants tested the first prototype of the individualized LONG-REMI program in four consecutive weekly sessions. The usability and satisfaction of the experience were evaluated using the VAS scale at the end of the intervention. Emotions were evaluated using the PANAS scale before and at the end of the intervention.ResultsData from 56 participants were analysed. For all participants, usability and satisfaction were highly perceived, with scores of 7.75±1.88 and 8.38±1.57, respectively. The positive affect subscale PANAS showed significant changes (28.86±8.88 before the intervention versus 36.70±9.43 post intervention, Z = −4.18, P = 0.000). There were no significant changes in the PANAS negative affect subscale.ConclusionsThe first prototype of the LONG-REMI technological application can be used by older people both with and without cognitive impairment. This has the potential to be an instrument for future cognitive therapies with stimulating activities and benefits for emotions.     

Social Influences in Sequential Decision Making

People often make decisions in a social environment. The present work examines social influence on people’s decisions in a sequential decision-making situation. In the first experimental study, we implemented an information cascade paradigm, illustrating that people infer information from decisions of others and use this information to make their own decisions. We followed a cognitive modeling approach to elicit the weight people give to social as compared to private individual information. The proposed social influence model shows that participants overweight their own private information relative to social information, contrary to the normative Bayesian account. In our second study, we embedded the abstract decision problem of Study 1 in a medical decision-making problem. We examined whether in a medical situation people also take others’ authority into account in addition to the information that their decisions convey. The social influence model illustrates that people weight social information differentially according to the authority of other decision makers. The influence of authority was strongest when an authority''s decision contrasted with private information. Both studies illustrate how the social environment provides sources of information that people integrate differently for their decisions.     

An Examination of the Generalizability of Motor Costs

Most approaches to understanding human motor control assume that people maximize their rewards while minimizing their motor efforts. This tradeoff between potential rewards and a sense of effort is quantified with a cost function. While the rewards can change across tasks, our sense of effort is assumed to remain constant and characterize how the nervous system organizes motor control. As such, when a proposed cost function compares well with data it is argued to be the underlying cause of a motor behavior, and not simply a fit to the data. Implicit in this proposition is the assumption that this cost function can then predict new motor behaviors. Here we examined this idea and asked whether an inferred cost function in one setting could explain subject’s behavior in settings that differed dynamically but had identical rewards. We found that the pattern of behavior observed across settings was similar to our predictions of optimal behavior. However, we could not conclude that this behavior was consistent with a conserved sense of effort. These results suggest that the standard forms for quantifying cost may not be sufficient to accurately examine whether or not human motor behavior abides by optimality principles.     

Effect of Social Influence on Effort-Allocation for Monetary Rewards

Though decades of research have shown that people are highly influenced by peers, few studies have directly assessed how the value of social conformity is weighed against other types of costs and benefits. Using an effort-based decision-making paradigm with a novel social influence manipulation, we measured how social influence affected individuals’ decisions to allocate effort for monetary rewards during trials with either high or low probability of receiving a reward. We found that information about the effort-allocation of peers modulated participant choices, specifically during conditions of low probability of obtaining a reward. This suggests that peer influence affects effort-based choices to obtain rewards especially under conditions of risk. This study provides evidence that people value social conformity in addition to other costs and benefits when allocating effort, and suggests that neuroeconomic studies that assess trade-offs between effort and reward should consider social environment as a factor that can influence decision-making.     

Incentives for collecting Gaharu (fungal-infected wood ofaquilaria Spp.; thymelaeaceae) in east Kalimantan

The economic importance of gaharu is assessed in three villages on the Bahau River in north-central Borneo to gain insights about the incentives for harvesting and management of a valuable nontimber forest product. Three indicators of economic value—level and proportion of income, returns to labor, and proportion of gaharu collecting households per village—are used to demonstrate the multiple incentives that NTFP income can generate. The concept of incentive logic is developed as an analytical technique to show how economic values can be linked to incentives for different types of management actions. The article discusses how incentives from gaharu income were most likely linked to the stake local people had in the resource, their preferences about which forest product to harvest, and their willingness to engage in collective action. These incentives contributed to sustainability to the extent they induced actions that reduce threats to the resource. The article suggests that an understanding of the influence of economic incentives on people’s resource management can be improved by recognizing three factors: the multiple incentives created by an income, the logical link of those incentives to a management action, and the influence of other sociocultural and biophysical factors on management.     

Determining the contributions of divisive and subtractive feedback in the Hodgkin-Huxley model

The Hodgkin-Huxley (HH) model is the basis for numerous neural models. There are two negative feedback processes in the HH model that regulate rhythmic spiking. The first is an outward current with an activation variable n that has an opposite influence to the excitatory inward current and therefore provides subtractive negative feedback. The other is the inactivation of an inward current with an inactivation variable h that reduces the amount of positive feedback and therefore provides divisive feedback. Rhythmic spiking can be obtained with either negative feedback process, so we ask what is gained by having two feedback processes. We also ask how the different negative feedback processes contribute to spiking. We show that having two negative feedback processes makes the HH model more robust to changes in applied currents and conductance densities than models that possess only one negative feedback variable. We also show that the contributions made by the subtractive and divisive feedback variables are not static, but depend on time scales and conductance values. In particular, they contribute differently to the dynamics in Type I versus Type II neurons.     

Prosocial behaviour emerges independent of reciprocity in cottontop tamarins

The cooperative breeding hypothesis posits that cooperatively breeding species are motivated to act prosocially, that is, to behave in ways that provide benefits to others, and that cooperative breeding has played a central role in the evolution of human prosociality. However, investigations of prosocial behaviour in cooperative breeders have produced varying results and the mechanisms contributing to this variation are unknown. We investigated whether reciprocity would facilitate prosocial behaviour among cottontop tamarins, a cooperatively breeding primate species likely to engage in reciprocal altruism, by comparing the number of food rewards transferred to partners who had either immediately previously provided or denied rewards to the subject. Subjects were also tested in a non-social control condition. Overall, results indicated that reciprocity increased food transfers. However, temporal analyses revealed that when the tamarins'' behaviour was evaluated in relation to the non-social control, results were best explained by (i) an initial depression in the transfer of rewards to partners who recently denied rewards, and (ii) a prosocial effect that emerged late in sessions independent of reciprocity. These results support the cooperative breeding hypothesis, but suggest a minimal role for positive reciprocity, and emphasize the importance of investigating proximate temporal mechanisms underlying prosocial behaviour.     

The Value of Emotion: How Does Episodic Prospection Modulate Delay Discounting?

Background

Humans often show impatience when making intertemporal choice for monetary rewards, preferring small rewards delivered immediately to larger rewards delivered after a delay, which reflects a fundamental psychological principle: delay discounting. However, we propose that episodic prospection humans can vividly envisage exerts a strong and broad influence on individuals'' delay discounting. Specifically, episodic prospection may affect individuals'' intertemporal choice by the negative or positive emotion of prospection.

Methodology/Principal Findings

The present study explored how episodic prospection modulated delay discounting by emotion. Study 1 showed that participants were more inclined to choose the delayed but larger rewards when they imaged positive future events than when they did not image events; Study 2 showed that participants were more inclined to choose the immediate but smaller rewards when they imaged negative future events than when they did not image events; In contrast, study 3 showed that choice preferences of participants when they imaged neutral future events were the same as when they did not image events.

Conclusions/Significance

By manipulating the emotion valence of episodic prospection, our findings suggested that positive emotion made individuals tend to choose delayed rewards, while negative emotion made individuals tend to choose immediate rewards. Only imaging events with neutral emotion did not affect individuals'' choice preference. Thus, the valence of imaged future events'' emotion might play an important role in individuals'' intertemporal choice. It is possible that the valence of emotion may affect the changed direction (promote or inhibit) of individuals'' delay discounting, while the ability to image future events affects the changed degree of individuals'' delay discounting.