Behavioural Flexibility and Personality in Zebra Finches

Consistent individual differences in behaviour of animals, that is, personalities, are both widespread and widely studied, but very few studies also include cognitive traits in this context. Animal personality has recently been integrated into the pace‐of‐life‐syndrome hypothesis, relating individual behavioural traits to life history. Variation in cognitive traits could be explained well by this theoretical framework. A risk‐reward trade‐off might lead to different cognitive types: Active birds that learn fast, take risks and probably have a fast lifestyle and less active, slow learning birds that are risk averse but thereby perform better in reversal learning as they probably pay more attention to external cues. We investigated the performance of zebra finches (Taeniopygia guttata) in a cognitively challenging reversal learning task and linked this to two personality traits: activity and fearfulness. Male birds were better in reversal learning than females. While no personality‐related differences occurred in the initial learning of our task, more active and fearful birds relearned the cue–reward association faster. While birds of different sex might have revealed different risk‐taking strategies in the training, our findings do not reveal the expected direction of a risk‐reward trade‐off in the reversal learning. It seems likely that a more general and personality‐related cognitive ability might improve performance across different tasks. The linkage between personality and cognition documented here could hence suggest that cognitive traits are indeed part of an overall pace‐of‐life syndrome.     

Neural mechanisms underlying motivation of mental versus physical effort

Mental and physical efforts, such as paying attention and lifting weights, have been shown to involve different brain systems. These cognitive and motor systems, respectively, include cortical networks (prefronto-parietal and precentral regions) as well as subregions of the dorsal basal ganglia (caudate and putamen). Both systems appeared sensitive to incentive motivation: their activity increases when we work for higher rewards. Another brain system, including the ventral prefrontal cortex and the ventral basal ganglia, has been implicated in encoding expected rewards. How this motivational system drives the cognitive and motor systems remains poorly understood. More specifically, it is unclear whether cognitive and motor systems can be driven by a common motivational center or if they are driven by distinct, dedicated motivational modules. To address this issue, we used functional MRI to scan healthy participants while performing a task in which incentive motivation, cognitive, and motor demands were varied independently. We reasoned that a common motivational node should (1) represent the reward expected from effort exertion, (2) correlate with the performance attained, and (3) switch effective connectivity between cognitive and motor regions depending on task demand. The ventral striatum fulfilled all three criteria and therefore qualified as a common motivational node capable of driving both cognitive and motor regions of the dorsal striatum. Thus, we suggest that the interaction between a common motivational system and the different task-specific systems underpinning behavioral performance might occur within the basal ganglia.     

The Impact of Financial Reward Contingencies on Cognitive Function Profiles in Adult ADHD

Objectives

Although it is well established that cognitive performance in children with attention-deficit/hyperactivity disorder (ADHD) is affected by reward and that key deficits associated with the disorder may thereby be attenuated or even compensated, this phenomenon in adults with ADHD has thus far not been addressed. Therefore, the aim of the present study was to examine the motivating effect of financial reward on task performance in adults with ADHD by focusing on the domains of executive functioning, attention, time perception, and delay aversion.

Methods

We examined male and female adults aged 18–40 years with ADHD (n = 38) along with a matched control group (n = 40) using six well-established experimental paradigms.

Results

Impaired performance in the ADHD group was observed for stop-signal omission errors, n-back accuracy, reaction time variability in the continuous performance task, and time reproduction accuracy, and reward normalized time reproduction accuracy. Furthermore, when rewarded, subjects with ADHD exhibited longer reaction times and fewer false positives in the continuous performance task, which suggests the use of strategies to prevent impulsivity errors.

Conclusions

Taken together, our results support the existence of both cognitive and motivational mechanisms for the disorder, which is in line with current models of ADHD. Furthermore, our data suggest cognitive strategies of “stopping and thinking” as a possible underlying mechanism for task improvement that seems to be mediated by reward, which highlights the importance of the interaction between motivation and cognition in adult ADHD.     

Impaired working memory,cognitive flexibility and reward processing in mice genetically lacking Gpr88: Evidence for a key role for Gpr88 in multiple cortico-striatal-thalamic circuits

The GPR88 orphan G protein-coupled receptor is expressed throughout the striatum, being preferentially localised in medium spiny neurons. It is also present in lower densities in frontal cortex and thalamus. Rare mutations in humans suggest a role in cognition and motor function, while common variants are associated with psychosis. Here we evaluate the influence of genetic deletion of GPR88 upon performance in translational tasks interrogating motivation, reward evaluation and cognitive function. In an automated radial arm maze ‘N-back’ working memory task, Gpr88 KO mice showed impaired correct responding, suggesting a role for GPR88 receptors in working memory circuitry. Associative learning performance was similar to wild-type controls in a touchscreen task but performance was impaired at the reversal learning stage, suggesting cognitive inflexibility. Gpr88 KO mice showed higher breakpoints, reduced latencies and lengthened session time in a progressive ratio task consistent with enhanced motivation. Simultaneously, locomotor hyperactivity was apparent in this task, supporting previous findings of actions of GPR88 in a cortico-striatal-thalamic motor loop. Evidence for a role of GPR88 in reward processing was demonstrated in a touchscreen-based equivalent of the Iowa gambling task. Although both Gpr88 KO and wild-type mice showed a preference for an optimum contingency choice, Gpr88 KO mice selected more risky choices at the expense of more advantageous lower risk options. Together these novel data suggest that striatal GPR88 receptors influence activity in a range of procedures integrated by prefrontal, orbitofrontal and anterior cingulate cortico-striatal-thalamic loops leading to altered cognitive, motivational and reward evaluation processes.     

Relief as a reward: hedonic and neural responses to safety from pain

Relief fits the definition of a reward. Unlike other reward types the pleasantness of relief depends on the violation of a negative expectation, yet this has not been investigated using neuroimaging approaches. We hypothesized that the degree of negative expectation depends on state (dread) and trait (pessimism) sensitivity. Of the brain regions that are involved in mediating pleasure, the nucleus accumbens also signals unexpected reward and positive prediction error. We hypothesized that accumbens activity reflects the level of negative expectation and subsequent pleasant relief. Using fMRI and two purpose-made tasks, we compared hedonic and BOLD responses to relief with responses during an appetitive reward task in 18 healthy volunteers. We expected some similarities in task responses, reflecting common neural substrates implicated across reward types. However, we also hypothesized that relief responses would differ from appetitive rewards in the nucleus accumbens, since only relief pleasantness depends on negative expectations. The results confirmed these hypotheses. Relief and appetitive reward task activity converged in the ventromedial prefrontal cortex, which also correlated with appetitive reward pleasantness ratings. In contrast, dread and pessimism scores correlated with relief but not with appetitive reward hedonics. Moreover, only relief pleasantness covaried with accumbens activation. Importantly, the accumbens signal appeared to specifically reflect individual differences in anticipation of the adverse event (dread, pessimism) but was uncorrelated to appetitive reward hedonics. In conclusion, relief differs from appetitive rewards due to its reliance on negative expectations, the violation of which is reflected in relief-related accumbens activation.     

Locus coeruleus neurons encode the subjective difficulty of triggering and executing actions

The brain stem noradrenergic nucleus locus coeruleus (LC) is involved in various costly processes: arousal, stress, and attention. Recent work has pointed toward an implication in physical effort, and indirect evidence suggests that the LC could be also involved in cognitive effort. To assess the dynamic relation between LC activity, effort production, and difficulty, we recorded the activity of 193 LC single units in 5 monkeys performing 2 discounting tasks (a delay discounting task and a force discounting task), as well as a simpler target detection task where conditions were matched for difficulty and only differed in terms of sensory-motor processes. First, LC neurons displayed a transient activation both when monkeys initiated an action and when exerting force. Second, the magnitude of the activation scaled with the associated difficulty, and, potentially, the corresponding amount of effort produced, both for decision and force production. Indeed, at action initiation in both discounting tasks, LC activation increased in conditions associated with lower average engagement rate, i.e., those requiring more cognitive control to trigger the response. Decision-related activation also scaled with response time (RT), over and above task parameters, in line with the idea that it reflects the amount of resources (here time) spent on the decision process. During force production, LC activation only scaled with the amount of force produced in the force discounting task, but not in the control target detection task, where subjective difficulty was equivalent across conditions. Our data show that LC neurons dynamically track the amount of effort produced to face both cognitive and physical challenges with a subsecond precision. This works provides key insight into effort processing and the contribution of the noradrenergic system, which is affected in several pathologies where effort is impaired, including Parkinson disease and depression.

Compared to reward, the neural basis of effort remains poorly understood. This study uses neurophysiological recordings in behaving macaques to show that locus coeruleus noradrenergic neurons provide information about both cognitive and physical effort, a few hundred milliseconds after it had been exerted.     

Humans can adopt optimal discounting strategy under real-time constraints

Critical to our many daily choices between larger delayed rewards, and smaller more immediate rewards, are the shape and the steepness of the function that discounts rewards with time. Although research in artificial intelligence favors exponential discounting in uncertain environments, studies with humans and animals have consistently shown hyperbolic discounting. We investigated how humans perform in a reward decision task with temporal constraints, in which each choice affects the time remaining for later trials, and in which the delays vary at each trial. We demonstrated that most of our subjects adopted exponential discounting in this experiment. Further, we confirmed analytically that exponential discounting, with a decay rate comparable to that used by our subjects, maximized the total reward gain in our task. Our results suggest that the particular shape and steepness of temporal discounting is determined by the task that the subject is facing, and question the notion of hyperbolic reward discounting as a universal principle.     

Transfer of the Nonmatch-to-Goal rule in Monkeys across Cognitive Domains

To solve novel problems, it is advantageous to abstract relevant information from past experience to transfer on related problems. To study whether macaque monkeys were able to transfer an abstract rule across cognitive domains, we trained two monkeys on a nonmatch-to-goal (NMTG) task. In the object version of the task (O-NMTG), the monkeys were required to choose between two object-like stimuli, which differed either only in shape or in shape and color. For each choice, they were required to switch from their previously chosen object-goal to a different one. After they reached a performance level of over 90% correct on the O-NMTG task, the monkeys were tested for rule transfer on a spatial version of the task (S-NMTG). To receive a reward, the monkeys had to switch from their previously chosen location to a different one. In both the O-NMTG and S-NMTG tasks, there were four potential choices, presented in pairs from trial-to-trial. We found that both monkeys transferred successfully the NMTG rule within the first testing session, showing effective transfer of the learned rule between two cognitive domains.     

Dopamine neurons can represent context-dependent prediction error

Midbrain dopamine (DA) neurons are thought to encode reward prediction error. Reward prediction can be improved if any relevant context is taken into account. We found that monkey DA neurons can encode a context-dependent prediction error. In the first noncontextual task, a light stimulus was randomly followed by reward, with a fixed equal probability. The response of DA neurons was positively correlated with the number of preceding unrewarded trials and could be simulated by a conventional temporal difference (TD) model. In the second contextual task, a reward-indicating light stimulus was presented with the probability that, while fixed overall, was incremented as a function of the number of preceding unrewarded trials. The DA neuronal response then was negatively correlated with this number. This history effect corresponded to the prediction error based on the conditional probability of reward and could be simulated only by implementing the relevant context into the TD model.     

Functional connectivity between prefrontal cortex and striatum estimated by phase locking value

The interplay between the prefrontal cortex (PFC) and striatum has an important role in cognitive processes. To investigate interactive functions between the two areas in reward processing, we recorded local field potentials (LFPs) simultaneously from the two areas of two monkeys performing a reward prediction task (large reward vs small reward). The power of the LFPs was calculated in three frequency bands: the beta band (15–29 Hz), the low gamma band (30–49 Hz), and the high gamma band (50–100 Hz). We found that both the PFC and striatum encoded the reward information in the beta band. The reward information was also found in the high gamma band in the PFC, not in the striatum. We further calculated the phase-locking value (PLV) between two LFP signals to measure the phase synchrony between the PFC and striatum. It was found that significant differences occurred between PLVs in different task periods and in different frequency bands. The PLVs in small reward condition were significant higher than that in large reward condition in the beta band. In contrast, the PLVs in the high gamma band were stronger in large reward trials than in small trials. These results suggested that the functional connectivity between the PFC and striatum depended on the task periods and reward conditions. The beta synchrony between the PFC and striatum may regulate behavioral outputs of the monkeys in the small reward condition.     

Corvids can decide if a future exchange is worth waiting for

Evidence for time-dependent calculations about future rewards is scarce in non-human animals. In non-human primates, only great apes are comparable with humans. Still, some species wait for several minutes to obtain a better reward in delayed exchange tasks. Corvids have been shown to match with non-human primates in some time-related tasks. Here, we investigate a delay of gratification in two corvid species, the carrion crow (Corvus corone) and the common raven (Corvus corax), in an exchange task. Results show that corvids success decreases quickly as delay increases, with a maximal delay of up to 320 s (more than 5 min). The decision to wait rests both on the quality of the prospective reward and the time required to obtain it. Corvids also apply tactics (placing the reward on the ground or caching it) that probably alleviate costs of waiting and distract their attention during waiting. These findings contrast previous results on delayed gratification in birds and indicate that some species may perform comparably to primates.     

Behavioral Responses to Inequity in Reward Distribution and Working Effort in Crows and Ravens

Sensitivity to inequity is considered to be a crucial cognitive tool in the evolution of human cooperation. The ability has recently been shown also in primates and dogs, raising the question of an evolutionary basis of inequity aversion. We present first evidence that two bird species are sensitive to other individuals'' efforts and payoffs. In a token exchange task we tested both behavioral responses to inequity in the quality of reward (preferred versus non-preferred food) and to the absence of reward in the presence of a rewarded partner, in 5 pairs of corvids (6 crows, 4 ravens). Birds decreased their exchange performance when the experimental partner received the reward as a gift, which indicates that they are sensitive to other individuals'' working effort. They also decreased their exchange performance in the inequity compared with the equity condition. Notably, corvids refused to take the reward after a successful exchange more often in the inequity compared with the other conditions. Our findings indicate that awareness to other individuals'' efforts and payoffs may evolve independently of phylogeny in systems with a given degree of social complexity.     

Langer doorwerken: wellicht zo slecht nog niet? Een pilotstudie naar de relatie tussen werkzaam zijn en cognitief functioneren op latere leeftijd

Studies have convincingly shown that both physical and mental activity are positively associated with cognitive task performance in aging. Little is known, however, about whether still being employed or doing volunteer work, which obviously engages physical and/or mental activity, is similarly associated with cognitive ability at an older age. The current study explored this relationship in 28 volunteers aged sixty years and older. Participants completed a neuropsychological test battery, and data regarding the number of working hours (paid and voluntary) per week were collected. A total of 28 participants were included, 13 of whom worked three or more hours per week. As a group, these active participants achieved better episodic memory, sustained attention and psychomotor speed results. This study shows that older people who are still working demonstrate better neuropsychological task performance. An important question for future research concerns the causality of this relationship.     

Reward Sensitivity Modulates Brain Activity in the Prefrontal Cortex,ACC and Striatum during Task Switching

Current perspectives on cognitive control acknowledge that individual differences in motivational dispositions may modulate cognitive processes in the absence of reward contingencies. This work aimed to study the relationship between individual differences in Behavioral Activation System (BAS) sensitivity and the neural underpinnings involved in processing a switching cue in a task-switching paradigm. BAS sensitivity was hypothesized to modulate brain activity in frontal regions, ACC and the striatum. Twenty-eight healthy participants underwent fMRI while performing a switching task, which elicited activity in fronto-striatal regions during the processing of the switch cue. BAS sensitivity was negatively associated with activity in the lateral prefrontal cortex, anterior cingulate cortex and the ventral striatum. Combined with previous results, our data indicate that BAS sensitivity modulates the neurocognitive processes involved in task switching in a complex manner depending on task demands. Therefore, individual differences in motivational dispositions may influence cognitive processing in the absence of reward contingencies.     

COMT val158met predicts reward responsiveness in humans

A functional variant of the catechol‐O‐methyltransferase (COMT) gene [val158met (rs4680)] is frequently implicated in decision‐making and higher cognitive functions. It may achieve its effects by modulating dopamine‐related decision‐making and reward‐guided behaviour. Here we demonstrate that individuals with the met/met polymorphism have greater responsiveness to reward than carriers of the val allele and that this correlates with risk‐seeking behaviour. We assessed performance on a reward responsiveness task and the Balloon analogue risk task, which measure how participants (N = 70, western European, university and postgraduate students) respond to reward and take risks in the presence of available reward. Individuals with the met/met genotype (n = 19) showed significantly higher reward responsiveness, F_2,64 = 4.02, P = 0.02, and reward‐seeking behaviour, F(2,68) = 4.52, P = 0.01, than did either val/met (n = 25) or val/val (n = 26) carriers. These results highlight a scenario in which genotype‐dependent reward responsiveness shapes reward‐seeking, therefore suggesting a novel framework by which COMT may modulate behaviour.     

From molecule to market: steroid hormones and financial risk-taking

Little is known about the role of the endocrine system in financial decision-making. Here, we survey research on steroid hormones and their cognitive effects, and examine potential links to trader performance in the financial markets. Preliminary findings suggest that cortisol codes for risk and testosterone for reward. A key finding of this endocrine research is the different cognitive effects of acute versus chronic exposure to hormones: acutely elevated steroids may optimize performance on a range of tasks; but chronically elevated steroids may promote irrational risk-reward choices. We present a hypothesis suggesting that the irrational exuberance and pessimism observed during market bubbles and crashes may be mediated by steroid hormones. If hormones can exaggerate market moves, then perhaps the age and sex composition among traders and asset managers may affect the level of instability witnessed in the financial markets.     

Functional MRI in awake unrestrained dogs

Because of dogs' prolonged evolution with humans, many of the canine cognitive skills are thought to represent a selection of traits that make dogs particularly sensitive to human cues. But how does the dog mind actually work? To develop a methodology to answer this question, we trained two dogs to remain motionless for the duration required to collect quality fMRI images by using positive reinforcement without sedation or physical restraints. The task was designed to determine which brain circuits differentially respond to human hand signals denoting the presence or absence of a food reward. Head motion within trials was less than 1 mm. Consistent with prior reinforcement learning literature, we observed caudate activation in both dogs in response to the hand signal denoting reward versus no-reward.     

Hippocampal representation of touch-guided behavior in rats: persistent and independent traces of stimulus and reward location

Understanding the mechanisms by which sensory experiences are stored remains a compelling challenge for neuroscience. Previous work has described how the activity of neurons in the sensory cortex allows rats to discriminate the physical features of an object contacted with their whiskers. But to date there is no evidence about how neurons represent the behavioural significance of tactile stimuli, or how they are encoded in memory. To investigate these issues, we recorded single-unit firing and local field potentials from the CA1 region of hippocampus while rats performed a task in which tactile stimuli specified reward location. On each trial the rat touched a textured plate with its whiskers, and then turned towards the Left or Right water spout. Two textures were associated with each reward location. To determine the influence of the rat's position on sensory coding, we placed it on a second platform in the same room where it performed the identical texture discrimination task. Over 25 percent of the sampled neurons encoded texture identity--their firing differed for two stimuli associated with the same reward location--and over 50 percent of neurons encoded the reward location with which the stimuli were associated. The neuronal population carried texture and reward location signals continuously, from the moment of stimulus contact until the end of reward collection. The set of neurons discriminating between one texture pair was found to be independent of, and partially overlapping, the set of neurons encoding the discrimination between a different texture pair. In a given neuron, the presence of a tactile signal was uncorrelated with the presence, magnitude, or timing of reward location signals. These experiments indicate that neurons in CA1 form a texture representation independently of the action the stimulus is associated with and retain the stimulus representation through reward collection.     

Learning to Obtain Reward,but Not Avoid Punishment,Is Affected by Presence of PTSD Symptoms in Male Veterans: Empirical Data and Computational Model

Post-traumatic stress disorder (PTSD) symptoms include behavioral avoidance which is acquired and tends to increase with time. This avoidance may represent a general learning bias; indeed, individuals with PTSD are often faster than controls on acquiring conditioned responses based on physiologically-aversive feedback. However, it is not clear whether this learning bias extends to cognitive feedback, or to learning from both reward and punishment. Here, male veterans with self-reported current, severe PTSD symptoms (PTSS group) or with few or no PTSD symptoms (control group) completed a probabilistic classification task that included both reward-based and punishment-based trials, where feedback could take the form of reward, punishment, or an ambiguous “no-feedback” outcome that could signal either successful avoidance of punishment or failure to obtain reward. The PTSS group outperformed the control group in total points obtained; the PTSS group specifically performed better than the control group on reward-based trials, with no difference on punishment-based trials. To better understand possible mechanisms underlying observed performance, we used a reinforcement learning model of the task, and applied maximum likelihood estimation techniques to derive estimated parameters describing individual participants’ behavior. Estimations of the reinforcement value of the no-feedback outcome were significantly greater in the control group than the PTSS group, suggesting that the control group was more likely to value this outcome as positively reinforcing (i.e., signaling successful avoidance of punishment). This is consistent with the control group’s generally poorer performance on reward trials, where reward feedback was to be obtained in preference to the no-feedback outcome. Differences in the interpretation of ambiguous feedback may contribute to the facilitated reinforcement learning often observed in PTSD patients, and may in turn provide new insight into how pathological behaviors are acquired and maintained in PTSD.     

Promising high monetary rewards for future task performance increases intermediate task performance

In everyday life contexts and work settings, monetary rewards are often contingent on future performance. Based on research showing that the anticipation of rewards causes improved task performance through enhanced task preparation, the present study tested the hypothesis that the promise of monetary rewards for future performance would not only increase future performance, but also performance on an unrewarded intermediate task. Participants performed an auditory Simon task in which they responded to two consecutive tones. While participants could earn high vs. low monetary rewards for fast responses to every second tone, their responses to the first tone were not rewarded. Moreover, we compared performance under conditions in which reward information could prompt strategic performance adjustments (i.e., when reward information was presented for a relatively long duration) to conditions preventing strategic performance adjustments (i.e., when reward information was presented very briefly). Results showed that high (vs. low) rewards sped up both rewarded and intermediate, unrewarded responses, and the effect was independent of the duration of reward presentation. Moreover, long presentation led to a speed-accuracy trade-off for both rewarded and unrewarded tones, whereas short presentation sped up responses to rewarded and unrewarded tones without this trade-off. These results suggest that high rewards for future performance boost intermediate performance due to enhanced task preparation, and they do so regardless whether people respond to rewards in a strategic or non-strategic manner.