Frontal cortex and reward-guided learning and decision-making

Reward-guided decision-making and learning depends on distributed neural circuits with many components. Here we focus on recent evidence that suggests four frontal lobe regions make distinct contributions to reward-guided learning and decision-making: the lateral orbitofrontal cortex, the ventromedial prefrontal cortex and adjacent medial orbitofrontal cortex, anterior cingulate cortex, and the anterior lateral prefrontal cortex. We attempt to identify common themes in experiments with human participants and with animal models, which suggest roles that the areas play in learning about reward associations, selecting reward goals, choosing actions to obtain reward, and monitoring the potential value of switching to alternative courses of action.     

Social norm compliance as a signaling system. I. Studies of fitness-related attributions consequent on everyday norm violations

People attend to cues that convey information about social norms and try to comply with norms they believe are in force. Dispositions to comply with social norms are universal, suggesting that adherence to such norms is selectively advantageous. Possibly, compliance with social norms, however arbitrary these may be, serves a signaling function and is used to control attributions affecting fitness. To begin to test this hypothesis, we performed several experiments in which subjects watched videotapes of models violating everyday social norms and then rated those models on dimensions that would be relevant to the models' fitness, if subjects and models were socially interacting. In some experiments, violations of minor social norms significantly altered such ratings. Even subjects who failed to cite norm violations when given the opportunity nonetheless gave lower ratings to models as the result of norm violations. A manipulation that increased the salience of such norms increased the adverse effects of norms violations. The results are consistent with the hypothesis that norm compliance serves an important signaling function.     

Increased Neural Habituation in the Amygdala and Orbitofrontal Cortex in Social Anxiety Disorder Revealed by fMRI

A characterizing symptom of social anxiety disorder (SAD) is increased emotional reactivity towards potential social threat in combination with impaired emotion and stress regulation. While several neuroimaging studies have linked SAD with hyperreactivity in limbic brain regions when exposed to emotional faces, little is known about habituation in both the amygdala and neocortical regulation areas. 15 untreated SAD patients and 15 age- and gender-matched healthy controls underwent functional magnetic resonance imaging during repeated blocks of facial emotion () and object discrimination tasks (). Emotion processing networks were defined by a task-related contrast (). Linear regression was employed for assessing habituation effects in these regions. In both groups, the employed paradigm robustly activated the emotion processing and regulation network, including the amygdalae and orbitofrontal cortex (OFC). Statistically significant habituation effects were found in the amygdalae, OFC, and pulvinar thalamus of SAD patients. No such habituation was found in healthy controls. Concurrent habituation in the medial OFC and the amygdalae of SAD patients as shown in this study suggests intact functional integrity and successful short-term down-regulation of neural activation in brain areas responsible for emotion processing. Initial hyperactivation may be explained by an insufficient habituation to new stimuli during the first seconds of exposure. In addition, our results highlight the relevance of the orbitofrontal cortex in social anxiety disorders.     

Effects of orbitofrontal lesions on social behavior in a confined group of stump-tail macaques (Macaca speciosa)

We grouped 14 stump-tail macques, five males and nine females, and observed social behavior before subjecting four of them to resection of orbitofrontal cortex. Four monkeys also received control lesions of superior temporal cortex and the remaining animals served as unoperated controls. Observations of social behavior continued after surgery on the following behavior: Joining, grooming, self-grooming, threat, aggression, and presenting (total hours of observation equaled 154). Monkeys with orbitofrontal lesions showed decreases in threat and aggression but only one such monkey fell in dominance. Control (operated and unoperated) monkeys displayed little change in these behaviors. Monkeys with orbitofrontal lesions also increased joining and self-grooming but showed a decrease in grooming of others. Presenting did not change. The role of orbitofrontal cortex in modulating different aspects of social interaction is emphasized by these results. However, in primates this area of the frontal lobes appears to have its major influence in the emotional loading of such complex behaviors.     

Do descriptive social norms drive peer punishment? Conditional punishment strategies and their impact on cooperation

Peer punishment is widely considered a key mechanism supporting cooperation in human groups. Although much research shows that human behavior is shaped by the prevailing social norms, little is known about how punishment decisions are impacted by the social context. We present a set of large-scale incentivized experiments in which participants (999 American participants recruited via Amazon Mechanical Turk) could punish their partner conditional on either the level of cooperation or the level of punishment displayed by others who previously interacted in the same setting. While many participants punish independently of levels of cooperation or punishment, a substantial portion punishes free riding more severely when cooperation is more common (‘norm enforcement’), or when free riding is more severely punished by others (‘conformist punishment’). With a dynamic model we demonstrate that conditional punishment strategies can substantially promote cooperation. In particular, conformist punishment helps cooperation to gain a foothold in a population, and norm enforcement helps to maintain cooperation at high levels. Our results provide solid empirical evidence of conditional punishment strategies and illustrate their possible implications for the dynamics of human cooperation.     

Limbic frontal cortex in hominoids: A comparative study of area 13

The limbic frontal cortex forms part of the neural substrate responsible for emotional reactions to social stimuli. Area 13 is one of the cortical areas long known to be part of the posterior orbitofrontal cortex in several monkey species, such as the macaque. Its presence nevertheless in the human brain has been unclear, and the cortex of the frontal lobe of the great and lesser apes remains largely unknown. In this study area 13 was identified in human, chimpanzee, bonobo, gorilla, orangutan, and gibbon brains, and cortical maps were generated on the basis of its cytoarchitecture. Imaging techniques were used to characterize and quantify the microstructural organization of the area, and stereological tools were applied for estimates of the volume of area 13 in all species. Area 13 is conservative in its structure, and features such as size of cortical layers, density of neurons, and space available for connections are similar across hominoids with only subtle differences present. In contrast to the homogeneity found in its organization, variation is present in the relative size of this cortical area (as a percentage of total brain volume). The human and the bonobo include a complex orbitofrontal cortex and a relatively smaller area 13. On the contrary the orangutan stands out by having a shorter orbitofrontal region and a more expanded area 13. Differences in the organization and size of individual cortical areas involved in emotional reactions and social behavior can be related to behavioral specializations of each hominoid and to the evolution of emotions in hominids. Am J Phys Anthropol 106:129–155, 1998. © 1998 Wiley-Liss, Inc.     

Common Neural Mechanisms Underlying Reversal Learning by Reward and Punishment

Impairments in flexible goal-directed decisions, often examined by reversal learning, are associated with behavioral abnormalities characterized by impulsiveness and disinhibition. Although the lateral orbital frontal cortex (OFC) has been consistently implicated in reversal learning, it is still unclear whether this region is involved in negative feedback processing, behavioral control, or both, and whether reward and punishment might have different effects on lateral OFC involvement. Using a relatively large sample (N = 47), and a categorical learning task with either monetary reward or moderate electric shock as feedback, we found overlapping activations in the right lateral OFC (and adjacent insula) for reward and punishment reversal learning when comparing correct reversal trials with correct acquisition trials, whereas we found overlapping activations in the right dorsolateral prefrontal cortex (DLPFC) when negative feedback signaled contingency change. The right lateral OFC and DLPFC also showed greater sensitivity to punishment than did their left homologues, indicating an asymmetry in how punishment is processed. We propose that the right lateral OFC and anterior insula are important for transforming affective feedback to behavioral adjustment, whereas the right DLPFC is involved in higher level attention control. These results provide insight into the neural mechanisms of reversal learning and behavioral flexibility, which can be leveraged to understand risky behaviors among vulnerable populations.     

A neural basis for social cooperation

Cooperation based on reciprocal altruism has evolved in only a small number of species, yet it constitutes the core behavioral principle of human social life. The iterated Prisoner's Dilemma Game has been used to model this form of cooperation. We used fMRI to scan 36 women as they played an iterated Prisoner's Dilemma Game with another woman to investigate the neurobiological basis of cooperative social behavior. Mutual cooperation was associated with consistent activation in brain areas that have been linked with reward processing: nucleus accumbens, the caudate nucleus, ventromedial frontal/orbitofrontal cortex, and rostral anterior cingulate cortex. We propose that activation of this neural network positively reinforces reciprocal altruism, thereby motivating subjects to resist the temptation to selfishly accept but not reciprocate favors.     

Two neurocomputational building blocks of social norm compliance

Current explanatory frameworks for social norms pay little attention to why and how brains might carry out computational functions that generate norm compliance behavior. This paper expands on existing literature by laying out the beginnings of a neurocomputational framework for social norms and social cognition, which can be the basis for advancing our understanding of the nature and mechanisms of social norms. Two neurocomputational building blocks are identified that might constitute the core of the mechanism of norm compliance. They consist of Bayesian and reinforcement learning systems. It is sketched why and how the concerted activity of these systems can generate norm compliance by minimization of three specific kinds of prediction-errors.     

Functions of the orbitofrontal and pregenual cingulate cortex in taste, olfaction, appetite and emotion

Complementary neurophysiological recordings in macaques and functional neuroimaging in humans show that the primary taste cortex in the rostral insula and adjoining frontal operculum provides separate and combined representations of the taste, temperature, and texture (including viscosity and fat texture) of food in the mouth independently of hunger and thus of reward value and pleasantness. One synapse on, in the orbitofrontal cortex, these sensory inputs are for some neurons combined by learning with olfactory and visual inputs. Different neurons respond to different combinations, providing a rich representation of the sensory properties of food. The representation of taste and other food-related stimuli in the orbitofrontal cortex of macaques is found from its lateral border throughout area 13 to within 7 mm of the midline, and in humans the representation of food-related and other pleasant stimuli is found particularly in the medial orbitofrontal cortex. In the orbitofrontal cortex, feeding to satiety with one food decreases the responses of these neurons to that food, but not to other foods, showing that sensory-specific satiety is computed in the primate (including human) orbitofrontal cortex. Consistently, activation of parts of the human orbitofrontal cortex correlates with subjective ratings of the pleasantness of the taste and smell of food. Cognitive factors, such as a word label presented with an odour, influence the pleasantness of the odour, and the activation produced by the odour in the orbitofrontal cortex. Food intake is thus controlled by building a multimodal representation of the sensory properties of food in the orbitofrontal cortex, and gating this representation by satiety signals to produce a representation of the pleasantness or reward value of food which drives food intake. A neuronal representation of taste is also found in the pregenual cingulate cortex, which receives inputs from the orbitofrontal cortex, and in humans many pleasant stimuli activate the pregenual cingulate cortex, pointing towards this as an important area in motivation and emotion.     

Neural Basis of Anticipatory Anxiety Reappraisals

Reappraisal is a well-known emotion regulation strategy. Recent neuroimaging studies suggest that reappraisal recruits both medial and lateral prefrontal brain regions. However, few studies have investigated neural representation of reappraisals associated with anticipatory anxiety, and the specific nature of the brain activity underlying this process remains unclear. We used functional magnetic resonance imaging (fMRI) to investigate neural activity associated with reappraisals of transient anticipatory anxiety. Although transient anxiety activated mainly subcortical regions, reappraisals targeting the anxiety were associated with increased activity in the medial and lateral prefrontal regions (including the orbitofrontal and anterior cingulate cortices). Reappraisal decreased fear circuit activity (including the amygdala and thalamus). Correlational analysis demonstrated that reductions in subjective anxiety associated with reappraisal were correlated with orbitofrontal and anterior cingulate cortex activation. Reappraisal recruits medial and lateral prefrontal regions; particularly the orbitofrontal and anterior cingulate cortices are associated with successful use of this emotion regulation strategy.     

Does dishonesty really invite third-party punishment? Results of a more stringent test

Many experiments have demonstrated that people are willing to incur cost to punish norm violators even when they are not directly harmed by the violation. Such altruistic third-party punishment is often considered an evolutionary underpinning of large-scale human cooperation. However, some scholars argue that previously demonstrated altruistic third-party punishment against fairness-norm violations may be an experimental artefact. For example, envy-driven retaliatory behaviour (i.e. spite) towards better-off unfair game players may be misidentified as altruistic punishment. Indeed, a recent experiment demonstrated that participants ceased to inflict third-party punishment against an unfair player once a series of key methodological problems were systematically controlled for. Noticing that a previous finding regarding apparently altruistic third-party punishment against honesty-norm violations may have been subject to methodological issues, we used a different and what we consider to be a more sound design to evaluate these findings. Third-party punishment against dishonest players withstood this more stringent test.     

Relationship between Personality Traits and Brain Reward Responses when Playing on a Team

Cooperation is an integral part of human social life and we often build teams to achieve certain goals. However, very little is currently understood about emotions with regard to cooperation. Here, we investigated the impact of social context (playing alone versus playing on a team) on emotions while winning or losing a game. We hypothesized that activity in the reward network is modulated by the social context and that personality characteristics might impact team play. We conducted an event-related functional magnetic resonance imaging experiment that involved a simple game of dice. In the team condition, the participant played with a partner against another two-person team. In the single-player condition, the participant played alone against another player. Our results revealed that reward processing in the right amygdala was modulated by the social context. The main effect of outcome (gains versus losses) was associated with increased responses in the reward network. We also found that differences in the reward-related neural response due to social context were associated with specific personality traits. When playing on a team, increased activity in the amygdala during winning was a unique function of openness, while decreased activity in the ventromedial prefrontal cortex and ventral striatum during losing was associated with extraversion and conscientiousness, respectively. In conclusion, we provide evidence that working on a team influences the affective value of a negative outcome by attenuating the negative response associated with it in the amygdala. Our results also show that brain reward responses in a social context are affected by personality traits related to teamwork.     

Distinct neurocomputational mechanisms support informational and socially normative conformity

A change of mind in response to social influence could be driven by informational conformity to increase accuracy, or by normative conformity to comply with social norms such as reciprocity. Disentangling the behavioural, cognitive, and neurobiological underpinnings of informational and normative conformity have proven elusive. Here, participants underwent fMRI while performing a perceptual task that involved both advice-taking and advice-giving to human and computer partners. The concurrent inclusion of 2 different social roles and 2 different social partners revealed distinct behavioural and neural markers for informational and normative conformity. Dorsal anterior cingulate cortex (dACC) BOLD response tracked informational conformity towards both human and computer but tracked normative conformity only when interacting with humans. A network of brain areas (dorsomedial prefrontal cortex (dmPFC) and temporoparietal junction (TPJ)) that tracked normative conformity increased their functional coupling with the dACC when interacting with humans. These findings enable differentiating the neural mechanisms by which different types of conformity shape social changes of mind.

When we change our mind in response to other people’s opinion, we may be motivated to be correct or to have a good relationship with others. This fMRI study disentangles the neurobiological underpinnings of these two different motives in the human brain, and shows that the second motive is absent when we interact with computers.     

Punishment as a Means of Competition: Implications for Strong Reciprocity Theory

Strong negative reciprocity, that is, sanctions imposed on norm violators at the punisher’s own expense, has powerful cooperation-enhancing effects in both real-life and experimental game situations. However, it is plausible that punishment may obtain alternative roles depending on social context and the personality characteristics of participants. We examined the occurrence of punishing behavior among 80 subjects in a strongly competitive Public Goods game setting. Despite the punishment condition, the amount of the contributions decreased steadily during the game. The amount of contributions had no significant effect on received and imposed punishments. The results indicate that certain social contexts (in this case, intensive competition) exert modifying effects on the role that punishment takes on. Subjects punished each other in order to achieve a higher rank and a financially better outcome. Punishment primarily functioned as a means of rivalry, instead of as a way of second-order cooperation, as strong reciprocity suggests. These results indicate the need for the possible modification of the social conditions of punishment mechanisms described by the strong reciprocity theory as an evolutionary explanation of human cooperation.     

The Neuroanatomical Basis of Panic Disorder and Social Phobia in Schizophrenia: A Voxel Based Morphometric Study

ObjectiveIt is known that there is a high prevalence of certain anxiety disorders among schizophrenic patients, especially panic disorder and social phobia. However, the neural underpinnings of the comorbidity of such anxiety disorders and schizophrenia remain unclear. Our study aims to determine the neuroanatomical basis of the co-occurrence of schizophrenia with panic disorder and social phobia.MethodsVoxel-based morphometry was used in order to examine brain structure and to measure between-group differences, comparing magnetic resonance images of 20 anxious patients, 20 schizophrenic patients, 20 schizophrenic patients with comorbid anxiety, and 20 healthy control subjects.ResultsCompared to the schizophrenic patients, we observed smaller grey-matter volume (GMV) decreases in the dorsolateral prefrontal cortex and precentral gyrus in the schizophrenic-anxiety group. Additionally, the schizophrenic group showed significantly reduced GMV in the dorsolateral prefrontal cortex, precentral gyrus, orbitofrontal cortex, temporal gyrus and angular/inferior parietal gyrus when compared to the control group.ConclusionsOur findings suggest that the comorbidity of schizophrenia with panic disorder and social phobia might be characterized by specific neuroanatomical and clinical alterations that may be related to maladaptive emotion regulation related to anxiety. Even thought our findings need to be replicated, our study suggests that the identification of neural abnormalities involved in anxiety, schizophrenia and schizophrenia-anxiety may lead to an improved diagnosis and management of these conditions.     

A comparative volumetric analysis of the amygdaloid complex and basolateral division in the human and ape brain

The amygdaloid complex functions to facilitate effective appraisal of the social environment and is an essential component of the neural systems subserving social behavior. Despite its critical role in mediating social interaction, the amygdaloid complex has not attracted the same attention as the isocortex in most evolutionary analyses. We performed a comparative analysis of the amygdaloid complex in the hominoids to address the lack of comparative information available for this structure in the hominoid brain. We demarcated the amygdaloid complex and the three nuclei constituting its basolateral division, the lateral, basal, and accessory basal nuclei, in 12 histological series representing all six hominoid species. The volumes obtained for these areas were subjected to allometric analyses to determine whether any species deviated from expected values based on the other hominoids. Differences between groups were addressed using nonparametric comparisons of means. The human lateral nucleus was larger than predicted for an ape of human brain size and occupied the majority of the basolateral division, whereas the basal nucleus was the largest of the basolateral nuclei in all ape species. In orangutans the amygdala and basolateral division were smaller than in the African apes. While the gorilla had a smaller than predicted lateral nucleus, its basal and accessory basal nuclei were larger than predicted. These differences may reflect volumetric changes occurring in interconnected cortical areas, specifically the temporal lobe and orbitofrontal cortex, which also subserve social behavior and cognition, suggesting that this system may be acted upon in hominoid and hominid evolution.     

Placebo in emotional processing--induced expectations of anxiety relief activate a generalized modulatory network

Placebo analgesia and reward processing share several features. For instance, expectations have a strong influence on the subsequent emotional experience of both. Recent imaging data indicate similarities in the underlying neuronal network. We hypothesized that placebo analgesia is a special case of reward processing and that placebo treatment could modulate emotional perception in the same way as does pain perception. The behavioral part of this study indicates that placebo treatment has an effect on how subjects perceive unpleasant pictures. Furthermore, event-related fMRI demonstrated that the same modulatory network, including the rostral anterior cingulate cortex and the lateral orbitofrontal cortex, is involved in both emotional placebo and placebo analgesia. These effects were correlated with the reported placebo effect and were predicted by the amount of treatment expectation induced on a previous day. Thus, the placebo effect may be considered to be a general process of modulation induced by the subjects' expectations.     

Frequency Change Detection in Human Auditory Cortex

We offer a model of how human cortex detects changes in the auditory environment. Auditory change detection has recently been the object of intense investigation via the mismatch negativity (MMN). MMN is a preattentive response to sudden changes in stimulation, measured noninvasively in the electroencephalogram (EEG) and the magnetoencephalogram (MEG). It is elicited in the oddball paradigm, where infrequent deviant tones intersperse a series of repetitive standard tones. However, little apart from the participation of tonotopically organized auditory cortex is known about the neural mechanisms underlying change detection and the MMN. In the present study, we investigate how poststimulus inhibition might account for MMN and compare the effects of adaptation with those of lateral inhibition in a model describing tonotopically organized cortex. To test the predictions of our model, we performed MEG and EEG measurements on human subjects and used both small- (<1/3 octave) and large- (>5 octaves) frequency differences between the standard and deviant tones. The experimental results bear out the prediction that MMN is due to both adaptation and lateral inhibition. Finally, we suggest that MMN might serve as a probe of what stimulus features are mapped by human auditory cortex.     

Orbitofrontal cortex and memory formation

Which one of the many regions of the anatomically heterogeneous prefrontal cortex is part of the critical core of the neural circuit for encoding? This positron emission tomography (PET) experiment measured changes in cerebral blood flow (CBF) in normal human participants during the presentation of abstract visual information in four conditions that varied in their encoding demands. As encoding increased across the different conditions, there was an increase in activity in the right orbitofrontal cortex and the right parahippocampal region. No significant activation peaks were present in any other region of the frontal or temporal lobe. These findings indicate that the orbitofrontal cortex, which is massively connected to the medial temporal cortex, is a critical frontal region for memory formation.