Learning to Obtain Reward,but Not Avoid Punishment,Is Affected by Presence of PTSD Symptoms in Male Veterans: Empirical Data and Computational Model

Post-traumatic stress disorder (PTSD) symptoms include behavioral avoidance which is acquired and tends to increase with time. This avoidance may represent a general learning bias; indeed, individuals with PTSD are often faster than controls on acquiring conditioned responses based on physiologically-aversive feedback. However, it is not clear whether this learning bias extends to cognitive feedback, or to learning from both reward and punishment. Here, male veterans with self-reported current, severe PTSD symptoms (PTSS group) or with few or no PTSD symptoms (control group) completed a probabilistic classification task that included both reward-based and punishment-based trials, where feedback could take the form of reward, punishment, or an ambiguous “no-feedback” outcome that could signal either successful avoidance of punishment or failure to obtain reward. The PTSS group outperformed the control group in total points obtained; the PTSS group specifically performed better than the control group on reward-based trials, with no difference on punishment-based trials. To better understand possible mechanisms underlying observed performance, we used a reinforcement learning model of the task, and applied maximum likelihood estimation techniques to derive estimated parameters describing individual participants’ behavior. Estimations of the reinforcement value of the no-feedback outcome were significantly greater in the control group than the PTSS group, suggesting that the control group was more likely to value this outcome as positively reinforcing (i.e., signaling successful avoidance of punishment). This is consistent with the control group’s generally poorer performance on reward trials, where reward feedback was to be obtained in preference to the no-feedback outcome. Differences in the interpretation of ambiguous feedback may contribute to the facilitated reinforcement learning often observed in PTSD patients, and may in turn provide new insight into how pathological behaviors are acquired and maintained in PTSD.     

Neural signatures of arbitration between Pavlovian and instrumental action selection

Pavlovian associations drive approach towards reward-predictive cues, and avoidance of punishment-predictive cues. These associations “misbehave” when they conflict with correct instrumental behavior. This raises the question of how Pavlovian and instrumental influences on behavior are arbitrated. We test a computational theory according to which Pavlovian influence will be stronger when inferred controllability of outcomes is low. Using a model-based analysis of a Go/NoGo task with human subjects, we show that theta-band oscillatory power in frontal cortex tracks inferred controllability, and that these inferences predict Pavlovian action biases. Functional MRI data revealed an inferior frontal gyrus correlate of action probability and a ventromedial prefrontal correlate of outcome valence, both of which were modulated by inferred controllability.     

Reduction of Pavlovian Bias in Schizophrenia: Enhanced Effects in Clozapine-Administered Patients

The negative symptoms of schizophrenia (SZ) are associated with a pattern of reinforcement learning (RL) deficits likely related to degraded representations of reward values. However, the RL tasks used to date have required active responses to both reward and punishing stimuli. Pavlovian biases have been shown to affect performance on these tasks through invigoration of action to reward and inhibition of action to punishment, and may be partially responsible for the effects found in patients. Forty-five patients with schizophrenia and 30 demographically-matched controls completed a four-stimulus reinforcement learning task that crossed action (“Go” or “NoGo”) and the valence of the optimal outcome (reward or punishment-avoidance), such that all combinations of action and outcome valence were tested. Behaviour was modelled using a six-parameter RL model and EEG was simultaneously recorded. Patients demonstrated a reduction in Pavlovian performance bias that was evident in a reduced Go bias across the full group. In a subset of patients administered clozapine, the reduction in Pavlovian bias was enhanced. The reduction in Pavlovian bias in SZ patients was accompanied by feedback processing differences at the time of the P3a component. The reduced Pavlovian bias in patients is suggested to be due to reduced fidelity in the communication between striatal regions and frontal cortex. It may also partially account for previous findings of poorer “Go-learning” in schizophrenia where “Go” responses or Pavlovian consistent responses are required for optimal performance. An attenuated P3a component dynamic in patients is consistent with a view that deficits in operant learning are due to impairments in adaptively using feedback to update representations of stimulus value.     

The Co-Evolution of Fairness Preferences and Costly Punishment

We study the co-evolutionary emergence of fairness preferences in the form of other-regarding behavior and its effect on the origination of costly punishment behavior in public good games. Our approach closely combines empirical results from three experiments with an evolutionary simulation model. In this way, we try to fill a gap between the evolutionary theoretical literature on cooperation and punishment on the one hand and the empirical findings from experimental economics on the other hand. As a principal result, we show that the evolution among interacting agents inevitably favors a sense for fairness in the form of “disadvantageous inequity aversion”. The evolutionary dominance and stability of disadvantageous inequity aversion is demonstrated by enabling agents to co-evolve with different self- and other-regarding preferences in a competitive environment with limited resources. Disadvantageous inequity aversion leads to the emergence of costly (“altruistic”) punishment behavior and quantitatively explains the level of punishment observed in contemporary lab experiments performed on subjects with a western culture. Our findings corroborate, complement, and interlink the experimental and theoretical literature that has shown the importance of other-regarding behavior in various decision settings.     

Evolutionary Establishment of Moral and Double Moral Standards through Spatial Interactions

Situations where individuals have to contribute to joint efforts or share scarce resources are ubiquitous. Yet, without proper mechanisms to ensure cooperation, the evolutionary pressure to maximize individual success tends to create a tragedy of the commons (such as over-fishing or the destruction of our environment). This contribution addresses a number of related puzzles of human behavior with an evolutionary game theoretical approach as it has been successfully used to explain the behavior of other biological species many times, from bacteria to vertebrates. Our agent-based model distinguishes individuals applying four different behavioral strategies: non-cooperative individuals (“defectors”), cooperative individuals abstaining from punishment efforts (called “cooperators” or “second-order free-riders”), cooperators who punish non-cooperative behavior (“moralists”), and defectors, who punish other defectors despite being non-cooperative themselves (“immoralists”). By considering spatial interactions with neighboring individuals, our model reveals several interesting effects: First, moralists can fully eliminate cooperators. This spreading of punishing behavior requires a segregation of behavioral strategies and solves the “second-order free-rider problem”. Second, the system behavior changes its character significantly even after very long times (“who laughs last laughs best effect”). Third, the presence of a number of defectors can largely accelerate the victory of moralists over non-punishing cooperators. Fourth, in order to succeed, moralists may profit from immoralists in a way that appears like an “unholy collaboration”. Our findings suggest that the consideration of punishment strategies allows one to understand the establishment and spreading of “moral behavior” by means of game-theoretical concepts. This demonstrates that quantitative biological modeling approaches are powerful even in domains that have been addressed with non-mathematical concepts so far. The complex dynamics of certain social behaviors become understandable as the result of an evolutionary competition between different behavioral strategies.     

Understanding Recurrent Crime as System-Immanent Collective Behavior

Containing the spreading of crime is a major challenge for society. Yet, since thousands of years, no effective strategy has been found to overcome crime. To the contrary, empirical evidence shows that crime is recurrent, a fact that is not captured well by rational choice theories of crime. According to these, strong enough punishment should prevent crime from happening. To gain a better understanding of the relationship between crime and punishment, we consider that the latter requires prior discovery of illicit behavior and study a spatial version of the inspection game. Simulations reveal the spontaneous emergence of cyclic dominance between “criminals”, “inspectors”, and “ordinary people” as a consequence of spatial interactions. Such cycles dominate the evolutionary process, in particular when the temptation to commit crime or the cost of inspection are low or moderate. Yet, there are also critical parameter values beyond which cycles cease to exist and the population is dominated either by a stable mixture of criminals and inspectors or one of these two strategies alone. Both continuous and discontinuous phase transitions to different final states are possible, indicating that successful strategies to contain crime can be very much counter-intuitive and complex. Our results demonstrate that spatial interactions are crucial for the evolutionary outcome of the inspection game, and they also reveal why criminal behavior is likely to be recurrent rather than evolving towards an equilibrium with monotonous parameter dependencies.     

Agency Modulates the Lateral and Medial Prefrontal Cortex Responses in Belief-Based Decision Making

Many real-life decisions in complex and changing environments are guided by the decision maker’s beliefs, such as her perceived control over decision outcomes (i.e., agency), leading to phenomena like the “illusion of control”. However, the neural mechanisms underlying the “agency” effect on belief-based decisions are not well understood. Using functional imaging and a card guessing game, we revealed that the agency manipulation (i.e., either asking the subjects (SG) or the computer (CG) to guess the location of the winning card) not only affected the size of subjects’ bets, but also their “world model” regarding the outcome dependency. Functional imaging results revealed that the decision-related activation in the lateral and medial prefrontal cortex (PFC) was significantly modulated by agency and previous outcome. Specifically, these PFC regions showed stronger activation when subjects made decisions after losses than after wins under the CG condition, but the pattern was reversed under the SG condition. Furthermore, subjects with high external attribution of negative events were more affected by agency at the behavioral and neural levels. These results suggest that the prefrontal decision-making system can be modulated by abstract beliefs, and are thus vulnerable to factors such as false agency and attribution.     

Binocular Glaucomatous Visual Field Loss and Its Impact on Visual Exploration - A Supermarket Study

Advanced glaucomatous visual field loss may critically interfere with quality of life. The purpose of this study was to (i) assess the impact of binocular glaucomatous visual field loss on a supermarket search task as an example of everyday living activities, (ii) to identify factors influencing the performance, and (iii) to investigate the related compensatory mechanisms. Ten patients with binocular glaucoma (GP), and ten healthy-sighted control subjects (GC) were asked to collect twenty different products chosen randomly in two supermarket racks as quickly as possible. The task performance was rated as “passed” or “failed” with regard to the time per correctly collected item. Based on the performance of control subjects, the threshold value for failing the task was defined as μ+3σ (in seconds per correctly collected item). Eye movements were recorded by means of a mobile eye tracker. Eight out of ten patients with glaucoma and all control subjects passed the task. Patients who failed the task needed significantly longer time (111.47 s ±12.12 s) to complete the task than patients who passed (64.45 s ±13.36 s, t-test, p<0.001). Furthermore, patients who passed the task showed a significantly higher number of glances towards the visual field defect (VFD) area than patients who failed (t-test, p<0.05). According to these results, glaucoma patients with defects in the binocular visual field display on average longer search times in a naturalistic supermarket task. However, a considerable number of patients, who compensate by frequent glancing towards the VFD, showed successful task performance. Therefore, systematic exploration of the VFD area seems to be a “time-effective” compensatory mechanism during the present supermarket task.     

Feedback and incentive effects on the decrease of interbeat interval

Johnston and Lethem (1981) have hypothesized that when subjects attempt to decrease their interbeat interval (i.e., increase their heart rate) maximally, then interbeat interval feedback has a purely motivational role, but when they attempt to decrease their interbeat interval by a precise amount, such feedback has a primarily informational role. This was tested by comparing the performance of 16 subjects on both types of task. It was predicted that additional monetary incentives would reduce the difference between feedback and no-feedback conditions when subjects were attempting to reduce interbeat interval maximally but not when attempting to reduce it by a specific amount. This prediction was not supported. Incentive was found to aid performance on the maximal interbeat interval decrease task, but this was independent of the effects of feedback. Only feedback effects were detected on the specific interbeat interval decrease task.     

Assessing the Perception of Trunk Movements in Military Personnel with Chronic Non-Specific Low Back Pain Using a Virtual Mirror

Chronic pain, including chronic non-specific low back pain (CNSLBP), is often associated with body perception disturbances, but these have generally been assessed under static conditions. The objective of this study was to use a “virtual mirror” that scaled visual movement feedback to assess body perception during active movement in military personnel with CNSLBP (n = 15) as compared to military healthy control subjects (n = 15). Subjects performed a trunk flexion task while sitting and standing in front of a large screen displaying a full-body virtual mirror-image (avatar) in real-time. Avatar movements were scaled to appear greater, identical, or smaller than the subjects’ actual movements. A total of 126 trials with 11 different scaling factors were pseudo-randomized across 6 blocks. After each trial, subjects had to decide whether the avatar’s movements were “greater” or “smaller” than their own movements. Based on this two-alternative forced choice paradigm, a psychophysical curve was fitted to the data for each subject, and several metrics were derived from this curve. In addition, task adherence (kinematics) and virtual reality immersion were assessed. Groups displayed a similar ability to discriminate between different levels of movement scaling. Still, subjects with CNSLBP showed an abnormal performance and tended to overestimate their own movements (a right-shifted psychophysical curve). Subjects showed adequate task adherence, and on average virtual reality immersion was reported to be very good. In conclusion, these results extend previous work in patients with CNSLBP, and denote an important relationship between body perception, movement and pain. As such, the assessment of body perception during active movement can offer new avenues for understanding and managing body perception disturbances and abnormal movement patterns in patients with pain.     

Clinical Heterogeneity of Duchenne Muscular Dystrophy (DMD): Definition of Sub-Phenotypes and Predictive Criteria by Long-Term Follow-Up

Background

To explore clinical heterogeneity of Duchenne muscular dystrophy (DMD), viewed as a major obstacle to the interpretation of therapeutic trials

Methodology/Principal Findings

A retrospective single institution long-term follow-up study was carried out in DMD patients with both complete lack of muscle dystrophin and genotyping. An exploratory series (series 1) was used to assess phenotypic heterogeneity and to identify early criteria predicting future outcome; it included 75 consecutive steroid-free patients, longitudinally evaluated for motor, respiratory, cardiac and cognitive functions (median follow-up: 10.5 yrs). A validation series (series 2) was used to test robustness of the selected predictive criteria; it included 34 more routinely evaluated patients (age>12 yrs). Multivariate analysis of series 1 classified 70/75 patients into 4 clusters with distinctive intellectual and motor outcomes: A (early infantile DMD, 20%): severe intellectual and motor outcomes; B (classical DMD, 28%): intermediate intellectual and poor motor outcome; C (moderate pure motor DMD, 22%): normal intelligence and delayed motor impairment; and D (severe pure motor DMD, 30%): normal intelligence and poor motor outcome. Group A patients had the most severe respiratory and cardiac involvement. Frequency of mutations upstream to exon 30 increased from group A to D, but genotype/phenotype correlations were restricted to cognition (IQ>71: OR 7.7, 95%CI 1.6–20.4, p<0.003). Diagnostic accuracy tests showed that combination of “clinical onset <2 yrs” with “mental retardation” reliably assigned patients to group A (sensitivity 0.93, specificity 0.98). Combination of “lower limb MMT score>6 at 8 yrs” with “normal or borderline mental status” reliably assigned patients to group C (sensitivity: 1, specificity: 0.94). These criteria were also predictive of “early infantile DMD” and “moderate pure motor DMD” in series 2.

Conclusions/Significance

DMD can be divided into 4 sub-phenotypes differing by severity of muscle and brain dysfunction. Simple early criteria can be used to include patients with similar outcomes in future therapeutic trials.     

Temperament and Impulsivity Predictors of Smoking Cessation Outcomes

Aims

Temperament and impulsivity are powerful predictors of addiction treatment outcomes. However, a comprehensive assessment of these features has not been examined in relation to smoking cessation outcomes.

Methods

Naturalistic prospective study. Treatment-seeking smokers (n = 140) were recruited as they engaged in an occupational health clinic providing smoking cessation treatment between 2009 and 2013. Participants were assessed at baseline with measures of temperament (Temperament and Character Inventory), trait impulsivity (Barratt Impulsivity Scale), and cognitive impulsivity (Go/No Go, Delay Discounting and Iowa Gambling Task). The outcome measure was treatment status, coded as “dropout” versus “relapse” versus “abstinence” at 3, 6, and 12 months endpoints. Participants were telephonically contacted and reminded of follow-up face to face assessments at each endpoint. The participants that failed to answer the phone calls or self-reported discontinuation of treatment and failed to attend the upcoming follow-up session were coded as dropouts. The participants that self-reported continuing treatment, and successfully attended the upcoming follow-up session were coded as either “relapse” or “abstinence”, based on the results of smoking behavior self-reports cross-validated with co-oximetry hemoglobin levels. Multinomial regression models were conducted to test whether temperament and impulsivity measures predicted dropout and relapse relative to abstinence outcomes.

Results

Higher scores on temperament dimensions of novelty seeking and reward dependence predicted poorer retention across endpoints, whereas only higher scores on persistence predicted greater relapse. Higher scores on the trait dimension of non-planning impulsivity but not performance on cognitive impulsivity predicted poorer retention. Higher non-planning impulsivity and poorer performance in the Iowa Gambling Task predicted greater relapse at 3 and 6 months and 6 months respectively.

Conclusion

Temperament measures, and specifically novelty seeking and reward dependence, predict smoking cessation treatment retention, whereas persistence, non-planning impulsivity and poor decision-making predict smoking relapse.     

Money Walks: Implicit Mobility Behavior and Financial Well-Being

Traditional financial decision systems (e.g. credit) had to rely on explicit individual traits like age, gender, job type, and marital status, while being oblivious to spatio-temporal mobility or the habits of the individual involved. Emerging trends in geo-aware and mobile payment systems, and the resulting “big data,” present an opportunity to study human consumption patterns across space and time. Taking inspiration from animal behavior studies that have reported significant interconnections between animal spatio-temporal “foraging” behavior and their life outcomes, we analyzed a corpus of hundreds of thousands of human economic transactions and found that financial outcomes for individuals are intricately linked with their spatio-temporal traits like exploration, engagement, and elasticity. Such features yield models that are 30% to 49% better at predicting future financial difficulties than the comparable demographic models.     

Cooperating elephants mitigate competition until the stakes get too high

Cooperation is ubiquitous in the animal kingdom as it aims to maximize benefits through joint action. Selection, however, may also favor competitive behaviors that could violate cooperation. How animals mitigate competition is hotly debated, with particular interest in primates and little attention paid thus far to nonprimates. Using a loose-string pulling apparatus, we explored cooperative and competitive behavior, as well as mitigation of the latter, in semi-wild Asian elephants (Elephas maximus). Our results showed that elephants first maintained a very high cooperation rate (average = 80.8% across 45 sessions). Elephants applied “block,” “fight back,” “leave,” “move side,” and “submission” as mitigation strategies and adjusted these strategies according to their affiliation and rank difference with competition initiators. They usually applied a “fight back” mitigation strategy as a sanction when competition initiators were low ranking or when they had a close affiliation, but were submissive if the initiators were high ranking or when they were not closely affiliated. However, when the food reward was limited, the costly competitive behaviors (“monopoly” and “fight”) increased significantly, leading to a rapid breakdown in cooperation. The instability of elephant cooperation as a result of benefit reduction mirrors that of human society, suggesting that similar fundamental principles may underlie the evolution of cooperation across species.

This study shows that in a task requiring coordinated pulling, elephants compete for access to food but work to mitigate competition in order to maintain cooperation. If the cost of competition becomes too high, however, cooperation breaks down entirely. This behavior mirrors that seen in humans and other great apes, suggesting that certain cooperative mechanisms are not unique to primates.     

Joint Cognition: Thought Contagion and the Consequences of Cooperation when Sharing the Task of Random Sequence Generation

Generating random number sequences is a popular psychological task often used to measure executive functioning. We explore random generation under “joint cognition” instructions; pairs of participants take turns to compile a shared response sequence. Across three studies, we point to six key findings from this novel format. First, there are both costs and benefits from group performance. Second, repetition avoidance occurs in dyadic as well as individual production settings. Third, individuals modify their choices in a dyadic situation such that the pair becomes the unit of psychological function. Fourth, there is immediate contagion of sequence stereotypy amongst the pairs (i.e., each contributor “owns” their partner’s response). Fifth, dyad effects occur even when participants know their partner is not interacting with them (Experiment 2). Sixth, ironically, directing participants’ efforts away from their shared task responsibility can actually benefit conjoint performance (Experiment 3). These results both constrain models of random generation and illuminate processes of joint cognition.     

Embodied Emotion Modulates Neural Signature of Performance Monitoring

Background

Recent research on the “embodiment of emotion” implies that experiencing an emotion may involve perceptual, somatovisceral, and motor feedback aspects. For example, manipulations of facial expression and posture appear to induce emotional states and influence how affective information is processed. The present study investigates whether performance monitoring, a cognitive process known to be under heavy control of the dopaminergic system, is modulated by induced facial expressions. In particular, we focused on the error-related negativity, an electrophysiological correlate of performance monitoring.

Methods/Principal Findings

During a choice reaction task, participants held a Chinese chop stick either horizontally between the teeth (“smile” condition) or, in different runs, vertically (“no smile”) with the upper lip. In a third control condition, no chop stick was used (“no stick”). It could be shown on a separate sample that the facial feedback procedure is feasible to induce mild changes in positive affect. In the ERP sample, the smile condition, hypothesized to lead to an increase in dopaminergic activity, was associated with a decrease of ERN amplitude relative to “no smile” and “no stick” conditions.

Conclusion

Embodying emotions by induced facial expressions leads to a changes in the neural correlates of error detection. We suggest that this is due to the joint influence of the dopaminergic system on positive affect and performance monitoring.     

Network Effects of Risk Behavior Change Following Prophylactic Interventions

We formulated a network-based model to understand how risk behavior change in conjunction with failure of prophylactic interventions can lead to unintended outcomes where “less (intervention) is more (effective).” Our model captures the distinction between one- and two-sided risk behavior change. In one-sided situations (e.g. influenza/H1N1) it is sufficient for either individual in an interaction to exhibit risk behavior change whereas in two-sided situations (e.g. AIDS/HIV) it is necessary for both individuals in the interaction to exhibit risk behavior change, for a potential transmission of the disease. A central discovery is that this phenomenon occurs at differing levels of intervention coverage depending upon the “sidedness” of the interaction. We find that for one-sided interactions, sufficiently high vaccination coverage is necessary for mitigating the effects of risk behavior; for two-sided interactions, it is essential to combine prophylactic treatments with programs aimed at reducing risky behavior. Furthermore, again dependent on the “sidedness,” targeting highly connected nodes can be strictly worse than uniformly random interventions at the same level of coverage.     

Monitoring DNA Contamination in Handled vs. Directly Excavated Ancient Human Skeletal Remains

Bones, teeth and hair are often the only physical evidence of human or animal presence at an archaeological site; they are also the most widely used sources of samples for ancient DNA (aDNA) analysis. Unfortunately, the DNA extracted from ancient samples, already scarce and highly degraded, is widely susceptible to exogenous contaminations that can affect the reliability of aDNA studies. We evaluated the molecular effects of sample handling on five human skeletons freshly excavated from a cemetery dated between the 11 to the 14^th century. We collected specimens from several skeletal areas (teeth, ribs, femurs and ulnas) from each individual burial. We then divided the samples into two different sets: one labeled as “virgin samples” (i.e. samples that were taken by archaeologists under contamination-controlled conditions and then immediately sent to the laboratory for genetic analyses), and the second called “lab samples”(i.e. samples that were handled without any particular precautions and subject to normal washing, handling and measuring procedures in the osteological lab). Our results show that genetic profiles from “lab samples” are incomplete or ambiguous in the different skeletal areas while a different outcome is observed in the “virgin samples” set. Generally, all specimens from different skeletal areas in the exception of teeth present incongruent results between “lab” and “virgin” samples. Therefore teeth are less prone to contamination than the other skeletal areas we analyzed and may be considered a material of choice for classical aDNA studies. In addition, we showed that bones can also be a good candidate for human aDNA analysis if they come directly from the excavation site and are accompanied by a clear taphonomic history.     

Cognitive Demands of Lower Paleolithic Toolmaking

Stone tools provide some of the most abundant, continuous, and high resolution evidence of behavioral change over human evolution, but their implications for cognitive evolution have remained unclear. We investigated the neurophysiological demands of stone toolmaking by training modern subjects in known Paleolithic methods (“Oldowan”, “Acheulean”) and collecting structural and functional brain imaging data as they made technical judgments (outcome prediction, strategic appropriateness) about planned actions on partially completed tools. Results show that this task affected neural activity and functional connectivity in dorsal prefrontal cortex, that effect magnitude correlated with the frequency of correct strategic judgments, and that the frequency of correct strategic judgments was predictive of success in Acheulean, but not Oldowan, toolmaking. This corroborates hypothesized cognitive control demands of Acheulean toolmaking, specifically including information monitoring and manipulation functions attributed to the "central executive" of working memory. More broadly, it develops empirical methods for assessing the differential cognitive demands of Paleolithic technologies, and expands the scope of evolutionary hypotheses that can be tested using the available archaeological record.