The function and organization of lateral prefrontal cortex: a test of competing hypotheses

The present experiment tested three hypotheses regarding the function and organization of lateral prefrontal cortex (PFC). The first account (the information cascade hypothesis) suggests that the anterior-posterior organization of lateral PFC is based on the timing with which cue stimuli reduce uncertainty in the action selection process. The second account (the levels-of-abstraction hypothesis) suggests that the anterior-posterior organization of lateral PFC is based on the degree of abstraction of the task goals. The current study began by investigating these two hypotheses, and identified several areas of lateral PFC that were predicted to be active by both the information cascade and levels-of-abstraction accounts. However, the pattern of activation across experimental conditions was inconsistent with both theoretical accounts. Specifically, an anterior area of mid-dorsolateral PFC exhibited sensitivity to experimental conditions that, according to both accounts, should have selectively engaged only posterior areas of PFC. We therefore investigated a third possible account (the adaptive context maintenance hypothesis) that postulates that both posterior and anterior regions of PFC are reliably engaged in task conditions requiring active maintenance of contextual information, with the temporal dynamics of activity in these regions flexibly tracking the duration of maintenance demands. Activity patterns in lateral PFC were consistent with this third hypothesis: regions across lateral PFC exhibited transient activation when contextual information had to be updated and maintained in a trial-by-trial manner, but sustained activation when contextual information had to be maintained over a series of trials. These findings prompt a reconceptualization of current views regarding the anterior-posterior organization of lateral PFC, but do support other findings regarding the active maintenance role of lateral PFC in sequential working memory paradigms.     

Rapid Plasticity in the Prefrontal Cortex during Affective Associative Learning

MultiCS conditioning is an affective associative learning paradigm, in which affective categories consist of many similar and complex stimuli. Comparing visual processing before and after learning, recent MultiCS conditioning studies using time-sensitive magnetoencephalography (MEG) revealed enhanced activation of prefrontal cortex (PFC) regions towards emotionally paired versus neutral stimuli already during short-latency processing stages (i.e., 50 to 80 ms after stimulus onset). The present study aimed at showing that this rapid differential activation develops as a function of the acquisition and not the extinction of the emotional meaning associated with affectively paired stimuli. MEG data of a MultiCS conditioning study were analyzed with respect to rapid changes in PFC activation towards aversively (electric shock) paired and unpaired faces that occurred during the learning of stimulus-reinforcer contingencies. Analyses revealed an increased PFC activation towards paired stimuli during 50 to 80 ms already during the acquisition of contingencies, which emerged after a single pairing with the electric shock. Corresponding changes in stimulus valence could be observed in ratings of hedonic valence, although participants did not seem to be aware of contingencies. These results suggest rapid formation and access of emotional stimulus meaning in the PFC as well as a great capacity for adaptive and highly resolving learning in the brain under challenging circumstances.     

A rostro-caudal gradient of structured sequence processing in the left inferior frontal gyrus

In this paper, we present two novel perspectives on the function of the left inferior frontal gyrus (LIFG). First, a structured sequence processing perspective facilitates the search for functional segregation within the LIFG and provides a way to express common aspects across cognitive domains including language, music and action. Converging evidence from functional magnetic resonance imaging and transcranial magnetic stimulation studies suggests that the LIFG is engaged in sequential processing in artificial grammar learning, independently of particular stimulus features of the elements (whether letters, syllables or shapes are used to build up sequences). The LIFG has been repeatedly linked to processing of artificial grammars across all different grammars tested, whether they include non-adjacent dependencies or mere adjacent dependencies. Second, we apply the sequence processing perspective to understand how the functional segregation of semantics, syntax and phonology in the LIFG can be integrated in the general organization of the lateral prefrontal cortex (PFC). Recently, it was proposed that the functional organization of the lateral PFC follows a rostro-caudal gradient, such that more abstract processing in cognitive control is subserved by more rostral regions of the lateral PFC. We explore the literature from the viewpoint that functional segregation within the LIFG can be embedded in a general rostro-caudal abstraction gradient in the lateral PFC. If the lateral PFC follows a rostro-caudal abstraction gradient, then this predicts that the LIFG follows the same principles, but this prediction has not yet been tested or explored in the LIFG literature. Integration might provide further insights into the functional architecture of the LIFG and the lateral PFC.     

Interference in ballistic motor learning: specificity and role of sensory error signals

Humans are capable of learning numerous motor skills, but newly acquired skills may be abolished by subsequent learning. Here we ask what factors determine whether interference occurs in motor learning. We speculated that interference requires competing processes of synaptic plasticity in overlapping circuits and predicted specificity. To test this, subjects learned a ballistic motor task. Interference was observed following subsequent learning of an accuracy-tracking task, but only if the competing task involved the same muscles and movement direction. Interference was not observed from a non-learning task suggesting that interference requires competing learning. Subsequent learning of the competing task 4 h after initial learning did not cause interference suggesting disruption of early motor memory consolidation as one possible mechanism underlying interference. Repeated transcranial magnetic stimulation (rTMS) of corticospinal motor output at intensities below movement threshold did not cause interference, whereas suprathreshold rTMS evoking motor responses and (re)afferent activation did. Finally, the experiments revealed that suprathreshold repetitive electrical stimulation of the agonist (but not antagonist) peripheral nerve caused interference. The present study is, to our knowledge, the first to demonstrate that peripheral nerve stimulation may cause interference. The finding underscores the importance of sensory feedback as error signals in motor learning. We conclude that interference requires competing plasticity in overlapping circuits. Interference is remarkably specific for circuits involved in a specific movement and it may relate to sensory error signals.     

Sequence learning recodes cortical representations instead of strengthening initial ones

We contrast two computational models of sequence learning. The associative learner posits that learning proceeds by strengthening existing association weights. Alternatively, recoding posits that learning creates new and more efficient representations of the learned sequences. Importantly, both models propose that humans act as optimal learners but capture different statistics of the stimuli in their internal model. Furthermore, these models make dissociable predictions as to how learning changes the neural representation of sequences. We tested these predictions by using fMRI to extract neural activity patterns from the dorsal visual processing stream during a sequence recall task. We observed that only the recoding account can explain the similarity of neural activity patterns, suggesting that participants recode the learned sequences using chunks. We show that associative learning can theoretically store only very limited number of overlapping sequences, such as common in ecological working memory tasks, and hence an efficient learner should recode initial sequence representations.     

Lateral interactions among phosphatidylcholine and phosphatidylethanolamine head groups in phospholipid monolayers and bilayers

We develop theory for the lateral interactions among the zwitterionic head groups of phospholipids in monolayers and bilayers, particularly phosphatidylcholine (PC) and phosphatidylethanolamine (PE). With the P- end of the head group anchored at the water/hydrocarbon interface, a balance of two effects dictates the angle that the P--N+ dipole makes with respect to the plane of the bilayer: N+ is driven toward water due to the (Born) electrostatic free energy, but the hydrophobic effect drives the methyl and methylene groups around the N+ charge toward the hydrocarbon. The only adjustable parameter of the model is the average fluctuation of the oil/water interface or, alternatively, the dielectric constant of the hydrocarbon phase. The model predicts that at 5 degrees C the head group dipole should lie largely in the bilayer plane, in accord with X-ray, neutron diffraction, and NMR studies. The theory makes the novel prediction that the N+ end of the dipole becomes increasingly submerged in hydrocarbon with increasing temperature, leading to strongly enhanced lateral repulsion between PC head groups. This prediction is in good agreement with second and third viral coefficients of monolayer lateral pressures, and with the temperature dependence of the former. The theoretical model is consistent with head group fluctuations measured by neutron diffraction of PC and PE bilayers. Because PE has a smaller hydrophobic cluster near N+, its lateral repulsion should be much smaller and less temperature dependent than for PC, also in agreement with equation-of-state measurements. This suggests why at high density PE monolayers have higher melting temperatures than PC monolayers and more propensity for reversed curvature.     

Phospholipid interactions in model membrane systems. II. Theory.

We describe statistical thermodynamic theory for the lateral interactions among phospholipid head groups in monolayers and bilayers. Extensive monolayer experiments show that at low surface densities, PC head groups have strong lateral repulsions which increase considerably with temperature, whereas PE interactions are much weaker and have no significant temperature dependence (see the preceding paper). In previous work, we showed that the second virial coefficients for these interactions can be explained by: (a) steric repulsions among the head groups, and (b) a tilting of the P-N+ dipole of PC so that the N+ end enters the oil phase, to an extent that increases with temperature. It was also predicted that PE interactions should be weaker and less temperature dependent because the N+ terminal of the PE head-group is hydrophilic, hence, it is tilted into the water phase, so dipolar contributions among PE's are negligible due to the high dielectric constant of water. In the present work, we broaden the theory to treat phospholipid interactions up to higher lateral surface densities. We generalize the Hill interfacial virial expansion to account for dipoles and to include the third virial term. We show that to account for the large third virial coefficients for both PC and PE requires that the short range lateral attractions among the head groups also be taken into account. In addition, the third virial coefficient includes fluctuating head group dipoles, computed by Monte Carlo integration assuming pairwise additivity of the instantaneous pair potentials. We find that because the dipole fluctuations are correlated, the average triplet interactions do not equal the sum of the average dipole pair potentials. This is important for predicting, the magnitude and the independence of temperature of the third virial coefficients for PC. The consistency of the theory with data of both the second and the third virial coefficients extends the applicability of the head-group model to semiconcentrated monolayers, in agreement with the surface potential data in the foregoing paper.     

Uncertainty–guided learning with scaled prediction errors in the basal ganglia

To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We show that the new model has an advantage over conventional reinforcement learning models in a value tracking task, and approaches a theoretic limit of performance provided by the Kalman filter. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. In the proposed network, dopaminergic neurons encode reward prediction errors scaled by standard deviation of rewards. We show that such scaling may arise if the striatal neurons learn the standard deviation of rewards and modulate the activity of dopaminergic neurons. The model is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.     

Kamin Blocking Is Associated with Reduced Medial-Frontal Gyrus Activation: Implications for Prediction Error Abnormality in Schizophrenia

The following study used 3-T functional magnetic resonance imaging (fMRI) to investigate the neural signature of Kamin blocking. Kamin blocking is an associative learning phenomenon seen where prior association of a stimulus (A) with an outcome blocks subsequent learning to an added stimulus (B) when both stimuli are later presented together (AB) with the same outcome. While there are a number of theoretical explanations of Kamin blocking, it is widely considered to exemplify the use of prediction error in learning, where learning occurs in proportion to the difference between expectation and outcome. In Kamin blocking as stimulus A fully predicts the outcome no prediction error is generated by the addition of stimulus B to form the compound stimulus AB, hence learning about it is “blocked”. Kamin blocking is disrupted in people with schizophrenia, their relatives and healthy individuals with high psychometrically-defined schizotypy. This disruption supports suggestions that abnormal prediction error is a core deficit that can help to explain the symptoms of schizophrenia. The present study tested 9 healthy volunteers on an f-MRI adaptation of Oades'' “mouse in the house task”, the only task measuring Kamin blocking that shows disruption in schizophrenia patients that has been independently replicated. Participant''s Kamin blocking scores were found to inversely correlate with Kamin-blocking-related activation within the prefrontal cortex, specifically the medial frontal gyrus. The medial frontal gyrus has been associated with the psychological construct of uncertainty, which we suggest is consistent with disrupted Kamin blocking and demonstrated in people with schizophrenia. These data suggest that the medial frontal gyrus merits further investigation as a potential locus of reduced Kamin blocking and abnormal prediction error in schizophrenia.     

Interfering with theories of sleep and memory: sleep, declarative memory, and associative interference

Mounting behavioral evidence in humans supports the claim that sleep leads to improvements in recently acquired, nondeclarative memories. Examples include motor-sequence learning; visual-discrimination learning; and perceptual learning of a synthetic language. In contrast, there are limited human data supporting a benefit of sleep for declarative (hippocampus-mediated) memory in humans (for review, see). This is particularly surprising given that animal models (e.g.,) and neuroimaging studies (e.g.,) predict that sleep facilitates hippocampus-based memory consolidation. We hypothesized that we could unmask the benefits of sleep by challenging the declarative memory system with competing information (interference). This is the first study to demonstrate that sleep protects declarative memories from subsequent associative interference, and it has important implications for understanding the neurobiology of memory consolidation.     

Molecular mechanisms of fear learning and memory

Pavlovian fear conditioning is a particularly useful behavioral paradigm for exploring the molecular mechanisms of learning and memory because a well-defined response to a specific environmental stimulus is produced through associative learning processes. Synaptic plasticity in the lateral nucleus of the amygdala (LA) underlies this form of associative learning. Here, we summarize the molecular mechanisms that contribute to this synaptic plasticity in the context of auditory fear conditioning, the form of fear conditioning best understood at the molecular level. We discuss the neurotransmitter systems and signaling cascades that contribute to three phases of auditory fear conditioning: acquisition, consolidation, and reconsolidation. These studies suggest that multiple intracellular signaling pathways, including those triggered by activation of Hebbian processes and neuromodulatory receptors, interact to produce neural plasticity in the LA and behavioral fear conditioning. Collectively, this body of research illustrates the power of fear conditioning as a model system for characterizing the mechanisms of learning and memory in mammals and potentially for understanding fear-related disorders, such as PTSD and phobias.     

Suppressing Emotions Impairs Subsequent Stroop Performance and Reduces Prefrontal Brain Activation

Abundant behavioral evidence suggests that the ability to self-control is limited, and that any exertion of self-control will increase the likelihood of subsequent self-control failures. Here we investigated the neural correlates underlying the aftereffects of self-control on future control processes using functional magnetic resonance imaging (fMRI). An initial act of self-control (suppressing emotions) impaired subsequent performance in a second task requiring control (Stroop task). On the neural level, increased activity during emotion suppression was followed by a relative decrease in activity during the Stroop task in a cluster in the right lateral prefrontal cortex (PFC) including the dorsolateral prefrontal cortex (DLPFC), an area engaged in the effortful implementation of control. There was no reliable evidence for reduced activity in the medial frontal cortex (MFC) including the anterior cingulate cortex (ACC), which is involved in conflict detection processes and has previously also been implicated in self-control. Follow-up analyses showed that the detected cluster in the right lateral PFC and an area in the MFC were involved in both the emotion suppression task and the Stroop task, but only the cluster in the right lateral PFC showed reduced activation after emotion suppression during the Stroop task. Reduced activity in lateral prefrontal areas relevant for the implementation of control may be a critical consequence of prior self-control exertion if the respective areas are involved in both self-control tasks.     

Priming of associative long-term depression in the dentate gyrus by theta frequency synaptic activity.

Associative long-term synaptic depression (LTD) was investigated utilizing negatively correlated activity patterns in the medial and lateral perforant path inputs to the dentate gyrus in anesthetized rats. Normally only nonassociative, or heterosynaptic, LTD is elicited in naive pathways. We report here, however, that associative LTD in the lateral path is readily induced after being "primed" by a brief period of lateral path synaptic activity at a theta rhythm frequency (5 Hz). Priming of associative LTD lasts at least 2 hr and is not seen following priming activity at non-theta frequencies (1 and 15 Hz). N-methyl-D-aspartate receptor activation is critical for establishing the priming effect, but not for the subsequent induction of the associative LTD. These data suggest that theta rhythm activity in the dentate gyrus may predispose the system to a specific form of synaptic plasticity, associative LTD.     

Neural mechanisms of transient and sustained cognitive control during task switching

A hybrid blocked and event-related functional magnetic resonance imaging (fMRI) study decomposed brain activity during task switching into sustained and transient components. Contrasting task-switching blocks against single-task blocks revealed sustained activation in right anterior prefrontal cortex (PFC). Contrasting task-switch trials against task-repeat and single-task trials revealed activation in left lateral PFC and left superior parietal cortex. In both sets of regions, activation dynamics were strongly modulated by trial-by-trial fluctuations in response speed. In addition, right anterior PFC activity selectively covaried with the magnitude of mixing cost (i.e., task-repeat versus single-task trial performance), and left superior parietal activity selectively covaried with the magnitude of the switching cost (i.e., task-switch versus task-repeat trial performance). These results indicate a functional double dissociation in brain regions supporting different components of cognitive control during task switching and suggest that both sustained and transient control processes mediate the behavioral performance costs of task switching.     

Stimulus-dependent adjustment of reward prediction error in the midbrain

Previous reports have described that neural activities in midbrain dopamine areas are sensitive to unexpected reward delivery and omission. These activities are correlated with reward prediction error in reinforcement learning models, the difference between predicted reward values and the obtained reward outcome. These findings suggest that the reward prediction error signal in the brain updates reward prediction through stimulus-reward experiences. It remains unknown, however, how sensory processing of reward-predicting stimuli contributes to the computation of reward prediction error. To elucidate this issue, we examined the relation between stimulus discriminability of the reward-predicting stimuli and the reward prediction error signal in the brain using functional magnetic resonance imaging (fMRI). Before main experiments, subjects learned an association between the orientation of a perceptually salient (high-contrast) Gabor patch and a juice reward. The subjects were then presented with lower-contrast Gabor patch stimuli to predict a reward. We calculated the correlation between fMRI signals and reward prediction error in two reinforcement learning models: a model including the modulation of reward prediction by stimulus discriminability and a model excluding this modulation. Results showed that fMRI signals in the midbrain are more highly correlated with reward prediction error in the model that includes stimulus discriminability than in the model that excludes stimulus discriminability. No regions showed higher correlation with the model that excludes stimulus discriminability. Moreover, results show that the difference in correlation between the two models was significant from the first session of the experiment, suggesting that the reward computation in the midbrain was modulated based on stimulus discriminability before learning a new contingency between perceptually ambiguous stimuli and a reward. These results suggest that the human reward system can incorporate the level of the stimulus discriminability flexibly into reward computations by modulating previously acquired reward values for a typical stimulus.     

Implicit Value Updating Explains Transitive Inference Performance: The Betasort Model

Transitive inference (the ability to infer that B > D given that B > C and C > D) is a widespread characteristic of serial learning, observed in dozens of species. Despite these robust behavioral effects, reinforcement learning models reliant on reward prediction error or associative strength routinely fail to perform these inferences. We propose an algorithm called betasort, inspired by cognitive processes, which performs transitive inference at low computational cost. This is accomplished by (1) representing stimulus positions along a unit span using beta distributions, (2) treating positive and negative feedback asymmetrically, and (3) updating the position of every stimulus during every trial, whether that stimulus was visible or not. Performance was compared for rhesus macaques, humans, and the betasort algorithm, as well as Q-learning, an established reward-prediction error (RPE) model. Of these, only Q-learning failed to respond above chance during critical test trials. Betasort’s success (when compared to RPE models) and its computational efficiency (when compared to full Markov decision process implementations) suggests that the study of reinforcement learning in organisms will be best served by a feature-driven approach to comparing formal models.     

Accounting for Dynamic Fluctuations across Time when Examining fMRI Test-Retest Reliability: Analysis of a Reward Paradigm in the EMBARC Study

Longitudinal investigation of the neural correlates of reward processing in depression may represent an important step in defining effective biomarkers for antidepressant treatment outcome prediction, but the reliability of reward-related activation is not well understood. Thirty-seven healthy control participants were scanned using fMRI while performing a reward-related guessing task on two occasions, approximately one week apart. Two main contrasts were examined: right ventral striatum (VS) activation fMRI BOLD signal related to signed prediction errors (PE) and reward expectancy (RE). We also examined bilateral visual cortex activation coupled to outcome anticipation. Significant VS PE-related activity was observed at the first testing session, but at the second testing session, VS PE-related activation was significantly reduced. Conversely, significant VS RE-related activity was observed at time 2 but not time 1. Increases in VS RE-related activity from time 1 to time 2 were significantly associated with decreases in VS PE-related activity from time 1 to time 2 across participants. Intraclass correlations (ICCs) in VS were very low. By contrast, visual cortex activation had much larger ICCs, particularly in individuals with high quality data. Dynamic changes in brain activation are widely predicted, and failure to account for these changes could lead to inaccurate evaluations of the reliability of functional MRI signals. Conventional measures of reliability cannot distinguish between changes specified by algorithmic models of neural function and noisy signal. Here, we provide evidence for the former possibility: reward-related VS activations follow the pattern predicted by temporal difference models of reward learning but have low ICCs.     

The role of prefrontal dopamine D1 receptors in the neural mechanisms of associative learning

Dopamine is thought to play a major role in learning. However, while dopamine D1 receptors (D1Rs) in the prefrontal cortex (PFC) have been shown to modulate working memory-related neural activity, their role in the cellular basis of learning is unknown. We recorded activity from multiple electrodes while injecting the D1R antagonist SCH23390 in the lateral PFC as monkeys learned visuomotor associations. Blocking D1Rs impaired learning of novel associations and decreased cognitive flexibility but spared performance of already familiar associations. This suggests a greater role for prefrontal D1Rs in learning new, rather than performing familiar, associations. There was a corresponding greater decrease in neural selectivity and increase in alpha and beta oscillations in local field potentials for novel than for familiar associations. Our results suggest that weak stimulation of D1Rs observed in aging and psychiatric disorders may impair learning and PFC function by reducing neural selectivity and exacerbating neural oscillations associated with inattention and cognitive deficits.     

Levels of integration in cognitive control and sequence processing in the prefrontal cortex

Cognitive control is necessary to flexibly act in changing environments. Sequence processing is needed in language comprehension to build the syntactic structure in sentences. Functional imaging studies suggest that sequence processing engages the left ventrolateral prefrontal cortex (PFC). In contrast, cognitive control processes additionally recruit bilateral rostral lateral PFC regions. The present study aimed to investigate these two types of processes in one experimental paradigm. Sequence processing was manipulated using two different sequencing rules varying in complexity. Cognitive control was varied with different cue-sets that determined the choice of a sequencing rule. Univariate analyses revealed distinct PFC regions for the two types of processing (i.e. sequence processing: left ventrolateral PFC and cognitive control processing: bilateral dorsolateral and rostral PFC). Moreover, in a common brain network (including left lateral PFC and intraparietal sulcus) no interaction between sequence and cognitive control processing was observed. In contrast, a multivariate pattern analysis revealed an interaction of sequence and cognitive control processing, such that voxels in left lateral PFC and parietal cortex showed different tuning functions for tasks involving different sequencing and cognitive control demands. These results suggest that the difference between the process of rule selection (i.e. cognitive control) and the process of rule-based sequencing (i.e. sequence processing) find their neuronal underpinnings in distinct activation patterns in lateral PFC. Moreover, the combination of rule selection and rule sequencing can shape the response of neurons in lateral PFC and parietal cortex.