Practicing coarse orientation discrimination improves orientation signals in macaque cortical area v4

Practice improves the performance in visual tasks, but mechanisms underlying this adult brain plasticity are unclear. Single-cell studies reported no [1], weak [2], or moderate [3, 4] perceptual learning-related changes in macaque visual areas V1 and V4, whereas none were found in middle temporal (MT) [5]. These conflicting results and modeling of human (e.g., [6, 7]) and monkey data [8] suggested that changes in the readout of visual cortical signals underlie perceptual learning, rather than changes in these signals. In the V4 learning studies, monkeys discriminated small differences in orientation, whereas in the MT study, the animals discriminated opponent motion directions. Analogous to the latter study, we trained monkeys to discriminate static orthogonal orientations masked by noise. V4 neurons showed robust increases in their capacity to discriminate the trained orientations during the course of the training. This effect was observed during discrimination and passive fixation but specifically for the trained orientations. The improvement in neural discrimination was due to decreased response variability and an increase of the difference between the mean responses for the two trained orientations. These findings demonstrate that perceptual learning in a coarse discrimination task indeed can change the response properties of a cortical sensory area.     

Is Model Fitting Necessary for Model-Based fMRI?

Model-based analysis of fMRI data is an important tool for investigating the computational role of different brain regions. With this method, theoretical models of behavior can be leveraged to find the brain structures underlying variables from specific algorithms, such as prediction errors in reinforcement learning. One potential weakness with this approach is that models often have free parameters and thus the results of the analysis may depend on how these free parameters are set. In this work we asked whether this hypothetical weakness is a problem in practice. We first developed general closed-form expressions for the relationship between results of fMRI analyses using different regressors, e.g., one corresponding to the true process underlying the measured data and one a model-derived approximation of the true generative regressor. Then, as a specific test case, we examined the sensitivity of model-based fMRI to the learning rate parameter in reinforcement learning, both in theory and in two previously-published datasets. We found that even gross errors in the learning rate lead to only minute changes in the neural results. Our findings thus suggest that precise model fitting is not always necessary for model-based fMRI. They also highlight the difficulty in using fMRI data for arbitrating between different models or model parameters. While these specific results pertain only to the effect of learning rate in simple reinforcement learning models, we provide a template for testing for effects of different parameters in other models.     

Perceptual learning and decision-making in human medial frontal cortex

The dominant view that perceptual learning is accompanied by changes in early sensory representations has recently been challenged. Here we tested the idea that perceptual learning can be accounted for by reinforcement learning involving changes in higher decision-making areas. We trained subjects on an orientation discrimination task involving feedback over 4 days, acquiring fMRI data on the first and last day. Behavioral improvements were well explained by a reinforcement learning model in which learning leads to enhanced readout of sensory information, thereby establishing noise-robust representations of decision variables. We find stimulus orientation encoded in early visual and higher cortical regions such as lateral parietal cortex and anterior cingulate cortex (ACC). However, only activity patterns in the ACC tracked changes in decision variables during learning. These results provide strong evidence for perceptual learning-related changes in higher order areas and suggest that perceptual and reward learning are based on a common neurobiological mechanism.     

Interplay between Short- and Long-Term Plasticity in Cell-Assembly Formation

Various hippocampal and neocortical synapses of mammalian brain show both short-term plasticity and long-term plasticity, which are considered to underlie learning and memory by the brain. According to Hebb’s postulate, synaptic plasticity encodes memory traces of past experiences into cell assemblies in cortical circuits. However, it remains unclear how the various forms of long-term and short-term synaptic plasticity cooperatively create and reorganize such cell assemblies. Here, we investigate the mechanism in which the three forms of synaptic plasticity known in cortical circuits, i.e., spike-timing-dependent plasticity (STDP), short-term depression (STD) and homeostatic plasticity, cooperatively generate, retain and reorganize cell assemblies in a recurrent neuronal network model. We show that multiple cell assemblies generated by external stimuli can survive noisy spontaneous network activity for an adequate range of the strength of STD. Furthermore, our model predicts that a symmetric temporal window of STDP, such as observed in dopaminergic modulations on hippocampal neurons, is crucial for the retention and integration of multiple cell assemblies. These results may have implications for the understanding of cortical memory processes.     

Striatal and Hippocampal Involvement in Motor Sequence Chunking Depends on the Learning Strategy

Motor sequences can be learned using an incremental approach by starting with a few elements and then adding more as training evolves (e.g., learning a piano piece); conversely, one can use a global approach and practice the whole sequence in every training session (e.g., shifting gears in an automobile). Yet, the neural correlates associated with such learning strategies in motor sequence learning remain largely unexplored to date. Here we used functional magnetic resonance imaging to measure the cerebral activity of individuals executing the same 8-element sequence after they completed a 4-days training regimen (2 sessions each day) following either a global or incremental strategy. A network comprised of striatal and fronto-parietal regions was engaged significantly regardless of the learning strategy, whereas the global training regimen led to additional cerebellar and temporal lobe recruitment. Analysis of chunking/grouping of sequence elements revealed a common prefrontal network in both conditions during the chunk initiation phase, whereas execution of chunk cores led to higher mediotemporal activity (involving the hippocampus) after global than incremental training. The novelty of our results relate to the recruitment of mediotemporal regions conditional of the learning strategy. Thus, the present findings may have clinical implications suggesting that the ability of patients with lesions to the medial temporal lobe to learn and consolidate new motor sequences may benefit from using an incremental strategy.     

Structural and Functional Changes in Subcortical Vascular Mild Cognitive Impairment: A Combined Voxel-Based Morphometry and Resting-State fMRI Study

The present study aimed to investigate changes in structural gray matter (GM) volume and functional amplitude of spontaneous low-frequency oscillations (LFO) and functional connectivity density in patients with subcortical vascular mild cognitive impairment (svMCI). Structural MRI and resting-sate functional MRI data were collected from 26 svMCI patients and 28 age- and gender-matched healthy controls. Structurally, widespread GM atrophy was found in the svMCI patients that resided primarily in frontal (e.g., the superior and middle frontal gyri and medial prefrontal cortex) and temporal (the superior and inferior temporal gyri) brain regions as well as several subcortical brain sites (e.g., the thalamus and the caudate). Functionally, svMCI-related changes were predominantly found in the default mode network (DMN). Compared with the healthy controls, the svMCI patients exhibited decreased LFO amplitudes in the anterior part of the DMN (e.g., the medial prefrontal cortex), whereas increased LFO amplitudes in the posterior part of the DMN (e.g., the posterior cingulate/precuneus). As for functional connectivity density, the DMN regions (e.g., the posterior cingulate/precuneus, the medial prefrontal cortex and the middle temporal gyrus) consistently exhibited decreased functional connectivity. Finally, the overall patterns of functional alterations in LFO amplitudes and functional connectivity density remained little changed after controlling for structural GM volume losses, which suggests that functional abnormalities can be only partly explained by morphological GM volume changes. Together, our results indicate that svMCI patients exhibit widespread abnormalities in both structural GM volume and functional intrinsic brain activity, which have important implications in understanding the pathophysiological mechanism of svMCI.     

Coexistence of Reward and Unsupervised Learning During the Operant Conditioning of Neural Firing Rates

A fundamental goal of neuroscience is to understand how cognitive processes, such as operant conditioning, are performed by the brain. Typical and well studied examples of operant conditioning, in which the firing rates of individual cortical neurons in monkeys are increased using rewards, provide an opportunity for insight into this. Studies of reward-modulated spike-timing-dependent plasticity (RSTDP), and of other models such as R-max, have reproduced this learning behavior, but they have assumed that no unsupervised learning is present (i.e., no learning occurs without, or independent of, rewards). We show that these models cannot elicit firing rate reinforcement while exhibiting both reward learning and ongoing, stable unsupervised learning. To fix this issue, we propose a new RSTDP model of synaptic plasticity based upon the observed effects that dopamine has on long-term potentiation and depression (LTP and LTD). We show, both analytically and through simulations, that our new model can exhibit unsupervised learning and lead to firing rate reinforcement. This requires that the strengthening of LTP by the reward signal is greater than the strengthening of LTD and that the reinforced neuron exhibits irregular firing. We show the robustness of our findings to spike-timing correlations, to the synaptic weight dependence that is assumed, and to changes in the mean reward. We also consider our model in the differential reinforcement of two nearby neurons. Our model aligns more strongly with experimental studies than previous models and makes testable predictions for future experiments.     

Thinking Outside the Euclidean Box: Riemannian Geometry and Inter-Temporal Decision-Making

Inter-temporal decisions involves assigning values to various payoffs occurring at different temporal distances. Past research has used different approaches to study these decisions made by humans and animals. For instance, considering that people discount future payoffs at a constant rate (e.g., exponential discounting) or at variable rate (e.g., hyperbolic discounting). In this research, we question the widely assumed, but seldom questioned, notion across many of the existing approaches that the decision space, where the decision-maker perceives time and monetary payoffs, is a Euclidean space. By relaxing the rigid assumption of Euclidean space, we propose that the decision space is a more flexible Riemannian space of Constant Negative Curvature. We test our proposal by deriving a discount function, which uses the distance in the Negative Curvature space instead of Euclidean temporal distance. The distance function includes both perceived values of time as well as money, unlike past work which has considered just time. By doing so we are able to explain many of the empirical findings in inter-temporal decision-making literature. We provide converging evidence for our proposal by estimating the curvature of the decision space utilizing manifold learning algorithm and showing that the characteristics (i.e., metric properties) of the decision space resembles those of the Negative Curvature space rather than the Euclidean space. We conclude by presenting new theoretical predictions derived from our proposal and implications for how non-normative behavior is defined.     

Double dissociation of PKC and AC manipulations on operant and classical learning in Drosophila

Learning about relationships between stimuli (i.e., classical conditioning [1]) and learning about consequences of one's own behavior (i.e., operant conditioning [2]) constitute the major part of our predictive understanding of the world. Since these forms of learning were recognized as two separate types 80 years ago [3], a recurrent concern has been the issue of whether one biological process can account for both of them [4, 5, 6, 7, 8, 9]. Today, we know the anatomical structures required for successful learning in several different paradigms, e.g., operant and classical processes can be localized to different brain regions in rodents [9] and an identified neuron in Aplysia shows opposite biophysical changes after operant and classical training, respectively [5]. We also know to some detail the molecular mechanisms underlying some forms of learning and memory consolidation. However, it is not known whether operant and classical learning can be distinguished at the molecular level. Therefore, we investigated whether genetic manipulations could differentiate between operant and classical learning in Drosophila. We found a double dissociation of protein kinase C and adenylyl cyclase on operant and classical learning. Moreover, the two learning systems interacted hierarchically such that classical predictors were learned preferentially over operant predictors.     

Learning Multisensory Integration and Coordinate Transformation via Density Estimation

Sensory processing in the brain includes three key operations: multisensory integration—the task of combining cues into a single estimate of a common underlying stimulus; coordinate transformations—the change of reference frame for a stimulus (e.g., retinotopic to body-centered) effected through knowledge about an intervening variable (e.g., gaze position); and the incorporation of prior information. Statistically optimal sensory processing requires that each of these operations maintains the correct posterior distribution over the stimulus. Elements of this optimality have been demonstrated in many behavioral contexts in humans and other animals, suggesting that the neural computations are indeed optimal. That the relationships between sensory modalities are complex and plastic further suggests that these computations are learned—but how? We provide a principled answer, by treating the acquisition of these mappings as a case of density estimation, a well-studied problem in machine learning and statistics, in which the distribution of observed data is modeled in terms of a set of fixed parameters and a set of latent variables. In our case, the observed data are unisensory-population activities, the fixed parameters are synaptic connections, and the latent variables are multisensory-population activities. In particular, we train a restricted Boltzmann machine with the biologically plausible contrastive-divergence rule to learn a range of neural computations not previously demonstrated under a single approach: optimal integration; encoding of priors; hierarchical integration of cues; learning when not to integrate; and coordinate transformation. The model makes testable predictions about the nature of multisensory representations.     

The human nucleus accumbens is highly susceptible to G protein down-regulation by methamphetamine and heroin

Although the nucleus accumbens is assumed to be a critical brain "pleasure center," its function in humans is unknown. As animal data suggest that a unique feature of this small brain area is its high sensitivity to down-regulation of an inhibitory G protein by drugs of abuse, we compared G protein levels in postmortem nucleus accumbens with those in seven other brain regions of chronic users of cocaine, methamphetamine, and heroin, and of matched controls. Biochemical changes were restricted to the nucleus accumbens in which concentrations of G(alpha)1 and/or G(alpha)2 were reduced by 32-49% in the methamphetamine and heroin users. This selective responsiveness to these abused drugs implies a special role for the human nucleus accumbens in mechanisms of drug reinforcement and suggests that some features of the drug-dependent state (e.g., tolerance) might be related to inhibition of G(alpha)1-linked receptor activity.     

Rational inattention and tonic dopamine

Slow-timescale (tonic) changes in dopamine (DA) contribute to a wide variety of processes in reinforcement learning, interval timing, and other domains. Furthermore, changes in tonic DA exert distinct effects depending on when they occur (e.g., during learning vs. performance) and what task the subject is performing (e.g., operant vs. classical conditioning). Two influential theories of tonic DA—the average reward theory and the Bayesian theory in which DA controls precision—have each been successful at explaining a subset of empirical findings. But how the same DA signal performs two seemingly distinct functions without creating crosstalk is not well understood. Here we reconcile the two theories under the unifying framework of ‘rational inattention,’ which (1) conceptually links average reward and precision, (2) outlines how DA manipulations affect this relationship, and in so doing, (3) captures new empirical phenomena. In brief, rational inattention asserts that agents can increase their precision in a task (and thus improve their performance) by paying a cognitive cost. Crucially, whether this cost is worth paying depends on average reward availability, reported by DA. The monotonic relationship between average reward and precision means that the DA signal contains the information necessary to retrieve the precision. When this information is needed after the task is performed, as presumed by Bayesian inference, acute manipulations of DA will bias behavior in predictable ways. We show how this framework reconciles a remarkably large collection of experimental findings. In reinforcement learning, the rational inattention framework predicts that learning from positive and negative feedback should be enhanced in high and low DA states, respectively, and that DA should tip the exploration-exploitation balance toward exploitation. In interval timing, this framework predicts that DA should increase the speed of the internal clock and decrease the extent of interference by other temporal stimuli during temporal reproduction (the central tendency effect). Finally, rational inattention makes the new predictions that these effects should be critically dependent on the controllability of rewards, that post-reward delays in intertemporal choice tasks should be underestimated, and that average reward manipulations should affect the speed of the clock—thus capturing empirical findings that are unexplained by either theory alone. Our results suggest that a common computational repertoire may underlie the seemingly heterogeneous roles of DA.     

Large Scale Brain Functional Networks Support Sentence Comprehension: Evidence from Both Explicit and Implicit Language Tasks

Previous studies have indicated that sentences are comprehended via widespread brain regions in the fronto-temporo-parietal network in explicit language tasks (e.g., semantic congruency judgment tasks), and through restricted temporal or frontal regions in implicit language tasks (e.g., font size judgment tasks). This discrepancy has raised questions regarding a common network for sentence comprehension that acts regardless of task effect and whether different tasks modulate network properties. To this end, we constructed brain functional networks based on 27 subjects’ fMRI data that was collected while performing explicit and implicit language tasks. We found that network properties and network hubs corresponding to the implicit language task were similar to those associated with the explicit language task. We also found common hubs in occipital, temporal and frontal regions in both tasks. Compared with the implicit language task, the explicit language task resulted in greater global efficiency and increased integrated betweenness centrality of the left inferior frontal gyrus, which is a key region related to sentence comprehension. These results suggest that brain functional networks support both explicit and implicit sentence comprehension; in addition, these two types of language tasks may modulate the properties of brain functional networks.     

Thunderstruck: The ACDC model of flexible sequences and rhythms in recurrent neural circuits

Adaptive sequential behavior is a hallmark of human cognition. In particular, humans can learn to produce precise spatiotemporal sequences given a certain context. For instance, musicians can not only reproduce learned action sequences in a context-dependent manner, they can also quickly and flexibly reapply them in any desired tempo or rhythm without overwriting previous learning. Existing neural network models fail to account for these properties. We argue that this limitation emerges from the fact that sequence information (i.e., the position of the action) and timing (i.e., the moment of response execution) are typically stored in the same neural network weights. Here, we augment a biologically plausible recurrent neural network of cortical dynamics to include a basal ganglia-thalamic module which uses reinforcement learning to dynamically modulate action. This “associative cluster-dependent chain” (ACDC) model modularly stores sequence and timing information in distinct loci of the network. This feature increases computational power and allows ACDC to display a wide range of temporal properties (e.g., multiple sequences, temporal shifting, rescaling, and compositionality), while still accounting for several behavioral and neurophysiological empirical observations. Finally, we apply this ACDC network to show how it can learn the famous “Thunderstruck” song intro and then flexibly play it in a “bossa nova” rhythm without further training.     

A neural systems analysis of adaptive navigation

In the field of the neurobiology of learning, significant emphasis has been placed on understanding neural plasticity within a single structure (or synapse type) as it relates to a particular type of learning mediated by a particular brain area. To appreciate fully the breadth of the plasticity responsible for complex learning phenomena, it is imperative that we also examine the neural mechanisms of the behavioral instantiation of learned information, how motivational systems interact, and how past memories affect the learning process. To address this issue, we describe a model of complex learning (rodent adaptive navigation) that could be used to study dynamically interactive neural systems. Adaptive navigation depends on the efficient integration of external and internal sensory information with motivational systems to arrive at the most effective cognitive and/or behavioral strategies. We present evidence consistent with the view that during navigation: 1) the limbic thalamus and limbic cortex is primarily responsible for the integration of current and expected sensory information, 2) the hippocampal-septal-hypothalamic system provides a mechanism whereby motivational perspectives bias sensory processing, and 3) the amygdala-prefrontal-striatal circuit allows animals to evaluate the expected reinforcement consequences of context-dependent behavioral responses. Although much remains to be determined regarding the nature of the interactions among neural systems, new insights have emerged regarding the mechanisms that underlie flexible and adaptive behavioral responses.     

Resting-State fMRI Activity Predicts Unsupervised Learning and Memory in an Immersive Virtual Reality Environment

In the real world, learning often proceeds in an unsupervised manner without explicit instructions or feedback. In this study, we employed an experimental paradigm in which subjects explored an immersive virtual reality environment on each of two days. On day 1, subjects implicitly learned the location of 39 objects in an unsupervised fashion. On day 2, the locations of some of the objects were changed, and object location recall performance was assessed and found to vary across subjects. As prior work had shown that functional magnetic resonance imaging (fMRI) measures of resting-state brain activity can predict various measures of brain performance across individuals, we examined whether resting-state fMRI measures could be used to predict object location recall performance. We found a significant correlation between performance and the variability of the resting-state fMRI signal in the basal ganglia, hippocampus, amygdala, thalamus, insula, and regions in the frontal and temporal lobes, regions important for spatial exploration, learning, memory, and decision making. In addition, performance was significantly correlated with resting-state fMRI connectivity between the left caudate and the right fusiform gyrus, lateral occipital complex, and superior temporal gyrus. Given the basal ganglia''s role in exploration, these findings suggest that tighter integration of the brain systems responsible for exploration and visuospatial processing may be critical for learning in a complex environment.     

Reinforcement and learning

Evidence has been accumulating to support the process of reinforcement as a potential mechanism in speciation. In many species, mate choice decisions are influenced by cultural factors, including learned mating preferences (sexual imprinting) or learned mate attraction signals (e.g., bird song). It has been postulated that learning can have a strong impact on the likelihood of speciation and perhaps on the process of reinforcement, but no models have explicitly considered learning in a reinforcement context. We review the evidence that suggests that learning may be involved in speciation and reinforcement, and present a model of reinforcement via learned preferences. We show that not only can reinforcement occur when preferences are learned by imprinting, but that such preferences can maintain species differences easily in comparison with both autosomal and sex-linked genetically inherited preferences. We highlight the need for more explicit study of the connection between the behavioral process of learning and the evolutionary process of reinforcement in natural systems.     

The mind through chick eyes: memory,cognition and anticipation

To understand the animal mind, we have to reconstruct how animals recognize the external world through their own eyes. For the reconstruction to be realistic, explanations must be made both in their proximate causes (brain mechanisms) as well as ultimate causes (evolutionary backgrounds). Here, we review recent advances in the behavioral, psychological, and system-neuroscience studies accomplished using the domestic chick as subjects. Diverse behavioral paradigms are compared (such as filial imprinting, sexual imprinting, one-trial passive avoidance learning, and reinforcement operant conditioning) in their behavioral characterizations (development, sensory and motor aspects of functions, fitness gains) and relevant brain mechanisms. We will stress that common brain regions are shared by these distinct paradigms, particularly those in the ventral telencephalic structures such as AIv (in the archistriatum) and LPO (in the medial striatum). Neuronal ensembles in these regions could code the chick's anticipation for forthcoming events, particularly the quality/quantity and the temporal proximity of rewards. Without the internal representation of the anticipated proximity in LPO, behavioral tolerance will be lost, and the chick makes impulsive choice for a less optimized option. Functional roles of these regions proved compatible with their anatomical counterparts in the mammalian brain, thus suggesting that the neural systems linking between the memorized past and the anticipated future have remained highly conservative through the evolution of the amniotic vertebrates during the last 300 million years. With the conservative nature in mind, research efforts should be oriented toward a unifying theory, which could explain behavioral deviations from optimized foraging, such as "na?ve curiosity," "contra-freeloading," "Concorde fallacy," and "altruism."     

Hearing and seeing meaning in speech and gesture: insights from brain and behaviour

As we speak, we use not only the arbitrary form–meaning mappings of the speech channel but also motivated form–meaning correspondences, i.e. iconic gestures that accompany speech (e.g. inverted V-shaped hand wiggling across gesture space to demonstrate walking). This article reviews what we know about processing of semantic information from speech and iconic gestures in spoken languages during comprehension of such composite utterances. Several studies have shown that comprehension of iconic gestures involves brain activations known to be involved in semantic processing of speech: i.e. modulation of the electrophysiological recording component N400, which is sensitive to the ease of semantic integration of a word to previous context, and recruitment of the left-lateralized frontal–posterior temporal network (left inferior frontal gyrus (IFG), medial temporal gyrus (MTG) and superior temporal gyrus/sulcus (STG/S)). Furthermore, we integrate the information coming from both channels recruiting brain areas such as left IFG, posterior superior temporal sulcus (STS)/MTG and even motor cortex. Finally, this integration is flexible: the temporal synchrony between the iconic gesture and the speech segment, as well as the perceived communicative intent of the speaker, modulate the integration process. Whether these findings are special to gestures or are shared with actions or other visual accompaniments to speech (e.g. lips) or other visual symbols such as pictures are discussed, as well as the implications for a multimodal view of language.     

When Feelings Arise with Meanings: How Emotion and Meaning of a Native Language Affect Second Language Processing in Adult Learners

To determine when and how L2 learners start to process L2 words affectively and semantically, we conducted a longitudinal study on their interaction in adult L2 learners. In four test sessions, spanning half a year of L2 learning, we monitored behavioral and ERP learning-related changes for one and the same set of words by means of a primed lexical-decision paradigm with L1 primes and L2 targets. Sensitivity rates, accuracy rates, RTs, and N400 amplitude to L2 words and pseudowords improved significantly across sessions. A semantic priming effect (e.g, prime “driver”facilitating response to target “street”) was found in accuracy rates and RTs when collapsing Sessions 1 to 4, while this effect modulated ERP amplitudes within the first 300 ms of L2 target processing. An overall affective priming effect (e.g., “sweet” facilitating”taste”) was also found in RTs and ERPs (posterior P1). Importantly, the ERPs showed an L2 valence effect across sessions (e.g., positive words were easier to process than neutral words), indicating that L2 learners were sensitive to L2 affective meaning. Semantic and affective priming interacted in the N400 time-window only in Session 4, implying that they affected meaning integration during L2 immersion together. The results suggest that L1 and L2 are initially processed semantically and affectively via relatively separate channels that are more and more linked contingent on L2 exposure.