Representational drift as a window into neural and behavioural plasticity

Large-scale recordings of neural activity over days and weeks have revealed that neural representations of familiar tasks, precepts and actions continually evolve without obvious changes in behaviour. We hypothesise that this steady drift in neural activity and accompanying physiological changes is due in part to the continuous application of a learning rule at the cellular and population level. Explicit predictions of this drift can be found in neural network models that use iterative learning to optimise weights. Drift therefore provides a measurable signal that can reveal systems–level properties of biological plasticity mechanisms, such as their precision and effective learning rates.     

Social behavior in context: Hormonal modulation of behavioral plasticity and social competence

In social species animals should fine-tune the expression of their social behavior to social environments in order to avoid the costs of engaging in costly social interactions. Therefore, social competence, defined as the ability of an animal to optimize the expression of its social behavior as a function of the available social information, should be considered as a performance trait that impacts on the Darwinian fitness of the animal. Social competence is based on behavioral plasticity which, in turn, can be achieved by different neural mechanisms of plasticity, namely by rewiring or by biochemically switching nodes of a putative neural network underlying social behavior. Since steroid hormones respond to social interactions and have receptors extensively expressed in the social behavioral neural network, it is proposed that steroids play a key role in the hormonal modulation of social plasticity. Here, we propose a reciprocal model for the action of androgens on short-term behavioral plasticity and review a set of studies conducted in our laboratory using an African cichlid fish (Oreochromis mossambicus) that provide support for it. Androgens are shown to be implicated as physiological mediators in a wide range of social phenomena that promote social competence, namely by adjusting the behavioral response to the nature of the intruder and the presence of third parties (dear enemy and audience effects), by anticipating territorial intrusions (bystander effect and conditioning of the territorial response), and by modifying future behavior according to prior experience of winning (winner effect). The rapid behavioral actions of socially induced short-term transient changes in androgens indicate that these effects are most likely mediated by nongenomic mechanisms. The fact that the modulation of rapid changes in behavior is open to the influence of circulating levels of androgens, and is not exclusively achieved by changes in central neuromodulators, suggests functional relevance of integrating body parameters in the behavioral response. Thus, the traditional view of seeing neural circuits as unique causal agents of behavior should be updated to a brain-body-environment perspective, in which these neural circuits are embodied and the behavioral performance (and outcomes as fitness) depends on a dynamic relationship between the different levels. In this view hormones play a major role as behavioral modulators.     

Spatiotemporal Computations of an Excitable and Plastic Brain: Neuronal Plasticity Leads to Noise-Robust and Noise-Constructive Computations

It is a long-established fact that neuronal plasticity occupies the central role in generating neural function and computation. Nevertheless, no unifying account exists of how neurons in a recurrent cortical network learn to compute on temporally and spatially extended stimuli. However, these stimuli constitute the norm, rather than the exception, of the brain''s input. Here, we introduce a geometric theory of learning spatiotemporal computations through neuronal plasticity. To that end, we rigorously formulate the problem of neural representations as a relation in space between stimulus-induced neural activity and the asymptotic dynamics of excitable cortical networks. Backed up by computer simulations and numerical analysis, we show that two canonical and widely spread forms of neuronal plasticity, that is, spike-timing-dependent synaptic plasticity and intrinsic plasticity, are both necessary for creating neural representations, such that these computations become realizable. Interestingly, the effects of these forms of plasticity on the emerging neural code relate to properties necessary for both combating and utilizing noise. The neural dynamics also exhibits features of the most likely stimulus in the network''s spontaneous activity. These properties of the spatiotemporal neural code resulting from plasticity, having their grounding in nature, further consolidate the biological relevance of our findings.     

Serotonin, inhibition, and negative mood

Pavlovian predictions of future aversive outcomes lead to behavioral inhibition, suppression, and withdrawal. There is considerable evidence for the involvement of serotonin in both the learning of these predictions and the inhibitory consequences that ensue, although less for a causal relationship between the two. In the context of a highly simplified model of chains of affectively charged thoughts, we interpret the combined effects of serotonin in terms of pruning a tree of possible decisions, (i.e., eliminating those choices that have low or negative expected outcomes). We show how a drop in behavioral inhibition, putatively resulting from an experimentally or psychiatrically influenced drop in serotonin, could result in unexpectedly large negative prediction errors and a significant aversive shift in reinforcement statistics. We suggest an interpretation of this finding that helps dissolve the apparent contradiction between the fact that inhibition of serotonin reuptake is the first-line treatment of depression, although serotonin itself is most strongly linked with aversive rather than appetitive outcomes and predictions.     

Learning Multisensory Integration and Coordinate Transformation via Density Estimation

Sensory processing in the brain includes three key operations: multisensory integration—the task of combining cues into a single estimate of a common underlying stimulus; coordinate transformations—the change of reference frame for a stimulus (e.g., retinotopic to body-centered) effected through knowledge about an intervening variable (e.g., gaze position); and the incorporation of prior information. Statistically optimal sensory processing requires that each of these operations maintains the correct posterior distribution over the stimulus. Elements of this optimality have been demonstrated in many behavioral contexts in humans and other animals, suggesting that the neural computations are indeed optimal. That the relationships between sensory modalities are complex and plastic further suggests that these computations are learned—but how? We provide a principled answer, by treating the acquisition of these mappings as a case of density estimation, a well-studied problem in machine learning and statistics, in which the distribution of observed data is modeled in terms of a set of fixed parameters and a set of latent variables. In our case, the observed data are unisensory-population activities, the fixed parameters are synaptic connections, and the latent variables are multisensory-population activities. In particular, we train a restricted Boltzmann machine with the biologically plausible contrastive-divergence rule to learn a range of neural computations not previously demonstrated under a single approach: optimal integration; encoding of priors; hierarchical integration of cues; learning when not to integrate; and coordinate transformation. The model makes testable predictions about the nature of multisensory representations.     

A neural systems analysis of adaptive navigation

In the field of the neurobiology of learning, significant emphasis has been placed on understanding neural plasticity within a single structure (or synapse type) as it relates to a particular type of learning mediated by a particular brain area. To appreciate fully the breadth of the plasticity responsible for complex learning phenomena, it is imperative that we also examine the neural mechanisms of the behavioral instantiation of learned information, how motivational systems interact, and how past memories affect the learning process. To address this issue, we describe a model of complex learning (rodent adaptive navigation) that could be used to study dynamically interactive neural systems. Adaptive navigation depends on the efficient integration of external and internal sensory information with motivational systems to arrive at the most effective cognitive and/or behavioral strategies. We present evidence consistent with the view that during navigation: 1) the limbic thalamus and limbic cortex is primarily responsible for the integration of current and expected sensory information, 2) the hippocampal-septal-hypothalamic system provides a mechanism whereby motivational perspectives bias sensory processing, and 3) the amygdala-prefrontal-striatal circuit allows animals to evaluate the expected reinforcement consequences of context-dependent behavioral responses. Although much remains to be determined regarding the nature of the interactions among neural systems, new insights have emerged regarding the mechanisms that underlie flexible and adaptive behavioral responses.     

Hebbian learning in parallel and modular memories

Many cognitive and sensorimotor functions in the brain involve parallel and modular memory subsystems that are adapted by activity-dependent Hebbian synaptic plasticity. This is in contrast to the multilayer perceptron model of supervised learning where sensory information is presumed to be integrated by a common pool of hidden units through backpropagation learning. Here we show that Hebbian learning in parallel and modular memories is more advantageous than backpropagation learning in lumped memories in two respects: it is computationally much more efficient and structurally much simpler to implement with biological neurons. Accordingly, we propose a more biologically relevant neural network model, called a tree-like perceptron, which is a simple modification of the multilayer perceptron model to account for the general neural architecture, neuronal specificity, and synaptic learning rule in the brain. The model features a parallel and modular architecture in which adaptation of the input-to-hidden connection follows either a Hebbian or anti-Hebbian rule depending on whether the hidden units are excitatory or inhibitory, respectively. The proposed parallel and modular architecture and implicit interplay between the types of synaptic plasticity and neuronal specificity are exhibited by some neocortical and cerebellar systems. Received: 13 October 1996 / Accepted in revised form: 16 October 1997     

A continuous-time neural model for sequential action

Action selection, planning and execution are continuous processes that evolve over time, responding to perceptual feedback as well as evolving top-down constraints. Existing models of routine sequential action (e.g. coffee- or pancake-making) generally fall into one of two classes: hierarchical models that include hand-built task representations, or heterarchical models that must learn to represent hierarchy via temporal context, but thus far lack goal-orientedness. We present a biologically motivated model of the latter class that, because it is situated in the Leabra neural architecture, affords an opportunity to include both unsupervised and goal-directed learning mechanisms. Moreover, we embed this neurocomputational model in the theoretical framework of the theory of event coding (TEC), which posits that actions and perceptions share a common representation with bidirectional associations between the two. Thus, in this view, not only does perception select actions (along with task context), but actions are also used to generate perceptions (i.e. intended effects). We propose a neural model that implements TEC to carry out sequential action control in hierarchically structured tasks such as coffee-making. Unlike traditional feedforward discrete-time neural network models, which use static percepts to generate static outputs, our biological model accepts continuous-time inputs and likewise generates non-stationary outputs, making short-timescale dynamic predictions.     

Numerical modelling of plasticity induced by transcranial magnetic stimulation

We use neural field theory and spike-timing dependent plasticity to make a simple but biophysically reasonable model of long-term plasticity changes in the cortex due to transcranial magnetic stimulation (TMS). We show how common TMS protocols can be captured and studied within existing neural field theory. Specifically, we look at repetitive TMS protocols such as theta burst stimulation and paired-pulse protocols. Continuous repetitive protocols result mostly in depression, but intermittent repetitive protocols in potentiation. A paired pulse protocol results in depression at short ( < ～ 10 ms) and long ( > ～ 100 ms) interstimulus intervals, but potentiation for mid-range intervals. The model is sensitive to the choice of neural populations that are driven by the TMS pulses, and to the parameters that describe plasticity, which may aid interpretation of the high variability in existing experimental results. Driving excitatory populations results in greater plasticity changes than driving inhibitory populations. Modelling also shows the merit in optimizing a TMS protocol based on an individual’s electroencephalogram. Moreover, the model can be used to make predictions about protocols that may lead to improvements in repetitive TMS outcomes.     

Reorganization and plasticity in the adult brain during learning of motor skills

On the basis of brain imaging studies, Doyon and Ungerleider recently proposed a model describing the cerebral plasticity that occurs in both cortico-striatal and cortico-cerebellar systems of the adult brain during learning of new motor skilled behaviors. This theoretical framework makes several testable predictions with regards to the contribution of these neural systems based on the phase (fast, slow, consolidation, automatization, and retention) and nature of the motor learning processes (motor sequence versus motor adaptation) acquired through repeated practice. There has been recent behavioral, lesion and additional neuroimaging studies that have addressed the assumptions made in this theory that will help in the revision of this model.     

Learning through maps: functional significance of topographic organization in primary sensory cortex.

The presence of "maps" in sensory cortex is a hallmark of the mammalian nervous system, but the functional significance of topographic organization has been called into question by physiological studies claiming that patterns of neural behavioral activity transcend topographic boundaries. This paper discusses recent behavioral and physiological studies suggesting that, when animals or human subjects learn perceptual tasks, the neural modifications associated with the learning are distributed according to the spatial arrangement of the primary sensory cortical map. Topographical cortical representations of sensory events, therefore, appear to constitute a true structural framework for information processing and plasticity.     

The neural basis of following advice

Learning by following explicit advice is fundamental for human cultural evolution, yet the neurobiology of adaptive social learning is largely unknown. Here, we used simulations to analyze the adaptive value of social learning mechanisms, computational modeling of behavioral data to describe cognitive mechanisms involved in social learning, and model-based functional magnetic resonance imaging (fMRI) to identify the neurobiological basis of following advice. One-time advice received before learning had a sustained influence on people's learning processes. This was best explained by social learning mechanisms implementing a more positive evaluation of the outcomes from recommended options. Computer simulations showed that this "outcome-bonus" accumulates more rewards than an alternative mechanism implementing higher initial reward expectation for recommended options. fMRI results revealed a neural outcome-bonus signal in the septal area and the left caudate. This neural signal coded rewards in the absence of advice, and crucially, it signaled greater positive rewards for positive and negative feedback after recommended rather than after non-recommended choices. Hence, our results indicate that following advice is intrinsically rewarding. A positive correlation between the model's outcome-bonus parameter and amygdala activity after positive feedback directly relates the computational model to brain activity. These results advance the understanding of social learning by providing a neurobiological account for adaptive learning from advice.     

Sequence learning recodes cortical representations instead of strengthening initial ones

We contrast two computational models of sequence learning. The associative learner posits that learning proceeds by strengthening existing association weights. Alternatively, recoding posits that learning creates new and more efficient representations of the learned sequences. Importantly, both models propose that humans act as optimal learners but capture different statistics of the stimuli in their internal model. Furthermore, these models make dissociable predictions as to how learning changes the neural representation of sequences. We tested these predictions by using fMRI to extract neural activity patterns from the dorsal visual processing stream during a sequence recall task. We observed that only the recoding account can explain the similarity of neural activity patterns, suggesting that participants recode the learned sequences using chunks. We show that associative learning can theoretically store only very limited number of overlapping sequences, such as common in ecological working memory tasks, and hence an efficient learner should recode initial sequence representations.     

Specificity of sensorimotor learning and the neural code: Neuronal representations in the primary motor cortex

Human studies show that the learning of a new sensorimotor mapping that requires adaptation to directional errors is local and generalizes poorly to untrained directions. We trained monkeys to learn new visuomotor rotations for only one target in space and recorded neuronal activity in the primary motor cortex before, during and after learning. Similar to humans, the monkeys showed poor transfer of learning to other directions, as observed by behavioral aftereffects for untrained directions. To test for internal representations underlying these changes, we compared two features of neuronal activity before and after learning: changes in firing rates and changes in information content. Specific elevations of firing rate were only observed in a subpopulation of cells in the motor cortex with directional properties corresponding to the locally learned rotation; namely cells only showed plasticity if their preferred direction was near the training one. We applied measures from information theory to probe for learning-related changes in the neuronal code. Single cells conveyed more information about the direction of movement and this specific improvement in encoding was correlated with an increase in the slope of the neurons' tuning curve. Further, the improved information after learning enabled a more accurate reconstruction of movement direction from neuronal populations. Our findings suggest a neural mechanism for the confined generalization of a newly acquired internal model by showing a tight relationship between the locality of learning and the properties of neurons. They also provide direct evidence for improvement in the neural code as a result of learning.     

A neural circuit model of emotional learning using two pathways with different granularity and speed of information processing

We propose a neural circuit model of emotional learning using two pathways with different granularity and speed of information processing. In order to derive a precise time process, we utilized a spiking model neuron proposed by Izhikevich and spike-timing-dependent synaptic plasticity (STDP) of both excitatory and inhibitory synapses. We conducted computer simulations to evaluate the proposed model. We demonstrate some aspects of emotional learning from the perspective of the time process. The agreement of the results with the previous behavioral experiments suggests that the structure and learning process of the proposed model are appropriate.     

A neural field model of the somatosensory cortex: formation, maintenance and reorganization of ordered topographic maps

We investigate the formation and maintenance of ordered topographic maps in the primary somatosensory cortex as well as the reorganization of representations after sensory deprivation or cortical lesion. We consider both the critical period (postnatal) where representations are shaped and the post-critical period where representations are maintained and possibly reorganized. We hypothesize that feed-forward thalamocortical connections are an adequate site of plasticity while cortico-cortical connections are believed to drive a competitive mechanism that is critical for learning. We model a small skin patch located on the distal phalangeal surface of a digit as a set of 256 Merkel ending complexes (MEC) that feed a computational model of the primary somatosensory cortex (area 3b). This model is a two-dimensional neural field where spatially localized solutions (a.k.a. bumps) drive cortical plasticity through a Hebbian-like learning rule. Simulations explain the initial formation of ordered representations following repetitive and random stimulations of the skin patch. Skin lesions as well as cortical lesions are also studied and results confirm the possibility to reorganize representations using the same learning rule and depending on the type of the lesion. For severe lesions, the model suggests that cortico-cortical connections may play an important role in complete recovery.     

Beetles, boxes and brain cells: neural mechanisms underlying valuation and learning

Sensory cues in the environment can predict the availability of reward. Through experience, humans and animals learn these predictions and use them to guide their actions. For example, we can learn to discriminate chanterelles from ordinary champignons through experience. Assuming the development of a taste for the complex and lingering flavors of chanterelles, we therefore learn to value the same action--picking mushrooms--differentially depending upon the appearance of a mushroom. One major goal of cognitive neuroscience is to understand the neural mechanisms that underlie this sort of learning. Because the acquisition of rewards motivates much behavior, recent efforts have focused on describing the neural signals related to learning the value of stimuli and actions. Neurons in the basal ganglia, in midbrain dopamine areas, in frontal and parietal cortices and in other brain areas, all modulate their activity in relation to aspects of learning. By training monkeys on various behavioral tasks, recent studies have begun to characterize how neural signals represent distinct processes, such as the timing of events, motivation, absolute (objective) and relative (subjective) valuation, and the formation of associative links between stimuli and potential actions. In addition, a number of studies have either further characterized dopamine signals or sought to determine how such signaling might interact with target structures, such as the striatum and rhinal cortex, to underlie learning.     

Using Graph Components Derived from an Associative Concept Dictionary to Predict fMRI Neural Activation Patterns that Represent the Meaning of Nouns

In this study, we introduce an original distance definition for graphs, called the Markov-inverse-F measure (MiF). This measure enables the integration of classical graph theory indices with new knowledge pertaining to structural feature extraction from semantic networks. MiF improves the conventional Jaccard and/or Simpson indices, and reconciles both the geodesic information (random walk) and co-occurrence adjustment (degree balance and distribution). We measure the effectiveness of graph-based coefficients through the application of linguistic graph information for a neural activity recorded during conceptual processing in the human brain. Specifically, the MiF distance is computed between each of the nouns used in a previous neural experiment and each of the in-between words in a subgraph derived from the Edinburgh Word Association Thesaurus of English. From the MiF-based information matrix, a machine learning model can accurately obtain a scalar parameter that specifies the degree to which each voxel in (the MRI image of) the brain is activated by each word or each principal component of the intermediate semantic features. Furthermore, correlating the voxel information with the MiF-based principal components, a new computational neurolinguistics model with a network connectivity paradigm is created. This allows two dimensions of context space to be incorporated with both semantic and neural distributional representations.     

Parallel processing in an identified neural circuit: the Aplysia californica gill-withdrawal response model system

The response of the gill of Aplysia calfornica Cooper to weak to moderate tactile stimulation of the siphon, the gill-withdrawal response or GWR, has been an important model system for work aimed at understanding the relationship between neural plasticity and simple forms of non-associative and associative learning. Interest in the GWR has been based largely on the hypothesis that the response could be explained adequately by parallel monosynaptic reflex arcs between six parietovisceral ganglion (PVG) gill motor neurons (GMNs) and a cluster of sensory neurons termed the LE cluster. This hypothesis, the Kupfermann-Kandel model, made clear, falsifiable predictions that have stimulated experimental work for many years. Here, we review tests of three predictions of the Kupfermann-Kandel model: (1) that the GWR is a simple, reflexive behaviour graded with stimulus intensity; (2) that central nervous system (CNS) pathways are necessary and sufficient for the GWR; and (3) that activity in six identified GMNs is sufficient to account for the GWR. The available data suggest that (1) a variety of action patterns occur in the context of the GWR; (2) the PVG is not necessary and the diffuse peripheral nervous system (PNS) is sufficient to mediate these action patterns; and (3) the role of any individual GMN in the behaviour varies. Both the control of gill-withdrawal responses, and plasticity in these responses, are broadly distributed across both PNS and CNS pathways. The Kupfermann-Kandel model is inconsistent with the available data and therefore stands rejected. There is, no known causal connection or correlation between the observed plasticity at the identified synapses in this system and behavioural changes during non-associative and associative learning paradigms. Critical examination of these well-studied central pathways suggests that they represent a 'wetware' neural network, architecturally similar to the neural network models of the widely used 'Perceptron' and/or 'Back-propagation' type. Such models may offer a more biologically realistic representation of nervous system organisation than has been thought. In this model, the six parallel GMNs of the CNS correspond to a hidden layer within one module of the gill-control system. That is, the gill-control system appears to be organised as a distributed system with several parallel modules, some of which are neural networks in their own right. A new model is presented here which predicts that the six GMNs serve as components of a 'push-pull' gain control system, along with known but largely unidentified inhibitory motor neurons from the PVG. This 'push-pull' gain control system sets the responsiveness of the peripheral gill motor system. Neither causal nor correlational links between specific forms of neural plasticity and behavioural plasticity have been demonstrated in the GWR model system. However, the GWR model system does provide an opportunity to observe and describe directly the physiological and biochemical mechanisms of distributed representation and parallel processing in a largely identifiable 'wetware' neural network.     

Evolution and plasticity: Divergence of song discrimination is faster in birds with innate song than in song learners in Neotropical passerine birds

Plasticity is often thought to accelerate trait evolution and speciation. For example, plasticity in birdsong may partially explain why clades of song learners are more diverse than related clades with innate song. This “song learning” hypothesis predicts that (1) differences in song traits evolve faster in song learners, and (2) behavioral discrimination against allopatric song (a proxy for premating reproductive isolation) evolves faster in song learners. We tested these predictions by analyzing acoustic traits and conducting playback experiments in allopatric Central American sister pairs of song learning oscines (N = 42) and nonlearning suboscines (N = 27). We found that nonlearners evolved mean acoustic differences slightly faster than did leaners, and that the mean evolutionary rate of song discrimination was 4.3 times faster in nonlearners than in learners. These unexpected results may be a consequence of significantly greater variability in song traits in song learners (by 54–79%) that requires song‐learning oscines to evolve greater absolute differences in song before achieving the same level of behavioral song discrimination as nonlearning suboscines. This points to “a downside of learning” for the evolution of species discrimination, and represents an important example of plasticity reducing the rate of evolution and diversification by increasing variability.