Shaping embodied neural networks for adaptive goal-directed behavior

The acts of learning and memory are thought to emerge from the modifications of synaptic connections between neurons, as guided by sensory feedback during behavior. However, much is unknown about how such synaptic processes can sculpt and are sculpted by neuronal population dynamics and an interaction with the environment. Here, we embodied a simulated network, inspired by dissociated cortical neuronal cultures, with an artificial animal (an animat) through a sensory-motor loop consisting of structured stimuli, detailed activity metrics incorporating spatial information, and an adaptive training algorithm that takes advantage of spike timing dependent plasticity. By using our design, we demonstrated that the network was capable of learning associations between multiple sensory inputs and motor outputs, and the animat was able to adapt to a new sensory mapping to restore its goal behavior: move toward and stay within a user-defined area. We further showed that successful learning required proper selections of stimuli to encode sensory inputs and a variety of training stimuli with adaptive selection contingent on the animat's behavior. We also found that an individual network had the flexibility to achieve different multi-task goals, and the same goal behavior could be exhibited with different sets of network synaptic strengths. While lacking the characteristic layered structure of in vivo cortical tissue, the biologically inspired simulated networks could tune their activity in behaviorally relevant manners, demonstrating that leaky integrate-and-fire neural networks have an innate ability to process information. This closed-loop hybrid system is a useful tool to study the network properties intermediating synaptic plasticity and behavioral adaptation. The training algorithm provides a stepping stone towards designing future control systems, whether with artificial neural networks or biological animats themselves.     

Arc/Arg3.1 mediates homeostatic synaptic scaling of AMPA receptors

Homeostatic plasticity may compensate for Hebbian forms of synaptic plasticity, such as long-term potentiation (LTP) and depression (LTD), by scaling neuronal output without changing the relative strength of individual synapses. This delicate balance between neuronal output and distributed synaptic weight may be necessary for maintaining efficient encoding of information across neuronal networks. Here, we demonstrate that Arc/Arg3.1, an immediate-early gene (IEG) that is rapidly induced by neuronal activity associated with information encoding in the brain, mediates homeostatic synaptic scaling of AMPA type glutamate receptors (AMPARs) via its ability to activate a novel and selective AMPAR endocytic pathway. High levels of Arc/Arg3.1 block the homeostatic increases in AMPAR function induced by chronic neuronal inactivity. Conversely, loss of Arc/Arg3.1 results in increased AMPAR function and abolishes homeostatic scaling of AMPARs. These observations, together with evidence that Arc/Arg3.1 is required for memory consolidation, reveal the importance of Arc/Arg3.1's dynamic expression as it exerts continuous and precise control over synaptic strength and cellular excitability.     

MiR‐134‐dependent regulation of Pumilio‐2 is necessary for homeostatic synaptic depression

Neurons employ a set of homeostatic plasticity mechanisms to counterbalance altered levels of network activity. The molecular mechanisms underlying homeostatic plasticity in response to increased network excitability are still poorly understood. Here, we describe a sequential homeostatic synaptic depression mechanism in primary hippocampal neurons involving miRNA‐dependent translational regulation. This mechanism consists of an initial phase of synapse elimination followed by a reinforcing phase of synaptic downscaling. The activity‐regulated microRNA miR‐134 is necessary for both synapse elimination and the structural rearrangements leading to synaptic downscaling. Results from miR‐134 inhibition further uncover a differential requirement for GluA1/2 subunits for the functional expression of homeostatic synaptic depression. Downregulation of the miR‐134 target Pumilio‐2 in response to chronic activity, which selectively occurs in the synapto‐dendritic compartment, is required for miR‐134‐mediated homeostatic synaptic depression. We further identified polo‐like kinase 2 (Plk2) as a novel target of Pumilio‐2 involved in the control of GluA2 surface expression. In summary, we have described a novel pathway of homeostatic plasticity that stabilizes neuronal circuits in response to increased network activity.     

Homeostatic synaptic plasticity: from single synapses to neural circuits

Homeostatic synaptic plasticity remains an enigmatic form of synaptic plasticity. Increasing interest on the topic has fuelled a surge of recent studies that have identified key molecular players and the signaling pathways involved. However, the new findings also highlight our lack of knowledge concerning some of the basic properties of homeostatic synaptic plasticity. In this review we address how homeostatic mechanisms balance synaptic strengths between the presynaptic and the postsynaptic terminals and across synapses that share the same postsynaptic neuron.     

Network Self-Organization Explains the Statistics and Dynamics of Synaptic Connection Strengths in Cortex

The information processing abilities of neural circuits arise from their synaptic connection patterns. Understanding the laws governing these connectivity patterns is essential for understanding brain function. The overall distribution of synaptic strengths of local excitatory connections in cortex and hippocampus is long-tailed, exhibiting a small number of synaptic connections of very large efficacy. At the same time, new synaptic connections are constantly being created and individual synaptic connection strengths show substantial fluctuations across time. It remains unclear through what mechanisms these properties of neural circuits arise and how they contribute to learning and memory. In this study we show that fundamental characteristics of excitatory synaptic connections in cortex and hippocampus can be explained as a consequence of self-organization in a recurrent network combining spike-timing-dependent plasticity (STDP), structural plasticity and different forms of homeostatic plasticity. In the network, associative synaptic plasticity in the form of STDP induces a rich-get-richer dynamics among synapses, while homeostatic mechanisms induce competition. Under distinctly different initial conditions, the ensuing self-organization produces long-tailed synaptic strength distributions matching experimental findings. We show that this self-organization can take place with a purely additive STDP mechanism and that multiplicative weight dynamics emerge as a consequence of network interactions. The observed patterns of fluctuation of synaptic strengths, including elimination and generation of synaptic connections and long-term persistence of strong connections, are consistent with the dynamics of dendritic spines found in rat hippocampus. Beyond this, the model predicts an approximately power-law scaling of the lifetimes of newly established synaptic connection strengths during development. Our results suggest that the combined action of multiple forms of neuronal plasticity plays an essential role in the formation and maintenance of cortical circuits.     

Sensory experience modifies spontaneous state dynamics in a large-scale barrel cortical model

Experimental evidence suggests that spontaneous neuronal activity may shape and be shaped by sensory experience. However, we lack information on how sensory experience modulates the underlying synaptic dynamics and how such modulation influences the response of the network to future events. Here we study whether spike-timing-dependent plasticity (STDP) can mediate sensory-induced modifications in the spontaneous dynamics of a new large-scale model of layers II, III and IV of the rodent barrel cortex. Our model incorporates significant physiological detail, including the types of neurons present, the probabilities and delays of connections, and the STDP profiles at each excitatory synapse. We stimulated the neuronal network with a protocol of repeated sensory inputs resembling those generated by the protraction-retraction motion of whiskers when rodents explore their environment, and studied the changes in network dynamics. By applying dimensionality reduction techniques to the synaptic weight space, we show that the initial spontaneous state is modified by each repetition of the stimulus and that this reverberation of the sensory experience induces long-term, structured modifications in the synaptic weight space. The post-stimulus spontaneous state encodes a memory of the stimulus presented, since a different dynamical response is observed when the network is presented with shuffled stimuli. These results suggest that repeated exposure to the same sensory experience could induce long-term circuitry modifications via 'Hebbian' STDP plasticity.     

Phosphorylation of AMPA receptors is required for sensory deprivation-induced homeostatic synaptic plasticity

Sensory experience, and the lack thereof, can alter the function of excitatory synapses in the primary sensory cortices. Recent evidence suggests that changes in sensory experience can regulate the synaptic level of Ca(2+)-permeable AMPA receptors (CP-AMPARs). However, the molecular mechanisms underlying such a process have not been determined. We found that binocular visual deprivation, which is a well-established in vivo model to produce multiplicative synaptic scaling in visual cortex of juvenile rodents, is accompanied by an increase in the phosphorylation of AMPAR GluR1 (or GluA1) subunit at the serine 845 (S845) site and the appearance of CP-AMPARs at synapses. To address the role of GluR1-S845 in visual deprivation-induced homeostatic synaptic plasticity, we used mice lacking key phosphorylation sites on the GluR1 subunit. We found that mice specifically lacking the GluR1-S845 site (GluR1-S845A mutants), which is a substrate of cAMP-dependent kinase (PKA), show abnormal basal excitatory synaptic transmission and lack visual deprivation-induced homeostatic synaptic plasticity. We also found evidence that increasing GluR1-S845 phosphorylation alone is not sufficient to produce normal multiplicative synaptic scaling. Our study provides concrete evidence that a GluR1 dependent mechanism, especially S845 phosphorylation, is a necessary pre-requisite step for in vivo homeostatic synaptic plasticity.     

Homeostatic synaptic plasticity: local and global mechanisms for stabilizing neuronal function

Neural circuits must maintain stable function in the face of many plastic challenges, including changes in synapse number and strength, during learning and development. Recent work has shown that these destabilizing influences are counterbalanced by homeostatic plasticity mechanisms that act to stabilize neuronal and circuit activity. One such mechanism is synaptic scaling, which allows neurons to detect changes in their own firing rates through a set of calcium-dependent sensors that then regulate receptor trafficking to increase or decrease the accumulation of glutamate receptors at synaptic sites. Additional homeostatic mechanisms may allow local changes in synaptic activation to generate local synaptic adaptations, and network-wide changes in activity to generate network-wide adjustments in the balance between excitation and inhibition. The signaling pathways underlying these various forms of homeostatic plasticity are currently under intense scrutiny, and although dozens of molecular pathways have now been implicated in homeostatic plasticity, a clear picture of how homeostatic feedback is structured at the molecular level has not yet emerged. On a functional level, neuronal networks likely use this complex set of regulatory mechanisms to achieve homeostasis over a wide range of temporal and spatial scales.     

Molecular mechanisms of homeostatic plasticity in central pattern generator networks

Central pattern generator (CPG) networks rely on a balance of intrinsic and network properties to produce reliable, repeatable activity patterns. This balance is maintained by homeostatic plasticity where alterations in neuronal properties dynamically maintain appropriate neural output in the face of changing environmental conditions and perturbations. However, it remains unclear just how these neurons and networks can both monitor their ongoing activity and use this information to elicit homeostatic physiological responses to ensure robustness of output over time. Evidence exists that CPG networks use a mixed strategy of activity‐dependent, activity‐independent, modulator‐dependent, and synaptically regulated homeostatic plasticity to achieve this critical stability. In this review, we focus on some of the current understanding of the molecular pathways and mechanisms responsible for this homeostatic plasticity in the context of central pattern generator function, with a special emphasis on some of the smaller invertebrate networks that have allowed for extensive cellular‐level analyses that have brought recent insights to these questions.     

Modulation of hippocampal plasticity in learning and memory

Synaptic plasticity plays a central role in the study of neural mechanisms of learning and memory. Plasticity rules are not invariant over time but are under neuromodulatory control, enabling behavioral states to influence memory formation. Neuromodulation controls synaptic plasticity at network level by directing information flow, at circuit level through changes in excitation/inhibition balance, and at synaptic level through modulation of intracellular signaling cascades. Although most research has focused on modulation of principal neurons, recent progress has uncovered important roles for interneurons in not only routing information, but also setting conditions for synaptic plasticity. Moreover, astrocytes have been shown to both gate and mediate plasticity. These additional mechanisms must be considered for a comprehensive mechanistic understanding of learning and memory.     

Interplay between Short- and Long-Term Plasticity in Cell-Assembly Formation

Various hippocampal and neocortical synapses of mammalian brain show both short-term plasticity and long-term plasticity, which are considered to underlie learning and memory by the brain. According to Hebb’s postulate, synaptic plasticity encodes memory traces of past experiences into cell assemblies in cortical circuits. However, it remains unclear how the various forms of long-term and short-term synaptic plasticity cooperatively create and reorganize such cell assemblies. Here, we investigate the mechanism in which the three forms of synaptic plasticity known in cortical circuits, i.e., spike-timing-dependent plasticity (STDP), short-term depression (STD) and homeostatic plasticity, cooperatively generate, retain and reorganize cell assemblies in a recurrent neuronal network model. We show that multiple cell assemblies generated by external stimuli can survive noisy spontaneous network activity for an adequate range of the strength of STD. Furthermore, our model predicts that a symmetric temporal window of STDP, such as observed in dopaminergic modulations on hippocampal neurons, is crucial for the retention and integration of multiple cell assemblies. These results may have implications for the understanding of cortical memory processes.     

Homeostatic plasticity in the CNS: synaptic and intrinsic forms

The study of experience-dependent plasticity has been dominated by questions of how Hebbian plasticity mechanisms act during learning and development. This is unsurprising as Hebbian plasticity constitutes the most fully developed and influential model of how information is stored in neural circuits and how neural circuitry can develop without extensive genetic instructions. Yet Hebbian plasticity may not be sufficient for understanding either learning or development: the dramatic changes in synapse number and strength that can be produced by this kind of plasticity tend to threaten the stability of neural circuits. Recent work has suggested that, in addition to Hebbian plasticity, homeostatic regulatory mechanisms are active in a variety of preparations. These mechanisms alter both the synaptic connections between neurons and the intrinsic electrical properties of individual neurons, in such a way as to maintain some constancy in neuronal properties despite the changes wrought by Hebbian mechanisms. Here we review the evidence for homeostatic plasticity in the central nervous system, with special emphasis on results from cortical preparations.     

Synaptic Plasticity Controls Sensory Responses through Frequency-Dependent Gamma Oscillation Resonance

Synchronized gamma frequency oscillations in neural networks are thought to be important to sensory information processing, and their effects have been intensively studied. Here we describe a mechanism by which the nervous system can readily control gamma oscillation effects, depending selectively on visual stimuli. Using a model neural network simulation, we found that sensory response in the primary visual cortex is significantly modulated by the resonance between “spontaneous” and “stimulus-driven” oscillations. This gamma resonance can be precisely controlled by the synaptic plasticity of thalamocortical connections, and cortical response is regulated differentially according to the resonance condition. The mechanism produces a selective synchronization between the afferent and downstream neural population. Our simulation results explain experimental observations such as stimulus-dependent synchronization between the thalamus and the cortex at different oscillation frequencies. The model generally shows how sensory information can be selectively routed depending on its frequency components.     

Balanced networks under spike-time dependent plasticity

The dynamics of local cortical networks are irregular, but correlated. Dynamic excitatory–inhibitory balance is a plausible mechanism that generates such irregular activity, but it remains unclear how balance is achieved and maintained in plastic neural networks. In particular, it is not fully understood how plasticity induced changes in the network affect balance, and in turn, how correlated, balanced activity impacts learning. How do the dynamics of balanced networks change under different plasticity rules? How does correlated spiking activity in recurrent networks change the evolution of weights, their eventual magnitude, and structure across the network? To address these questions, we develop a theory of spike–timing dependent plasticity in balanced networks. We show that balance can be attained and maintained under plasticity–induced weight changes. We find that correlations in the input mildly affect the evolution of synaptic weights. Under certain plasticity rules, we find an emergence of correlations between firing rates and synaptic weights. Under these rules, synaptic weights converge to a stable manifold in weight space with their final configuration dependent on the initial state of the network. Lastly, we show that our framework can also describe the dynamics of plastic balanced networks when subsets of neurons receive targeted optogenetic input.     

Homeostatic control of synaptic rewiring in recurrent networks induces the formation of stable memory engrams

Brain networks store new memories using functional and structural synaptic plasticity. Memory formation is generally attributed to Hebbian plasticity, while homeostatic plasticity is thought to have an ancillary role in stabilizing network dynamics. Here we report that homeostatic plasticity alone can also lead to the formation of stable memories. We analyze this phenomenon using a new theory of network remodeling, combined with numerical simulations of recurrent spiking neural networks that exhibit structural plasticity based on firing rate homeostasis. These networks are able to store repeatedly presented patterns and recall them upon the presentation of incomplete cues. Storage is fast, governed by the homeostatic drift. In contrast, forgetting is slow, driven by a diffusion process. Joint stimulation of neurons induces the growth of associative connections between them, leading to the formation of memory engrams. These memories are stored in a distributed fashion throughout connectivity matrix, and individual synaptic connections have only a small influence. Although memory-specific connections are increased in number, the total number of inputs and outputs of neurons undergo only small changes during stimulation. We find that homeostatic structural plasticity induces a specific type of “silent memories”, different from conventional attractor states.     

Failed Stabilization for Long-Term Potentiation in the Auditory Cortex of Fmr1 Knockout Mice

Fragile X syndrome is a developmental disorder that affects sensory systems. A null mutation of the Fragile X Mental Retardation protein 1 (Fmr1) gene in mice has varied effects on developmental plasticity in different sensory systems, including normal barrel cortical plasticity, altered ocular dominance plasticity and grossly impaired auditory frequency map plasticity. The mutation also has different effects on long-term synaptic plasticity in somatosensory and visual cortical neurons, providing insights on how it may differentially affect the sensory systems. Here we present evidence that long-term potentiation (LTP) is impaired in the developing auditory cortex of the Fmr1 knockout (KO) mice. This impairment of synaptic plasticity is consistent with impaired frequency map plasticity in the Fmr1 KO mouse. Together, these results suggest a potential role of LTP in sensory map plasticity during early sensory development.     

Requirement for Plk2 in orchestrated ras and rap signaling, homeostatic structural plasticity, and memory

Ras and Rap small GTPases are important for synaptic plasticity and memory. However, their roles in homeostatic plasticity are unknown. Here, we report that polo-like kinase 2 (Plk2), a homeostatic suppressor of overexcitation, governs the activity of Ras and Rap via coordination of their regulatory proteins. Plk2 directs elimination of Ras activator RasGRF1 and Rap inhibitor SPAR via phosphorylation-dependent ubiquitin-proteasome degradation. Conversely, Plk2 phosphorylation stimulates Ras inhibitor SynGAP and Rap activator PDZGEF1. These Ras/Rap regulators perform complementary functions to downregulate dendritic spines and AMPA receptors following elevated activity, and their collective regulation by Plk2 profoundly stimulates Rap and suppresses Ras. Furthermore, perturbation of Plk2 disrupts Ras and Rap signaling, prevents homeostatic shrinkage and loss of dendritic spines, and impairs proper memory formation. Our study demonstrates a critical role of Plk2 in the synchronized tuning of Ras and Rap and underscores the functional importance of this regulation in homeostatic synaptic plasticity.     

Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses

The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their joint spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent theory for the evolution of network structure by combining fast spiking covariance with a slow evolution of synaptic weights. Through a finite-size expansion of network dynamics we obtain a low-dimensional set of nonlinear differential equations for the evolution of two-synapse connectivity motifs. With this theory in hand, we explore how the form of the plasticity rule drives the evolution of microcircuits in cortical networks. When potentiation and depression are in approximate balance, synaptic dynamics depend on weighted divergent, convergent, and chain motifs. For additive, Hebbian STDP these motif interactions create instabilities in synaptic dynamics that either promote or suppress the initial network structure. Our work provides a consistent theoretical framework for studying how spiking activity in recurrent networks interacts with synaptic plasticity to determine network structure.