Capacity of a model immune network

The capacity of a model immune network in terms of the number of different antigens that can be vaccinated against without any memory lost is computed and tested by numerical simulations. We also investigate memory loss and failure to vaccinate due to overcrowding the network with too many antigens. The computations are done for two different strategies for proliferation, one implying all the antigen specific clones and the second one being more thrifty.     

Holding Multiple Items in Short Term Memory: A Neural Mechanism

Human short term memory has a capacity of several items maintained simultaneously. We show how the number of short term memory representations that an attractor network modeling a cortical local network can simultaneously maintain active is increased by using synaptic facilitation of the type found in the prefrontal cortex. We have been able to maintain 9 short term memories active simultaneously in integrate-and-fire simulations where the proportion of neurons in each population, the sparseness, is 0.1, and have confirmed the stability of such a system with mean field analyses. Without synaptic facilitation the system can maintain many fewer memories active in the same network. The system operates because of the effectively increased synaptic strengths formed by the synaptic facilitation just for those pools to which the cue is applied, and then maintenance of this synaptic facilitation in just those pools when the cue is removed by the continuing neuronal firing in those pools. The findings have implications for understanding how several items can be maintained simultaneously in short term memory, how this may be relevant to the implementation of language in the brain, and suggest new approaches to understanding and treating the decline in short term memory that can occur with normal aging.     

A discrete fully recurrent network of max product units for associative memory and classification

This paper defines the truncated normalized max product operation for the transformation of states of a network and provides a method for solving a set of equations based on this operation. The operation serves as the transformation for the set of fully connected units in a recurrent network that otherwise might consist of linear threshold units. Component values of the state vector and outputs of the units take on the values in the set [0, 0.1,..., 0.9, 1]. The result is a much larger state space given a particular number of units and size of connection matrix than for a network based on threshold units. Since the operation defined here can form the basis of transformations in a recurrent network with a finite number of states, fixed points or cycles are possible and the network based on this operation for transformations can be used as an associative memory or pattern classifier with fixed points taking on the role of prototypes. Discrete fully recurrent networks have proven themselves to be very useful as associative memories and as classifiers. However they are often based on units that have binary states. The effect of this is that the data to be processed consisting of vectors in R(n) have to be converted to vectors in [0, 1]m with m much larger than n since binary encoding based on positional notation is not feasible. This implies a large increase in the number of components. The effect can be lessened by allowing more states for each unit in our network. The network proposed demonstrates those properties that are desirable in an associative memory very well as the simulations show.     

Physical models of neural networks

The state of art in computer modelling of neural networks with associative memory is reviewed. The available experimental data are considered on learning and memory of small neural systems, on isolated synapses and on molecular level. Computer simulations demonstrate that realistic models of neural ensembles exhibit properties which can be interpreted as image recognition, categorization, learning, prototype forming, etc. A bilayer model of associative neural network is proposed. One layer corresponds to the short-term memory, the other one to the long-term memory. Patterns are stored in terms of the synaptic strength matrix. We have studied the relaxational dynamics of neurons firing and suppression within the short-term memory layer under the influence of the long-term memory layer. The interaction among the layers has found to create a number of novel stable states which are not the learning patterns. These synthetic patterns may consist of elements belonging to different non-intersecting learning patterns. Within the framework of a hypothesis of selective and definite coding of images in brain one can interpret the observed effect as the "idea? generating" process.     

短时记忆的神经网络模型

提出一个带有指针环路的短时记忆神经网络模型,模型包含两个神经网络,其中一个是与长时记忆共有的存贮内容表达网络,另一个为短时指针神经元环路,由于指针环路仅作为记忆内容的临时指针,因此,仅用很少的存贮单元即可完成各种短时记忆任务,计算机仿真证明,本模型确能表现出短时记忆的存贮容量有限和组块编码两个基本特征。     

Extremely Dilute Modular Neuronal Networks: Neocortical Memory Retrieval Dynamics

A model of columnar networks of neocortical association areas is studied. The neuronal network is composed of many Hebbian autoassociators, or modules, each of which interacts with a relatively small number of the others, randomly chosen. Any module encodes and stores a number of elementary percepts, or features. Memory items, or patterns, are peculiar combinations of features sparsely distributed over the multi-modular network. Any feature stored in any module can be involved in several of the stored patterns; feature-sharing is in fact source of local ambiguities and, consequently, a potential cause of erroneous memory retrieval spreading through the model network in pattern completion tasks.The memory retrieval dynamics of the large modular autoassociator is investigated by combining mathematical analysis and numerical simulations. An oscillatory retrieval process is proposed that is very efficient in overcoming feature-sharing drawbacks; it requires a mechanism that modulates the robustness of local attractors to noise, and neuronal activity sparseness such that quiescent and active modules are about equally noisy to any post-synaptic module.Moreover, it is shown that statistical correlation between 'kinds' of features across the set of memory patterns can be exploited to obtain a more efficient achievement of memory retrieval capabilities.It is also shown that some spots of the network cannot be reached by retrieval activity spread if they are not directly cued by the stimulus. The locations of these activity isles depend on the pattern to retrieve, while their extension only depends (in large networks) on statistics of inter-modular connections and stored patterns. The existence of activity isles determines an upper-bound to retrieval quality that does not depend on the specific retrieval dynamics adopted, nor on whether feature-sharing is permitted. The oscillatory retrieval process nearly saturates this bound.     

Acetylcholine and associative memory in the piriform cortex

The significance of cholinergic modulation for associative memory performance in the piriform cortex was examined in a study combining cellular neurophysiology in brain slices with realistic biophysical network simulations. Three different physiological effects of acetylcholine were identified at the single-cell level: suppression of neuronal adaptation, suppression of synaptic transmission in the intrinsic fibers layer, and activity-dependent increase in synaptic strength. Biophysical simulations show how these three effects are joined together to enhance learning and recall performance of the cortical network. Furthermore, our data suggest that activity-dependent synaptic decay during learning is a crucial factor in determining learning capability of the cortical network. Accordingly, it is predicted that acetylcholine should also enhance long-term depression in the piriform cortex.     

On the dynamic organization of memory

The interaction of memory structures and retrieval dynamics is discussed. A mathematical model for associative free recall is presented to support the view that the organization of simple processing units plays an important role in the retrieval of memory traces. Computer simulations show that "flexibility" and "fidelity" of the dynamics strongly depend on the network structure, the amplification and decay parameters, and the noise term.     

Parallel processing for associative and neuronal networks

We present a parallel processing network, consisting of nine microcomputers, for neuron-network simulations and for the realization of an associative computer memory. We add some remarks on the present possibilities to implement larger associative networks and on parallel processing strategies in general.     

海马记忆功能的神经网络模型

综合神经心理学,神经生理学、解剖学与神经网络研究的成果,提出一个海马记忆功能的神经网络模型。模型由三个神经网络所组成;海马的ＣＡ１和ＣＡ３网络和大脑皮层联合区,ＣＡ３的功能是将不同感觉输入联合起来,而ＣＡ１的作用是将它们结成一个单一的记忆。而大脑皮层则是长期记忆的部位。在ＶＡＸ１１／７５０上进行计算机仿真,仿真证明模型有近期及长期记忆功能,破坏模拟海马的部分,模型显示出与顺行性遗忘症相似的特性。在     

Towards biologically constrained attractor models of schizophrenia

Alterations in neuromodulation or synaptic transmission in biophysical attractor network models, as proposed by the dominant dopaminergic and glutamatergic theories of schizophrenia, successfully mimic working memory (WM) deficits in people with schizophrenia (PSZ). Yet, multiple, often opposing alterations in memory circuits can lead to the same behavioral patterns in these network models. Here, we critically revise the computational and experimental literature that links NMDAR hypofunction to WM precision loss in PSZ. We show in network simulations that currently available experimental evidence cannot set apart competing biophysical accounts. Critical points to resolve are the effects of increases vs. decreases in E/I ratio (e.g. through NMDAR blockade) on firing rate tuning and shared noise modulations and possible concomitant deficits in short-term plasticity. We argue that these concerted experimental and computational efforts will lead to a better understanding of the neurobiology underlying cognitive deficits in PSZ.     

A Three-Threshold Learning Rule Approaches the Maximal Capacity of Recurrent Neural Networks

Understanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is the attractor neural network scenario, whose prototype is the Hopfield model. The model simplicity and the locality of the synaptic update rules come at the cost of a poor storage capacity, compared with the capacity achieved with perceptron learning algorithms. Here, by transforming the perceptron learning rule, we present an online learning rule for a recurrent neural network that achieves near-maximal storage capacity without an explicit supervisory error signal, relying only upon locally accessible information. The fully-connected network consists of excitatory binary neurons with plastic recurrent connections and non-plastic inhibitory feedback stabilizing the network dynamics; the memory patterns to be memorized are presented online as strong afferent currents, producing a bimodal distribution for the neuron synaptic inputs. Synapses corresponding to active inputs are modified as a function of the value of the local fields with respect to three thresholds. Above the highest threshold, and below the lowest threshold, no plasticity occurs. In between these two thresholds, potentiation/depression occurs when the local field is above/below an intermediate threshold. We simulated and analyzed a network of binary neurons implementing this rule and measured its storage capacity for different sizes of the basins of attraction. The storage capacity obtained through numerical simulations is shown to be close to the value predicted by analytical calculations. We also measured the dependence of capacity on the strength of external inputs. Finally, we quantified the statistics of the resulting synaptic connectivity matrix, and found that both the fraction of zero weight synapses and the degree of symmetry of the weight matrix increase with the number of stored patterns.     

Sequential configuration model for firing patterns in local neural networks

This paper presents a sequential configuration model to represent the coordinated firing patterns of memory traces in groups of neurons in local networks. Computer simulations are used to study the dynamic properties of memory traces selectively retrieved from networks in which multiple memory traces have been embedded according to the sequential configuration model. Distinct memory traces which utilize the same neurons, but differ only in temporal sequencing are selectively retrievable. Firing patterns of constituent neurons of retrieved memory traces exhibit the main properties of neurons observed in multi microelectrode recordings. The paper shows how to adjust relative synaptic weightings so as to control the disruptive influences of cross-talk in multipy-embedded networks. The theoretical distinction between (primarily anatomical) beds and (primarily physiological) realizations underlines the fundamentally stochastic nature of network firing patterns, and allows the definition of 4 degrees of clarity of retrieved memory traces.     

具有竞争指针的短时记忆神经网络模型

在我们以前提出的短时记忆神经网络模型基础上［３］,我们在新模型中引入突触竞争机制,提出了一个新的短时记忆神经网络模型。模型仍由两个神经网络所组成;其一为与长时记忆共有的信息内容表达网络,另一个为指针神经元环路。由于表达区神经元与指针神经元间的突触权重的竞争,使得模型可以表现出由干扰引起的短时记忆的遗忘。相应于自由回忆序列位置效应和汉字组块两个心理学实验,对模型做了计算机仿真。仿真结果显示模型的行为与两个心理实验定量地符合得很好。由此表明现在的模型更合适于作为短时记忆的模型。     

Parallel network simulations with NEURON

The NEURON simulation environment has been extended to support parallel network simulations. Each processor integrates the equations for its subnet over an interval equal to the minimum (interprocessor) presynaptic spike generation to postsynaptic spike delivery connection delay. The performance of three published network models with very different spike patterns exhibits superlinear speedup on Beowulf clusters and demonstrates that spike communication overhead is often less than the benefit of an increased fraction of the entire problem fitting into high speed cache. On the EPFL IBM Blue Gene, almost linear speedup was obtained up to 100 processors. Increasing one model from 500 to 40,000 realistic cells exhibited almost linear speedup on 2000 processors, with an integration time of 9.8 seconds and communication time of 1.3 seconds. The potential for speed-ups of several orders of magnitude makes practical the running of large network simulations that could otherwise not be explored. Action Editor: Alain Destexhe     

The mechanism for the complexation and dissociation between siRNA and PMAL: a molecular dynamics simulation study based on a coarse-grained model

Abstract

The capacity of silencing genes makes small interfering RNA (siRNA) becomes potential candidates for curing many fatal diseases. Due to the low stability and delivery efficiency of siRNA, the design of amphiphilic carrier for siRNA delivery is vital for the practical gene therapy. In the present work, we explored how the complexation and dissociation of siRNA with poly (maleic anhydride-alt-1-decene) substituted with 3-(dimethylamino) propylamine (PMAL), which is a recent synthesised amphiphilic polymer and can be used in delivery of siRNAs and proteins, using traditional molecular dynamics simulations, together with steered molecular dynamics simulations. It was shown that the complexation of siRNA with PMALs can spontaneously occur, no matter what unit number of PMAL is. PMALs of different unit numbers form micelle-like structures and interact with siRNA surface. With the increase of unit number, PMAL becomes more flexible and interacts with siRNA from attachment to entanglement. The dissociation of PMAL from siRNA is an energy-consuming process. The free energy difference increases with the unit number of PMAL. The free energy for dissociation involves both the stretch of PMAL and the separation of PMAL from siRNA. Therefore, an optimal unit number of PMAL is critical for the delivery efficiency of siRNA when PMAL is used as carrier. In present work, when the radius of gyration of PMAL approaches to that of siRNA, PMAL gives a favoured both complexation and dissociation between siRNA and PMAL. Finally, we propose the mechanism of complexation and dissociation of PMAL with siRNA. The above simulation established a molecular insight of the interaction between siRNA and PMAL and was helpful for the design and applications of new PMAL-based polymers as siRNA delivery carriers.     

Memory Capacity of Networks with Stochastic Binary Synapses

In standard attractor neural network models, specific patterns of activity are stored in the synaptic matrix, so that they become fixed point attractors of the network dynamics. The storage capacity of such networks has been quantified in two ways: the maximal number of patterns that can be stored, and the stored information measured in bits per synapse. In this paper, we compute both quantities in fully connected networks of N binary neurons with binary synapses, storing patterns with coding level , in the large and sparse coding limits (). We also derive finite-size corrections that accurately reproduce the results of simulations in networks of tens of thousands of neurons. These methods are applied to three different scenarios: (1) the classic Willshaw model, (2) networks with stochastic learning in which patterns are shown only once (one shot learning), (3) networks with stochastic learning in which patterns are shown multiple times. The storage capacities are optimized over network parameters, which allows us to compare the performance of the different models. We show that finite-size effects strongly reduce the capacity, even for networks of realistic sizes. We discuss the implications of these results for memory storage in the hippocampus and cerebral cortex.     

Homeostatic control of synaptic rewiring in recurrent networks induces the formation of stable memory engrams

Brain networks store new memories using functional and structural synaptic plasticity. Memory formation is generally attributed to Hebbian plasticity, while homeostatic plasticity is thought to have an ancillary role in stabilizing network dynamics. Here we report that homeostatic plasticity alone can also lead to the formation of stable memories. We analyze this phenomenon using a new theory of network remodeling, combined with numerical simulations of recurrent spiking neural networks that exhibit structural plasticity based on firing rate homeostasis. These networks are able to store repeatedly presented patterns and recall them upon the presentation of incomplete cues. Storage is fast, governed by the homeostatic drift. In contrast, forgetting is slow, driven by a diffusion process. Joint stimulation of neurons induces the growth of associative connections between them, leading to the formation of memory engrams. These memories are stored in a distributed fashion throughout connectivity matrix, and individual synaptic connections have only a small influence. Although memory-specific connections are increased in number, the total number of inputs and outputs of neurons undergo only small changes during stimulation. We find that homeostatic structural plasticity induces a specific type of “silent memories”, different from conventional attractor states.     

A Predictive Reinforcement Model of Dopamine Neurons for Learning Approach Behavior

A neural network model of how dopamine and prefrontal cortex activity guides short- and long-term information processing within the cortico-striatal circuits during reward-related learning of approach behavior is proposed. The model predicts two types of reward-related neuronal responses generated during learning: (1) cell activity signaling errors in the prediction of the expected time of reward delivery and (2) neural activations coding for errors in the prediction of the amount and type of reward or stimulus expectancies. The former type of signal is consistent with the responses of dopaminergic neurons, while the latter signal is consistent with reward expectancy responses reported in the prefrontal cortex. It is shown that a neural network architecture that satisfies the design principles of the adaptive resonance theory of Carpenter and Grossberg (1987) can account for the dopamine responses to novelty, generalization, and discrimination of appetitive and aversive stimuli. These hypotheses are scrutinized via simulations of the model in relation to the delivery of free food outside a task, the timed contingent delivery of appetitive and aversive stimuli, and an asymmetric, instructed delay response task.