A model cortical circuit for the storage of temporal sequences

Despite the fact that temporal information processing is of particular significance in biological memory systems, not much has yet been explored about how these systems manage to store temporal information involved in sequences of stimuli. A neural network model capable of learning and recalling temporal sequences is proposed, based on a neural mechanism in which the sequences are expanded into a series of periodic rectangular oscillations. Thus, the mathematical framework underlying the model, to some extent, is concerned with the Walsh function series. The oscillatory activities generated by the interplay between excitatory and inhibitory neuron pools are transmitted to another neuron pool whose role in learning and retrieval is to modify the rhythms and phases of the rectangular oscillations. Thus, a basic functional neural circuit involves three different neuron pools. The modifiability of rhythms and phases is incorporated into the model with the aim of improving the quality of the retrieval. Numerical simulations were conducted to show the characteristic features of the learning as well as the performance of the model in memory recall.     

Diffusion models for firing of a neuron with varying threshold

A stochastic model for the firing of a neuron with refractory properties is treated analytically. Refractory behavior is modeled by a threshold function θ(t) which is infinite immediately after the neuron fires, as well as during the absolute refractory period, and then decreases monotonically to the quiescent threshold level, θ_∞, during the relative refractory period. Using Wald's identity, input-output relations are derived analytically for the exponential threshold which has a time constant equal to the membrane time constant. A method for computing these relations for a general threshold is presented and is explicitly used for the general exponential threshold and the Hagiwara threshold, θ(t) = θ_∞e^α/t, where a is a constant.     

A nonlinear autoregressive Volterra model of the Hodgkin–Huxley equations

We propose a new variant of Volterra-type model with a nonlinear auto-regressive (NAR) component that is a suitable framework for describing the process of AP generation by the neuron membrane potential, and we apply it to input-output data generated by the Hodgkin–Huxley (H–H) equations. Volterra models use a functional series expansion to describe the input-output relation for most nonlinear dynamic systems, and are applicable to a wide range of physiologic systems. It is difficult, however, to apply the Volterra methodology to the H–H model because is characterized by distinct subthreshold and suprathreshold dynamics. When threshold is crossed, an autonomous action potential (AP) is generated, the output becomes temporarily decoupled from the input, and the standard Volterra model fails. Therefore, in our framework, whenever membrane potential exceeds some threshold, it is taken as a second input to a dual-input Volterra model. This model correctly predicts membrane voltage deflection both within the subthreshold region and during APs. Moreover, the model naturally generates a post-AP afterpotential and refractory period. It is known that the H–H model converges to a limit cycle in response to a constant current injection. This behavior is correctly predicted by the proposed model, while the standard Volterra model is incapable of generating such limit cycle behavior. The inclusion of cross-kernels, which describe the nonlinear interactions between the exogenous and autoregressive inputs, is found to be absolutely necessary. The proposed model is general, non-parametric, and data-derived.     

Statistical study on reverberation between two mutually excitatory neuron groups

The reverberation that occurs between two neuron groups, which have excitatory mono-synaptic random connections with each other can be studied theoretically by employing a model neuron, which expresses well the characters of a real neuron. In this model we consider three effects, which are; the effect of the summation of the excitatory post-synaptic potential (EPSP) of neurons; the effect of the spontaneous firing of neurons as a noise in groups and the effect of the relative refractory period of neurons. As a result, it is shown that under the effect of the summation of the EPSP of neurons and the effect of the noise, the systematic threshold p theta takes the same value as is observed in practice. The effect of the relative refractory period has been considered in order to explain the low speed of the increase in firing activity, as observed in the reverberating system. It suppresses slightly the speed of the increase in firing activity (pi) in the system. Moreover, the speed can be suppressed by making the refractory effect strong according to the increase of pi. However, the initial increase of pi at a high speed that was observed in the experiment cannot be explained simply by the effect of the refractoriness, even if it were the absolute refractoriness.     

An extended model for a spiking neuron class

This paper proposes an extension to the model of a spiking neuron for information processing in artificial neural networks, developing a new approach for the dynamic threshold of the integrate-and-fire neuron. This new approach invokes characteristics of biological neurons such as the behavior of chemical synapses and the receptor field. We demonstrate how such a digital model of spiking neurons can solve complex nonlinear classification with a single neuron, performing experiments for the classical XOR problem. Compared with rate-coded networks and the classical integrate-and-fire model, the trained network demonstrated faster information processing, requiring fewer neurons and shorter learning periods. The extended model validates all the logic functions of biological neurons when such functions are necessary for the proper flow of binary codes through a neural network.     

A neural circuit model of emotional learning using two pathways with different granularity and speed of information processing

We propose a neural circuit model of emotional learning using two pathways with different granularity and speed of information processing. In order to derive a precise time process, we utilized a spiking model neuron proposed by Izhikevich and spike-timing-dependent synaptic plasticity (STDP) of both excitatory and inhibitory synapses. We conducted computer simulations to evaluate the proposed model. We demonstrate some aspects of emotional learning from the perspective of the time process. The agreement of the results with the previous behavioral experiments suggests that the structure and learning process of the proposed model are appropriate.     

A simple computer model of excitable synaptically connected neurons

 The space-lumped two-variable neuron model is studied. Extension of the neural model by adding a simple synaptic current allows the demonstration of neural interactions. The production of synchronous burst activity in this simple two-neuron excitatory loop is modeled, including the influence of random background excitatory input. The ability of the neuron model to integrate inputs spatially and temporally is shown. Two refractory periods after stimuli were identified and their role in burst cessation is demonstrated. Our findings show that simple neural units without long-lasting membrane processes are capable of generating long lasting patterns of activity. The results of simulation of simple background activity suggest that an increase in background activity tends to cause decreased activity of the network. This phenomenon, as well as the existence of two refractory periods, allows for burst cessation without inhibition in this simple model. Received: 6 December 1996/Accepted in revised form: 7 April 1997     

A model for nonlinear stochastic behavior of the pupil

In binocular fusion, pairs of left and right stimuli yielding the same brightness perception constitute an equibrightness curve in a coordinate system whose ordinate and abscissa correspond to the left and right stimulus strengths. A neural network model is presented to elucidate the characteristics of the curve. According to the model, Fechner's paradox is due to the threshold characteristics of the neuron. If the shapes or movements are radically different between the left and right stimuli, the retinal rivalry is caused. That is, only the left stimulus is perceived at one moment and the right stimulus at another moment. The period of left or right eye dominance alternates randomly from time to time. The distribution of the period is approximate to the gamma distribution. In order to account for this fact, a neural network model is proposed, which consists of a pair of neurons receiving inputs with stochastic fluctuations. The computer simulation was carried out with satisfactory results. The model of retinal rivalry is integrated with that of brightness perception.     

On some properties of stochastic information processes in neurons and neuron populations

The information in the nervous spike trains and its processing by neural units are discussed. In these problems, our attention is focused on the stochastic properties of neurons and neuron populations. There are three subjects in this paper, which are the spontaneous type neuron, the forced type neuron and the reciprocal inhibitory pairs.

1. The spontaneous type neuron produces spikes without excitatory inputs. The mathematical model has the following assumptions. The neuron potential (NP) has the fluctuation and obeys the Ornstein-Uhlenbeck process, because the N P is not so perfectly random as that of the Wiener process but has an attraction to the rest value. The threshold varies exponentially and the NP has the constant lower limit. When the NP reaches the threshold, the neuron fires and the NP is reset to a certain position. After a firing, an absolute refractory period exists. In discussing the stochastic properties of neurons, the transition probability density function and the first passage time density function are the important quantities, which are governed by the Kolmogorov's equations. Although they can be set up easily, we can rarely obtain the analytical solutions in time domain. Moreover, they cover only simple properties. Hence the numerical analysis is performed and a good deal of fair results are obtained and discussed.
2. The forced type neuron has input pulse trains which are assumed to be based on the Poisson process. Other assumptions and methods are almost the same as above except the diffusion approximation of the stochastic process. In this case, we encounter the inhomogeneous process due to the pulse-frequency-modulation, whose first passage time density reveals the multimodal distribution. The numerical analysis is also tried, and the output spike interval density is further discussed in the case of the periodic modulation.
3. Two types of reciprocal inhibitory pairs are discussed. The first type has two excitatory driving inputs which are mutually independent. The second type has one common excitatory input but it advances in two ways, one of which has a time lag. The neuron dynamics is the same as that of the forced type neuron and each neuron has an identical structure. The inputs are assumed to be based on the Poisson process and the inhibition occurs when the companion neuron fires. In this case, the equations of the probability density functions are not obtained. Hence the computer simulation is tried and it is observed that the stochastic rhythm emerges in spite of the temporally homogeneous inputs. Furthermore, the case of inhomogeneous inputs is discussed.

     

On the role of anatomy in learning by the cerebellar cortex

The properties due to the location of neurons, synapses, and possibly even synaptic channels, in neuron networks are still unknown. Our preliminary results suggest that not only the interconnections but also the relative positions of the different elements in the network are of importance in the learning process in the cerebellar cortex. We have used neural field equations to investigate the mechanisms of learning in the hierarchical neural network. The numerical resolution of these equations reveals two important properties: (i) The hierarchical structure of this network has the expected effect on learning because the flow of information at the neuronal level is controlled by the heterosynaptic effect through the synaptic density-connectivity function, i.e. the action potential field variable is controlled by the synaptic efficacy field variable at different points of the neuron. (ii) The geometry of the system involves different velocities of propagation along different fibers, i.e. different delays between cells, and thus has a stabilizing effect on the dynamics, allowing the Purkinje output to reach a given value. The field model proposed should be useful in the study of the spatial properties of hierarchical biological systems.     

A Boolean complete neural model of adaptive behavior

A multi-layered neural assembly is developed which has the capability of learning arbitrary Boolean functions. Though the model neuron is more powerful than those previously considered, assemblies of neurons are needed to detect non-linearly separable patterns. Algorithms for learning at the neuron and assembly level are described. The model permits multiple output systems to share a common memory. Learned evaluation allows sequences of actions to be organized. Computer simulations demonstrate the capabilities of the model.     

锥体神经元的连接模式对突触后局部电位的依赖

脑皮层的功能连接模式与突触可塑性密切相关,受突触空间分布和刺激模式等多种因素的影响。尽管越来越多的证据表明突触可塑性不仅受突触后动作电位而且还受突触后局部树突电位的影响,但是目前尚不清楚神经元的功能连接模式是否和怎样依赖于突触后局部电位的。为此,本文建立了一个无需硬边界设置的、突触后局部膜电位依赖的可塑性模型。该模型具有突触强度的自平衡能力并且能够再现多种突触可塑性实验结果。基于该模型对两个锥体神经元的功能连接模式进行仿真的结果表明,当突触后局部电位都处于亚阈值时两个神经元无功能连接,如果一个神经元的突触后膜电位高于阈值电位则产生向该神经元的单向连接,当两个神经元的突触后膜电位都超过阈值电位时则产生双向连接,说明突触后局部膜电位分布是神经元功能连接模式形成的关键。研究结果加深了神经网络连接模式形成机制的理解,对学习和记忆的研究具有重要意义。     

The enzymatic neuron as a reaction-diffusion network of cyclic nucleotides

Recent evidence suggests that the cyclic nucleotides play a central role in the intracellular processing of neural signals. The dynamics of this system may be seen as a realization of the enzymatic neuron model. Enzymatic neurons are formal neurons which map binary afferent signals into patterns of excitation across an abstract membrane. The distribution of enzyme-like elements called excitases enables a set of local threshold functions to determine the firing activity of the neuron. This paper analyzes the basic properties of enzymatic neurons in a simple continuous-time framework, and shows how they may be presented as reaction-diffusion networks which model the cyclic nucleotide system. We present the results of computer simulations of this neuron and discuss its implications for selectional learning and its relation to conventional two-factor systems. One fundamental property of the reaction-diffusion neuron is its so-called “double-dynamics” property; examination of this property and its contribution to the computing power of the neuron provides some insight into the obscure relation between microscopic and macroscopic models of computation.     

A Spike-Timing Pattern Based Neural Network Model for the Study of Memory Dynamics

It is well accepted that the brain''s computation relies on spatiotemporal activity of neural networks. In particular, there is growing evidence of the importance of continuously and precisely timed spiking activity. Therefore, it is important to characterize memory states in terms of spike-timing patterns that give both reliable memory of firing activities and precise memory of firing timings. The relationship between memory states and spike-timing patterns has been studied empirically with large-scale recording of neuron population in recent years. Here, by using a recurrent neural network model with dynamics at two time scales, we construct a dynamical memory network model which embeds both fast neural and synaptic variation and slow learning dynamics. A state vector is proposed to describe memory states in terms of spike-timing patterns of neural population, and a distance measure of state vector is defined to study several important phenomena of memory dynamics: partial memory recall, learning efficiency, learning with correlated stimuli. We show that the distance measure can capture the timing difference of memory states. In addition, we examine the influence of network topology on learning ability, and show that local connections can increase the network''s ability to embed more memory states. Together theses results suggest that the proposed system based on spike-timing patterns gives a productive model for the study of detailed learning and memory dynamics.     

Some properties of randomly connected networks of neuron-like elements with refractory

Dynamic properties of randomly connected networks consisting of neuron-like elements with refractory are investigated from a macroscopic point of view. Equations describing the transition of the activity level of the network — a macroscopic state — are derived under some hypotheses on the stochastic properties of the network. The equations are characterized by a set of parameters which are determined by distributions of the threshold values of elements and the weighting values of connection between elements. It is shown that a network behaves like a monostable, bistable or astable circuit when its refractory period is less than one time unit and that a network is monostable or bistable when its refractory period is longer than two time units. An oscillatory network, on the other hand, is always realized if the network has a feedback mechanism which decreases the excitability of neurons when high activity level is sustained. Some results of computer simulation of randomly connected neuron networks are also presented.     

Addition of song-related neurons in swamp sparrows coincides with memorization, not production, of learned songs

During song learning in birds, neurons are added to some song nuclei and lost from others. Previous studies have been unable to distinguish whether these neural changes are uniquely associated with memorizing a song model (sensory acquisition) or vocal practice (sensorimotor learning). In this study we measured changes in neuron number within song nuclei of swamp sparrows, a species in which the two phases of song learning are nonoverlapping. Male swamp sparrows were collected as hatchlings and tape-tutored from approximately 22 to 62 days of age. Swamp sparrows memorize about 60% of their song material during this period, but do not begin practicing this learned material until approximately 275 days of age. Birds were sacrificed at 23, 41, 61, 71, 274, or 340 days of age. During sensory acquisition, neuron number increased drastically in both the caudal nucleus of the ventral hyperstriatum (HVc) and Area X. The period of sensorimotor learning was not associated with any further changes in neuron number within these regions. We were unable to detect any significant changes in neuron number within the magnocellular nucleus of the neostriatum or the robust nucleus of the archistriatum during either stage of song learning. These results raise the possibility that ongoing addition of HVc and Area X neurons may encourage, and thereby temporally restrict, song acquisition.     

Functional model of biological neural networks

A functional model of biological neural networks, called temporal hierarchical probabilistic associative memory (THPAM), is proposed in this paper. THPAM comprises functional models of dendritic trees for encoding inputs to neurons, a first type of neuron for generating spike trains, a second type of neuron for generating graded signals to modulate neurons of the first type, supervised and unsupervised Hebbian learning mechanisms for easy learning and retrieving, an arrangement of dendritic trees for maximizing generalization, hardwiring for rotation-translation-scaling invariance, and feedback connections with different delay durations for neurons to make full use of present and past informations generated by neurons in the same and higher layers. These functional models and their processing operations have many functions of biological neural networks that have not been achieved by other models in the open literature and provide logically coherent answers to many long-standing neuroscientific questions. However, biological justifications of these functional models and their processing operations are required for THPAM to qualify as a macroscopic model (or low-order approximate) of biological neural networks.     

An improved genetic algorithm based fuzzy-tuned neural network

This paper presents a fuzzy-tuned neural network, which is trained by an improved genetic algorithm (GA). The fuzzy-tuned neural network consists of a neural-fuzzy network and a modified neural network. In the modified neural network, a neuron model with two activation functions is used so that the degree of freedom of the network function can be increased. The neural-fuzzy network governs some of the parameters of the neuron model. It will be shown that the performance of the proposed fuzzy-tuned neural network is better than that of the traditional neural network with a similar number of parameters. An improved GA is proposed to train the parameters of the proposed network. Sets of improved genetic operations are presented. The performance of the improved GA will be shown to be better than that of the traditional GA. Some application examples are given to illustrate the merits of the proposed neural network and the improved GA.     

Contribution to the probabilistic theory of neural nets: II. Facilitation and threshold phenomena

The output curve of a single neuron with a threshold of response with respect to the frequency of the stimuli is derived. If the stimuli are regularly spaced in time, the output curve has discontinuities. If the threshold and/or refractory period are sufficiently large, the output curve approaches the “all-or-none” curve. In the case of completely randomized stimuli, the output curve is sigmoid. The equation of this curve is derived and some properties are studied. Threshold and “all-or-none” effects can be achieved by “pyramiding” neurons of this type to converge on neurons of higher order.