A discrete time neural network model with spiking neurons: II: Dynamics with noise

We provide rigorous and exact results characterizing the statistics of spike trains in a network of leaky Integrate-and-Fire neurons, where time is discrete and where neurons are submitted to noise, without restriction on the synaptic weights. We show the existence and uniqueness of an invariant measure of Gibbs type and discuss its properties. We also discuss Markovian approximations and relate them to the approaches currently used in computational neuroscience to analyse experimental spike trains statistics.     

Energy coding in neural network with inhibitory neurons

This paper aimed at assessing and comparing the effects of the inhibitory neurons in the neural network on the neural energy distribution, and the network activities in the absence of the inhibitory neurons to understand the nature of neural energy distribution and neural energy coding. Stimulus, synchronous oscillation has significant difference between neural networks with and without inhibitory neurons, and this difference can be quantitatively evaluated by the characteristic energy distribution. In addition, the synchronous oscillation difference of the neural activity can be quantitatively described by change of the energy distribution if the network parameters are gradually adjusted. Compared with traditional method of correlation coefficient analysis, the quantitative indicators based on nervous energy distribution characteristics are more effective in reflecting the dynamic features of the neural network activities. Meanwhile, this neural coding method from a global perspective of neural activity effectively avoids the current defects of neural encoding and decoding theory and enormous difficulties encountered. Our studies have shown that neural energy coding is a new coding theory with high efficiency and great potential.     

A modified Zeeman model for producing HRV signals and its application to ECG signal generation

Developing a mathematical model for the artificial generation of electrocardiogram (ECG) signals is a subject that has been widely investigated. One of the challenges is to generate ECG signals with a wide range of waveforms, power spectra and variations in heart rate variability (HRV)--all of which are important indexes of human heart functions. In this paper we present a comprehensive model for generating such artificial ECG signals. We incorporate into our model the effects of respiratory sinus arrhythmia, Mayer waves and the important very low-frequency component in the power spectrum of HRV. We use a new modified Zeeman model for generating the time series for HRV, and a single cycle of ECG is produced by using a simple neural network. The importance of the work is the model's ability to produce artificial ECG signals that resemble experimental recordings under various physiological conditions. As such the model provides a useful tool to simulate and analyse the main characteristics of ECG, such as its power spectrum and HRV under different conditions. Potential applications of this model include using the generated ECG as a flexible signal source to assess the effectiveness of a diagnostic ECG signal-processing device.     

On the stationary state of a network of inhibitory spiking neurons

The background activity of a cortical neural network is modeled by a homogeneous integrate-and-fire network with unreliable inhibitory synapses. For the case of fast synapses, numerical and analytical calculations show that the network relaxes into a stationary state of high attention. The majority of the neurons has a membrane potential just below the threshold; as a consequence the network can react immediately – on the time scale of synaptic transmission- on external pulses. The neurons fire with a low rate and with a broad distribution of interspike intervals. Firing events of the total network are correlated over short time periods. The firing rate increases linearly with external stimuli. In the limit of infinitely large networks, the synaptic noise decreases to zero. Nevertheless, the distribution of interspike intervals remains broad. Action Editor: Misha Tsodyks     

On the sensitive dependence on initial conditions of the dynamics of networks of spiking neurons

We have previously formulated an abstract dynamical system for networks of spiking neurons and derived a formal result that identifies the criterion for its dynamics, without inputs, to be “sensitive to initial conditions”. Since formal results are applicable only to the extent to which their assumptions are valid, we begin this article by demonstrating that the assumptions are indeed reasonable for a wide range of networks, particularly those that lack overarching structure. A notable aspect of the criterion is the finding that sensitivity does not necessarily arise from randomness of connectivity or of connection strengths, in networks. The criterion guides us to cases that decouple these aspects: we present two instructive examples of networks, one with random connectivity and connection strengths, yet whose dynamics is insensitive, and another with structured connectivity and connection strengths, yet whose dynamics is sensitive. We then argue based on the criterion and the gross electrophysiology of the cortex that the dynamics of cortical networks ought to be almost surely sensitive under conditions typically found there. We supplement this with two examples of networks modeling cortical columns with widely differing qualitative dynamics, yet with both exhibiting sensitive dependence. Next, we use the criterion to construct a network that undergoes bifurcation from sensitive dynamics to insensitive dynamics when the value of a control parameter is varied. Finally, we extend the formal result to networks driven by stationary input spike trains, deriving a superior criterion than previously reported. Action Editor: John Rinzel     

An STDP training algorithm for a spiking neural network with dynamic threshold neurons

This paper proposes a supervised training algorithm for Spiking Neural Networks (SNNs) which modifies the Spike Timing Dependent Plasticity (STDP)learning rule to support both local and network level training with multiple synaptic connections and axonal delays. The training algorithm applies the rule to two and three layer SNNs, and is benchmarked using the Iris and Wisconsin Breast Cancer (WBC) data sets. The effectiveness of hidden layer dynamic threshold neurons is also investigated and results are presented.     

A computational model of visual marking using an inter-connected network of spiking neurons: the spiking search over time & space model (sSoTS).

In the real world, visual information is selected over time as well as space, when we prioritise new stimuli for attention. Watson and Humphreys [Watson, D., Humphreys, G.W., 1997. Visual marking: prioritizing selection for new objects by top-down attentional inhibition of old objects. Psychological Review 104, 90-122] presented evidence that new information in search tasks is prioritised by (amongst other processes) active ignoring of old items - a process they termed visual marking. In this paper we present, for the first time, an explicit computational model of visual marking using biologically plausible activation functions. The "spiking search over time and space" model (sSoTS) incorporates different synaptic components (NMDA, AMPA, GABA) and a frequency adaptation mechanism based on [Ca(2+)] sensitive K(+) current. This frequency adaptation current can act as a mechanism that suppresses the previously attended items. We show that, when coupled with a process of active inhibition applied to old items, frequency adaptation leads to old items being de-prioritised (and new items prioritised) across time in search. Furthermore, the time course of these processes mimics the time course of the preview effect in human search. The results indicate that the sSoTS model can provide a biologically plausible account of human search over time as well as space.     

A neural network model of attention-modulated neurodynamics

Visual attention appears to modulate cortical neurodynamics and synchronization through various cholinergic mechanisms. In order to study these mechanisms, we have developed a neural network model of visual cortex area V4, based on psychophysical, anatomical and physiological data. With this model, we want to link selective visual information processing to neural circuits within V4, bottom-up sensory input pathways, top-down attention input pathways, and to cholinergic modulation from the prefrontal lobe. We investigate cellular and network mechanisms underlying some recent analytical results from visual attention experimental data. Our model can reproduce the experimental findings that attention to a stimulus causes increased gamma-frequency synchronization in the superficial layers. Computer simulations and STA power analysis also demonstrate different effects of the different cholinergic attention modulation action mechanisms.     

Testing of information condensation in a model reverberating spiking neural network

Information about external world is delivered to the brain in the form of structured in time spike trains. During further processing in higher areas, information is subjected to a certain condensation process, which results in formation of abstract conceptual images of external world, apparently, represented as certain uniform spiking activity partially independent on the input spike trains details. Possible physical mechanism of condensation at the level of individual neuron was discussed recently. In a reverberating spiking neural network, due to this mechanism the dynamics should settle down to the same uniform/ periodic activity in response to a set of various inputs. Since the same periodic activity may correspond to different input spike trains, we interpret this as possible candidate for information condensation mechanism in a network. Our purpose is to test this possibility in a network model consisting of five fully connected neurons, particularly, the influence of geometric size of the network, on its ability to condense information. Dynamics of 20 spiking neural networks of different geometric sizes are modelled by means of computer simulation. Each network was propelled into reverberating dynamics by applying various initial input spike trains. We run the dynamics until it becomes periodic. The Shannon's formula is used to calculate the amount of information in any input spike train and in any periodic state found. As a result, we obtain explicit estimate of the degree of information condensation in the networks, and conclude that it depends strongly on the net's geometric size.     

Geometric properties of a class of piecewise affine biological network models

The purpose of this report is to investigate some dynamical properties common to several biological systems. A model is chosen, which consists of a system of piecewise affine differential equations. Such a model has been previously studied in the context of gene regulation and neural networks, as well as biochemical kinetics. Unlike most of these studies, nonuniform decay rates and several thresholds per variable are assumed, thus considering a more realistic model. This model is investigated with the aid of a geometric formalism. We first provide an analysis of a continuous-space, discrete-time dynamical system equivalent to the initial one, by the way of a transition map. This is similar to former studies. Especially, the analysis of periodic trajectories is carried out in the case of multiple thresholds, thus extending previous results, which all concerned the restricted case of binary systems. The piecewise affine structure of such models is then used to provide a partition of the phase space, in terms of explicit cells. Allowed transitions between these cells define a language on a finite alphabet. Some words are proved to be forbidden in this language, thus improving the knowledge on such systems in terms of symbolic dynamics. More precisely, we show that taking these forbidden words into account leads to a dynamical system with strictly lower topological entropy. This holds for a class of systems, characterized by the presence of a splitting box, with additional conditions. We conclude after an illustrative three-dimensional example.     

A linear model for characterization of synchronization frequencies of neural networks

The synchronization frequency of neural networks and its dynamics have important roles in deciphering the working mechanisms of the brain. It has been widely recognized that the properties of functional network synchronization and its dynamics are jointly determined by network topology, network connection strength, i.e., the connection strength of different edges in the network, and external input signals, among other factors. However, mathematical and computational characterization of the relationships between network synchronization frequency and these three important factors are still lacking. This paper presents a novel computational simulation framework to quantitatively characterize the relationships between neural network synchronization frequency and network attributes and input signals. Specifically, we constructed a series of neural networks including simulated small-world networks, real functional working memory network derived from functional magnetic resonance imaging, and real large-scale structural brain networks derived from diffusion tensor imaging, and performed synchronization simulations on these networks via the Izhikevich neuron spiking model. Our experiments demonstrate that both of the network synchronization strength and synchronization frequency change according to the combination of input signal frequency and network self-synchronization frequency. In particular, our extensive experiments show that the network synchronization frequency can be represented via a linear combination of the network self-synchronization frequency and the input signal frequency. This finding could be attributed to an intrinsically-preserved principle in different types of neural systems, offering novel insights into the working mechanism of neural systems.     

Directional operations in the motor cortex modeled by a neural network of spiking neurons

A neural network with realistically modeled, spiking neurons is proposed to model ensemble operations of directionally tuned neurons in the motor cortex. The model reproduces well directional operations previously identified experimentally, including the prediction of the direction of an upcoming movement in reaching tasks and the rotation of the neuronal population vector in a directional transformation task.     

A spiking neural network model of the midbrain superior colliculus that generates saccadic motor commands

Single-unit recordings suggest that the midbrain superior colliculus (SC) acts as an optimal controller for saccadic gaze shifts. The SC is proposed to be the site within the visuomotor system where the nonlinear spatial-to-temporal transformation is carried out: the population encodes the intended saccade vector by its location in the motor map (spatial), and its trajectory and velocity by the distribution of firing rates (temporal). The neurons’ burst profiles vary systematically with their anatomical positions and intended saccade vectors, to account for the nonlinear main-sequence kinematics of saccades. Yet, the underlying collicular mechanisms that could result in these firing patterns are inaccessible to current neurobiological techniques. Here, we propose a simple spiking neural network model that reproduces the spike trains of saccade-related cells in the intermediate and deep SC layers during saccades. The model assumes that SC neurons have distinct biophysical properties for spike generation that depend on their anatomical position in combination with a center–surround lateral connectivity. Both factors are needed to account for the observed firing patterns. Our model offers a basis for neuronal algorithms for spatiotemporal transformations and bio-inspired optimal controllers.     

Simulander: A neural network model for the orientation movement of salamanders

Simulander is a feedforward neural network simulating the orientation movement of salamanders. The orientation movement is part of the prey capture behavior; it is performed with the head alone. Simulander is a network which consists of 300 neurons incorporating several cytoarchitectonic and electrophysiological features of the salamander brain. The network is trained by means of an evolution strategy. Although only 100 tectum neurons with fairly large receptive fields are used (coarse coding), Simulander is able to localize an irregularly moving prey precisely. It is demonstrated that large receptive field neurons are important for successful prey localization. The removal of a model tectum hemisphere leads to a network which accounts for investigations made in living monocular salamanders. The model also yields an understanding of electrical stimulation experiments in toads.     

Travelling waves and EEG patterns during epileptic seizure: analysis with an integrate-and-fire neural network

Epilepsy is characterized by paradoxical patterns of neural activity. They may cause different types of electroencephalogram (EEG), which dynamically change in shape and frequency content during the temporal evolution of seizure. It is generally assumed that these epileptic patterns may originate in a network of strongly interconnected neurons, when excitation dominates over inhibition. The aim of this work is to use a neural network composed of 50 x 50 integrate-and-fire neurons to analyse which parameter alterations, at the level of synapse topology, may induce network instability and epileptic-like discharges, and to study the corresponding spatio-temporal characteristics of electrical activity in the network. We assume that a small group of central neurons is stimulated by a depolarizing current (epileptic focus) and that neurons are connected via a Mexican-hat topology of synapses. A signal representative of cortical EEG (ECoG) is simulated by summing the membrane potential changes of all neurons. A sensitivity analysis on the parameters describing the synapse topology shows that an increase in the strength and in spatial extension of excitatory vs. inhibitory synapses may cause the occurrence of travelling waves, which propagate along the network. These propagating waves may cause EEG patterns with different shape and frequency, depending on the particular parameter set used during the simulations. The resulting model EEG signals include irregular rhythms with large amplitude and a wide frequency content, low-amplitude high-frequency rapid discharges, isolated or repeated bursts, and low-frequency quasi-sinusoidal patterns. A slow progressive temporal variation in a single parameter may cause the transition from one pattern to another, thus generating a highly non-stationary signal which resembles that observed during ECoG measurements. These results may help to elucidate the mechanisms at the basis of some epileptic discharges, and to relate rapid changes in EEG patterns with the underlying alterations at the network level.     

Robust emergence of small-world structure in networks of spiking neurons

Spontaneous activity in biological neural networks shows patterns of dynamic synchronization. We propose that these patterns support the formation␣of a small-world structure—network connectivity␣optimal for distributed information processing. We␣present numerical simulations with connected Hindmarsh–Rose neurons in which, starting from random connection distributions, small-world networks evolve as a result of applying an adaptive rewiring rule. The rule connects pairs of neurons that tend fire in synchrony, and disconnects ones that fail to synchronize. Repeated application of the rule leads to small-world structures. This mechanism is robustly observed for bursting and irregular firing regimes.     

Computer control of pH and DO in a laboratory fermenter using a neural network technique

In this contribution, the advantages of the artificial neural network approach to the identification and control of a laboratory-scale biochemical reactor are demonstrated. It is very important to be able to maintain the levels of two process variables, pH and dissolved oxygen (DO) concentration, over the course of fermentation in biosystems control. A PC-supported, fully automated, multi-task control system has been designed and built by the authors. Forward and inverse neural process models are used to identify and control both the pH and the DO concentration in a fermenter containing a Saccharomyces cerevisiae based-culture. The models are trained off-line, using a modified back-propagation algorithm based on conjugate gradients. The inverse neural controller is augmented by a new adaptive term that results in a system with robust performance. Experimental results have confirmed that the regulatory and tracking performances of the control system proposed are good.     

Monotone and near-monotone biochemical networks

Monotone subsystems have appealing properties as components of larger networks, since they exhibit robust dynamical stability and predictability of responses to perturbations. This suggests that natural biological systems may have evolved to be, if not monotone, at least close to monotone in the sense of being decomposable into a “small” number of monotone components, In addition, recent research has shown that much insight can be attained from decomposing networks into monotone subsystems and the analysis of the resulting interconnections using tools from control theory. This paper provides an expository introduction to monotone systems and their interconnections, describing the basic concepts and some of the main mathematical results in a largely informal fashion. Supported in part by NSF Grants DMS-0504557 and DMS-0614371.     

A model for altered neural network dynamics related to prehension movements in Parkinson disease

In this paper, we present a neural network model of the interactions between cortex and the basal ganglia during prehensile movements. Computational neuroscience methods are used to explore the hypothesis that the altered kinematic patterns observed in Parkinson’s disease patients performing prehensile movements is mainly due to an altered neuronal activity located in the networks of cholinergic (ACh) interneurons of the striatum. These striatal cells, under a strong influence of the dopaminergic system, significantly contribute to the neural processing within the striatum and in the cortico-basal ganglia loops. In order to test this hypothesis, a large-scale model of neural interactions in the basal ganglia has been integrated with previous models accounting for the cortical organization of goal directed reaching and grasping movements in normal and perturbed conditions. We carry out a discussion of the model hypothesis validation by providing a control engineering analysis and by comparing results of real experiments with our simulation results in conditions resembling these original experiments.     

A case for spiking neural network simulation based on configurable multiple-FPGA systems

Recent neuropsychological research has begun to reveal that neurons encode information in the timing of spikes. Spiking neural network simulations are a flexible and powerful method for investigating the behaviour of neuronal systems. Simulation of the spiking neural networks in software is unable to rapidly generate output spikes in large-scale of neural network. An alternative approach, hardware implementation of such system, provides the possibility to generate independent spikes precisely and simultaneously output spike waves in real time, under the premise that spiking neural network can take full advantage of hardware inherent parallelism. We introduce a configurable FPGA-oriented hardware platform for spiking neural network simulation in this work. We aim to use this platform to combine the speed of dedicated hardware with the programmability of software so that it might allow neuroscientists to put together sophisticated computation experiments of their own model. A feed-forward hierarchy network is developed as a case study to describe the operation of biological neural systems (such as orientation selectivity of visual cortex) and computational models of such systems. This model demonstrates how a feed-forward neural network constructs the circuitry required for orientation selectivity and provides platform for reaching a deeper understanding of the primate visual system. In the future, larger scale models based on this framework can be used to replicate the actual architecture in visual cortex, leading to more detailed predictions and insights into visual perception phenomenon.