Fast and robust image segmentation by small-world neural oscillator networks

Inspired by the temporal correlation theory of brain functions, researchers have presented a number of neural oscillator networks to implement visual scene segmentation problems. Recently, it is shown that many biological neural networks are typical small-world networks. In this paper, we propose and investigate two small-world models derived from the well-known LEGION (locally excitatory and globally inhibitory oscillator network) model. To form a small-world network, we add a proper proportion of unidirectional shortcuts (random long-range connections) to the original LEGION model. With local connections and shortcuts, the neural oscillators can not only communicate with neighbors but also exchange phase information with remote partners. Model 1 introduces excitatory shortcuts to enhance the synchronization within an oscillator group representing the same object. Model 2 goes further to replace the global inhibitor with a sparse set of inhibitory shortcuts. Simulation results indicate that the proposed small-world models could achieve synchronization faster than the original LEGION model and are more likely to bind disconnected image regions belonging together. In addition, we argue that these two models are more biologically plausible.     

Robust emergence of small-world structure in networks of spiking neurons

Spontaneous activity in biological neural networks shows patterns of dynamic synchronization. We propose that these patterns support the formation␣of a small-world structure—network connectivity␣optimal for distributed information processing. We␣present numerical simulations with connected Hindmarsh–Rose neurons in which, starting from random connection distributions, small-world networks evolve as a result of applying an adaptive rewiring rule. The rule connects pairs of neurons that tend fire in synchrony, and disconnects ones that fail to synchronize. Repeated application of the rule leads to small-world structures. This mechanism is robustly observed for bursting and irregular firing regimes.     

Inhibition of rhythmic spiking by colored noise in neural systems

We study the effect of colored noise on the rhythmic spiking activity of neural systems in this paper. The phenomenon of the so-called inverse stochastic resonance , that is, noise with appropriate intensity suppresses the spiking activity in neural systems, is clearly observed in a special parameter regime. We find that the inhibition effect of colored noise is stronger than that of Gaussian white noise. Furthermore, our simulation results show that the inhibition effect of colored noise provides a useful mechanism for the generation of synchronized burst in type-2 mixed-feed-forward-feedback loop neuronal network motif, which indicates that such inhibition effect might have some biological implications.     

Voltage-stepping schemes for the simulation of spiking neural networks

The numerical simulation of spiking neural networks requires particular attention. On the one hand, time-stepping methods are generic but they are prone to numerical errors and need specific treatments to deal with the discontinuities of integrate-and-fire models. On the other hand, event-driven methods are more precise but they are restricted to a limited class of neuron models. We present here a voltage-stepping scheme that combines the advantages of these two approaches and consists of a discretization of the voltage state-space. The numerical simulation is reduced to a local event-driven method that induces an implicit activity-dependent time discretization (time-steps automatically increase when the neuron is slowly varying). We show analytically that such a scheme leads to a high-order algorithm so that it accurately approximates the neuronal dynamics. The voltage-stepping method is generic and can be used to simulate any kind of neuron models. We illustrate it on nonlinear integrate-and-fire models and show that it outperforms time-stepping schemes of Runge-Kutta type in terms of simulation time and accuracy.

Interactions between neural networks: a mechanism for tuning chaos and oscillations

We show that chaos and oscillations in a higher-order binary neural network can be tuned effectively using interactions between neural networks. Our results suggest that network interactions may be useful as a means of adjusting the level of dynamic activities in systems that employ chaos and oscillations for information processing, or as a means of suppressing oscillatory behaviors in systems that require stability. URL: http:// www.ntu.edu.sg/home/elpwang     

A stochastic population approach to the problem of stable recruitment hierarchies in spiking neural networks

Synchrony-driven recruitment learning addresses the question of how arbitrary concepts, represented by synchronously active ensembles, may be acquired within a randomly connected static graph of neuron-like elements. Recruitment learning in hierarchies is an inherently unstable process. This paper presents conditions on parameters for a feedforward network to ensure stable recruitment hierarchies. The parameter analysis is conducted by using a stochastic population approach to model a spiking neural network. The resulting network converges to activate a desired number of units at each stage of the hierarchy. The original recruitment method is modified first by increasing feedforward connection density for ensuring sufficient activation, then by incorporating temporally distributed feedforward delays for separating inputs temporally, and finally by limiting excess activation via lateral inhibition. The task of activating a desired number of units from a population is performed similarly to a temporal k-winners-take-all network.     

Dendritic synchrony and transient dynamics in a coupled oscillator model of the dopaminergic neuron

Transient increases in spontaneous firing rate of mesencephalic dopaminergic neurons have been suggested to act as a reward prediction error signal. A mechanism previously proposed involves subthreshold calcium-dependent oscillations in all parts of the neuron. In that mechanism, the natural frequency of oscillation varies with diameter of cell processes, so there is a wide variation of natural frequencies on the cell, but strong voltage coupling enforces a single frequency of oscillation under resting conditions. In previous work, mathematical analysis of a simpler system of oscillators showed that the chain of oscillators could produce transient dynamics in which the frequency of the coupled system increased temporarily, as seen in a biophysical model of the dopaminergic neuron. The transient dynamics was shown to be consequence of a slow drift along an invariant subset of phase space, with rate of drift given by a Lyapunov function. In this paper, we show that the same mathematical structure exists for the full biophysical model, giving physiological meaning to the slow drift and the Lyapunov function, which is shown to describe differences in intracellular calcium concentration in different parts of the cell. The duration of transients was long, being comparable to the time constant of calcium disposition. These results indicate that brief changes in input to the dopaminergic neuron can produce long lasting firing rate transients whose form is determined by intrinsic cell properties.     

Learning and coding of concepts in neural networks

Our environment consists of virtually an infinite number of scenarios in which we have to function. In order to respond properly to an incoming stimulus, the brain has first to analyze it, and to find out the basic familiar elements that are part of it. In other words, by using a library which contains a relatively small number of basic concepts, the brain analyzes the multitude of incoming events. Some of those basic concepts are innate, but many of them must be learned, in order to accommodate for the arbitrary environment around us. A classifying box is defined as the neural network that finds out the familiar concepts that are present in an incoming stimulus. Models for classifying boxes are introduced, and possible mechanisms by which they may establish their libraries of concepts are suggested, and then compared and evaluated by computer simulations.     

Probabilistic polynomial dynamical systems for reverse engineering of gene regulatory networks

Elucidating the structure and/or dynamics of gene regulatory networks from experimental data is a major goal of systems biology. Stochastic models have the potential to absorb noise, account for un-certainty, and help avoid data overfitting. Within the frame work of probabilistic polynomial dynamical systems, we present an algorithm for the reverse engineering of any gene regulatory network as a discrete, probabilistic polynomial dynamical system. The resulting stochastic model is assembled from all minimal models in the model space and the probability assignment is based on partitioning the model space according to the likeliness with which a minimal model explains the observed data. We used this method to identify stochastic models for two published synthetic network models. In both cases, the generated model retains the key features of the original model and compares favorably to the resulting models from other algorithms.     

Timing as an intrinsic property of neural networks: evidence from in vivo and in vitro experiments

The discrimination and production of temporal patterns on the scale of hundreds of milliseconds are critical to sensory and motor processing. Indeed, most complex behaviours, such as speech comprehension and production, would be impossible in the absence of sophisticated timing mechanisms. Despite the importance of timing to human learning and cognition, little is known about the underlying mechanisms, in particular whether timing relies on specialized dedicated circuits and mechanisms or on general and intrinsic properties of neurons and neural circuits. Here, we review experimental data describing timing and interval-selective neurons in vivo and in vitro. We also review theoretical models of timing, focusing primarily on the state-dependent network model, which proposes that timing in the subsecond range relies on the inherent time-dependent properties of neurons and the active neural dynamics within recurrent circuits. Within this framework, time is naturally encoded in populations of neurons whose pattern of activity is dynamically changing in time. Together, we argue that current experimental and theoretical studies provide sufficient evidence to conclude that at least some forms of temporal processing reflect intrinsic computations based on local neural network dynamics.     

Deterministic convergence of chaos injection-based gradient method for training feedforward neural networks

It has been shown that, by adding a chaotic sequence to the weight update during the training of neural networks, the chaos injection-based gradient method (CIBGM) is superior to the standard backpropagation algorithm. This paper presents the theoretical convergence analysis of CIBGM for training feedforward neural networks. We consider both the case of batch learning as well as the case of online learning. Under mild conditions, we prove the weak convergence, i.e., the training error tends to a constant and the gradient of the error function tends to zero. Moreover, the strong convergence of CIBGM is also obtained with the help of an extra condition. The theoretical results are substantiated by a simulation example.     

Geometric properties of a class of piecewise affine biological network models

The purpose of this report is to investigate some dynamical properties common to several biological systems. A model is chosen, which consists of a system of piecewise affine differential equations. Such a model has been previously studied in the context of gene regulation and neural networks, as well as biochemical kinetics. Unlike most of these studies, nonuniform decay rates and several thresholds per variable are assumed, thus considering a more realistic model. This model is investigated with the aid of a geometric formalism. We first provide an analysis of a continuous-space, discrete-time dynamical system equivalent to the initial one, by the way of a transition map. This is similar to former studies. Especially, the analysis of periodic trajectories is carried out in the case of multiple thresholds, thus extending previous results, which all concerned the restricted case of binary systems. The piecewise affine structure of such models is then used to provide a partition of the phase space, in terms of explicit cells. Allowed transitions between these cells define a language on a finite alphabet. Some words are proved to be forbidden in this language, thus improving the knowledge on such systems in terms of symbolic dynamics. More precisely, we show that taking these forbidden words into account leads to a dynamical system with strictly lower topological entropy. This holds for a class of systems, characterized by the presence of a splitting box, with additional conditions. We conclude after an illustrative three-dimensional example.     

Synchronization and stochastic resonance of the small-world neural network based on the CPG

According to biological knowledge, the central nervous system controls the central pattern generator (CPG) to drive the locomotion. The brain is a complex system consisting of different functions and different interconnections. The topological properties of the brain display features of small-world network. The synchronization and stochastic resonance have important roles in neural information transmission and processing. In order to study the synchronization and stochastic resonance of the brain based on the CPG, we establish the model which shows the relationship between the small-world neural network (SWNN) and the CPG. We analyze the synchronization of the SWNN when the amplitude and frequency of the CPG are changed and the effects on the CPG when the SWNN’s parameters are changed. And we also study the stochastic resonance on the SWNN. The main findings include: (1) When the CPG is added into the SWNN, there exists parameters space of the CPG and the SWNN, which can make the synchronization of the SWNN optimum. (2) There exists an optimal noise level at which the resonance factor Q gets its peak value. And the correlation between the pacemaker frequency and the dynamical response of the network is resonantly dependent on the noise intensity. The results could have important implications for biological processes which are about interaction between the neural network and the CPG.     

Non-Gaussian noise optimized spiking activity of Hodgkin–Huxley neurons on random complex networks

In this paper, we numerically study how the NGN's deviation q from Gaussian noise (q = 1) affects the spike coherence and synchronization of 60 coupled Hodgkin–Huxley (HH) neurons driven by a periodic sinusoidal stimulus on random complex networks. It is found that the effect of the deviation depends on the network randomness p (the fraction of random shortcuts): for larger p (p > 0.15), the spiking regularity keeps being improved with increasing q; while, for smaller p (p < 0.15), the spiking regularity can reach the best performance at an optimal intermediate q value, indicating the occurrence of “deviation-optimized spike coherence”. The synchronization becomes enhanced with decreasing q, and the enhancing extent for a random HH neuron network is stronger than for a regular one. These behaviors show that the spike coherence and synchronization of the present HH neurons on random networks can be more strongly enhanced by various other types of external noise than by Gaussian noise, whereby the neuron firings may behave more periodically in time and more synchronously in space. Our results provide the constructive roles of the NGN on the spiking activity of the present system of HH neuron networks.     

Regulation of rhythm genesis by volume-limited,astroglia-like signals in neural networks

Rhythmic activity of the brain often depends on synchronized spiking of interneuronal networks interacting with principal neurons. The quest for physiological mechanisms regulating network synchronization has therefore been firmly focused on synaptic circuits. However, it has recently emerged that synaptic efficacy could be influenced by astrocytes that release signalling molecules into their macroscopic vicinity. To understand how this volume-limited synaptic regulation can affect oscillations in neural populations, here we explore an established artificial neural network mimicking hippocampal basket cells receiving inputs from pyramidal cells. We find that network oscillation frequencies and average cell firing rates are resilient to changes in excitatory input even when such changes occur in a significant proportion of participating interneurons, be they randomly distributed or clustered in space. The astroglia-like, volume-limited regulation of excitatory synaptic input appears to better preserve network synchronization (compared with a similar action evenly spread across the network) while leading to a structural segmentation of the network into cell subgroups with distinct firing patterns. These observations provide us with some previously unknown insights into the basic principles of neural network control by astroglia.     

Simulated neural dynamics of decision-making in an auditory delayed match-to-sample task

An important goal of research on the cognitive neuroscience of decision-making is to produce a comprehensive model of behavior that flows from perception to action with all of the intermediate steps defined. To understand the mechanisms of perceptual decision-making for an auditory discrimination experiment, we connected a large-scale, neurobiologically realistic auditory pattern recognition model to a three-layer decision-making model and simulated an auditory delayed match-to-sample (DMS) task. In each trial of our simulated DMS task, pairs of stimuli were compared each stimulus being a sequence of three frequency-modulated tonal-contour segments, and a "match" or "nonmatch" button was pressed. The model's simulated response times and the different patterns of neural responses (transient, sustained, increasing) are consistent with experimental data and the simulated neurophysiological activity provides insights into the neural interactions from perception to action in the auditory DMS task.     

Novel tracking function of moving target using chaotic dynamics in a recurrent neural network model

Chaotic dynamics introduced in a recurrent neural network model is applied to controlling an object to track a moving target in two-dimensional space, which is set as an ill-posed problem. The motion increments of the object are determined by a group of motion functions calculated in real time with firing states of the neurons in the network. Several cyclic memory attractors that correspond to several simple motions of the object in two-dimensional space are embedded. Chaotic dynamics introduced in the network causes corresponding complex motions of the object in two-dimensional space. Adaptively real-time switching of control parameter results in constrained chaos (chaotic itinerancy) in the state space of the network and enables the object to track a moving target along a certain trajectory successfully. The performance of tracking is evaluated by calculating the success rate over 100 trials with respect to nine kinds of trajectories along which the target moves respectively. Computer experiments show that chaotic dynamics is useful to track a moving target. To understand the relations between these cases and chaotic dynamics, dynamical structure of chaotic dynamics is investigated from dynamical viewpoint.