Dynamic connections in neural networks

Massively parallel (neural-like) networks are receiving increasing attention as a mechanism for expressing information processing models. By exploiting powerful primitive units and stability-preserving construction rules, various workers have been able to construct and test quite complex models, particularly in vision research. But all of the detailed technical work was concerned with the structure and behavior offixed networks. The purpose of this paper is to extend the methodology to cover several aspects of change and memory.     

Improved neuronal models for studying neural networks

Previous neuronal models used for the study of neural networks are considered. Equations are developed for a model which includes: 1) a normalized range of firing rates with decreased sensitivity at large excitatory or large inhibitory input levels, 2) a single rate constant for the increase in firing rate following step changes in the input, 3) one or more rate constants, as required to fit experimental data for the adaptation of firing rates to maintained inputs. Computed responses compare well with the types of neuronal responses observed experimentally. Depending on the parameters, overdamped increases and decreases, damped oscillatory or maintained oscillatory changes in firing rate are observed to step changes in the input. The integrodifferential equations describing the neuronal models can be represented by a set of first-order differential equations. Steady-state solutions for these equations can be obtained for constant inputs, as well as the stability of the solutions to small perturbations. The linear frequency response function is derived for sufficiently small time-varying inputs. The linear responses are also compared with the computed solutions for larger non-linear responses.     

Timing of neural commands: a model study with neuronal networks

Networks constructed of biologically realistic model neurons (neuroids) were used to study how in a neural assembly using pulse (interval)-coded information slow rhythmical oscillations with possible mode transitions might occur and how the efferent commands might be structured and their phase-shifts created. The simulations show that slow oscillations (in the hertz range) can be derived from reverberatory spiking in relatively short closed loops (fewer than ten neuroids) with the inputs protected against disturbing afferent signals and the outputs coupled by convergence on a common neuroid. Slow oscillations can be modified by a tonic activity entering the network; this activity changes the transmission time in the coupled loops involved. The structuring of the regulatory commands (in the millisecond range) was achieved by simulation of sequential activity propagation in a non-ring neuronal assembly supervised by a tonic activity in a set of inputs. The tonic activity acted as an instructive signal influencing the pattern of the functional connectivity in such a way that a particular efferent command was generated by the instructed network. Received: 3 March 1997 / Accepted in revised form: 15 July 1997     

Information characteristics of dual-layer neuronal networks with binary plastic synapses

Information properties of bilayer neuron nets which fulfil the function of hetero- or autoassociative memory were investigated. It was shown that with optimal choice of the net parameters its efficiency coefficient is close to the earlier calculated one for a single associative-neuron-like element, and the quality coefficient is near 1.     

Experimental analysis of neuronal dynamics in cultured cortical networks and transitions between different patterns of activity

A general method for developing data-based, stochastic nonlinear models of neurons by means of extended functional series expansions was applied to neural activities of pigeon auditory nerve fibers responding to Gaussian white noise stimuli. To determine Wiener series representations of the investigated neurons the fast orthogonal search algorithm was used. The results suggest that nonlinearities are only instantaneous and that the signal transduction of the investigated sensory system can be described by cascades of dynamic linear and static nonlinear devices. However, only slight improvements result from the nonlinear terms. Considerable improvements are, nevertheless, possible by generalizing the ordinary Wiener series, so that prior neural activity can be taken into account. These extended series were used to develop stochastic models of spiking neurons. The models are able to generate realistic interspike interval distributions and rate-intensity functions. Finally, it will be shown that the irregularity in real and modeled action potential trains has advantages concerning the decoding of neural responses. Received: 16 April 1996 / Accepted in revised form: 23 October 1996     

Reticular activation and the dynamics of neuronal networks

It is postulated that during arousal the cortical system is driven by a spatially and temporally noisy signal arising from non-specific reticulo-cortical pathways. An elementary unit of cortical neuroanatomy is assumed, which permits non-linear dynamics to be represented by stochastic linear equations. Under these assumptions the resonant modes of the system of cortical dendrites approach thermodynamic equilibrium. Specific sensory signals perturb the dendritic system about equilibrium, generate low frequency, linear, non-dispersive waves corresponding to the EEG, which in turn regulate action potential sequences, and instantiate internal inputs to the dendritic field. A large and distributed memory capacity in axo-synaptic couplings, resistance to interference between functionally separate logical operations, and a very large next-state function set emerge as properties of the network. The model is able to explain the close association of the EEG with cognition, the channel of low capacity corresponding to the field of immediate attention, the low overall correlation of action potentials with EEG, and specificity of action potentials in some neurons during particular cognitive activity. Predictions made from hypothesis include features of thermal equilibrium in EEG (determinable by autoregression) and expectation that the cortical evoked response can be accounted for as the response to a sensory impulse of specific time characteristics.     

Identifying controlling nodes in neuronal networks in different scales

Recent studies have detected hubs in neuronal networks using degree, betweenness centrality, motif and synchronization and revealed the importance of hubs in their structural and functional roles. In addition, the analysis of complex networks in different scales are widely used in physics community. This can provide detailed insights into the intrinsic properties of networks. In this study, we focus on the identification of controlling regions in cortical networks of cats' brain in microscopic, mesoscopic and macroscopic scales, based on single-objective evolutionary computation methods. The problem is investigated by considering two measures of controllability separately. The impact of the number of driver nodes on controllability is revealed and the properties of controlling nodes are shown in a statistical way. Our results show that the statistical properties of the controlling nodes display a concave or convex shape with an increase of the allowed number of controlling nodes, revealing a transition in choosing driver nodes from the areas with a large degree to the areas with a low degree. Interestingly, the community Auditory in cats' brain, which has sparse connections with other communities, plays an important role in controlling the neuronal networks.     

Calcium and synaptic dynamics underlying reverberatory activity in neuronal networks

Persistent activity is postulated to drive neural network plasticity and learning. To investigate its underlying cellular mechanisms, we developed a biophysically tractable model that explains the emergence, sustenance and eventual termination of short-term persistent activity. Using the model, we reproduced the features of reverberating activity that were observed in small (50-100 cells) networks of cultured hippocampal neurons, such as the appearance of polysynaptic current clusters, the typical inter-cluster intervals, the typical duration of reverberation, and the response to changes in extra-cellular ionic composition. The model relies on action potential-triggered residual pre-synaptic calcium, which we suggest plays an important role in sustaining reverberations. We show that reverberatory activity is maintained by enhanced asynchronous transmitter release from pre-synaptic terminals, which in itself depends on the dynamics of residual pre-synaptic calcium. Hence, asynchronous release, rather than being a 'synaptic noise', can play an important role in network dynamics. Additionally, we found that a fast timescale synaptic depression is responsible for oscillatory network activation during reverberations, whereas the onset of a slow timescale depression leads to the termination of reverberation. The simplicity of our model enabled a number of predictions that were confirmed by additional analyses of experimental manipulations.     

On the dynamics of the spontaneous activity in neuronal networks

Most neuronal networks, even in the absence of external stimuli, produce spontaneous bursts of spikes separated by periods of reduced activity. The origin and functional role of these neuronal events are still unclear. The present work shows that the spontaneous activity of two very different networks, intact leech ganglia and dissociated cultures of rat hippocampal neurons, share several features. Indeed, in both networks: i) the inter-spike intervals distribution of the spontaneous firing of single neurons is either regular or periodic or bursting, with the fraction of bursting neurons depending on the network activity; ii) bursts of spontaneous spikes have the same broad distributions of size and duration; iii) the degree of correlated activity increases with the bin width, and the power spectrum of the network firing rate has a 1/f behavior at low frequencies, indicating the existence of long-range temporal correlations; iv) the activity of excitatory synaptic pathways mediated by NMDA receptors is necessary for the onset of the long-range correlations and for the presence of large bursts; v) blockage of inhibitory synaptic pathways mediated by GABA(A) receptors causes instead an increase in the correlation among neurons and leads to a burst distribution composed only of very small and very large bursts. These results suggest that the spontaneous electrical activity in neuronal networks with different architectures and functions can have very similar properties and common dynamics.     

Some new stability properties of dynamic neural networks with different time-scales

Dynamic neural networks with different time-scales include the aspects of fast and slow phenomenons. Some applications require that the equilibrium points of these networks to be stable. The main contribution of the paper is that Lyapunov function and singularly perturbed technique are combined to access several new stable properties of different time-scales neural networks. Exponential stability and asymptotic stability are obtained by sector and bound conditions. Compared to other papers, these conditions are simpler. Numerical examples are given to demonstrate the effectiveness of the theoretical results.     

Stability analysis of memristor-based fractional-order neural networks with different memductance functions

Cognitive Neurodynamics - In this paper, the problem of the existence, uniqueness and uniform stability of memristor-based fractional-order neural networks (MFNNs) with two different types of...     

Clustering with neural networks

Partitioning a set ofN patterns in ad-dimensional metric space intoK clusters — in a way that those in a given cluster are more similar to each other than the rest — is a problem of interest in many fields, such as, image analysis, taxonomy, astrophysics, etc. As there are approximatelyK ^N/K! possible ways of partitioning the patterns amongK clusters, finding the best solution is beyond exhaustive search whenN is large. We show that this problem, in spite of its exponential complexity, can be formulated as an optimization problem for which very good, but not necessarily optimal, solutions can be found by using a Hopfield model of neural networks. To obtain a very good solution, the network must start from many randomly selected initial states. The network is simulated on the MPP, a 128 × 128 SIMD array machine, where we use the massive parallelism not only in solving the differential equations that govern the evolution of the network, but also in starting the network from many initial states at once thus obtaining many solutions in one run. We achieve speedups of two to three orders of magnitude over serial implementations and the promise through Analog VLSI implementations of further speedups of three to six orders of magnitude.Supported by a National Research Council-NASA Research Associatship     

Coping with variability in small neuronal networks

Experimental and corresponding modeling studies indicate that there is a 2- to 5-fold variation of intrinsic and synaptic parameters across animals while functional output is maintained. Here, we review experiments, using the heartbeat central pattern generator (CPG) in medicinal leeches, which explore the consequences of animal-to-animal variation in synaptic strength for coordinated motor output. We focus on a set of segmental heart motor neurons that all receive inhibitory synaptic input from the same four premotor interneurons. These four premotor inputs fire in a phase progression and the motor neurons also fire in a phase progression because of differences in synaptic strength profiles of the four inputs among segments. Our work tested the hypothesis that functional output is maintained in the face of animal-to-animal variation in the absolute strength of connections because relative strengths of the four inputs onto particular motor neurons is maintained across animals. Our experiments showed that relative strength is not strictly maintained across animals even as functional output is maintained, and animal-to-animal variations in strength of particular inputs do not correlate strongly with output phase. Further experiments measured the precise temporal pattern of the premotor inputs, the segmental synaptic strength profiles of their connections onto motor neurons, and the temporal pattern (phase progression) of those motor neurons all in the same animal for a series of 12 animals. The analysis of input and output in this sample of 12 individuals suggests that the number (four) of inputs to each motor neuron and the variability of the temporal pattern of input from the CPG across individuals weaken the influence of the strength of individual inputs. Moreover, the temporal pattern of the output varies as much across individuals as that of the input. Essentially, each animal arrives at a unique solution for how the network produces functional output.     

Passivity of memristor-based BAM neural networks with different memductance and uncertain delays

This paper addresses the passivity problem for a class of memristor-based bidirectional associate memory (BAM) neural networks with uncertain time-varying delays. In particular, the proposed memristive BAM neural networks is formulated with two different types of memductance functions. By constructing proper Lyapunov–Krasovskii functional and using differential inclusions theory, a new set of sufficient condition is obtained in terms of linear matrix inequalities which guarantee the passivity criteria for the considered neural networks. Finally, two numerical examples are given to illustrate the effectiveness of the proposed theoretical results.     

Pattern recognition in neural networks with competing dynamics: coexistence of fixed-point and cyclic attractors

We study the properties of the dynamical phase transition occurring in neural network models in which a competition between associative memory and sequential pattern recognition exists. This competition occurs through a weighted mixture of the symmetric and asymmetric parts of the synaptic matrix. Through a generating functional formalism, we determine the structure of the parameter space at non-zero temperature and near saturation (i.e., when the number of stored patterns scales with the size of the network), identifying the regions of high and weak pattern correlations, the spin-glass solutions, and the order-disorder transition between these regions. This analysis reveals that, when associative memory is dominant, smooth transitions appear between high correlated regions and spurious states. In contrast when sequential pattern recognition is stronger than associative memory, the transitions are always discontinuous. Additionally, when the symmetric and asymmetric parts of the synaptic matrix are defined in terms of the same set of patterns, there is a discontinuous transition between associative memory and sequential pattern recognition. In contrast, when the symmetric and asymmetric parts of the synaptic matrix are defined in terms of independent sets of patterns, the network is able to perform both associative memory and sequential pattern recognition for a wide range of parameter values.     

Global dynamics of a class of complex valued neural networks

In this paper activation dynamics of a complex valued neural network has been studied. Sufficient conditions for global exponential stability of a unique equilibrium are obtained. Our results show that in the serial mode of operation, the network converges to a stable state.     

Response of de- and hyperpolarizing effects in neuronal networks with different mechanisms for terminating activity

The response of stochastically organized homogeneous neuronal networks and others reacting by pre- and postsynaptic inhibition to external de- and hyperpolarizing effects of different intensity and time course were compared using a simulated mathematical computer model. The area of lasting depolarizing effects extended in those with pre- and postsynaptic inhibition compared with heterogeneous neuronal networks. Effects of brief de- and hyperpolarizing effects in a network depended on fluctuations in the level of the activity it displays; depolarizing effects may shorten and hyperpolarizing influences prolong the activity period of networks. The effects of a shortened network activity stage due to brief depolarizing influences was also more marked in networks with preand postsynaptic inhibition. Findings from this research would lead to the deduction that the presence of additional negative feedback circuits in the form of presynaptic or postsynaptic inhibitory series promotes better control in the mechanisms terminating network activity.A. A. Bogomolets Institute of Physiology, Academy of Sciences of the Ukrainian SSR, Kiev. Translated from Neirofiziologiya, Vol. 20, No. 4, pp. 438–447, July–August, 1988.