Stability analysis of memristor-based fractional-order neural networks with different memductance functions

Cognitive Neurodynamics - In this paper, the problem of the existence, uniqueness and uniform stability of memristor-based fractional-order neural networks (MFNNs) with two different types of...     

Stability and synchronization analysis of inertial memristive neural networks with time delays

This paper is concerned with the problem of stability and pinning synchronization of a class of inertial memristive neural networks with time delay. In contrast to general inertial neural networks, inertial memristive neural networks is applied to exhibit the synchronization and stability behaviors due to the physical properties of memristors and the differential inclusion theory. By choosing an appropriate variable transmission, the original system can be transformed into first order differential equations. Then, several sufficient conditions for the stability of inertial memristive neural networks by using matrix measure and Halanay inequality are derived. These obtained criteria are capable of reducing computational burden in the theoretical part. In addition, the evaluation is done on pinning synchronization for an array of linearly coupled inertial memristive neural networks, to derive the condition using matrix measure strategy. Finally, the two numerical simulations are presented to show the effectiveness of acquired theoretical results.     

Global gene expression analysis by combinatorial optimization

Generally, there is a trade-off between methods of gene expression analysis that are precise but labor-intensive, e.g. RT-PCR, and methods that scale up to global coverage but are not quite as quantitative, e.g. microarrays. In the present paper, we show how how a known method of gene expression profiling (K. Kato, Nucleic Acids Res. 23, 3685-3690 (1995)), which relies on a fairly small number of steps, can be turned into a global gene expression measurement by advanced data post-processing, with potentially little loss of accuracy. Post-processing here entails solving an ancillary combinatorial optimization problem. Validation is performed on in silico experiments generated from the FANTOM data base of full-length mouse cDNA. We present two variants of the method. One uses state-of-the-art commercial software for solving problems of this kind, the other a code developed by us specifically for this purpose, released in the public domain under GPL license.     

Model‐guided combinatorial optimization of complex synthetic gene networks

Constructing gene circuits that satisfy quantitative performance criteria has been a long‐standing challenge in synthetic biology. Here, we show a strategy for optimizing a complex three‐gene circuit, a novel proportional miRNA biosensor, using predictive modeling to initiate a search in the phase space of sensor genetic composition. We generate a library of sensor circuits using diverse genetic building blocks in order to access favorable parameter combinations and uncover specific genetic compositions with greatly improved dynamic range. The combination of high‐throughput screening data and the data obtained from detailed mechanistic interrogation of a small number of sensors was used to validate the model. The validated model facilitated further experimentation, including biosensor reprogramming and biosensor integration into larger networks, enabling in principle arbitrary logic with miRNA inputs using normal form circuits. The study reveals how model‐guided generation of genetic diversity followed by screening and model validation can be successfully applied to optimize performance of complex gene networks without extensive prior knowledge.     

On the probabilistic optimization of spiking neural networks

The construction of a Spiking Neural Network (SNN), i.e. the choice of an appropriate topology and the configuration of its internal parameters, represents a great challenge for SNN based applications. Evolutionary Algorithms (EAs) offer an elegant solution for these challenges and methods capable of exploring both types of search spaces simultaneously appear to be the most promising ones. A variety of such heterogeneous optimization algorithms have emerged recently, in particular in the field of probabilistic optimization. In this paper, a literature review on heterogeneous optimization algorithms is presented and an example of probabilistic optimization of SNN is discussed in detail. The paper provides an experimental analysis of a novel Heterogeneous Multi-Model Estimation of Distribution Algorithm (hMM-EDA). First, practical guidelines for configuring the method are derived and then the performance of hMM-EDA is compared to state-of-the-art optimization algorithms. Results show hMM-EDA as a light-weight, fast and reliable optimization method that requires the configuration of only very few parameters. Its performance on a synthetic heterogeneous benchmark problem is highly competitive and suggests its suitability for the optimization of SNN.     

Activity analysis of neural networks

In a series of articles (Leung et al., 1973, 1974; Ogztöreli, 1972, 1975, 1978, 1979; Stein et al., 1974) we have investigated some of the physiologically significant properties of a general neural model. In these papers the nature of the oscillations occuring in the model has been briefly analyzed by omitting the effects of the discrete time-lags in the interaction of neurons, although these time-lags were incoporated in the general model. In the present work we investigate the effects of the time-lags on the oscillations which are intrinsic to the neural model, depending on the structural parameters such as external inputs, interaction coefficients, self-inhibition, self-excitation and selfadaptation coefficients. The numerical solution of the neural model, the computation of the steady-state solutions and the natural modes of the oscillations around the steady-state solutions are described.This work was partly supported by the Natural Sciences and Engineering Research Council of Canada under Grant NRC-A-4345 through the University of Alberta     

State-space analysis of time-varying higher-order spike correlation for multiple neural spike train data

Precise spike coordination between the spiking activities of multiple neurons is suggested as an indication of coordinated network activity in active cell assemblies. Spike correlation analysis aims to identify such cooperative network activity by detecting excess spike synchrony in simultaneously recorded multiple neural spike sequences. Cooperative activity is expected to organize dynamically during behavior and cognition; therefore currently available analysis techniques must be extended to enable the estimation of multiple time-varying spike interactions between neurons simultaneously. In particular, new methods must take advantage of the simultaneous observations of multiple neurons by addressing their higher-order dependencies, which cannot be revealed by pairwise analyses alone. In this paper, we develop a method for estimating time-varying spike interactions by means of a state-space analysis. Discretized parallel spike sequences are modeled as multi-variate binary processes using a log-linear model that provides a well-defined measure of higher-order spike correlation in an information geometry framework. We construct a recursive Bayesian filter/smoother for the extraction of spike interaction parameters. This method can simultaneously estimate the dynamic pairwise spike interactions of multiple single neurons, thereby extending the Ising/spin-glass model analysis of multiple neural spike train data to a nonstationary analysis. Furthermore, the method can estimate dynamic higher-order spike interactions. To validate the inclusion of the higher-order terms in the model, we construct an approximation method to assess the goodness-of-fit to spike data. In addition, we formulate a test method for the presence of higher-order spike correlation even in nonstationary spike data, e.g., data from awake behaving animals. The utility of the proposed methods is tested using simulated spike data with known underlying correlation dynamics. Finally, we apply the methods to neural spike data simultaneously recorded from the motor cortex of an awake monkey and demonstrate that the higher-order spike correlation organizes dynamically in relation to a behavioral demand.     

Classification of bacterial species from proteomic data using combinatorial approaches incorporating artificial neural networks, cluster analysis and principal components analysis

MOTIVATION: Robust computer algorithms are required to interpret the vast amounts of proteomic data currently being produced and to generate generalized models which are applicable to 'real world' scenarios. One such scenario is the classification of bacterial species. These vary immensely, some remaining remarkably stable whereas others are extremely labile showing rapid mutation and change. Such variation makes clinical diagnosis difficult and pathogens may be easily misidentified. RESULTS: We applied artificial neural networks (Neuroshell 2) in parallel with cluster analysis and principal components analysis to surface enhanced laser desorption/ionization (SELDI)-TOF mass spectrometry data with the aim of accurately identifying the bacterium Neisseria meningitidis from species within this genus and other closely related taxa. A subset of ions were identified that allowed for the consistent identification of species, classifying >97% of a separate validation subset of samples into their respective groups. AVAILABILITY: Neuroshell 2 is commercially available from Ward Systems.     

Stability of delayed memristive neural networks with time-varying impulses

This paper addresses the stability problem on the memristive neural networks with time-varying impulses. Based on the memristor theory and neural network theory, the model of the memristor-based neural network is established. Different from the most publications on memristive networks with fixed-time impulse effects, we consider the case of time-varying impulses. Both the destabilizing and stabilizing impulses exist in the model simultaneously. Through controlling the time intervals of the stabilizing and destabilizing impulses, we ensure the effect of the impulses is stabilizing. Several sufficient conditions for the globally exponentially stability of memristive neural networks with time-varying impulses are proposed. The simulation results demonstrate the effectiveness of the theoretical results.     

Artificial neural networks for solution scattering data analysis

1. Download : Download high-res image (142KB)
2. Download : Download full-size image

     

Dynamic neural networks as a tool for the online optimization of industrial fermentation

A system for online optimization of industrial fermentation based on a model with dynamic neural networks is described. The developed dynamic neural network, consisting of adapted neurons to consider the process dynamics, can model the complex, non-linear fermentation of beer in order to predict the future process. The predicted trajectories of gravity, pH, and diacetyl are in agreement with the experimental data measured at an automated pilot fermenter. It was possible to predict the future course of the batch fermentation as soon as 12 h of process data were available. In combination with the variational principle, the process model was used to optimize productivity. The temperature trajectory is optimized using a cost functional, including technical and technological conditions of the brewery in order to reduce the process time by steady product quality. The results show a reduction of the process time of up to 20%, which leads to an increase in utilization capacity.     

Genetic optimization of combinatorial libraries

Most agrochemical and pharmaceutical companies have set up high-throughput screening programs which require large numbers of compounds to screen. Combinatorial libraries provide an attractive way to deliver these compounds. A single combinatorial library with four variable positions can yield more than 10(12) potential compounds, if one assumes that about 1000 reagents are available for each position. This is far more than any high-throughput screening facility can afford to screen. We have proposed a method for iterative compound selection from large databases, which identifies the most active compounds by examining only a small fraction of the database. In this article, we describe the extension of this method to the problem of selecting compounds from large combinatorial libraries. Copyright 1998 John Wiley & Sons, Inc.     

The applicability of recurrent neural networks for biological sequence analysis

Selection of machine learning techniques requires a certain sensitivity to the requirements of the problem. In particular, the problem can be made more tractable by deliberately using algorithms that are biased toward solutions of the requisite kind. In this paper, we argue that recurrent neural networks have a natural bias toward a problem domain of which biological sequence analysis tasks are a subset. We use experiments with synthetic data to illustrate this bias. We then demonstrate that this bias can be exploitable using a data set of protein sequences containing several classes of subcellular localization targeting peptides. The results show that, compared with feed forward, recurrent neural networks will generally perform better on sequence analysis tasks. Furthermore, as the patterns within the sequence become more ambiguous, the choice of specific recurrent architecture becomes more critical.     

The complexity of linkage analysis with neural networks

As the focus of genome-wide scans for disease loci have shifted from simple Mendelian traits to genetically complex traits, researchers have begun to consider new alternative ways to detect linkage that will consider more than the marginal effects of a single disease locus at a time. One interesting new method is to train a neural network on a genome-wide data set in order to search for the best non-linear relationship between identity-by-descent sharing among affected siblings at markers and their disease status. We investigate here the repeatability of the neural network results from run to run, and show that the results obtained by multiple runs of the neural network method may differ quite a bit. This is most likely due to the fact that training a neural network involves minimizing an error function with a multitude of local minima.     

Stability of two cluster solutions in pulse coupled networks of neural oscillators

Phase response curves (PRCs) have been widely used to study synchronization in neural circuits comprised of pacemaking neurons. They describe how the timing of the next spike in a given spontaneously firing neuron is affected by the phase at which an input from another neuron is received. Here we study two reciprocally coupled clusters of pulse coupled oscillatory neurons. The neurons within each cluster are presumed to be identical and identically pulse coupled, but not necessarily identical to those in the other cluster. We investigate a two cluster solution in which all oscillators are synchronized within each cluster, but in which the two clusters are phase locked at nonzero phase with each other. Intuitively, one might expect this solution to be stable only when synchrony within each isolated cluster is stable, but this is not the case. We prove rigorously the stability of the two cluster solution and show how reciprocal coupling can stabilize synchrony within clusters that cannot synchronize in isolation. These stability results for the two cluster solution suggest a mechanism by which reciprocal coupling between brain regions can induce local synchronization via the network feedback loop.     

Cooperative recurrent modular neural networks for constrained optimization: a survey of models and applications

Constrained optimization problems arise in a wide variety of scientific and engineering applications. Since several single recurrent neural networks when applied to solve constrained optimization problems for real-time engineering applications have shown some limitations, cooperative recurrent neural network approaches have been developed to overcome drawbacks of these single recurrent neural networks. This paper surveys in details work on cooperative recurrent neural networks for solving constrained optimization problems and their engineering applications, and points out their standing models from viewpoint of both convergence to the optimal solution and model complexity. We provide examples and comparisons to shown advantages of these models in the given applications.     

Stability of dead zone bidirectional associative memory neural networks involving time delays

A mathematical model describing the dynamical interactions of bidirectional associative memory networks involving transmission delays is considered. The influence of a dead zone or a zone of noactivation on the global stability is investigated and various easily verifiable sets of sufficient conditions are established. The asymptotic nature of solutions when the given system of equations does not possess an equilibrium pattern is discussed.     

A combinatorial optimization approach for diverse motif finding applications

Background  

Discovering approximately repeated patterns, or motifs, in biological sequences is an important and widely-studied problem in computational molecular biology. Most frequently, motif finding applications arise when identifying shared regulatory signals within DNA sequences or shared functional and structural elements within protein sequences. Due to the diversity of contexts in which motif finding is applied, several variations of the problem are commonly studied.     

Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks

Recurrent neural networks (RNNs) are widely used in computational neuroscience and machine learning applications. In an RNN, each neuron computes its output as a nonlinear function of its integrated input. While the importance of RNNs, especially as models of brain processing, is undisputed, it is also widely acknowledged that the computations in standard RNN models may be an over-simplification of what real neuronal networks compute. Here, we suggest that the RNN approach may be made computationally more powerful by its fusion with Bayesian inference techniques for nonlinear dynamical systems. In this scheme, we use an RNN as a generative model of dynamic input caused by the environment, e.g. of speech or kinematics. Given this generative RNN model, we derive Bayesian update equations that can decode its output. Critically, these updates define a 'recognizing RNN' (rRNN), in which neurons compute and exchange prediction and prediction error messages. The rRNN has several desirable features that a conventional RNN does not have, e.g. fast decoding of dynamic stimuli and robustness to initial conditions and noise. Furthermore, it implements a predictive coding scheme for dynamic inputs. We suggest that the Bayesian inversion of RNNs may be useful both as a model of brain function and as a machine learning tool. We illustrate the use of the rRNN by an application to the online decoding (i.e. recognition) of human kinematics.