Stability analysis of memristor-based fractional-order neural networks with different memductance functions

Cognitive Neurodynamics - In this paper, the problem of the existence, uniqueness and uniform stability of memristor-based fractional-order neural networks (MFNNs) with two different types of...     

Stability and synchronization analysis of inertial memristive neural networks with time delays

This paper is concerned with the problem of stability and pinning synchronization of a class of inertial memristive neural networks with time delay. In contrast to general inertial neural networks, inertial memristive neural networks is applied to exhibit the synchronization and stability behaviors due to the physical properties of memristors and the differential inclusion theory. By choosing an appropriate variable transmission, the original system can be transformed into first order differential equations. Then, several sufficient conditions for the stability of inertial memristive neural networks by using matrix measure and Halanay inequality are derived. These obtained criteria are capable of reducing computational burden in the theoretical part. In addition, the evaluation is done on pinning synchronization for an array of linearly coupled inertial memristive neural networks, to derive the condition using matrix measure strategy. Finally, the two numerical simulations are presented to show the effectiveness of acquired theoretical results.     

Global gene expression analysis by combinatorial optimization

Generally, there is a trade-off between methods of gene expression analysis that are precise but labor-intensive, e.g. RT-PCR, and methods that scale up to global coverage but are not quite as quantitative, e.g. microarrays. In the present paper, we show how how a known method of gene expression profiling (K. Kato, Nucleic Acids Res. 23, 3685-3690 (1995)), which relies on a fairly small number of steps, can be turned into a global gene expression measurement by advanced data post-processing, with potentially little loss of accuracy. Post-processing here entails solving an ancillary combinatorial optimization problem. Validation is performed on in silico experiments generated from the FANTOM data base of full-length mouse cDNA. We present two variants of the method. One uses state-of-the-art commercial software for solving problems of this kind, the other a code developed by us specifically for this purpose, released in the public domain under GPL license.     

Model‐guided combinatorial optimization of complex synthetic gene networks

Constructing gene circuits that satisfy quantitative performance criteria has been a long‐standing challenge in synthetic biology. Here, we show a strategy for optimizing a complex three‐gene circuit, a novel proportional miRNA biosensor, using predictive modeling to initiate a search in the phase space of sensor genetic composition. We generate a library of sensor circuits using diverse genetic building blocks in order to access favorable parameter combinations and uncover specific genetic compositions with greatly improved dynamic range. The combination of high‐throughput screening data and the data obtained from detailed mechanistic interrogation of a small number of sensors was used to validate the model. The validated model facilitated further experimentation, including biosensor reprogramming and biosensor integration into larger networks, enabling in principle arbitrary logic with miRNA inputs using normal form circuits. The study reveals how model‐guided generation of genetic diversity followed by screening and model validation can be successfully applied to optimize performance of complex gene networks without extensive prior knowledge.     

On the probabilistic optimization of spiking neural networks

The construction of a Spiking Neural Network (SNN), i.e. the choice of an appropriate topology and the configuration of its internal parameters, represents a great challenge for SNN based applications. Evolutionary Algorithms (EAs) offer an elegant solution for these challenges and methods capable of exploring both types of search spaces simultaneously appear to be the most promising ones. A variety of such heterogeneous optimization algorithms have emerged recently, in particular in the field of probabilistic optimization. In this paper, a literature review on heterogeneous optimization algorithms is presented and an example of probabilistic optimization of SNN is discussed in detail. The paper provides an experimental analysis of a novel Heterogeneous Multi-Model Estimation of Distribution Algorithm (hMM-EDA). First, practical guidelines for configuring the method are derived and then the performance of hMM-EDA is compared to state-of-the-art optimization algorithms. Results show hMM-EDA as a light-weight, fast and reliable optimization method that requires the configuration of only very few parameters. Its performance on a synthetic heterogeneous benchmark problem is highly competitive and suggests its suitability for the optimization of SNN.     

Activity analysis of neural networks

In a series of articles (Leung et al., 1973, 1974; Ogztöreli, 1972, 1975, 1978, 1979; Stein et al., 1974) we have investigated some of the physiologically significant properties of a general neural model. In these papers the nature of the oscillations occuring in the model has been briefly analyzed by omitting the effects of the discrete time-lags in the interaction of neurons, although these time-lags were incoporated in the general model. In the present work we investigate the effects of the time-lags on the oscillations which are intrinsic to the neural model, depending on the structural parameters such as external inputs, interaction coefficients, self-inhibition, self-excitation and selfadaptation coefficients. The numerical solution of the neural model, the computation of the steady-state solutions and the natural modes of the oscillations around the steady-state solutions are described.This work was partly supported by the Natural Sciences and Engineering Research Council of Canada under Grant NRC-A-4345 through the University of Alberta     

State-space analysis of time-varying higher-order spike correlation for multiple neural spike train data

Precise spike coordination between the spiking activities of multiple neurons is suggested as an indication of coordinated network activity in active cell assemblies. Spike correlation analysis aims to identify such cooperative network activity by detecting excess spike synchrony in simultaneously recorded multiple neural spike sequences. Cooperative activity is expected to organize dynamically during behavior and cognition; therefore currently available analysis techniques must be extended to enable the estimation of multiple time-varying spike interactions between neurons simultaneously. In particular, new methods must take advantage of the simultaneous observations of multiple neurons by addressing their higher-order dependencies, which cannot be revealed by pairwise analyses alone. In this paper, we develop a method for estimating time-varying spike interactions by means of a state-space analysis. Discretized parallel spike sequences are modeled as multi-variate binary processes using a log-linear model that provides a well-defined measure of higher-order spike correlation in an information geometry framework. We construct a recursive Bayesian filter/smoother for the extraction of spike interaction parameters. This method can simultaneously estimate the dynamic pairwise spike interactions of multiple single neurons, thereby extending the Ising/spin-glass model analysis of multiple neural spike train data to a nonstationary analysis. Furthermore, the method can estimate dynamic higher-order spike interactions. To validate the inclusion of the higher-order terms in the model, we construct an approximation method to assess the goodness-of-fit to spike data. In addition, we formulate a test method for the presence of higher-order spike correlation even in nonstationary spike data, e.g., data from awake behaving animals. The utility of the proposed methods is tested using simulated spike data with known underlying correlation dynamics. Finally, we apply the methods to neural spike data simultaneously recorded from the motor cortex of an awake monkey and demonstrate that the higher-order spike correlation organizes dynamically in relation to a behavioral demand.     

Stability of delayed memristive neural networks with time-varying impulses

This paper addresses the stability problem on the memristive neural networks with time-varying impulses. Based on the memristor theory and neural network theory, the model of the memristor-based neural network is established. Different from the most publications on memristive networks with fixed-time impulse effects, we consider the case of time-varying impulses. Both the destabilizing and stabilizing impulses exist in the model simultaneously. Through controlling the time intervals of the stabilizing and destabilizing impulses, we ensure the effect of the impulses is stabilizing. Several sufficient conditions for the globally exponentially stability of memristive neural networks with time-varying impulses are proposed. The simulation results demonstrate the effectiveness of the theoretical results.     

Classification of bacterial species from proteomic data using combinatorial approaches incorporating artificial neural networks, cluster analysis and principal components analysis

MOTIVATION: Robust computer algorithms are required to interpret the vast amounts of proteomic data currently being produced and to generate generalized models which are applicable to 'real world' scenarios. One such scenario is the classification of bacterial species. These vary immensely, some remaining remarkably stable whereas others are extremely labile showing rapid mutation and change. Such variation makes clinical diagnosis difficult and pathogens may be easily misidentified. RESULTS: We applied artificial neural networks (Neuroshell 2) in parallel with cluster analysis and principal components analysis to surface enhanced laser desorption/ionization (SELDI)-TOF mass spectrometry data with the aim of accurately identifying the bacterium Neisseria meningitidis from species within this genus and other closely related taxa. A subset of ions were identified that allowed for the consistent identification of species, classifying >97% of a separate validation subset of samples into their respective groups. AVAILABILITY: Neuroshell 2 is commercially available from Ward Systems.     

Dynamic neural networks as a tool for the online optimization of industrial fermentation

A system for online optimization of industrial fermentation based on a model with dynamic neural networks is described. The developed dynamic neural network, consisting of adapted neurons to consider the process dynamics, can model the complex, non-linear fermentation of beer in order to predict the future process. The predicted trajectories of gravity, pH, and diacetyl are in agreement with the experimental data measured at an automated pilot fermenter. It was possible to predict the future course of the batch fermentation as soon as 12 h of process data were available. In combination with the variational principle, the process model was used to optimize productivity. The temperature trajectory is optimized using a cost functional, including technical and technological conditions of the brewery in order to reduce the process time by steady product quality. The results show a reduction of the process time of up to 20%, which leads to an increase in utilization capacity.     

Genetic optimization of combinatorial libraries

Most agrochemical and pharmaceutical companies have set up high-throughput screening programs which require large numbers of compounds to screen. Combinatorial libraries provide an attractive way to deliver these compounds. A single combinatorial library with four variable positions can yield more than 10(12) potential compounds, if one assumes that about 1000 reagents are available for each position. This is far more than any high-throughput screening facility can afford to screen. We have proposed a method for iterative compound selection from large databases, which identifies the most active compounds by examining only a small fraction of the database. In this article, we describe the extension of this method to the problem of selecting compounds from large combinatorial libraries. Copyright 1998 John Wiley & Sons, Inc.     

The complexity of linkage analysis with neural networks

As the focus of genome-wide scans for disease loci have shifted from simple Mendelian traits to genetically complex traits, researchers have begun to consider new alternative ways to detect linkage that will consider more than the marginal effects of a single disease locus at a time. One interesting new method is to train a neural network on a genome-wide data set in order to search for the best non-linear relationship between identity-by-descent sharing among affected siblings at markers and their disease status. We investigate here the repeatability of the neural network results from run to run, and show that the results obtained by multiple runs of the neural network method may differ quite a bit. This is most likely due to the fact that training a neural network involves minimizing an error function with a multitude of local minima.     

Cooperative recurrent modular neural networks for constrained optimization: a survey of models and applications

Constrained optimization problems arise in a wide variety of scientific and engineering applications. Since several single recurrent neural networks when applied to solve constrained optimization problems for real-time engineering applications have shown some limitations, cooperative recurrent neural network approaches have been developed to overcome drawbacks of these single recurrent neural networks. This paper surveys in details work on cooperative recurrent neural networks for solving constrained optimization problems and their engineering applications, and points out their standing models from viewpoint of both convergence to the optimal solution and model complexity. We provide examples and comparisons to shown advantages of these models in the given applications.     

Stability of two cluster solutions in pulse coupled networks of neural oscillators

Phase response curves (PRCs) have been widely used to study synchronization in neural circuits comprised of pacemaking neurons. They describe how the timing of the next spike in a given spontaneously firing neuron is affected by the phase at which an input from another neuron is received. Here we study two reciprocally coupled clusters of pulse coupled oscillatory neurons. The neurons within each cluster are presumed to be identical and identically pulse coupled, but not necessarily identical to those in the other cluster. We investigate a two cluster solution in which all oscillators are synchronized within each cluster, but in which the two clusters are phase locked at nonzero phase with each other. Intuitively, one might expect this solution to be stable only when synchrony within each isolated cluster is stable, but this is not the case. We prove rigorously the stability of the two cluster solution and show how reciprocal coupling can stabilize synchrony within clusters that cannot synchronize in isolation. These stability results for the two cluster solution suggest a mechanism by which reciprocal coupling between brain regions can induce local synchronization via the network feedback loop.     

Stability of dead zone bidirectional associative memory neural networks involving time delays

A mathematical model describing the dynamical interactions of bidirectional associative memory networks involving transmission delays is considered. The influence of a dead zone or a zone of noactivation on the global stability is investigated and various easily verifiable sets of sufficient conditions are established. The asymptotic nature of solutions when the given system of equations does not possess an equilibrium pattern is discussed.     

A combinatorial optimization approach for diverse motif finding applications

Background  

Discovering approximately repeated patterns, or motifs, in biological sequences is an important and widely-studied problem in computational molecular biology. Most frequently, motif finding applications arise when identifying shared regulatory signals within DNA sequences or shared functional and structural elements within protein sequences. Due to the diversity of contexts in which motif finding is applied, several variations of the problem are commonly studied.     

Integration of life cycle assessment,artificial neural networks,and metaheuristic optimization algorithms for optimization of tomato-based cropping systems in Iran

Purpose

The main purpose of this study was to evaluate the use of an integrated life cycle assessment (LCA), artificial neural network, and metaheuristic optimization model to improve the sustainability of tomato-based cropping systems in Iran. The model outputs the combination of input usage in a tomato cropping system, which leads to the highest economic output and the least environmental impact.

Methods

The LCA inventory was created using data from 114 open-field tomato farms in the Alborz Province of Iran during one growing period in 2015. Among all management components, the main focus was on irrigation management systems. The optimization problem was designed by integrating three indicators: carbon footprint (CF), benefit-cost ratio (BCR), and energy use efficiency (EUE) as the objective of field tomato production. The functional unit was 1 kg of tomato aligned with the system boundary of the cradle to market life cycle. Three artificial neural networks (ANNs) were applied to model relationships between the inputs and three indices (CF, BCR, and EUE) as the objective functions. Multi-objective genetic algorithm (MOGA) and multi-objective particle swarm optimization (MOPSO) were used to minimize the CF and maximize the BCR and EUE indicators. The abovementioned aims have been pursued by developing codes in MATLAB software.

Results and discussion

CF, BCR, and EUE were calculated to be 0.26 kg CO_2?eq (kg tomato)^?1, 1.8, and 0.5, respectively. MOGA results envisage the possibility of an increase of 86% and 50% in the EUE and BCR and a 43% reduction in the CF of tomato production systems. Moreover, EUE and BCR increased by 83% and 49%, and CF was reduced by 39% from the optimum results obtained from the MOPSO algorithm. It was revealed that in order to optimize field tomato production with the target objectives of this study, a large additional use for irrigation pipes, plastic, and machinery in comparison to current situation is required, while a large reduction of biocide, chemical fertilizer, and electricity consumption is indispensable.

Conclusions

According to the results of our study, it was concluded that the optimal solutions require a modernization of irrigation systems and a decrease in the consumption of chemical fertilizers and pesticides. The implementation of management options for such solutions is discussed.

     

Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks

Recurrent neural networks (RNNs) are widely used in computational neuroscience and machine learning applications. In an RNN, each neuron computes its output as a nonlinear function of its integrated input. While the importance of RNNs, especially as models of brain processing, is undisputed, it is also widely acknowledged that the computations in standard RNN models may be an over-simplification of what real neuronal networks compute. Here, we suggest that the RNN approach may be made computationally more powerful by its fusion with Bayesian inference techniques for nonlinear dynamical systems. In this scheme, we use an RNN as a generative model of dynamic input caused by the environment, e.g. of speech or kinematics. Given this generative RNN model, we derive Bayesian update equations that can decode its output. Critically, these updates define a 'recognizing RNN' (rRNN), in which neurons compute and exchange prediction and prediction error messages. The rRNN has several desirable features that a conventional RNN does not have, e.g. fast decoding of dynamic stimuli and robustness to initial conditions and noise. Furthermore, it implements a predictive coding scheme for dynamic inputs. We suggest that the Bayesian inversion of RNNs may be useful both as a model of brain function and as a machine learning tool. We illustrate the use of the rRNN by an application to the online decoding (i.e. recognition) of human kinematics.     

Synchrony measures for biological neural networks

 Synchronous firing of a population of neurons has been observed in many experimental preparations; in addition, various mathematical neural network models have been shown, analytically or numerically, to contain stable synchronous solutions. In order to assess the level of synchrony of a particular network over some time interval, quantitative measures of synchrony are needed. We develop here various synchrony measures which utilize only the spike times of the neurons; these measures are applicable in both experimental situations and in computer models. Using a mathematical model of the CA3 region of the hippocampus, we evaluate these synchrony measures and compare them with pictorial representations of network activity. We illustrate how synchrony is lost and synchrony measures change as heterogeneity amongst cells increases. Theoretical expected values of the synchrony measures for different categories of network solutions are derived and compared with results of simulations. Received: 6 June 1994/Accepted in revised form: 13 January 1995