An optimization framework of biological dynamical systems

Different biological dynamics are often described by different mathematical equations. On the other hand, some mathematical models describe many biological dynamics universally. Here, we focus on three biological dynamics: the Lotka-Volterra equation, the Hopfield neural networks, and the replicator equation. We describe these three dynamical models using a single optimization framework, which is constructed with employing the Riemannian geometry. Then, we show that the optimization structures of these dynamics are identical, and the differences among the three dynamics are only in the constraints of the optimization. From this perspective, we discuss the unified view for biological dynamics. We also discuss the plausible categorizations, the fundamental nature, and the efficient modeling of the biological dynamics, which arise from the optimization perspective of the dynamical systems.     

Models of Hopfield-type quaternion neural networks and their energy functions

Recently models of neural networks that can directly deal with complex numbers, complex-valued neural networks, have been proposed and several studies on their abilities of information processing have been done. Furthermore models of neural networks that can deal with quaternion numbers, which is the extension of complex numbers, have also been proposed. However they are all multilayer quaternion neural networks. This paper proposes models of fully connected recurrent quaternion neural networks, Hopfield-type quaternion neural networks. Since quaternion numbers are non-commutative on multiplication, some different models can be considered. We investigate dynamics of these proposed models from the point of view of the existence of an energy function and derive their conditions for existence.     

人工神经网络模型及其在农业和生态学研究中的应用

对于一些复杂的农业生态系统,人们对其生态过程了解较少,且这些系统的不确定性和模糊性较大,用传统的方法难以模拟这些系统的行为,神经网络模型因为能较精确地模拟这些系统的行为,而引起生态学者们的广泛兴趣。该文着重介绍了误差逆传神经网络模型的结构、算法及其在农业和生态学中的应用研究。误差逆传神经网络模型一般采用三层神经网络模型结构,三层的神经网络模型能模拟任意复杂程度的连续函数,而且因为它的结构小而不容易产生与训练数据的过度吻合。误差逆传神经网络模型算法的主要特征是:利用当前的输入误差对权值进行调整。在生态学和农业研究中,误差逆传神经网络模型通常作为非线性函数模拟器用于预测作物产量、生物生产量、生物与环境之间的关系等。已有的研究表明:误差逆传神经网络模型的模拟精度要远远高于多元线性方程,类似于非线性方程,而在样本量足够的情况下,有一定的外推能力。但是误差逆传神经网络模型需要大量的样本量来保证所求取参数的可靠性,但这在实际研究中很难做到,因而限制了误差逆传神经网络模型的应用。近年来人们提出了强制训练停止、复合模型等多种技术来提高误差逆传神经网络模型的外推能力,也提出了Garson算法、敏感性分析以及随机化检验等技术对误差逆传神经网络模型的机理进行解释。误差逆传神经网络模型的真正优势在于模拟人们了解较少或不确定性和模糊性较大系统的行为,这些是传统模型所无法实现的,因而是对传统机理模型的重要补充。     

Learning Universal Computations with Spikes

Providing the neurobiological basis of information processing in higher animals, spiking neural networks must be able to learn a variety of complicated computations, including the generation of appropriate, possibly delayed reactions to inputs and the self-sustained generation of complex activity patterns, e.g. for locomotion. Many such computations require previous building of intrinsic world models. Here we show how spiking neural networks may solve these different tasks. Firstly, we derive constraints under which classes of spiking neural networks lend themselves to substrates of powerful general purpose computing. The networks contain dendritic or synaptic nonlinearities and have a constrained connectivity. We then combine such networks with learning rules for outputs or recurrent connections. We show that this allows to learn even difficult benchmark tasks such as the self-sustained generation of desired low-dimensional chaotic dynamics or memory-dependent computations. Furthermore, we show how spiking networks can build models of external world systems and use the acquired knowledge to control them.     

Optimization of a fermentation medium using neural networks and genetic algorithms

Artificial neural networks and genetic algorithms are used to model and optimize a fermentation medium for the production of the enzyme hydantoinase by Agrobacterium radiobacter. Experimental data reported in the literature were used to build two neural network models. The concentrations of four medium components served as inputs to the neural network models, and hydantoinase or cell concentration served as a single output of each model. Genetic algorithms were used to optimize the input space of the neural network models to find the optimum settings for maximum enzyme and cell production. Using this procedure, two artificial intelligence techniques have been effectively integrated to create a powerful tool for process modeling and optimization.     

Exponential input-to-state stability of recurrent neural networks with multiple time-varying delays

In this paper, input-to-state stability problems for a class of recurrent neural networks model with multiple time-varying delays are concerned with. By utilizing the Lyapunov–Krasovskii functional method and linear matrix inequalities techniques, some sufficient conditions ensuring the exponential input-to-state stability of delayed network systems are firstly obtained. Two numerical examples and its simulations are given to illustrate the efficiency of the derived results.     

The need for stochastic replication of ecological neural networks

Artificial neural networks are becoming increasingly popular as predictive statistical tools in ecosystem ecology and as models of signal processing in behavioural and evolutionary ecology. We demonstrate here that a commonly used network in ecology, the three-layer feed-forward network, trained with the backpropagation algorithm, can be extremely sensitive to the stochastic variation in training data that results from random sampling of the same underlying statistical distribution, with networks converging to several distinct predictive states. Using a random walk procedure to sample error-weight space, and Sammon dimensional reduction of weight arrays, we demonstrate that these different predictive states are not artefactual, due to local minima, but lie at the base of major error troughs in the error-weight surface. We further demonstrate that various gross weight compositions can produce the same predictive state, suggesting the analogy of weight space as a 'patchwork' of multiple predictive states. Our results argue for increased inclusion of stochastic training replication and analysis into ecological and behavioural applications of artificial neural networks.     

基于神经网络的生化过程预估优化控制

综观历史与现状,生化过程数学模型的建立是很困难的。不多的一些数学模型也往往由于精度低,应用范围窄而无法在实际中应用。这是由于生化过程的机理非常复杂,具有高度非线性和时变特性。并且不同于一般物理过程的是生化过程是个物理上不可逆的过程。近几年来,人工神经网络(ANN)得到了迅速发展,并被广泛应用到各个领域。同样,ANN也为生化过程控制提供了一种新方法。本文以工业生产中发酵过程的补料控制为例,叙述了ANN如何用干生化过程预估和优化控制。相应地,对结果进行了分析讨论。     

Maximum Likelihood Neural Network Prediction Models

Neural networks have received much attention in recent years mostly by non-statisticians. The purpose of this paper is to incorporate neural networks in a non-linear regression model and obtain maximum likelihood estimates of the network parameters using a standard Newton-Raphson algorithm. We use maximum likelihood estimators instead of the usual back-propagation technique and compare the neural network predictions with predictions of quadratic regression models and with non-parametric nearest neighbor predictions. These comparisons are made using data generated from a variety of functions. Because of the number of parameters involved, neural network models can easily over-fit the data, hence validation of results is crucial.     

Neural networks for perceptual processing: from simulation tools to theories

Neural networks are modelling tools that are, in principle, able to capture the input-output behaviour of arbitrary systems that may include the dynamics of animal populations or brain circuits. While a neural network model is useful if it captures phenomenologically the behaviour of the target system in this way, its utility is amplified if key mechanisms of the model can be discovered, and identified with those of the underlying system. In this review, we first describe, at a fairly high level with minimal mathematics, some of the tools used in constructing neural network models. We then go on to discuss the implications of network models for our understanding of the system they are supposed to describe, paying special attention to those models that deal with neural circuits and brain systems. We propose that neural nets are useful for brain modelling if they are viewed in a wider computational framework originally devised by Marr. Here, neural networks are viewed as an intermediate mechanistic abstraction between 'algorithm' and 'implementation', which can provide insights into biological neural representations and their putative supporting architectures.     

Neural network-based prediction of mutation-induced protein stability changes in Staphylococcal nuclease at 20 residue positions

Protein-based therapeutics are playing an increasingly important role in the treatment of diseases, including diabetes and cancer. The viability of these treatments, however, are highly dependent on the stability of the therapeutic, since stability affects both the shelf life of the therapeutic as well as its active life in the body. Stability engineering can, therefore, be used to increase the effectiveness of protein-based therapeutics. Computational methods of protein stability prediction have been under development for about a decade, but complex molecular interactions make stability prediction difficult and computationally intensive. A rapid computational method of protein stability prediction is developed using feed-forward neural networks and used to predict mutation-induced stability changes in Staphylococcal nuclease. The input to the neural network consisted of sequences of evolutionarily based amino acid similarity scores that were obtained through the comparison of the amino acids in a mutation containing sequence to their positional counterparts in the baseline wild-type amino acid sequence. A training set was created which consisted of similarity score sequences, for which the stabilities of the corresponding amino acid sequences were known, paired with the relative stabilities of the sequences to that of the baseline. Back-propagation of error was used to train the network to output accurate relative stability scores for the sequences in the training set. Neural network-based relative stability predictions for 55 sequences containing mutation combinations not found in the training set had an accuracy of 92.8%.     

Neural networks for clustered and longitudinal data using mixed effects models

Although most statistical methods for the analysis of longitudinal data have focused on retrospective models of association, new advances in mobile health data have presented opportunities for predicting future health status by leveraging an individual's behavioral history alongside data from similar patients. Methods that incorporate both individual-level and sample-level effects are critical to using these data to its full predictive capacity. Neural networks are powerful tools for prediction, but many assume input observations are independent even when they are clustered or correlated in some way, such as in longitudinal data. Generalized linear mixed models (GLMM) provide a flexible framework for modeling longitudinal data but have poor predictive power particularly when the data are highly nonlinear. We propose a generalized neural network mixed model that replaces the linear fixed effect in a GLMM with the output of a feed-forward neural network. The model simultaneously accounts for the correlation structure and complex nonlinear relationship between input variables and outcomes, and it utilizes the predictive power of neural networks. We apply this approach to predict depression and anxiety levels of schizophrenic patients using longitudinal data collected from passive smartphone sensor data.     

Backpropagation algorithm adaptation parameters using learning automata

Despite of the many successful applications of backpropagation for training multi-layer neural networks, it has many drawbocks. For complex problems it may require a long time to train the networks, and it may not train at all. Long training time can be the result of the non-optimal parameters. It is not easy to choose appropriate value of the parameters for a particular problem. In this paper, by interconnection of fixed structure learning automata (FSLA) to the feedforward neural networks, we apply learning automata (LA) scheme for adjusting these parameters based on the observation of random response of neural networks. The main motivation in using learning automata as an adaptation algorithm is to use its capability of global optimization when dealing with multi-modal surface. The feasibility of proposed method is shown through simulations on three learning problems: exclusive-or, encoding problem, and digit recognition. The simulation results show that the adaptation of these parameters using this method not only increases the convergence rate of learning but it increases the likelihood of escaping from the local minima.     

基于神经网络和遗传算法的木糖醇发酵培养基优化研究

发酵过程机理复杂、影响因素众多。菌种的生理生化特性及发酵的工艺确定之后 ,适宜的培养基配方成了发酵水平、原料成本高低的决定因素。为了优化培养基配方 ,采用遗传算法是一种行之有效的方法。遗传算法 (ＧＡ)是基于达尔文进化论和孟德尔遗传学说来实现随机、自适应、并行性全局搜索的一种无须数学模型的优化算法。与其它搜索方法相比 ,ＧＡ的优越性主要有 :(1)在搜索过程中ＧＡ不易陷入局部最优 ,即使所定义的目标函数非连续、不规则或伴有噪声 ,它也能以很大的概率找到全局最优解 ;(2 )由于ＧＡ固有的并行性 ,使得它非常适合于大规模并…     

Artificial neural network based experimental design procedure for enhancing fermentation development

Conventional experimental design techniques are available to assist in the optimization of fermentation processes, but due to the nonlinearities in the bioprocess, they are limited in their effectiveness. This problem is further complicated with recombinant systems as a result of the additional complexities of the process. This article describes a general strategy using artificial neural networks as an alternative approach to fermentation process development laboratory are presented for the neural network based procedures. (c) 1994 John Wiley & Sons, Inc.     

Neural network designs for poly- β-hydroxybutyrate production optimization under simulated industrial conditions

Improvement of the fermentation efficiency of poly--hydroxybutyrate (PHB) may make it competitive with chemically synthesized petroleum-based polymers. One step toward this is optimization of fluid dispersion and the feed rates to a fed-batch bioreactor. In a recent study using a fermentation model, dispersion corresponding to a Peclet number of 20 was shown to maximize the productivity of PHB. Here further improvement has been investigated using neural optimization. A comparison of seven neural topologies has shown that while feed-forward and radial basis neural networks are computationally efficient, recurrent networks generate higher concentrations of PHB. All networks enhanced the productivity by 16–93% over model-based optimization.     

变系数变时滞BAM神经网络概周期解的存在性与全局吸引性

利用指数二分性、Banach不动点定理与微分不等式分析技巧,在不要求激活函数有界的条件下,给出了变系数变时滞的BAM神经网络概周期解的存在唯一性和全局吸引性的充分条件．所得结果推广和改进了相应文献的结果。对设计BAM神经网络概周期振荡有重要意义．     

Inhibition in the nervous system: Models of its roles in choice and context determination

Neural inhibition has often been regarded as playing an important role in stabilizing and tuning the responses of networks of excitatory neurons. Some partial quantitative bases for this qualitative notion are discussed in the context of current neural network models. Such neural network principles as associative learning, competition, opponent processing, and interlevel resonant feedback are explained and related to behavioral and neurochemical data. Tentative analogies of parts of these model networks with specific neurotransmitter systems are explored; these analogies are likely to become more precise as the networks are further refined.Special issue dedicated to Dr. Eugene Roberts.     

Reliable prediction of T-cell epitopes using neural networks with novel sequence representations

In this paper we describe an improved neural network method to predict T-cell class I epitopes. A novel input representation has been developed consisting of a combination of sparse encoding, Blosum encoding, and input derived from hidden Markov models. We demonstrate that the combination of several neural networks derived using different sequence-encoding schemes has a performance superior to neural networks derived using a single sequence-encoding scheme. The new method is shown to have a performance that is substantially higher than that of other methods. By use of mutual information calculations we show that peptides that bind to the HLA A*0204 complex display signal of higher order sequence correlations. Neural networks are ideally suited to integrate such higher order correlations when predicting the binding affinity. It is this feature combined with the use of several neural networks derived from different and novel sequence-encoding schemes and the ability of the neural network to be trained on data consisting of continuous binding affinities that gives the new method an improved performance. The difference in predictive performance between the neural network methods and that of the matrix-driven methods is found to be most significant for peptides that bind strongly to the HLA molecule, confirming that the signal of higher order sequence correlation is most strongly present in high-binding peptides. Finally, we use the method to predict T-cell epitopes for the genome of hepatitis C virus and discuss possible applications of the prediction method to guide the process of rational vaccine design.     

Fine-tuning and the stability of recurrent neural networks

A central criticism of standard theoretical approaches to constructing stable, recurrent model networks is that the synaptic connection weights need to be finely-tuned. This criticism is severe because proposed rules for learning these weights have been shown to have various limitations to their biological plausibility. Hence it is unlikely that such rules are used to continuously fine-tune the network in vivo. We describe a learning rule that is able to tune synaptic weights in a biologically plausible manner. We demonstrate and test this rule in the context of the oculomotor integrator, showing that only known neural signals are needed to tune the weights. We demonstrate that the rule appropriately accounts for a wide variety of experimental results, and is robust under several kinds of perturbation. Furthermore, we show that the rule is able to achieve stability as good as or better than that provided by the linearly optimal weights often used in recurrent models of the integrator. Finally, we discuss how this rule can be generalized to tune a wide variety of recurrent attractor networks, such as those found in head direction and path integration systems, suggesting that it may be used to tune a wide variety of stable neural systems.