Parameter estimation of a monod-type model. Part II: Use of a neural network with associative memories

Summary On the basis of a previous study related with parameter identifiability and sensitivity analysis of a Monod-type model, a parameter estimation method based on Artificial Neural Networks (ANN) with Associative Memories (AMs) is presented. A combination of an iterative procedure and a convergence index given by AMs allows to confirm the nature of relations existing between state variables and parameters which were found in the first part of the study. The convergence criterion is particularly well adapted to showing various influences of state variables on parameter estimation of such a model.     

Izhikevich神经元网络的同步与联想记忆

联想记忆是人脑的一项重要功能。以Izhikevich神经元模型为节点,构建神经网络,神经元之间采用全连结的方式;以神经元群体的时空编码（spatio-temporal coding）理论研究所构建神经网络的联想记忆功能。在加入高斯白噪声的情况下,调节网络中神经元之间的连接强度的大小,当连接强度和噪声强度达到一个阈值时网络中部分神经元同步放电,实现了存储模式的联想记忆与恢复。仿真结果表明,神经元之间的连接强度在联想记忆的过程中发挥了重要的作用,噪声可以促使神经元间的同步放电,有助于神经网络实现存储模式的联想记忆与恢复。     

Quantitative measurement of two-component pH-sensitive colorimetric spectra using multilayer neural networks

The purpose of this research was to develop a noise tolerant and faster processing approach for in vivo and in vitro spectrophotometric applications where distorted spectra are difficult to interpret quantitatively. A PC based multilayer neural network with a sigmoid activation function and a generalized delta learning rule was trained with a two component (protonated and unprotonated form) pH-dependent spectrum generated from microspectrophotometry of the vital dye neutral red (NR). The network makes use of the digitized absorption spectrum between 375 and 675 nm. The number of nodes in the input layer was determined by the required resolution. The number of output nodes determined the step size of the quantization value used to distinguish the input spectra (i.e. defined the number of distinct output steps). Mathematic analysis provided the conditions for which this network is guaranteed to converge. Simulation results showed that features of the input spectrum were successfully identified and stored in the weight matrix of the input and hidden layers. After convergent training with typical spectra, a calibration curve was constructed to interpret the output layer activity and therefore, predict interpolated pH values of unknown spectra. With its built-in redundant presentation, this approach needed no preprocessing procedures (baseline correction or intensive signal averaging) normally used in multicomponent analyses. The identification of unknown spectra with the activities of the output layer is a one step process using the convergent weight matrix. After learning from examples, real time applications can be accomplished without solving multiple linear equations as in the multiple linear regression method. This method can be generalized to pattern oriented sensory information processing and multi-sensor data fusion for quantitative measurement purposes.     

Stimulus design for model selection and validation in cell signaling

Mechanism-based chemical kinetic models are increasingly being used to describe biological signaling. Such models serve to encapsulate current understanding of pathways and to enable insight into complex biological processes. One challenge in model development is that, with limited experimental data, multiple models can be consistent with known mechanisms and existing data. Here, we address the problem of model ambiguity by providing a method for designing dynamic stimuli that, in stimulus–response experiments, distinguish among parameterized models with different topologies, i.e., reaction mechanisms, in which only some of the species can be measured. We develop the approach by presenting two formulations of a model-based controller that is used to design the dynamic stimulus. In both formulations, an input signal is designed for each candidate model and parameterization so as to drive the model outputs through a target trajectory. The quality of a model is then assessed by the ability of the corresponding controller, informed by that model, to drive the experimental system. We evaluated our method on models of antibody–ligand binding, mitogen-activated protein kinase (MAPK) phosphorylation and de-phosphorylation, and larger models of the epidermal growth factor receptor (EGFR) pathway. For each of these systems, the controller informed by the correct model is the most successful at designing a stimulus to produce the desired behavior. Using these stimuli we were able to distinguish between models with subtle mechanistic differences or where input and outputs were multiple reactions removed from the model differences. An advantage of this method of model discrimination is that it does not require novel reagents, or altered measurement techniques; the only change to the experiment is the time course of stimulation. Taken together, these results provide a strong basis for using designed input stimuli as a tool for the development of cell signaling models.     

Variational learning for Generalized Associative Functional Networks in modeling dynamic process of plant growth

This paper presents a new statistical techniques — Bayesian Generalized Associative Functional Networks (GAFN), to model the dynamical plant growth process of greenhouse crops. GAFNs are able to incorporate the domain knowledge and data to model complex ecosystem. By use of the functional networks and Bayesian framework, the prior knowledge can be naturally embedded into the model, and the functional relationship between inputs and outputs can be learned during the training process. Our main interest is focused on the Generalized Associative Functional Networks (GAFNs), which are appropriate to model multiple variable processes. Three main advantages are obtained through the applications of Bayesian GAFN methods to modeling dynamic process of plant growth. Firstly, this approach provides a powerful tool for revealing some useful relationships between the greenhouse environmental factors and the plant growth parameters. Secondly, Bayesian GAFN can model Multiple-Input Multiple-Output (MIMO) systems from the given data, and presents a good generalization capability from the final single model for successfully fitting all 12 data sets over 5-year field experiments. Thirdly, the Bayesian GAFN method can also play as an optimization tool to estimate the interested parameter in the agro-ecosystem. In this work, two algorithms are proposed for the statistical inference of parameters in GAFNs. Both of them are based on the variational inference, also called variational Bayes (VB) techniques, which may provide probabilistic interpretations for the built models. VB-based learning methods are able to yield estimations of the full posterior probability of model parameters. Synthetic and real-world examples are implemented to confirm the validity of the proposed methods.     

Using input minimization to train a cerebellar model to simulate regulation of smooth pursuit

Cerebellar learning appears to be driven by motor error, but whether or not error signals are provided by climbing fibers (CFs) remains a matter of controversy. Here we show that a model of the cerebellum can be trained to simulate the regulation of smooth pursuit eye movements by minimizing its inputs from parallel fibers (PFs), which carry various signals including error and efference copy. The CF spikes act as “learn now” signals. The model can be trained to simulate the regulation of smooth pursuit of visual objects following circular or complex trajectories and provides insight into how Purkinje cells might encode pursuit parameters. In minimizing both error and efference copy, the model demonstrates how cerebellar learning through PF input minimization (InMin) can make movements more accurate and more efficient. An experimental test is derived that would distinguish InMin from other models of cerebellar learning which assume that CFs carry error signals.     

Learning and Long-Term Retention of Large-Scale Artificial Languages

Recovering discrete words from continuous speech is one of the first challenges facing language learners. Infants and adults can make use of the statistical structure of utterances to learn the forms of words from unsegmented input, suggesting that this ability may be useful for bootstrapping language-specific cues to segmentation. It is unknown, however, whether performance shown in small-scale laboratory demonstrations of “statistical learning” can scale up to allow learning of the lexicons of natural languages, which are orders of magnitude larger. Artificial language experiments with adults can be used to test whether the mechanisms of statistical learning are in principle scalable to larger lexicons. We report data from a large-scale learning experiment that demonstrates that adults can learn words from unsegmented input in much larger languages than previously documented and that they retain the words they learn for years. These results suggest that statistical word segmentation could be scalable to the challenges of lexical acquisition in natural language learning.     

Deriving the probabilities of water loss and ammonia loss for amino acids from tandem mass spectra

In protein identification through tandem mass spectrometry, it is critical to accurately predict the theoretical spectrum for a peptide sequence. The widely used prediction models, such as SEQUEST and MASCOT, ignore the intensity of the ions with important neutral losses, including water loss and ammonia loss. However, ignoring these neutral losses results in a significant deviation between the predicted theoretical spectrum and its experimental counterpart. Here, based on the "one peak, multiple explanations" observation, we proposed an expectation-maximization (EM) method to automatically learn the probabilities of water loss and ammonia loss for each amino acid. Then we employed these probabilities to design an improved statistical model for theoretical spectrum prediction. We implemented these methods and tested them on practical data. On a training set containing 1803 spectra, the experimental results show a good agreement with some known knowledge about neutral losses, such as the tendency of water loss from Asp, Glu, Ser, and Thr. Furthermore, on a testing set containing 941 spectra, the improved similarity between the experimental and predicted spectra demonstrates that this method can generate more reasonable predictions relative to the model that ignores neutral losses. As an application of the derived probabilities, we implemented a database searching method adopting the improved theoretical spectrum model with neutral loss ions estimated. Experimental results on Keller's data set demonstrate that this method can identify peptides more accurately than SEQUEST. In another application to validate SEQUEST's results, the reported peptide-spectrum pairs are reranked with respect to the similarity between experimental and predicted spectra. Experimental results on both LTQ and QSTAR data sets suggest that this reranking strategy can effectively distinguish the false negative predictions reported by SEQUEST.     

Analytical reconstruction of the neuronal input current from spike train data

The time course of the current driving action potential generation at a neuron investigated experimentally is in general not measurable directly. In this paper an indirect method is introduced that allows estimation of this unknown current time course using only spike train data. Assuming the leaky integrator model as valid for the action potential encoding site of the investigated neuron, the unknown input current is obtained by determining (analytically) a current time course that upon injection into the leaky integrator model evokes action potential sequences identical to those observed experimentally. Applications of this current-reconstruction procedure to neuronal output data obtained from a leaky integrator model showed that the procedure allows a good estimation of the underlying input current even if the membrane time constant of the investigated neuron is not known exactly. Additionally, an application of current reconstruction to experimental data obtained from a cat muscle spindle primary afferent subject to repeated -stimuli is demonstrated.     

A neural-network technique to learn concepts from electroencephalograms

A new technique is presented developed to learn multi-class concepts from clinical electroencephalograms (EEGs). A desired concept is represented as a neuronal computational model consisting of the input, hidden, and output neurons. In this model the hidden neurons learn independently to classify the EEG segments presented by spectral and statistical features. This technique has been applied to the EEG data recorded from 65 sleeping healthy newborns in order to learn a brain maturation concept of newborns aged between 35 and 51 weeks. The 39,399 and 19,670 segments from these data have been used for learning and testing the concept, respectively. As a result, the concept has correctly classified 80.1% of the testing segments or 87.7% of the 65 records.     

Probabilistic discovery of overlapping cellular processes and their regulation.

In this paper, we explore modeling overlapping biological processes. We discuss a probabilistic model of overlapping biological processes, gene membership in those processes, and an addition to that model that identifies regulatory mechanisms controlling process activation. A key feature of our approach is that we allow genes to participate in multiple processes, thus providing a more biologically plausible model for the process of gene regulation. We present algorithms to learn each model automatically from data, using only genomewide measurements of gene expression as input. We compare our results to those obtained by other approaches and show that significant benefits can be gained by modeling both the organization of genes into overlapping cellular processes and the regulatory programs of these processes. Moreover, our method successfully grouped genes known to function together, recovered many regulatory relationships that are known in the literature, and suggested novel hypotheses regarding the regulatory role of previously uncharacterized proteins.     

Application of the recommendation architecture to telecommunications network management.

The recommendation architecture has been proposed as a system architecture which can enable a system to learn to perform a complex combination of interrelated functions. The capability of a system with the recommendation architecture to learn to manage complex telecommunication backbone networks has been investigated. A network model with a number of nodes and links and carrying realistic but randomly generated traffic was used as the target for the management system. Traffic data taken from the model was used as input to the recommendation architecture system. The traffic data was organized into inputs once every 5 minutes, and the management system organized these inputs into a hierarchy of repetition similarity. It was demonstrated that the outputs of this hierarchy provided information on the condition of the network. This output information was a compressed version of the inputs which correlated with major network conditions.     

A recurrent network for landmark-based navigation

A new type of network is proposed that can be applied to landmark navigation. It solves the guidance task, that is, it finds a nonvisually marked location using knowledge concerning its spatial relation to other, visible landmarks. The path to the searched location is not disturbed if a landmark is not visible for some time. The network can also describe findings obtained by experiments with insects and rodents, where the position of the landmarks has been changed after training. In this net, recognition does not occur by searching for a match between a pattern seen and the same pattern being stored but by searching for a match between a pattern seen with a prediction calculated from different data. A simple extension allows a unique match of the landmarks seen with the items stored in memory. With this extension a recognition of the individual landmark is not necessary. A specific output unit of the network can be interpreted in such a way as to show properties of place cells found in vertebrates and the function of the network proposed here as to determine the input to a place cell. The model can explain the observation that a given place cell can also be active when the animal moves in a different environment. An extension is discussed of how the network could be exploited for recognition-triggered response that allows animals to follow fixed routes.     

蜜蜂对复合气味中特征因素的选择性记忆巩固的研究

目的 蜜蜂天生具有丰富的嗅觉辨识能力,觅食、交配、导航以及社交活动均依赖其嗅觉系统,是研究嗅觉感知和学习记忆的行为及神经机制的理想模型。蜜蜂既能够将某个复合气味作为一个整体也可以将复合气味的各组成成分进行辨别和区分,但是在特征依赖的联合记忆中依据何种原则进行加工并存储到长期记忆还不清楚。方法 本文利用特征阳性（feature positive：AB+,B-）和特征阴性（feature negative：AB-,B+）的奖赏性嗅觉条件化,训练蜜蜂对复合气味和成分气味的辨别,并检测蜜蜂对复合气味（AB）、成分气味（B）以及特征气味（A）的中长时记忆（3 h）和长时记忆（24 h）。结果 在特征阳性的奖赏性嗅觉条件化中,蜜蜂对训练过的气味可以形成稳定的中长时和长时记忆,并且对复合气味中的特征气味的记忆与复合气味的记忆呈现高度相似。但在特征阴性的奖赏性嗅觉条件化中,蜜蜂虽能够在3 h和24 h对训练过的两种气味具有显著的伸喙反应差异,且对特征阴性的气味无显著反应,但对复合气味的反应随时间的推移而增加。结论 实验结果表明,蜜蜂选择性地将与奖赏信息联合出现的气味巩固到长时记忆中,但并未依据特征成分加工储存到长时记忆中。奖赏信息预示着食物源,与生存息息相关,表明对环境信息进行选择性的记忆巩固加工并储存可能是低等动物高效地编码生存相关信息的重要策略。     

An analytical model for the estimation of chromosome substitution effects in the offspring of individuals heterozygous at a segregating marker locus

Summary Use of marker genes for quantitative traits has been suggested as a supplement to selection for livestock species. Linkage relationships can be estimated by using data from offspring of a heterozygous parent, if offspring can be positively assigned segregation of one or the other of the marker alleles. In field data, some data on offspring can be characterized and used to estimate the difference in chromosome substitution effects, but other matings result in uncertain transfer of the marker alleles. In this study, an alternative estimation procedure is proposed that would allow incorporation of data on all offspring of a heterozygous parent, even those where chromosome segregation is ambiguous. If the frequency of the marker alleles is known in the population of mates of a heterozygous individual, the mean and variance of the heterozygous offspring can be used in a generalized leastsquares model to estimate the chromosome substitution effect. When gene frequencies are not known, maximum likelihood estimates can be obtained from the data for use in a conditional estimate. Monte Carlo simulations of data following the assumed genetic model were analyzed as proposed, and parameter estimates were characterized. Estimates of chromosome substitution effects were reasonable approximations of input values. Distributions of t-statistics testing the null hypothesis of no difference between marked chromosome segments were unbiased, with only slightly larger variance than expected. Addition of data from heterozygous offspring improved the efficiency of detection of chromosome substitution effects by more than four times when marker gene frequencies were low.     

Associative memory in quaternionic Hopfield neural network

Associative memory networks based on quaternionic Hopfield neural network are investigated in this paper. These networks are composed of quaternionic neurons, and input, output, threshold, and connection weights are represented in quaternions, which is a class of hypercomplex number systems. The energy function of the network and the Hebbian rule for embedding patterns are introduced. The stable states and their basins are explored for the networks with three neurons and four neurons. It is clarified that there exist at most 16 stable states, called multiplet components, as the degenerated stored patterns, and each of these states has its basin in the quaternionic networks.     

Protein structure prediction: making AWSEM AWSEM‐ER by adding evolutionary restraints

Protein sequences have evolved to fold into functional structures, resulting in families of diverse protein sequences that all share the same overall fold. One can harness protein family sequence data to infer likely contacts between pairs of residues. In the current study, we combine this kind of inference from coevolutionary information with a coarse‐grained protein force field ordinarily used with single sequence input, the Associative memory, Water mediated, Structure and Energy Model (AWSEM), to achieve improved structure prediction. The resulting Associative memory, Water mediated, Structure and Energy Model with Evolutionary Restraints (AWSEM‐ER) yields a significant improvement in the quality of protein structure prediction over the single sequence prediction from AWSEM when a sufficiently large number of homologous sequences are available. Free energy landscape analysis shows that the addition of the evolutionary term shifts the free energy minimum to more native‐like structures, which explains the improvement in the quality of structures when performing predictions using simulated annealing. Simulations using AWSEM without coevolutionary information have proved useful in elucidating not only protein folding behavior, but also mechanisms of protein function. The success of AWSEM‐ER in de novo structure prediction suggests that the enhanced model opens the door to functional studies of proteins even when no experimentally solved structures are available.     

Error-tolerant pooling designs with inhibitors.

Pooling designs are used in clone library screening to efficiently distinguish positive clones from negative clones. Mathematically, a pooling design is just a nonadaptive group testing scheme which has been extensively studied in the literature. In some applications, there is a third category of clones called "inhibitors" whose effect is to neutralize positives. Specifically, the presence of an inhibitor in a pool dictates a negative outcome even though positives are present. Sequential group testing schemes, which can be modified to three-stage schemes, have been proposed for the inhibitor model, but it is unknown whether a pooling design (a one-stage scheme) exists. Another open question raised in the literature is whether the inhibitor model can treat unreliable pool outcomes. In this paper, we answer both open problems by giving a pooling design, as well as a two-stage scheme, for the inhibitor model with unreliable outcomes. The number of pools required by our schemes are quite comparable to the three-stage scheme.     

A multiorganism based method for Bayesian gene network estimation

The primary goal of this article is to infer genetic interactions based on gene expression data. A new method for multiorganism Bayesian gene network estimation is presented based on multitask learning. When the input datasets are sparse, as is the case in microarray gene expression data, it becomes difficult to separate random correlations from true correlations that would lead to actual edges when modeling the gene interactions as a Bayesian network. Multitask learning takes advantage of the similarity between related tasks, in order to construct a more accurate model of the underlying relationships represented by the Bayesian networks. The proposed method is tested on synthetic data to illustrate its validity. Then it is iteratively applied on real gene expression data to learn the genetic regulatory networks of two organisms with homologous genes.