Inference of biochemical network models in S-system using multiobjective optimization approach

MOTIVATION: The inference of biochemical networks, such as gene regulatory networks, protein-protein interaction networks, and metabolic pathway networks, from time-course data is one of the main challenges in systems biology. The ultimate goal of inferred modeling is to obtain expressions that quantitatively understand every detail and principle of biological systems. To infer a realizable S-system structure, most articles have applied sums of magnitude of kinetic orders as a penalty term in the fitness evaluation. How to tune a penalty weight to yield a realizable model structure is the main issue for the inverse problem. No guideline has been published for tuning a suitable penalty weight to infer a suitable model structure of biochemical networks. RESULTS: We introduce an interactive inference algorithm to infer a realizable S-system structure for biochemical networks. The inference problem is formulated as a multiobjective optimization problem to minimize simultaneously the concentration error, slope error and interaction measure in order to find a suitable S-system model structure and its corresponding model parameters. The multiobjective optimization problem is solved by the epsilon-constraint method to minimize the interaction measure subject to the expectation constraints for the concentration and slope error criteria. The theorems serve to guarantee the minimum solution for the epsilon-constrained problem to achieve the minimum interaction network for the inference problem. The approach could avoid assigning a penalty weight for sums of magnitude of kinetic orders.     

Gene regulatory network modeling via global optimization of high-order dynamic Bayesian network

ABSTRACT: BACKGROUND: Dynamic Bayesian network (DBN) is among the mainstream approaches for modeling various biological networks, including the gene regulatory network (GRN). Most current methods for learning DBN employ either local search such as hill-climbing, or a meta stochastic global optimization framework such as genetic algorithm or simulated annealing, which are only able to locate sub-optimal solutions. Further, current DBN applications have essentially been limited to small sized networks. RESULTS: To overcome the above difficulties, we introduce here a deterministic global optimization based DBN approach for reverse engineering genetic networks from time course gene expression data. For such DBN models that consist only of inter time slice arcs, we show that there exists a polynomial time algorithm for learning the globally optimal network structure. The proposed approach, named GlobalMIT+, employs the recently proposed information theoretic scoring metric named mutual information test (MIT). GlobalMIT+ is able to learn high-order time delayed genetic interactions, which are common to most biological systems. Evaluation of the approach using both synthetic and real data sets, including a 733 cyanobacterial gene expression data set, shows significantly improved performance over other techniques. CONCLUSIONS: Our studies demonstrate that deterministic global optimization approaches can infer large scale genetic networks.     

Novel recurrent neural network for modelling biological networks: Oscillatory p53 interaction dynamics

Understanding the control of cellular networks consisting of gene and protein interactions and their emergent properties is a central activity of Systems Biology research. For this, continuous, discrete, hybrid, and stochastic methods have been proposed. Currently, the most common approach to modelling accurate temporal dynamics of networks is ordinary differential equations (ODE). However, critical limitations of ODE models are difficulty in kinetic parameter estimation and numerical solution of a large number of equations, making them more suited to smaller systems. In this article, we introduce a novel recurrent artificial neural network (RNN) that addresses above limitations and produces a continuous model that easily estimates parameters from data, can handle a large number of molecular interactions and quantifies temporal dynamics and emergent systems properties. This RNN is based on a system of ODEs representing molecular interactions in a signalling network. Each neuron represents concentration change of one molecule represented by an ODE. Weights of the RNN correspond to kinetic parameters in the system and can be adjusted incrementally during network training. The method is applied to the p53-Mdm2 oscillation system – a crucial component of the DNA damage response pathways activated by a damage signal. Simulation results indicate that the proposed RNN can successfully represent the behaviour of the p53-Mdm2 oscillation system and solve the parameter estimation problem with high accuracy. Furthermore, we presented a modified form of the RNN that estimates parameters and captures systems dynamics from sparse data collected over relatively large time steps. We also investigate the robustness of the p53-Mdm2 system using the trained RNN under various levels of parameter perturbation to gain a greater understanding of the control of the p53-Mdm2 system. Its outcomes on robustness are consistent with the current biological knowledge of this system. As more quantitative data become available on individual proteins, the RNN would be able to refine parameter estimation and mapping of temporal dynamics of individual signalling molecules as well as signalling networks as a system. Moreover, RNN can be used to modularise large signalling networks.     

Sensitivity and specificity of inferring genetic regulatory interactions from microarray experiments with dynamic Bayesian networks

MOTIVATION: Bayesian networks have been applied to infer genetic regulatory interactions from microarray gene expression data. This inference problem is particularly hard in that interactions between hundreds of genes have to be learned from very small data sets, typically containing only a few dozen time points during a cell cycle. Most previous studies have assessed the inference results on real gene expression data by comparing predicted genetic regulatory interactions with those known from the biological literature. This approach is controversial due to the absence of known gold standards, which renders the estimation of the sensitivity and specificity, that is, the true and (complementary) false detection rate, unreliable and difficult. The objective of the present study is to test the viability of the Bayesian network paradigm in a realistic simulation study. First, gene expression data are simulated from a realistic biological network involving DNAs, mRNAs, inactive protein monomers and active protein dimers. Then, interaction networks are inferred from these data in a reverse engineering approach, using Bayesian networks and Bayesian learning with Markov chain Monte Carlo. RESULTS: The simulation results are presented as receiver operator characteristics curves. This allows estimating the proportion of spurious gene interactions incurred for a specified target proportion of recovered true interactions. The findings demonstrate how the network inference performance varies with the training set size, the degree of inadequacy of prior assumptions, the experimental sampling strategy and the inclusion of further, sequence-based information. AVAILABILITY: The programs and data used in the present study are available from http://www.bioss.sari.ac.uk/~dirk/Supplements     

A conceptual review on systems biology in health and diseases: from biological networks to modern therapeutics

Human physiology is an ensemble of various biological processes spanning from intracellular molecular interactions to the whole body phenotypic response. Systems biology endures to decipher these multi-scale biological networks and bridge the link between genotype to phenotype. The structure and dynamic properties of these networks are responsible for controlling and deciding the phenotypic state of a cell. Several cells and various tissues coordinate together to generate an organ level response which further regulates the ultimate physiological state. The overall network embeds a hierarchical regulatory structure, which when unusually perturbed can lead to undesirable physiological state termed as disease. Here, we treat a disease diagnosis problem analogous to a fault diagnosis problem in engineering systems. Accordingly we review the application of engineering methodologies to address human diseases from systems biological perspective. The review highlights potential networks and modeling approaches used for analyzing human diseases. The application of such analysis is illustrated in the case of cancer and diabetes. We put forth a concept of cell-to-human framework comprising of five modules (data mining, networking, modeling, experimental and validation) for addressing human physiology and diseases based on a paradigm of system level analysis. The review overtly emphasizes on the importance of multi-scale biological networks and subsequent modeling and analysis for drug target identification and designing efficient therapies.     

基因组尺度代谢网络研究进展

基因组尺度代谢网络从基因组序列出发,结合基因、蛋白质、代谢数据库和实验数据,从系统的角度定量研究生命体的代谢过程,了解各个组分之间的相互作用关系。这类网络模型对于生命活动理论研究和优良工程菌的构建都具有重要的理论和实践意义。以下结合作者的实际研究经验,对基因组尺度代谢网络从重构到模拟直至应用进行了较为详细的介绍,并讨论了一些目前存在的难题和未来的研究方向。     

An integer optimization algorithm for robust identification of non-linear gene regulatory networks

ABSTRACT: BACKGROUND: Reverse engineering gene networks and identifying regulatory interactions are integral to understanding cellular decision making processes. Advancement in high throughput experimental techniques has initiated innovative data driven analysis of gene regulatory networks. However, inherent noise associated with biological systems requires numerous experimental replicates for reliable conclusions. Furthermore, evidence of robust algorithms directly exploiting basic biological traits are few. Such algorithms are expected to be efficient in their performance and robust in their prediction. RESULTS: We have developed a network identification algorithm to accurately infer both the topology and strength of regulatory interactions from time series gene expression data in the presence of significant experimental noise and non-linear behavior. In this novel formulism, we have addressed data variability in biological systems by integrating network identification with the bootstrap resampling technique, hence predicting robust interactions from limited experimental replicates subjected to noise. Furthermore, we have incorporated non-linearity in gene dynamics using the S-system formulation. The basic network identification formulation exploits the trait of sparsity of biological interactions. Towards that, the identification algorithm is formulated as an integer-programming problem by introducing binary variables for each network component. The objective function is targeted to minimize the network connections subjected to the constraint of maximal agreement between the experimental and predicted gene dynamics. The developed algorithm is validated using both in-silico and experimental data-sets. These studies show that the algorithm can accurately predict the topology and connection strength of the in silico networks, as quantified by high precision and recall, and small discrepancy between the actual and predicted kinetic parameters. Furthermore, in both the in silico and experimental case studies, the predicted gene expression profiles are in very close agreement with the dynamics of the input data. CONCLUSIONS: Our integer programming algorithm effectively utilizes bootstrapping to identify robust gene regulatory networks from noisy, non-linear time-series gene expression data. With significant noise and non-linearities being inherent to biological systems, the present formulism, with the incorporation of network sparsity, is extremely relevant to gene regulatory networks, and while the formulation has been validated against in silico and E. Coli data, it can be applied to any biological system.     

Hub-centered gene network reconstruction using automatic relevance determination

Network inference deals with the reconstruction of biological networks from experimental data. A variety of different reverse engineering techniques are available; they differ in the underlying assumptions and mathematical models used. One common problem for all approaches stems from the complexity of the task, due to the combinatorial explosion of different network topologies for increasing network size. To handle this problem, constraints are frequently used, for example on the node degree, number of edges, or constraints on regulation functions between network components. We propose to exploit topological considerations in the inference of gene regulatory networks. Such systems are often controlled by a small number of hub genes, while most other genes have only limited influence on the network's dynamic. We model gene regulation using a Bayesian network with discrete, Boolean nodes. A hierarchical prior is employed to identify hub genes. The first layer of the prior is used to regularize weights on edges emanating from one specific node. A second prior on hyperparameters controls the magnitude of the former regularization for different nodes. The net effect is that central nodes tend to form in reconstructed networks. Network reconstruction is then performed by maximization of or sampling from the posterior distribution. We evaluate our approach on simulated and real experimental data, indicating that we can reconstruct main regulatory interactions from the data. We furthermore compare our approach to other state-of-the art methods, showing superior performance in identifying hubs. Using a large publicly available dataset of over 800 cell cycle regulated genes, we are able to identify several main hub genes. Our method may thus provide a valuable tool to identify interesting candidate genes for further study. Furthermore, the approach presented may stimulate further developments in regularization methods for network reconstruction from data.     

Inferring network interactions within a cell

The continuing growth in high-throughput data acquisition has led to a proliferation of network models to represent and analyse biological systems. These networks involve distinct interaction types detected by a combination of methods, ranging from directly observed physical interactions based in biochemistry to interactions inferred from phenotype measurements, genomic expression and comparative genomics. The discovery of interactions increasingly requires a blend of experimental and computational methods. Considering yeast as a model system, recent analytical methods are reviewed here and specific aims are proposed to improve network interaction inference and facilitate predictive biological modelling.     

A declarative constraint-based method for analyzing discrete genetic regulatory networks

Dynamical modeling has proven useful for understanding how complex biological processes emerge from the many components and interactions composing genetic regulatory networks (GRNs). However, the development of models is hampered by large uncertainties in both the network structure and parameter values. To remedy this problem, the models are usually developed through an iterative process based on numerous simulations, confronting model predictions with experimental data and refining the model structure and/or parameter values to repair the inconsistencies. In this paper, we propose an alternative to this generate-and-test approach. We present a four-step method for the systematic construction and analysis of discrete models of GRNs by means of a declarative approach. Instead of instantiating the models as in classical modeling approaches, the biological knowledge on the network structure and its dynamics is formulated in the form of constraints. The compatibility of the network structure with the constraints is queried and in case of inconsistencies, some constraints are relaxed. Common properties of the consistent models are then analyzed by means of dedicated languages. Two such languages are introduced in the paper. Removing questionable constraints or adding interesting ones allows to further analyze the models. This approach allows to identify the best experiments to be carried out, in order to discriminate sets of consistent models and refine our knowledge on the system functioning. We test the feasibility of our approach, by applying it to the re-examination of a model describing the nutritional stress response in the bacterium Escherichia coli.     

Analysis of Discrete Bioregulatory Networks Using Symbolic Steady States

A discrete model of a biological regulatory network can be represented by a discrete function that contains all available information on interactions between network components and the rules governing the evolution of the network in a finite state space. Since the state space size grows exponentially with the number of network components, analysis of large networks is a complex problem. In this paper, we introduce the notion of symbolic steady state that allows us to identify subnetworks that govern the dynamics of the original network in some region of state space. We state rules to explicitly construct attractors of the system from subnetwork attractors. Using the results, we formulate sufficient conditions for the existence of multiple attractors resp. a cyclic attractor based on the existence of positive resp. negative feedback circuits in the graph representing the structure of the system. In addition, we discuss approaches to finding symbolic steady states. We focus both on dynamics derived via synchronous as well as asynchronous update rules. Lastly, we illustrate the results by analyzing a model of T helper cell differentiation.     

Inference of signaling and gene regulatory networks by steady-state perturbation experiments: structure and accuracy

One of the fundamental problems of cell biology is the understanding of complex regulatory networks. Such networks are ubiquitous in cells and knowledge of their properties is essential for the understanding of cellular behavior. In earlier work (Kholodenko et al. (PNAS 99: 12841), it was shown how the structure of biological networks can be quantified from experimental measurements of steady-state concentrations of key intermediates as a result of perturbations using a simple algorithm called "unravelling". Here, we study the effect of experimental uncertainty on the accuracy of the inferred structure (i.e. whether interactions are excitatory or inhibitory) of the networks determined using the unravelling algorithm. We show that the accuracy of the network structure depends not only on the noise level but on the strength of the interactions within the network. In particular, both very small and very large values of the connection strengths lead to large uncertainty in the inferred network. We describe a powerful geometric tool for the intuitive understanding of the effect of experimental error on the qualitative accuracy of the inferred network. In addition, we show that the use of additional data beyond that needed to minimally constrain the network not only improves the accuracy of the inferred network, but also may allow the detection of situations in which the initial assumptions of unravelling with respect to the network and the perturbations have been violated. Our ideas are illustrated using the mitogen-activated protein kinase (MAPK) signaling network as an example.     

Inference of scale-free networks from gene expression time series

Quantitative time-series observation of gene expression is becoming possible, for example by cell array technology. However, there are no practical methods with which to infer network structures using only observed time-series data. As most computational models of biological networks for continuous time-series data have a high degree of freedom, it is almost impossible to infer the correct structures. On the other hand, it has been reported that some kinds of biological networks, such as gene networks and metabolic pathways, may have scale-free properties. We hypothesize that the architecture of inferred biological network models can be restricted to scale-free networks. We developed an inference algorithm for biological networks using only time-series data by introducing such a restriction. We adopt the S-system as the network model, and a distributed genetic algorithm to optimize models to fit its simulated results to observed time series data. We have tested our algorithm on a case study (simulated data). We compared optimization under no restriction, which allows for a fully connected network, and under the restriction that the total number of links must equal that expected from a scale free network. The restriction reduced both false positive and false negative estimation of the links and also the differences between model simulation and the given time-series data.     

Discovering Main Genetic Interactions with LABNet LAsso-Based Network Inference

Genome-wide association studies can potentially unravel the mechanisms behind complex traits and common genetic diseases. Despite the valuable results produced thus far, many questions remain unanswered. For instance, which specific genetic compounds are linked to the risk of the disease under investigation; what biological mechanism do they act through; or how do they interact with environmental and other external factors? The driving force of computational biology is the constantly growing amount of big data generated by high-throughput technologies. A practical framework that can deal with this abundance of information and that consent to discovering genetic associations and interactions is provided by means of networks. Unfortunately, high dimensionality, the presence of noise and the geometry of data can make the aforementioned problem extremely challenging. We propose a penalised linear regression approach that can deal with the aforementioned issues that affect genetic data. We analyse the gene expression profiles of individuals with a common trait to infer the network structure of interactions among genes. The permutation-based approach leads to more stable and reliable networks inferred from synthetic microarray data. We show that a higher number of permutations determines the number of predicted edges, improves the overall sensitivity and controls the number of false positives.     

Theoretical and computational studies of the glucose signaling pathways in yeast using global gene expression data

We have combined DNA microarray experiments with novel computational methods as a means of defining the topology of a biological signal transduction pathway. By DNA microarray techniques, we previously acquired data on expression over time of all genes in the yeast Saccharomyces following addition of glucose to wild-type cells and to cells mutated in one or more components of the Ras signaling network. In addition, we examined the time course of expression following activation of components of the Ras signaling network in the absence of glucose addition. In this current study, we have applied a novel theoretical and computational framework to these data to identify the network topology of the glucose signaling pathway in yeast and the role of Ras components in that network. The computational approach involves clustering genes by expression pattern, postulating a signaling network topology superstructure that includes all possible component interconnections and then evaluating the feasibility of the superstructure interconnections by optimization methods using Mixed Integer Linear Programming techniques. This approach is the first rigorous mathematical framework for addressing the biological network topology issue, and the novel formulation features the introduction of discrete variables for the connectivity and logical expressions that connect the experimental observations to the network structure. This analysis yields a topology for the glucose signaling pathway that is consistent with, and an extension of, known biological interactions in glucose signaling.     

The Evolutionary Dynamics of Protein-Protein Interaction Networks Inferred from the Reconstruction of Ancient Networks

Cellular functions are based on the complex interplay of proteins, therefore the structure and dynamics of these protein-protein interaction (PPI) networks are the key to the functional understanding of cells. In the last years, large-scale PPI networks of several model organisms were investigated. A number of theoretical models have been developed to explain both the network formation and the current structure. Favored are models based on duplication and divergence of genes, as they most closely represent the biological foundation of network evolution. However, studies are often based on simulated instead of empirical data or they cover only single organisms. Methodological improvements now allow the analysis of PPI networks of multiple organisms simultaneously as well as the direct modeling of ancestral networks. This provides the opportunity to challenge existing assumptions on network evolution. We utilized present-day PPI networks from integrated datasets of seven model organisms and developed a theoretical and bioinformatic framework for studying the evolutionary dynamics of PPI networks. A novel filtering approach using percolation analysis was developed to remove low confidence interactions based on topological constraints. We then reconstructed the ancient PPI networks of different ancestors, for which the ancestral proteomes, as well as the ancestral interactions, were inferred. Ancestral proteins were reconstructed using orthologous groups on different evolutionary levels. A stochastic approach, using the duplication-divergence model, was developed for estimating the probabilities of ancient interactions from today''s PPI networks. The growth rates for nodes, edges, sizes and modularities of the networks indicate multiplicative growth and are consistent with the results from independent static analysis. Our results support the duplication-divergence model of evolution and indicate fractality and multiplicative growth as general properties of the PPI network structure and dynamics.     

Reconstructing gene regulatory networks: from random to scale-free connectivity

The manipulation of organisms using combinations of gene knockout, RNAi and drug interaction experiments can be used to reveal regulatory interactions between genes. Several algorithms have been proposed that try to reconstruct the underlying regulatory networks from gene expression data sets arising from such experiments. Often these approaches assume that each gene has approximately the same number of interactions within the network, and the methods rely on prior knowledge, or the investigator's best guess, of the average network connectivity. Recent evidence points to scale-free properties in biological networks, however, where network connectivity follows a power-law distribution. For scale-free networks, the average number of regulatory interactions per gene does not satisfactorily characterise the network. With this in mind, a new reverse engineering approach is introduced that does not require prior knowledge of network connectivity and its performance is compared with other published algorithms using simulated gene expression data with biologically relevant network structures. Because this new approach does not make any assumptions about the distribution of network connections, it is suitable for application to scale-free networks.     

Sampling-rate-dependent probabilistic Boolean networks

External control of a genetic regulatory network is used for the purpose of avoiding undesirable states, such as those associated with a disease. To date, intervention has mainly focused on the external control of probabilistic Boolean networks via the associated discrete-time discrete-space Markov processes. Implementation of an intervention policy derived for probabilistic Boolean networks requires nearly continuous observation of the underlying biological system since precise application requires the observation of all transitions. In medical applications, as in many engineering problems, the process is sampled at discrete time intervals and a decision to intervene or not must be made at each sample point. In this work, sampling-rate-dependent probabilistic Boolean network is proposed as an extension of probabilistic Boolean network. The proposed framework is capable of capturing the sampling rate of the underlying system.     

Reverse engineering of gene regulatory networks: a finite state linear model

We propose a new model for describing gene regulatory networks that can capture discrete (Boolean) and continuous (differential) aspects of gene regulation. After giving some illustrations of the model, we study the problem of the reverse engineering of such networks, i.e., how to construct a network from gene expression data. We prove that for our model there exists an algorithm finding a network compatible with the given data. We demonstrate the model by simulating lambda-phage. We also describe some generalizations of the model, discuss their relevance to the real-world gene networks and formulate a number of open problems.