Inference of S-system models of genetic networks using a cooperative coevolutionary algorithm

MOTIVATION: To resolve the high-dimensionality of the genetic network inference problem in the S-system model, a problem decomposition strategy has been proposed. While this strategy certainly shows promise, it cannot provide a model readily applicable to the computational simulation of the genetic network when the given time-series data contain measurement noise. This is a significant limitation of the problem decomposition, given that our analysis and understanding of the genetic network depend on the computational simulation. RESULTS: We propose a new method for inferring S-system models of large-scale genetic networks. The proposed method is based on the problem decomposition strategy and a cooperative coevolutionary algorithm. As the subproblems divided by the problem decomposition strategy are solved simultaneously using the cooperative coevolutionary algorithm, the proposed method can be used to infer any S-system model ready for computational simulation. To verify the effectiveness of the proposed method, we apply it to two artificial genetic network inference problems. Finally, the proposed method is used to analyze the actual DNA microarray data.     

An intelligent two-stage evolutionary algorithm for dynamic pathway identification from gene expression profiles

From gene expression profiles, it is desirable to rebuild cellular dynamic regulation networks to discover more delicate and substantial functions in molecular biology, biochemistry, bioengineering and pharmaceutics. S-system model is suitable to characterize biochemical network systems and capable to analyze the regulatory system dynamics. However, inference of an S-system model of N-gene genetic networks has 2N(N+1) parameters in a set of non-linear differential equations to be optimized. This paper proposes an intelligent two-stage evolutionary algorithm (iTEA) to efficiently infer the S-system models of genetic networks from time-series data of gene expression. To cope with curse of dimensionality, the proposed algorithm consists of two stages where each uses a divide-and-conquer strategy. The optimization problem is first decomposed into N subproblems having 2(N+1) parameters each. At the first stage, each subproblem is solved using a novel intelligent genetic algorithm (IGA) with intelligent crossover based on orthogonal experimental design (OED). At the second stage, the obtained N solutions to the N subproblems are combined and refined using an OED-based simulated annealing algorithm for handling noisy gene expression profiles. The effectiveness of iTEA is evaluated using simulated expression patterns with and without noise running on a single-processor PC. It is shown that 1) IGA is efficient enough to solve subproblems; 2) IGA is significantly superior to the existing method SPXGA; and 3) iTEA performs well in inferring S-system models for dynamic pathway identification.     

Inference of S-system models of gene regulatory networks using immune algorithm

The S-system model is one of the nonlinear differential equation models of gene regulatory networks, and it can describe various dynamics of the relationships among genes. If we successfully infer rigorous S-system model parameters that describe a target gene regulatory network, we can simulate gene expressions mathematically. However, the problem of finding an optimal S-system model parameter is too complex to be solved analytically. Thus, some heuristic search methods that offer approximate solutions are needed for reducing the computational time. In previous studies, several heuristic search methods such as Genetic Algorithms (GAs) have been applied to the parameter search of the S-system model. However, they have not achieved enough estimation accuracy. One of the conceivable reasons is that the mechanisms to escape local optima. We applied an Immune Algorithm (IA) to search for the S-system parameters. IA is also a heuristic search method, which is inspired by the biological mechanism of acquired immunity. Compared to GA, IA is able to search large solution space, thereby avoiding local optima, and have multiple candidates of the solutions. These features work well for searching the S-system model. Actually, our algorithm showed higher performance than GA for both simulation and real data analyses.     

Inference of Vohradsky's Models of Genetic Networks by Solving Two-Dimensional Function Optimization Problems

The inference of a genetic network is a problem in which mutual interactions among genes are inferred from time-series of gene expression levels. While a number of models have been proposed to describe genetic networks, this study focuses on a mathematical model proposed by Vohradský. Because of its advantageous features, several researchers have proposed the inference methods based on Vohradský''s model. When trying to analyze large-scale networks consisting of dozens of genes, however, these methods must solve high-dimensional non-linear function optimization problems. In order to resolve the difficulty of estimating the parameters of the Vohradský''s model, this study proposes a new method that defines the problem as several two-dimensional function optimization problems. Through numerical experiments on artificial genetic network inference problems, we showed that, although the computation time of the proposed method is not the shortest, the method has the ability to estimate parameters of Vohradský''s models more effectively with sufficiently short computation times. This study then applied the proposed method to an actual inference problem of the bacterial SOS DNA repair system, and succeeded in finding several reasonable regulations.     

A direct substitution, equation error technique for solving the thermographic tomography problem

A new technique for solving the combined state and parameter estimation problem in thermographic tomography is presented. The technique involves the direct substitution of known skin temperatures into the finite difference form of the bio-heat transfer equation as formulated for solving an initial value problem with a convection boundary condition at the skin surface. These equations are then used to solve the inverse bio-heat transfer problem for the unknown subcutaneous tissue temperatures and physiological parameters. For a small number of nodal points, closed form algebraic solutions are obtained. For larger sets of equations, a hybrid technique is used in which the problem is initially posed as an unconstrained optimization problem in which the model equation error is minimized using the conjugate gradient descent technique to get close to a solution. Then a generalized Newton-Raphson technique was used to solve the equations. A numerical simulation of a one-dimensional problem is investigated both with and without noise superimposed on the input (transient) skin temperature data. The results show that the technique gives very accurate results if the skin temperature data contains little noise. It is also shown that if the physical properties of the tissue and the metabolism are known, that a given set of proper transient skin temperature inputs yields a unique solution for the unknown internal temperatures and blood perfusion rates. However, the similar problem with known blood perfusion rates and unknown metabolisms does not yield a unique solution for the internal temperatures and metabolisms.     

Symmetries of S-systems.

An S-system is a set of first-order nonlinear differential equations that all have the same structure: The derivative of a variable is equal to the difference of two products of power-law functions. S-systems have been used as models for a variety of problems, primarily in biology. In addition, S-systems possess the interesting property that large classes of differential equations can be recast exactly as S-systems, a feature that has been proven useful in statistics and numerical analysis. Here, simple criteria are introduced that determine whether an S-system possesses certain types of symmetries and how the underlying transformation groups can be constructed. If a transformation group exists, families of solutions can be characterized, the number of S-system equations necessary for solution can be reduced, and some boundary value problems can be reduced to initial value problems.     

A pattern-directed approach to flexible manufacturing: A framework for intelligent scheduling,learning, and control

The planning, scheduling, and control of manufacturing systems can all be viewed as problem-solving activities. In flexible manufacturing systems (FMSs), the computer program carrying out these problem-solving activities must additionally be able to handle the shorter lead time, the flexibility of job routing, the multiprocessing environment, the dynamic changing states, and the versatility of machines. This article presents an artificial intelligence (AI) method to perform manufacturing problem solving. Since the method is driven by manufacturing scenarios represented by symbolic patterns, it is referred to as pattern-directed. The method is based on three AI techniques. The first is the pattern-directed inference technique to capture the dynamic nature of FMSs. The second is the nonlinear planning technique to construct schedules and assign resources. The third is the inductive learning method to generate the pattern-directed heuristics. This article focuses on solving the FMS scheduling problem. In addition, this article reports the computation results to evaluate the utility of various heuristic functions, to identify important design parameters, and to analyze the resulting computational performance in using the pattern-directed approach for manufacturing problem-solving tasks such as scheduling.     

Maximum likelihood Jukes-Cantor triplets: analytic solutions

Maximum likelihood (ML) is a popular method for inferring a phylogenetic tree of the evolutionary relationship of a set of taxa, from observed homologous aligned genetic sequences of the taxa. Generally, the computation of the ML tree is based on numerical methods, which in a few cases, are known to converge to a local maximum on a tree, which is suboptimal. The extent of this problem is unknown, one approach is to attempt to derive algebraic equations for the likelihood equation and find the maximum points analytically. This approach has so far only been successful in the very simplest cases, of three or four taxa under the Neyman model of evolution of two-state characters. In this paper we extend this approach, for the first time, to four-state characters, the Jukes-Cantor model under a molecular clock, on a tree T on three taxa, a rooted triple. We employ spectral methods (Hadamard conjugation) to express the likelihood function parameterized by the path-length spectrum. Taking partial derivatives, we derive a set of polynomial equations whose simultaneous solution contains all critical points of the likelihood function. Using tools of algebraic geometry (the resultant of two polynomials) in the computer algebra packages (Maple), we are able to find all turning points analytically. We then employ this method on real sequence data and obtain realistic results on the primate-rodents divergence time.     

Bayesian estimation of dominance merits in noninbred populations by using Gibbs sampling with two reduced sets of mixed model equations

Henderson's mixed model equations system is generally required in a Gibbs sampling application. In two previous studies, we proposed two indirect solving approaches that give dominance values in an animal model context with no need to process all this system. The first one does not require D-1 and the second is based on processing the additive animal model residuals. In the present work, we show that these two methods can be handled iteratively. Since the Bayesian approach is now a widely used tool in estimation of genetic parameters, the main part of this work is devoted to a Gibbs sampling application that can be accelerated by means of the aforementioned indirect solving methods. Three replicates of a population data set are simulated in the paper to compare the applications and estimates. This shows effectively that the estimates given by implementing a Gibbs sampler with each of the two suggested solving methods are obtained with less computational time and are comparable to those given by considering the integral system, particularly when priors are more weighted.     

Three-dimensional reconstruction using an adaptive simultaneous algebraic reconstruction technique in electron tomography

Three-dimensional (3D) reconstruction of electron tomography (ET) has emerged as an important technique in analyzing structures of complex biological samples. However most of existing reconstruction methods are not suitable for extremely noisy and incomplete data conditions. We present an adaptive simultaneous algebraic reconstruction technique (ASART) in which a modified multilevel access scheme and an adaptive relaxation parameter adjustment method are developed to improve the quality of the reconstructed 3D structure. The reconstruction process is facilitated by using a column-sum substitution approach. This modified multilevel access scheme is adopted to arrange the order of projections so as to minimize the correlations between consecutive views within a limited angle range. In the adaptive relaxation parameter adjustment method, not only the weight matrix (as in the existing methods) but the gray levels of the pixels are employed to adjust the relaxation parameters so that the quality of the reconstruction is improved while the convergence process of the reconstruction is accelerated. In the column-sum substitution approach, the computation to obtain the reciprocal of the sum for the columns in each view is avoided so that the needed computations for each iteration can be reduced. Experimental results show that the proposed technique ASART is better based on objective quality measures than other methods, especially when data is noisy and limited in tilt angles. At the same time, the reconstruction by ASART outperforms that of simultaneous algebraic reconstruction technique (SART) in speed.     

Optimization of regulatory architectures in metabolic reaction networks

Successful biotechnological applications, such as amino acid production, have demonstrated significant improvement in bioprocess performance by genetic modifications of metabolic control architectures and enzyme expression levels. However, the stoichiometric complexity of metabolic pathways, along with their strongly nonlinear nature and regulatory coupling, necessitates the use of structured kinetic models to direct experimental applications and aid in quantitative understanding of cellular bioprocesses. A novel optimization problem is introduced here, the objective of which is to identify changes in the regulatory characteristics of pertinent enzymes and in their cellular content which should be implemented to optimize a particular metabolic process. The mathematical representation of the metabolic reaction networks used is the S-system representation, which at steady state is characterized by linear equations. Exploiting the linearity of the representation, we formulated the optimization problem as a mixed-integer linear programming (MILP) problem. This formulation allows the consideration of a regulatory superstructure that contains all alternative regulatory structures that can be considered for a given pathway. The proposed approach is developed and illustrated using a simple linear pathway. Application of the framework on a complicated pathway-namely, the xanthine monophosphate (XMP) and guanosine monophosphate (GMP) synthesis pathway-identified the modification of the regulatory architecture that, along with changes in enzyme expression levels, can increase the XMP and GMP concentration by over 114 times the reference value, which is 50 times more than could be achieved by changes in enzyme expression levels only. (c) 1996 John Wiley & Sons, Inc.     

Hierarchical genetic algorithm for near optimal feedforward neural network design

In this paper, we propose a genetic algorithm based design procedure for a multi layer feed forward neural network. A hierarchical genetic algorithm is used to evolve both the neural networks topology and weighting parameters. Compared with traditional genetic algorithm based designs for neural networks, the hierarchical approach addresses several deficiencies, including a feasibility check highlighted in literature. A multi objective cost function is used herein to optimize the performance and topology of the evolved neural network simultaneously. In the prediction of Mackey Glass chaotic time series, the networks designed by the proposed approach prove to be competitive, or even superior, to traditional learning algorithms for the multi layer Perceptron networks and radial basis function networks. Based upon the chosen cost function, a linear weight combination decision making approach has been applied to derive an approximated Pareto optimal solution set. Therefore, designing a set of neural networks can be considered as solving a two objective optimization problem.     

Biochemical systems theory: increasing predictive power by using second-order derivatives measurements

Models based on the power-law formalism provide a useful tool for analyzing metabolic systems. Within this methodology, the S-system variant furnishes the best strategy. In this paper we explore an extension of this formalism by considering second-order derivative terms of the Taylor series which the power-law is based upon. Results show that the S-system equations which include second-order Taylor coefficients give better accuracy in predicting the response of the system to a perturbation. Hence, models based on this new approach could provide a useful tool for quantitative purposes if one is able to measure the required derivatives experimentally. In particular we show the utility of this approach when it comes to discriminating between two mechanisms that are equivalent in the S-system a representation based on first-order coefficients. However, the loss of analytical tractability is a serious disadvantage for using this approach as a general tool for studying metabolic systems.     

An improved surface-based method for DNA computation

DNA computing is a novel method for solving a class of intractable computational problems, in which the computing time can grow exponentially with problem size. Up to now, many accomplishments have been achieved to improve its performance and increase its reliability, among which a surface-based method is an efficient candidate. In this paper, the surface-based approach proposed by Liu, Q., Wang, L., Frutos, A.G., Condon, A.E., Corn, R.M., and Smith, L.M., 2000, DNA computing on surfaces. Nature 403, 175-179 is analyzed and an improved surface-based method for DNA computation (i.e. the hybrid DNA/optical computing method) is proposed. Compared with Liu et al.'s approach, our method has some significant advantages such as low cost, short operating time, reusable surface and simple experimental steps. Moreover, the concept of combining easily patterned DNA computing steps with equally parallel, but generally uniform and not easily patterned optical computing steps is an important new direction.     

Decoupling dynamical systems for pathway identification from metabolic profiles

RATIONALE: Modern molecular biology is generating data of unprecedented quantity and quality. Particularly exciting for biochemical pathway modeling and proteomics are comprehensive, time-dense profiles of metabolites or proteins that are measurable, for instance, with mass spectrometry, nuclear magnetic resonance or protein kinase phosphorylation. These profiles contain a wealth of information about the structure and dynamics of the pathway or network from which the data were obtained. The retrieval of this information requires a combination of computational methods and mathematical models, which are typically represented as systems of ordinary differential equations. RESULTS: We show that, for the purpose of structure identification, the substitution of differentials with estimated slopes in non-linear network models reduces the coupled system of differential equations to several sets of decoupled algebraic equations, which can be processed efficiently in parallel or sequentially. The estimation of slopes for each time series of the metabolic or proteomic profile is accomplished with a 'universal function' that is computed directly from the data by cross-validated training of an artificial neural network (ANN). CONCLUSIONS: Without preprocessing, the inverse problem of determining structure from metabolic or proteomic profile data is challenging and computationally expensive. The combination of system decoupling and data fitting with universal functions simplifies this inverse problem very significantly. Examples show successful estimations and current limitations of the method. AVAILABILITY: A preliminary Web-based application for ANN smoothing is accessible at http://bioinformatics.musc.edu/webmetabol/. S-systems can be interactively analyzed with the user-friendly freeware PLAS (http://correio.cc.fc.ul.pt/~aenf/plas.html) or with the MATLAB module BSTLab (http://bioinformatics.musc.edu/bstlab/), which is currently being beta-tested.     

Dynamic optimization of bioprocesses: efficient and robust numerical strategies

The dynamic optimization (open loop optimal control) of non-linear bioprocesses is considered in this contribution. These processes can be described by sets of non-linear differential and algebraic equations (DAEs), usually subject to constraints in the state and control variables. A review of the available solution techniques for this class of problems is presented, highlighting the numerical difficulties arising from the non-linear, constrained and often discontinuous nature of these systems. In order to surmount these difficulties, we present several alternative stochastic and hybrid techniques based on the control vector parameterization (CVP) approach. The CVP approach is a direct method which transforms the original problem into a non-linear programming (NLP) problem, which must be solved by a suitable (efficient and robust) solver. In particular, a hybrid technique uses a first global optimization phase followed by a fast second phase based on a local deterministic method, so it can handle the nonconvexity of many of these NLPs. The efficiency and robustness of these techniques is illustrated by solving several challenging case studies regarding the optimal control of fed-batch bioreactors and other bioprocesses. In order to fairly evaluate their advantages, a careful and critical comparison with several other direct approaches is provided. The results indicate that the two-phase hybrid approach presents the best compromise between robustness and efficiency.     

An automated procedure for the extraction of metabolic network information from time series data

Novel high-throughput measurement techniques in vivo are beginning to produce dense high-quality time series which can be used to investigate the structure and regulation of biochemical networks. We propose an automated information extraction procedure which takes advantage of the unique S-system structure and supports model building from time traces, curve fitting, model selection, and structure identification based on parameter estimation. The procedure comprises of three modules: model Generation, parameter estimation or model Fitting, and model Selection (GFS algorithm). The GFS algorithm has been implemented in MATLAB and returns a list of candidate S-systems which adequately explain the data and guides the search to the most plausible model for the time series under study. By combining two strategies (namely decoupling and limiting connectivity) with methods of data smoothing, the proposed algorithm is scalable up to realistic situations of moderate size. We illustrate the proposed methodology with a didactic example.     

Hydrodynamics of bolus flow—An analytical approach to blood flow in capillaries

A model which was used by Prothero and Burton to simulate a particular configuration in capillary blood vessels is investigated from a hydrodynamic point of view. In this model, the erythrocytes are approximated by rigid pistons, and plasma is assumed to be an incompressible Newtonian fluid. An order of magnitude analysis using the physiologically realistic values for various parameters reduces the exact equations of motion to an equation describing the creeping motion of the fluid. An analytical approach to the solution of the equation is proposed and some results are reported here. The solution of the flow field is given in terms of a stream function which is represented by two infinite series composed of known functions. Two coupled infinite systems of algebraic equations determining the coefficients of the two series have been derived. This method of solution is proposed as an alternative to the entirely numerical procedure of solving the similar problem proposed by Bugliarelloet al. A limiting case of large aspect ratio (the ratio of the axial spacing of the two successive erythrocytes to the capillary diameter) is studied and the solution, valid away from the erythrocyte surface, has been obtained in simple form. It resembles the classical Poisenille flow, but the pressure gradient is related to the erythrocyte speed.     

Dynamics of Random Boolean Networks under Fully Asynchronous Stochastic Update Based on Linear Representation

A novel algebraic approach is proposed to study dynamics of asynchronous random Boolean networks where a random number of nodes can be updated at each time step (ARBNs). In this article, the logical equations of ARBNs are converted into the discrete-time linear representation and dynamical behaviors of systems are investigated. We provide a general formula of network transition matrices of ARBNs as well as a necessary and sufficient algebraic criterion to determine whether a group of given states compose an attractor of length in ARBNs. Consequently, algorithms are achieved to find all of the attractors and basins in ARBNs. Examples are showed to demonstrate the feasibility of the proposed scheme.