A continuous optimization approach for inferring parameters in mathematical models of regulatory networks

Background

The advances of systems biology have raised a large number of sophisticated mathematical models for describing the dynamic property of complex biological systems. One of the major steps in developing mathematical models is to estimate unknown parameters of the model based on experimentally measured quantities. However, experimental conditions limit the amount of data that is available for mathematical modelling. The number of unknown parameters in mathematical models may be larger than the number of observation data. The imbalance between the number of experimental data and number of unknown parameters makes reverse-engineering problems particularly challenging.

Results

To address the issue of inadequate experimental data, we propose a continuous optimization approach for making reliable inference of model parameters. This approach first uses a spline interpolation to generate continuous functions of system dynamics as well as the first and second order derivatives of continuous functions. The expanded dataset is the basis to infer unknown model parameters using various continuous optimization criteria, including the error of simulation only, error of both simulation and the first derivative, or error of simulation as well as the first and second derivatives. We use three case studies to demonstrate the accuracy and reliability of the proposed new approach. Compared with the corresponding discrete criteria using experimental data at the measurement time points only, numerical results of the ERK kinase activation module show that the continuous absolute-error criteria using both function and high order derivatives generate estimates with better accuracy. This result is also supported by the second and third case studies for the G1/S transition network and the MAP kinase pathway, respectively. This suggests that the continuous absolute-error criteria lead to more accurate estimates than the corresponding discrete criteria. We also study the robustness property of these three models to examine the reliability of estimates. Simulation results show that the models with estimated parameters using continuous fitness functions have better robustness properties than those using the corresponding discrete fitness functions.

Conclusions

The inference studies and robustness analysis suggest that the proposed continuous optimization criteria are effective and robust for estimating unknown parameters in mathematical models.

Electronic supplementary material

The online version of this article (doi:10.1186/1471-2105-15-256) contains supplementary material, which is available to authorized users.     

Design of experiments for the precise estimation of dose-response parameters: the Hill equation

Optimal experimental designs were evaluated for the precise estimation of parameters of the Hill model. The optimally effective designs were obtained by using the criterion of D-optimization. For the Hill model, optimal designs replicate 3 sampling points. These points were shown to be quite sensitive to the behavior of the experimental error. Since an investigator is often uncertain about error conditions in biological studies, a practical approach would use the sampling scheme calculated for an intermediate error condition. Thus, if the behavior of error variances is not known, precise parameters of the Hill model are obtained by choosing concentrations which yield fractional responses (responses divided by their asymptotic, maximum value) of 0.086, 0.581 and 1.0. When experimental constraints limit the maximum attainable concentration and response, all design points are lowered. Appropriate designs can be constructed based on the design which is optimal when constraints result in a maximum attainable fractional response of 0.5. The optimal designs were found to be robust when the parameter values assumed by the investigator did not equal their true values. The estimating efficiencies obtained by using two frequently applied designs were assessed. Uniformly spaced concentrations yielded imprecise parameters. Six-point, geometrically spaced designs gave generally good results. However, their estimating efficiency was generally exceeded by the recommended sampling schemes even in the presence of uncertainty about error conditions. The method exemplified in this paper can be used for other models.     

The Arabidopsis thaliana flower organ specification gene regulatory network determines a robust differentiation process

The Arabidopsis thaliana flower organ specification gene regulatory network (FOS-GRN) has been modeled previously as a discrete dynamical system, recovering as steady states configurations that match the genetic profiles described in primordial cells of inflorescence, sepals, petals, stamens and carpels during early flower development. In this study, we first update the FOS-GRN by adding interactions and modifying some rules according to new experimental data. A discrete model of this updated version of the network has a dynamical behavior identical to previous versions, under both wild type and mutant conditions, thus confirming its robustness. Then, we develop a continuous version of the FOS-GRN using a new methodology that builds upon previous proposals. The fixed point attractors of the discrete system are all observed in the continuous model, but the latter also contains new steady states that might correspond to genetic activation states present briefly during the early phases of flower development. We show that both the discrete and the continuous models recover the observed stable gene configurations observed in the inflorescence meristem, as well as the primordial cells of sepals, petals, stamens and carpels. Additionally, both models are subjected to perturbations in order to establish the nature of additional signals that may suffice to determine the experimentally observed order of appearance of floral organs. Our results thus describe a possible mechanism by which the network canalizes molecular signals and/or noise, thus conferring robustness to the differentiation process.     

Non-Iterative Least Squares Estimation of Missing Values in Replicated Latin Square Designs

In this paper an attempt has been made to estimate several missing values in replicated latin square designs. The explicit computable expressions for the non-iterative least squares estimates of the missing values are presented for particular patterns of missing values.     

基于卷积神经网络-长短期记忆神经网络模型利用光学体积描记术重建动脉血压波信号

目的 直接动脉血压（arterial blood pressure,ABP）连续监测是侵入式的,传统袖带式的间接血压测量法无法实现连续监测。既往利用光学体积描记术（photoplethysmography,PPG）实现了连续无创血压监测,但其为收缩压和舒张压的离散值,而非ABP波的连续值,本研究期望基于卷积神经网络-长短期记忆神经网络（CNN-LSTM）利用PPG信号波重建ABP波信号,实现连续无创血压监测。方法 构建CNN-LSTM混合神经网络模型,利用重症监护医学信息集（medical information mart for intensive care,MIMIC）中的PPG与ABP波同步记录信号数据,将PPG信号波经预处理降噪、归一化、滑窗分割后输入该模型,重建与之同步对应的ABP波信号。结果 使用窗口长度312的CNN-LSTM神经网络时,重建ABP值与实际ABP值间误差最小,平均绝对误差（mean absolute error,MAE）和均方根误差（root mean square error,RMSE）分别为2.79 mmHg和4.24 mmHg,余弦相似度最大,重建ABP值与实际ABP值一致性和相关性情况良好,符合美国医疗器械促进协会（Association for the Advancement of Medical Instrumentation,AAMI）标准。结论 CNN-LSTM混合神经网络可利用PPG信号波重建ABP波信号,实现连续无创血压监测。     

From Boolean Network Model to Continuous Model Helps in Design of Functional Circuits

Computational circuit design with desired functions in a living cell is a challenging task in synthetic biology. To achieve this task, numerous methods that either focus on small scale networks or use evolutionary algorithms have been developed. Here, we propose a two-step approach to facilitate the design of functional circuits. In the first step, the search space of possible topologies for target functions is reduced by reverse engineering using a Boolean network model. In the second step, continuous simulation is applied to evaluate the performance of these topologies. We demonstrate the usefulness of this method by designing an example biological function: the SOS response of E. coli. Our numerical results show that the desired function can be faithfully reproduced by candidate networks with different parameters and initial conditions. Possible circuits are ranked according to their robustness against perturbations in parameter and gene expressions. The biological network is among the candidate networks, yet novel designs can be generated. Our method provides a scalable way to design robust circuits that can achieve complex functions, and makes it possible to uncover design principles of biological networks.     

On the Relationship of Steady States of Continuous and Discrete Models Arising from Biology

For many biological systems that have been modeled using continuous and discrete models, it has been shown that such models have similar dynamical properties. In this paper, we prove that this happens in more general cases. We show that under some conditions there is a bijection between the steady states of continuous and discrete models arising from biological systems. Our results also provide a novel method to analyze certain classes of nonlinear models using discrete mathematics.     

Quantifying surface gradients with a 2-band Enhanced Vegetation Index (EVI2)

Quantification of spatial and temporal heterogeneity has been given much attention in order to link ecological patterns to processes. The patch mosaic model, as an operational paradigm, has led to major advances in the field of quantitative landscape ecology. However, it is more realistic to conceptualize landscapes based on continuous rather than discrete spatial heterogeneity. While a conceptual shift has been proposed to supplement the patch mosaic model, few studies use the surface gradient model as a context. This paper explores some comparatively less-utilized metrics to quantify surface gradients in a protected landscape in Central India. Since surface metrics would require continuous variables capable of representing landscape characteristics, this study also explores the utility of a 2-band Enhanced Vegetation Index (EVI2) as a gradient surface. Findings suggest EVI2 relates strongly with discrete land cover classes and thus has potential to describe landscape characteristics without incorporating error through subjectivity. Surface metrics used in this study show potential to be effectively used in landscape level studies. However, these metrics were not developed for landscape level studies and should be used with caution, especially when dealing with multi-scale patterns and processes. Nevertheless, with the rapidly emerging field of surface metrology more studies need to apply these tools to quantify surface gradients and test the robustness of such metrics.     

Density-structured models for plant population dynamics

Density-structured models are structured population models in which the state variable is the proportion of populations or sites in a small number of discrete density states. Although such models have rarely been used, they have the advantage that they are straightforward to parameterize, make few assumptions about population dynamics, and permit rapid data collection using coarse density assessment. In this article, we highlight their use in relating population dynamics to environmental variation and their robustness to measurement error. We show that density-structured models are able to accurately represent population dynamics under a wide range of conditions. We look at the effects of including a persistent seedbank and describe numerical approximations for the mean and variance of population size. For simulated data, we determine the extent to which the underlying continuous process may be inferred from density-structured data. Finally, we discuss issues of parameter estimation and applications for which these types of models may be useful.     

A comparative method for both discrete and continuous characters using the threshold model

The threshold model developed by Sewall Wright in 1934 can be used to model the evolution of two-state discrete characters along a phylogeny. The model assumes that there is a quantitative character, called liability, that is unobserved and that determines the discrete character according to whether the liability exceeds a threshold value. A Markov chain Monte Carlo algorithm is used to infer the evolutionary covariances of the liabilities for discrete characters, sampling liability values consistent with the phylogeny and with the observed data. The same approach can also be used for continuous characters by assuming that the tip species have values that have been observed. In this way, one can make a comparative-methods analysis that combines both discrete and continuous characters. Simulations are presented showing that the covariances of the liabilities are successfully estimated, although precision can be achieved only by using a large number of species, and we must always worry whether the covariances and the model apply throughout the group. An advantage of the threshold model is that the model can be straightforwardly extended to accommodate within-species phenotypic variation and allows an interface with quantitative-genetics models.     

Quantitative trait linkage analysis using Gaussian copulas

Mapping and identifying variants that influence quantitative traits is an important problem for genetic studies. Traditional QTL mapping relies on a variance-components (VC) approach with the key assumption that the trait values in a family follow a multivariate normal distribution. Violation of this assumption can lead to inflated type I error, reduced power, and biased parameter estimates. To accommodate nonnormally distributed data, we developed and implemented a modified VC method, which we call the "copula VC method," that directly models the nonnormal distribution using Gaussian copulas. The copula VC method allows the analysis of continuous, discrete, and censored trait data, and the standard VC method is a special case when the data are distributed as multivariate normal. Through the use of link functions, the copula VC method can easily incorporate covariates. We use computer simulations to show that the proposed method yields unbiased parameter estimates, correct type I error rates, and improved power for testing linkage with a variety of nonnormal traits as compared with the standard VC and the regression-based methods.     

Resolvability of fluorescence lifetime distributions using phase fluorometry.

The analysis of the fluorescence decay using discrete exponential components assumes that a small number of species is present. In the absence of a definite kinetic model or when a large number of species is present, the exponential analysis underestimates the uncertainty of the recovered lifetime values. A different approach to determine the lifetime of a population of molecules is the use of probability density functions and lifetime distributions. Fluorescence decay data from continuous distributions of exponentially decaying components were generated. Different magnitudes of error were added to the data to simulate experimental conditions. The resolvability of the distributional model was studied by fitting the simulated data to one and two exponentials. The maximum width of symmetric distributions (uniform, gaussian, and lorentzian), which cannot be distinguished from single and double exponential fits for statistical errors of 1 and 0.1%, were determined. The width limits are determined by the statistical error of the data. It is also shown that, in the frequency domain, the discrete exponential analysis does not uniformly weights all the components of a distribution. This systematic error is less important when probability and distribution functions are used to recover the decay. Finally, it is shown that real lifetime distributions can be proved using multimodal probability density functions. In the companion paper that follows we propose a physical approach, which provides lifetime distribution functions for the tryptophan decay in proteins. In the third companion paper (Alcala, J.R., E. Gratton, and F.J. Prendergast, 1987, Biophys. J., in press) we use the distribution functions obtained to fit data from the fluorescence decay of single tryptophan proteins.     

Adaptive designs with arbitrary dependence structure

Adaptive designs were originally developed for independent and uniformly distributed p‐values. There are trial settings where independence is not satisfied or where it may not be possible to check whether it is satisfied. In these cases, the test statistics and p‐values of each stage may be dependent. Since the probability of a type I error for a fixed adaptive design depends on the true dependence structure between the p‐values of the stages, control of the type I error rate might be endangered if the dependence structure is not taken into account adequately. In this paper, we address the problem of controlling the type I error rate in two‐stage adaptive designs if any dependence structure between the test statistics of the stages is admitted (worst case scenario). For this purpose, we pursue a copula approach to adaptive designs. For two‐stage adaptive designs without futility stop, we derive the probability of a type I error in the worst case, that is for the most adverse dependence structure between the p‐values of the stages. Explicit analytical considerations are performed for the class of inverse normal designs. A comparison with the significance level for independent and uniformly distributed p‐values is performed. For inverse normal designs without futility stop and equally weighted stages, it turns out that correcting for the worst case is too conservative as compared to a simple Bonferroni design.     

Locally Weighted Score Estimation for Quantile Classification in Binary Regression Models

One common use of binary response regression methods is classification based on an arbitrary probability threshold dictated by the particular application. Since this is given to us a priori, it is sensible to incorporate the threshold into our estimation procedure. Specifically, for the linear logistic model, we solve a set of locally weighted score equations, using a kernel-like weight function centered at the threshold. The bandwidth for the weight function is selected by cross validation of a novel hybrid loss function that combines classification error and a continuous measure of divergence between observed and fitted values; other possible cross-validation functions based on more common binary classification metrics are also examined. This work has much in common with robust estimation, but differs from previous approaches in this area in its focus on prediction, specifically classification into high- and low-risk groups. Simulation results are given showing the reduction in error rates that can be obtained with this method when compared with maximum likelihood estimation, especially under certain forms of model misspecification. Analysis of a melanoma dataset is presented to illustrate the use of the method in practice.     

Improving the Flexibility and Efficiency of Phase II Designs for Oncology Trials

Summary Phase II trials in oncology are usually conducted as single-arm two-stage designs with binary endpoints. Currently available adaptive design methods are tailored to comparative studies with continuous test statistics. Direct transfer of these methods to discrete test statistics results in conservative procedures and, therefore, in a loss in power. We propose a method based on the conditional error function principle that directly accounts for the discreteness of the outcome. It is shown how application of the method can be used to construct new phase II designs that are more efficient as compared to currently applied designs and that allow flexible mid-course design modifications. The proposed method is illustrated with a variety of frequently used phase II designs.     

微生物连续发酵酶催化动力系统的鲁棒性与并行计算

依据甘油连续发酵生产1,3-丙二醇的非线性酶催化动力系统.针对1,3-丙二醇可能存在的跨膜运输方式建立相应的动力学模型,提出了酶催化动力系统的定量鲁棒性定义,并建立了以鲁棒性为性能指标、非线性动力系统为主要约束的参数辨识模型.由于求解该辨识问题的数值计算量大,在普通的PC机上难以完成,因此本文构建了相应的并行算法.根据数值结果推断出1,3-丙二醇最有可能的跨膜运输方式,这对于进一步研究甘油连续发酵的机理具有重要的参考价值.     

Parametric and Parametrically Smoothed Distribution-Free Proportional Hazard Models with Discrete Data

This paper discusses discrete time proportional hazard models and suggests a new class of flexible hazard functions. Explicitly modeling the discreteness of data is important since standard continuous models are biased; allowing for flexibility in the hazard estimation is desirable since strong parametric restrictions are likely to be similarly misleading. Simulation compare continuous and discrete models when data are generated by grouping and demonstrate that simple approximations recover underlying hazards well and outperform nonparametric maximum likelihood estimates in term of mean squared error.     

A Realistic Model Under Which the Genetic Code is Optimal

The genetic code has a high level of error robustness. Using values of hydrophobicity scales as a proxy for amino acid character, and the mean square measure as a function quantifying error robustness, a value can be obtained for a genetic code which reflects the error robustness of that code. By comparing this value with a distribution of values belonging to codes generated by random permutations of amino acid assignments, the level of error robustness of a genetic code can be quantified. We present a calculation in which the standard genetic code is shown to be optimal. We obtain this result by (1) using recently updated values of polar requirement as input; (2) fixing seven assignments (Ile, Trp, His, Phe, Tyr, Arg, and Leu) based on aptamer considerations; and (3) using known biosynthetic relations of the 20 amino acids. This last point is reflected in an approach of subdivision (restricting the random reallocation of assignments to amino acid subgroups, the set of 20 being divided in four such subgroups). The three approaches to explain robustness of the code (specific selection for robustness, amino acid–RNA interactions leading to assignments, or a slow growth process of assignment patterns) are reexamined in light of our findings. We offer a comprehensive hypothesis, stressing the importance of biosynthetic relations, with the code evolving from an early stage with just glycine and alanine, via intermediate stages, towards 64 codons carrying todays meaning.     

Efficient two-sample designs for microarray experiments with biological replications

In the last years, biostatistical research has begun to apply linear models and design theory to develop efficient experimental designs and analysis tools for gene expression microarray data. With two-colour microarrays, direct comparisons of RNA-targets are possible and lead to incomplete block designs. In this setting, efficient designs for simple and factorial microarray experiments have mainly been proposed for technical replicates. But for biological replicates, which are crucial to obtain inference that can be generalised to a biological population, this question has only been discussed recently and is not fully solved yet. In this paper, we propose efficient designs for independent two-sample experiments using two-colour microarrays enabling biologists to measure their biological random samples in an efficient manner to draw generalisable conclusions. We give advice for experimental situations with differing group sizes and show the impact of different designs on the variance and degrees of freedom of the test statistics. The designs proposed in this paper can be evaluated using SAS PROC MIXED or S+/R lme.