An unbiased sensitivity analysis reveals important parameters controlling periodicity of circadian clock

To assess the importance of model parameters in kinetic models, sensitivity analysis is generally employed to provide key measures. However, it is quite often that no information is available for a significant number of parameters in biochemical models. Therefore, the results of sensitivity analysis that heavily rely on the accuracy of parameters are largely ambiguous. In this study, we propose a computational approach to determine the relative importance of parameters controlling the performance of the circadian clock in Drosophila. While previous attempts to sensitivity analysis largely depend on the knowledge of model parameters which are generally unknown, our study depicts a consistent picture of sensitivity assessment for a large number of parameters, even when the values of these parameters are not available in vivo. The resulting parametric sensitivity analysis suggests that PER/TIM negative loop is critical to maintain the stable periodicity of the circadian clock, which is consistent to the previously experimental and computational findings. Furthermore, our analysis generates a rich hypothesis of important parameters in the circadian clock that can be further tested experimentally. This approach can also be extended to assess the sensitivity of parameters in any biochemical system where a large number of parameters have unknown values. Biotechnol. Bioeng. 2010; 105: 250–259. © 2009 Wiley Periodicals, Inc.     

Dynamic Bayesian sensitivity analysis of a myocardial metabolic model

Dynamic compartmentalized metabolic models are identified by a large number of parameters, several of which are either non-physical or extremely difficult to measure. Typically, the available data and prior information is insufficient to fully identify the system. Since the models are used to predict the behavior of unobserved quantities, it is important to understand how sensitive the output of the system is to perturbations in the poorly identifiable parameters. Classically, it is the goal of sensitivity analysis to asses how much the output changes as a function of the parameters. In the case of dynamic models, the output is a function of time and therefore its sensitivity is a time dependent function. If the output is a differentiable function of the parameters, the sensitivity at one time instance can be computed from its partial derivatives with respect to the parameters. The time course of these partial derivatives describes how the sensitivity varies in time.When the model is not uniquely identifiable, or if the solution of the parameter identification problem is known only approximately, we may have not one, but a distribution of possible parameter values. This is always the case when the parameter identification problem is solved in a statistical framework. In that setting, the proper way to perform sensitivity analysis is to not rely on the values of the sensitivity functions corresponding to a single model, but to consider the distributed nature of the sensitivity functions, inherited from the distribution of the vector of the model parameters.In this paper we propose a methodology for analyzing the sensitivity of dynamic metabolic models which takes into account the variability of the sensitivity over time and across a sample. More specifically, we draw a representative sample from the posterior density of the vector of model parameters, viewed as a random variable. To interpret the output of this doubly varying sensitivity analysis, we propose visualization modalities particularly effective at displaying simultaneously variations over time and across a sample. We perform an analysis of the sensitivity of the concentrations of lactate and glycogen in cytosol, and of ATP, ADP, NAD⁺ and NADH in cytosol and mitochondria, to the parameters identifying a three compartment model for myocardial metabolism during ischemia.     

Bayesian analysis of amino acid substitution models

Models of amino acid substitution present challenges beyond those often faced with the analysis of DNA sequences. The alignments of amino acid sequences are often small, whereas the number of parameters to be estimated is potentially large when compared with the number of free parameters for nucleotide substitution models. Most approaches to the analysis of amino acid alignments have focused on the use of fixed amino acid models in which all of the potentially free parameters are fixed to values estimated from a large number of sequences. Often, these fixed amino acid models are specific to a gene or taxonomic group (e.g. the Mtmam model, which has parameters that are specific to mammalian mitochondrial gene sequences). Although the fixed amino acid models succeed in reducing the number of free parameters to be estimated--indeed, they reduce the number of free parameters from approximately 200 to 0--it is possible that none of the currently available fixed amino acid models is appropriate for a specific alignment. Here, we present four approaches to the analysis of amino acid sequences. First, we explore the use of a general time reversible model of amino acid substitution using a Dirichlet prior probability distribution on the 190 exchangeability parameters. Second, we then explore the behaviour of prior probability distributions that are'centred' on the rates specified by the fixed amino acid model. Third, we consider a mixture of fixed amino acid models. Finally, we consider constraints on the exchangeability parameters as partitions,similar to how nucleotide substitution models are specified, and place a Dirichlet process prior model on all the possible partitioning schemes.     

Sensitivity Analysis of an ENteric Immunity SImulator (ENISI)-Based Model of Immune Responses to Helicobacter pylori Infection

Agent-based models (ABM) are widely used to study immune systems, providing a procedural and interactive view of the underlying system. The interaction of components and the behavior of individual objects is described procedurally as a function of the internal states and the local interactions, which are often stochastic in nature. Such models typically have complex structures and consist of a large number of modeling parameters. Determining the key modeling parameters which govern the outcomes of the system is very challenging. Sensitivity analysis plays a vital role in quantifying the impact of modeling parameters in massively interacting systems, including large complex ABM. The high computational cost of executing simulations impedes running experiments with exhaustive parameter settings. Existing techniques of analyzing such a complex system typically focus on local sensitivity analysis, i.e. one parameter at a time, or a close “neighborhood” of particular parameter settings. However, such methods are not adequate to measure the uncertainty and sensitivity of parameters accurately because they overlook the global impacts of parameters on the system. In this article, we develop novel experimental design and analysis techniques to perform both global and local sensitivity analysis of large-scale ABMs. The proposed method can efficiently identify the most significant parameters and quantify their contributions to outcomes of the system. We demonstrate the proposed methodology for ENteric Immune SImulator (ENISI), a large-scale ABM environment, using a computational model of immune responses to Helicobacter pylori colonization of the gastric mucosa.     

Sensitivity analysis to identify key parameters influencing Salmonella infection dynamics in a pig batch

In the context of managed herds, epidemiological models usually take into account relatively complex interactions involving a high number of parameters. Some parameters may be uncertain and/or highly variable, especially epidemiological parameters. Their impact on the model outputs must then be assessed by a sensitivity analysis, allowing to identify key parameters. The prevalence over time is an output of particular interest in epidemiological models, so sensitivity analysis methods adapted to such dynamic output are needed.In this paper, such a sensitivity analysis method, based on a principal component analysis and on analysis of variance, is presented. It allows to compute a generalised sensitivity index for each parameter of a model representing Salmonella spread within a pig batch. The model is a stochastic discrete-time model describing the batch dynamics and movements between rearing rooms, from birth to slaughterhouse delivery. Four health states were introduced: Salmonella-free, seronegative shedder, seropositive shedder and seropositive carrier. The indirect transmission was modelled via an infection probability function depending on the quantity of Salmonella in the rearing room.Simulations were run according to a fractional factorial design enabling the estimation of main effects and two-factor interactions. For each of the 18 epidemiological parameters, four values were chosen, leading to 4096 scenarios. For each scenario, 15 replications were performed, leading to 61 440 simulations. The sensitivity analysis was then conducted on the seroprevalence output.The parameters governing the infection probability function and residual room contaminations were identified as key parameters. To control the Salmonella seroprevalence, efficient measures should therefore aim at these parameters. Moreover, the shedding rate and maternal protective factor also had a major impact. Therefore, further investigation on the protective effect of maternal or post-infection antibodies would be needed.     

A continuous optimization approach for inferring parameters in mathematical models of regulatory networks

Background

The advances of systems biology have raised a large number of sophisticated mathematical models for describing the dynamic property of complex biological systems. One of the major steps in developing mathematical models is to estimate unknown parameters of the model based on experimentally measured quantities. However, experimental conditions limit the amount of data that is available for mathematical modelling. The number of unknown parameters in mathematical models may be larger than the number of observation data. The imbalance between the number of experimental data and number of unknown parameters makes reverse-engineering problems particularly challenging.

Results

To address the issue of inadequate experimental data, we propose a continuous optimization approach for making reliable inference of model parameters. This approach first uses a spline interpolation to generate continuous functions of system dynamics as well as the first and second order derivatives of continuous functions. The expanded dataset is the basis to infer unknown model parameters using various continuous optimization criteria, including the error of simulation only, error of both simulation and the first derivative, or error of simulation as well as the first and second derivatives. We use three case studies to demonstrate the accuracy and reliability of the proposed new approach. Compared with the corresponding discrete criteria using experimental data at the measurement time points only, numerical results of the ERK kinase activation module show that the continuous absolute-error criteria using both function and high order derivatives generate estimates with better accuracy. This result is also supported by the second and third case studies for the G1/S transition network and the MAP kinase pathway, respectively. This suggests that the continuous absolute-error criteria lead to more accurate estimates than the corresponding discrete criteria. We also study the robustness property of these three models to examine the reliability of estimates. Simulation results show that the models with estimated parameters using continuous fitness functions have better robustness properties than those using the corresponding discrete fitness functions.

Conclusions

The inference studies and robustness analysis suggest that the proposed continuous optimization criteria are effective and robust for estimating unknown parameters in mathematical models.

Electronic supplementary material

The online version of this article (doi:10.1186/1471-2105-15-256) contains supplementary material, which is available to authorized users.     

Model analysis of flowering phenology in recombinant inbred lines of barley

A generic model for flowering phenology as a function of daily temperature and photoperiod was applied to predict differences of flowering times among 96 individuals (including the two parents) of a recombinant inbred line population in barley (Hordeum vulgare L.). Because of the large number of individuals to study, there is a need for simple ways to derive model parameters for each genotype. Therefore the number of genotype-specific parameters was reduced to four, namely f(o) (the minimum number of days to flowering at the optimum temperature and photoperiod), (1) and (2) (the development stages for the start and the end of the photoperiod-sensitive phase, respectively), and delta (the photoperiod sensitivity). Values of these parameters were estimated using a newly described methodological framework based on data from a photoperiod-controlled experiment where plants were mutually transferred between long-day and short-day environments at regular intervals. This modelling approach was tested in eight independent field environments of different sowing dates in two growing seasons. The four-parameter model predicted 37-67% of observed phenotypic variation in an environment, 76% of variation in across-environment mean days to flowering among the genotypes, and 96% of variation in across-genotype mean among the eight environments. When all the observations of the 96 genotypes across the eight environments were pooled, the model explained 81% of the total variation. Sensitivity analysis showed that all four model parameters were important for predicting differences in flowering time among the genotypes; but their relative importance differed and the ranking was in the order of f(o), delta, theta1, and theta2. This study highlighted the potential of using ecophysiological models to assist the genetic analysis of quantitative crop traits whose phenotype is often environment-dependent.     

Defining and detecting structural sensitivity in biological models: developing a new framework

When we construct mathematical models to represent biological systems, there is always uncertainty with regards to the model specification—whether with respect to the parameters or to the formulation of model functions. Sometimes choosing two different functions with close shapes in a model can result in substantially different model predictions: a phenomenon known in the literature as structural sensitivity, which is a significant obstacle to improving the predictive power of biological models. In this paper, we revisit the general definition of structural sensitivity, compare several more specific definitions and discuss their usefulness for the construction and analysis of biological models. Then we propose a general approach to reveal structural sensitivity with regards to certain system properties, which considers infinite-dimensional neighbourhoods of the model functions: a far more powerful technique than the conventional approach of varying parameters for a fixed functional form. In particular, we suggest a rigorous method to unearth sensitivity with respect to the local stability of systems’ equilibrium points. We present a method for specifying the neighbourhood of a general unknown function with \(n\) inflection points in terms of a finite number of local function properties, and provide a rigorous proof of its completeness. Using this powerful result, we implement our method to explore sensitivity in several well-known multicomponent ecological models and demonstrate the existence of structural sensitivity in these models. Finally, we argue that structural sensitivity is an important intrinsic property of biological models, and a direct consequence of the complexity of the underlying real systems.     

Quantitative performance metrics for robustness in circadian rhythms

MOTIVATION: Sensitivity analysis provides key measures that aid in unraveling the design principles responsible for the robust performance of biological networks. Such metrics allow researchers to investigate comprehensively model performance, to develop more realistic models, and to design informative experiments. However, sensitivity analysis of oscillatory systems focuses on period and amplitude characteristics, while biologically relevant effects on phase are neglected. RESULTS: Here, we introduce a novel set of phase-based sensitivity metrics for performance: period, phase, corrected phase and relative phase. Both state- and phase-based tools are applied to free-running Drosophila melanogaster and Mus musculus circadian models. Each metric produces unique sensitivity values used to rank parameters from least to most sensitive. Similarities among the resulting rank distributions strongly suggest a conservation of sensitivity with respect to parameter function and type. A consistent result, for instance, is that model performance of biological oscillators is more sensitive to global parameters than local (i.e. circadian specific) parameters. Discrepancies among these distributions highlight the individual metrics' definition of performance as specific parametric sensitivity values depend on the defined metric, or output. AVAILABILITY: An implementation of the algorithm in MATLAB (Mathworks, Inc.) is available from the authors. SUPPLEMENTARY INFORMATION: Supplementary Data are available at Bioinformatics online.     

Analysis of oxygen transport in a diffusion-limited model of engineered heart tissue

Cardiac tissue engineering has made notable progress in recent years with the advent of an experimental model based on neonatal cardiomyocytes entrapped in collage gels and purified basement membrane extract, known as "engineered heart tissues" (EHTs). EHTs are a formidable display of tissue-level contractile function and cellular-level differentiation, although they suffer greatly from mass transport limitations due to the high density of metabolically active cells and the diffusion-limited nature of the hydrogel. In this report, a mathematical model was developed to predict oxygen levels inside a one-dimensional, diffusion-limited model of EHT. These predictions were then compared to values measured in corresponding experiments with a hypoxia-sensitive stain (pimonidazole). EHTs were cast between two plastic discs, which allowed for mass transfer with the culture medium to occur in only the radial direction. EHTs were cultured for up to 36 h in the presence of pimonidazole, after which time they were snap-frozen, histologically sectioned, and stained for bound pimonidazole. Quantitative image analysis was performed to measure the distance from the culture medium at which hypoxia first occurs under various conditions. As tested by variation of simple design parameters, the trends in oxygen profiles predicted by the model are in reasonable agreement with those obtained experimentally, although a number of ambiguities related to the specific model parameters led to a general overprediction of oxygen concentrations. Based on the sensitivity analysis in the present study, it is concluded that diffusion-reaction models may offer relatively precise predictions of oxygen concentrations in diffusion-limited tissue constructs.     

Practical identifiability of biokinetic parameters of a model describing two-step nitrification in biofilms

Parameter estimation and model calibration are key problems in the application of biofilm models in engineering practice, where a large number of model parameters need to be determined usually based on experimental data with only limited information content. In this article, identifiability of biokinetic parameters of a biofilm model describing two-step nitrification was evaluated based solely on bulk phase measurements of ammonium, nitrite, and nitrate. In addition to evaluating the impact of experimental conditions and available measurements, the influence of mass transport limitation within the biofilm and the initial parameter values on identifiability of biokinetic parameters was evaluated. Selection of parameters for identifiability analysis was based on global mean sensitivities while parameter identifiability was analyzed using local sensitivity functions. At most, four of the six most sensitive biokinetic parameters were identifiable from results of batch experiments at bulk phase dissolved oxygen concentrations of 0.8 or 5 mg O(2)/L. High linear dependences between the parameters of the subsets (KO2,AOB,muAOB) and (KO2,NOB,muNOB) resulted in reduced identifiability. Mass transport limitation within the biofilm did not influence the number of identifiable parameters but, in fact, decreased collinearity between parameters, especially for parameters that are otherwise correlated (e.g., muAOB) and KO2,AOB, or muNOB and KO2,NOB). The choice of the initial parameter values had a significant impact on the identifiability of two parameter subsets, both including the parameters muAOB and KO2,AOB. Parameter subsets that did not include the subsets muAOB and KO2,AOB or muNOB and KO2,NOB were clearly identifiable independently of the choice of the initial parameter values.     

Reduced Models of Algae Growth

The simulation of biological systems is often plagued by a high level of noise in the data, as well as by models containing a large number of correlated parameters. As a result, the parameters are poorly identified by the data, and the reliability of the model predictions may be questionable. Bayesian sampling methods provide an avenue for proper statistical analysis in such situations. Nevertheless, simulations should employ models that, on the one hand, are reduced as much as possible, and, on the other hand, are still able to capture the essential features of the phenomena studied. Here, in the case of algae growth modeling, we show how a systematic model reduction can be done. The simplified model is analyzed from both theoretical and statistical points of view.     

Application of 1D blood flow models of the human arterial network to differential pressure predictions

A new application of 1D models of the human arterial network is proposed. We take advantage of the sensitivity of the models predictions for the pressure profiles within the main aorta to key model parameter values. We propose to use the patterns in the predicted differences from a base case as a way to infer to the most probable changes in the parameter values. We demonstrate this application using an impedance model that we have recently developed (Johnson, 2010). The input model parameters are all physiologically related, such as the geometric dimensions of large arteries, various blood properties, vessel elasticity, etc. and can therefore be patient specific. As a base case, nominal values from the literature are used. The necessary information to characterize the smaller arteries, arterioles, and capillaries is taken from a physical scaling model (West, 1999). Model predictions for the effective impedance of the human arterial system closely agree with experimental data available in the literature. The predictions for the pressure wave development along the main arteries are also found in qualitative agreement with previous published results. The model has been further validated against our own measured pressure data in the carotid and radial arteries, obtained from healthy individuals. Upon changes in the value of key model parameters, we show that the differences seen in the pressure profiles correspond to qualitatively different patterns for different parameters. This suggests the possibility of using the model in interpreting multiple pressure data of healthy/diseased individuals.     

Kinetic models for detection of toxicity in a microbial fuel cell based biosensor

Currently available models describing microbial fuel cell (MFC) polarization curves, do not describe the effect of the presence of toxic components. A bioelectrochemical model combined with enzyme inhibition kinetics, that describes the polarization curve of an MFC-based biosensor, was modified to describe four types of toxicity. To get a stable and sensitive sensor, the overpotential has to be controlled. Simulations with the four modified models were performed to predict the overpotential that gives the most sensitive sensor. These simulations were based on data and parameter values from experimental results under non-toxic conditions. Given the parameter values from experimental results, controlling the overpotential at 250 mV leads to a sensor that is most sensitive to components that influence the whole bacterial metabolism or that influence the substrate affinity constant (Km). Controlling the overpotential at 105 mV is the most sensitive setting for components influencing the ratio of biochemical over electrochemical reaction rate constants (K1), while an overpotential of 76 mV gives the most sensitive setting for components that influence the ratio of the forward over backward biochemical rate constants (K2). The sensitivity of the biosensor was also analyzed for robustness against changes in the model parameters other than toxicity. As an example, the tradeoff between sensitivity and robustness for the model describing changes on K1 (IK1) is presented. The biosensor is sensitive for toxic components and robust for changes in model parameter K2 when overpotential is controlled between 118 and 140 mV under the simulated conditions.     

Steady state analysis of influx and transbilayer distribution of ergosterol in the yeast plasma membrane

Mathematical modeling is now frequently used in outbreak investigations to understand underlying mechanisms of infectious disease dynamics, assess patterns in epidemiological data, and forecast the trajectory of epidemics. However, the successful application of mathematical models to guide public health interventions lies in the ability to reliably estimate model parameters and their corresponding uncertainty. Here, we present and illustrate a simple computational method for assessing parameter identifiability in compartmental epidemic models. We describe a parametric bootstrap approach to generate simulated data from dynamical systems to quantify parameter uncertainty and identifiability. We calculate confidence intervals and mean squared error of estimated parameter distributions to assess parameter identifiability. To demonstrate this approach, we begin with a low-complexity SEIR model and work through examples of increasingly more complex compartmental models that correspond with applications to pandemic influenza, Ebola, and Zika. Overall, parameter identifiability issues are more likely to arise with more complex models (based on number of equations/states and parameters). As the number of parameters being jointly estimated increases, the uncertainty surrounding estimated parameters tends to increase, on average, as well. We found that, in most cases, R0 is often robust to parameter identifiability issues affecting individual parameters in the model. Despite large confidence intervals and higher mean squared error of other individual model parameters, R0 can still be estimated with precision and accuracy. Because public health policies can be influenced by results of mathematical modeling studies, it is important to conduct parameter identifiability analyses prior to fitting the models to available data and to report parameter estimates with quantified uncertainty. The method described is helpful in these regards and enhances the essential toolkit for conducting model-based inferences using compartmental dynamic models.     

Cross-species studies for target validation.

The completion of the genome sequences of several model organisms and the recent development of high throughput procedures to map genes, expression patterns and interactions is providing a steadily increasing number of candidate target genes. The function of most of these genes still remains unknown. Therefore, there is a growing demand in genetically tractable animal models in which the function of individual factors can be studied in large scale, particularly of those that are thought to segregate with human disorders. In this paper, current methods to validate target gene function and the advantages of different model organisms are compared.     

Wasp dynamics: A colony model

Social wasps are significant pests in many parts of the world, constituting a threat to human health and indigenous fauna as well as a commercial cost to industries such as beekeeping. In New Zealand Vespula vulgaris and V. germanica are significant environmental pests in large areas of native forest and, while limited chemical control is possible, for many areas there is no practical control option available. As part of a larger- scale ecological investigation into wasp biology and control, a within-colony model has been developed. The model is a simulation of intermediate complexity for which the driving relationships are drawn from an extensive field data set and published information. Although the model is conceptually similar to previous models and yields similar parameter values and output, it is mechanistically different. In particular, two new mechanistic relationships are derived: (1) the rate at which workers build cells is related to the larva:worker ratio; and (2) the rate at which the queen lays eggs is related to the number of worker-sealed brood present in the colony. Sensitivity analysis supports a possible role for queen quality in colony dynamics although other factors may also be involved. Also, sensitivity analysis results in model behaviour that is different to previous models; a result of its more mechanistic relationships. The model is intended as a tool to improve our understanding of wasp ecology and help develop effective strategies for wasp control.     

Selection of optimal model for the DNA histogram by analysis of error of estimated parameters.

The ability of four different mathematical models of the DNA histogram to give accurate estimates for the fractions of cells in G1, S, and G2 + M has been investigated. The models studied differ in the form and number of parameters of the function used to represent cells in S-phase. Results obtained from simulated DNA histograms suggest that the standard deviations of the model parameters increase exponentially with the width of the G1 and G2 + M peaks of the histogram. Error analysis is presented as a method to select a model of optimal complexity in relation to the resolution provided by the data in a given set of DNA histograms. Introduction of additional parameters improves the agreement between model and data but may result in a less well-posed model. A model with an optimal number of parameters can therefore be found that will yield parameter estimates with the smallest possible standard deviations.