What Can We Learn from Global Sensitivity Analysis of Biochemical Systems?

Most biological models of intermediate size, and probably all large models, need to cope with the fact that many of their parameter values are unknown. In addition, it may not be possible to identify these values unambiguously on the basis of experimental data. This poses the question how reliable predictions made using such models are. Sensitivity analysis is commonly used to measure the impact of each model parameter on its variables. However, the results of such analyses can be dependent on an exact set of parameter values due to nonlinearity. To mitigate this problem, global sensitivity analysis techniques are used to calculate parameter sensitivities in a wider parameter space. We applied global sensitivity analysis to a selection of five signalling and metabolic models, several of which incorporate experimentally well-determined parameters. Assuming these models represent physiological reality, we explored how the results could change under increasing amounts of parameter uncertainty. Our results show that parameter sensitivities calculated with the physiological parameter values are not necessarily the most frequently observed under random sampling, even in a small interval around the physiological values. Often multimodal distributions were observed. Unsurprisingly, the range of possible sensitivity coefficient values increased with the level of parameter uncertainty, though the amount of parameter uncertainty at which the pattern of control was able to change differed among the models analysed. We suggest that this level of uncertainty can be used as a global measure of model robustness. Finally a comparison of different global sensitivity analysis techniques shows that, if high-throughput computing resources are available, then random sampling may actually be the most suitable technique.     

Evaluating the Use of Global Sensitivity Analysis in Dynamic MFA

Dynamic material flow analysis (MFA) provides information about material usage over time and consequent changes in material stocks and flows. In order to understand the effect of limited data quality and model assumptions on MFA results, the use of sensitivity analysis methods in dynamic MFA studies has been on the increase. So far, sensitivity analysis in dynamic MFA has been conducted by means of a one‐at‐a‐time method, which tests parameter perturbations individually and observes the outcomes on output. In contrast to that, variance‐based global sensitivity analysis decomposes the variance of the model output into fractions caused by the uncertainty or variability of input parameters. The present study investigates interaction and time‐delay effects of uncertain parameters on the output of an archetypal input‐driven dynamic material flow model using variance‐based global sensitivity analysis. The results show that determining the main (first‐order) effects of parameter variations is often sufficient in dynamic MFA because substantial effects attributed to the simultaneous variation of several parameters (higher‐order effects) do not appear for classical setups of dynamic material flow models. For models with time‐varying parameters, time‐delay effects of parameter variation on model outputs need to be considered, potentially boosting the computational cost of global sensitivity analysis. Finally, the implications of exploring the sensitivities of model outputs with respect to parameter variations in the archetypical model are used to derive model‐ and goal‐specific recommendations on choosing appropriate sensitivity analysis methods in dynamic MFA.     

Three algorithms for interpreting models consisting of ordinary differential equations: Sensitivity coefficients,sensitivity functions,global optimization

This paper reports the development and application of three powerful algorithms for the analysis and simulation of mathematical models consisting of ordinary differential equations. First, we describe an extended parameter sensitivity analysis: we measure the relative sensitivities of many dynamical behaviors of the model to perturbations of each parameter. We check sensitivities to parameter variation over both small and large ranges. These two extensions of a common technique have applications in parameter estimation and in experimental design. Second, we compute sensitivity functions, using an efficient algorithm requiring just one model simulation to obtain all sensitivities of state variables to all parameters as functions of time. We extend the analysis to a behavior which is not a state variable. Third, we present an unconstrained global optimization algorithm, and apply it in a novel way: we determine the input to the model, given an optimality criterion and typical outputs. The algorithm itself is an efficient one for high-order problems, and does not get stuck at local extrema. We apply the sensitivity analysis, sensitivity functions, and optimization algorithm to a sixth-order nonlinear ordinary differential equation model for human eye movements. This application shows that the algorithms are not only practicable for high-order models, but also useful as conceptual tools.     

Application of global sensitivity analysis to determine goals for design of experiments: an example study on antibody-producing cell cultures

Global sensitivity analysis (GSA) can be used to quantify the importance of model parameters and their interactions with respect to model output. In this study, the Sobol' method for GSA is applied to a dynamic model of monoclonal antibody-producing mammalian cell cultures in order to identify the parameters that need to be accurately determined experimentally. Our results show that most parameters have low sensitivity indices and exhibit strong interactions with one another. These parameters can be set at their nominal values and unnecessary experimentation can therefore be avoided. In contrast, certain parameters are identified as sensitive, necessitating their estimation given sufficiently rich experimental data. Moreover, parameter sensitivity varies during culture time in a biologically meaningful manner. In conclusion, GSA can serve as an excellent precursor to optimal experiment design.     

Evaluation of habitat suitability index models by global sensitivity and uncertainty analyses: a case study for submerged aquatic vegetation

Habitat suitability index (HSI) models are commonly used to predict habitat quality and species distributions and are used to develop biological surveys, assess reserve and management priorities, and anticipate possible change under different management or climate change scenarios. Important management decisions may be based on model results, often without a clear understanding of the level of uncertainty associated with model outputs. We present an integrated methodology to assess the propagation of uncertainty from both inputs and structure of the HSI models on model outputs (uncertainty analysis: UA) and relative importance of uncertain model inputs and their interactions on the model output uncertainty (global sensitivity analysis: GSA). We illustrate the GSA/UA framework using simulated hydrology input data from a hydrodynamic model representing sea level changes and HSI models for two species of submerged aquatic vegetation (SAV) in southwest Everglades National Park: Vallisneria americana (tape grass) and Halodule wrightii (shoal grass). We found considerable spatial variation in uncertainty for both species, but distributions of HSI scores still allowed discrimination of sites with good versus poor conditions. Ranking of input parameter sensitivities also varied spatially for both species, with high habitat quality sites showing higher sensitivity to different parameters than low‐quality sites. HSI models may be especially useful when species distribution data are unavailable, providing means of exploiting widely available environmental datasets to model past, current, and future habitat conditions. The GSA/UA approach provides a general method for better understanding HSI model dynamics, the spatial and temporal variation in uncertainties, and the parameters that contribute most to model uncertainty. Including an uncertainty and sensitivity analysis in modeling efforts as part of the decision‐making framework will result in better‐informed, more robust decisions.     

Sensitivity Analysis of an ENteric Immunity SImulator (ENISI)-Based Model of Immune Responses to Helicobacter pylori Infection

Agent-based models (ABM) are widely used to study immune systems, providing a procedural and interactive view of the underlying system. The interaction of components and the behavior of individual objects is described procedurally as a function of the internal states and the local interactions, which are often stochastic in nature. Such models typically have complex structures and consist of a large number of modeling parameters. Determining the key modeling parameters which govern the outcomes of the system is very challenging. Sensitivity analysis plays a vital role in quantifying the impact of modeling parameters in massively interacting systems, including large complex ABM. The high computational cost of executing simulations impedes running experiments with exhaustive parameter settings. Existing techniques of analyzing such a complex system typically focus on local sensitivity analysis, i.e. one parameter at a time, or a close “neighborhood” of particular parameter settings. However, such methods are not adequate to measure the uncertainty and sensitivity of parameters accurately because they overlook the global impacts of parameters on the system. In this article, we develop novel experimental design and analysis techniques to perform both global and local sensitivity analysis of large-scale ABMs. The proposed method can efficiently identify the most significant parameters and quantify their contributions to outcomes of the system. We demonstrate the proposed methodology for ENteric Immune SImulator (ENISI), a large-scale ABM environment, using a computational model of immune responses to Helicobacter pylori colonization of the gastric mucosa.     

生态模型的灵敏度分析

灵敏度分析用于定性或定量地评价模型参数误差对模型结果产生的影响,是模型参数化过程和模型校正过程中的有用工具,具有重要的生态学意义．灵敏度分析包括局部灵敏度分析和全局灵敏度分析．局部灵敏度分析只检验单个参数的变化对模型结果的影响程度;全局灵敏度分析则检验多个参数的变化对模型运行结果总的影响,并分析每一个参数及其参数之间相互作用对模型结果的影响．目前,在对生态模型的灵敏度分析中,越来越倾向于使用全局灵敏度分析的方法．但国内仍多采用局部灵敏度分析方法,很少采用全局灵敏度分析方法．文中详细论述了局部灵敏分析和全局灵敏度分析的主要方法(一次变换法、多元回归法、Morris法、Sobol’法、傅里叶幅度灵敏度检验法和傅里叶幅度灵敏度检验扩展法),希望能为国内生态模型的发展提供一个比较完善的灵敏度分析方法库．结合国内外的灵敏度分析发展现状,指出联合灵敏度研究、灵敏度共性研究及空间直观景观模型的灵敏度分析将为生态模型灵敏度分析研究中的热点和难点．     

Parameter Identifiability and Sensitivity Analysis Predict Targets for Enhancement of STAT1 Activity in Pancreatic Cancer and Stellate Cells

The present work exemplifies how parameter identifiability analysis can be used to gain insights into differences in experimental systems and how uncertainty in parameter estimates can be handled. The case study, presented here, investigates interferon-gamma (IFNγ) induced STAT1 signalling in two cell types that play a key role in pancreatic cancer development: pancreatic stellate and cancer cells. IFNγ inhibits the growth for both types of cells and may be prototypic of agents that simultaneously hit cancer and stroma cells. We combined time-course experiments with mathematical modelling to focus on the common situation in which variations between profiles of experimental time series, from different cell types, are observed. To understand how biochemical reactions are causing the observed variations, we performed a parameter identifiability analysis. We successfully identified reactions that differ in pancreatic stellate cells and cancer cells, by comparing confidence intervals of parameter value estimates and the variability of model trajectories. Our analysis shows that useful information can also be obtained from nonidentifiable parameters. For the prediction of potential therapeutic targets we studied the consequences of uncertainty in the values of identifiable and nonidentifiable parameters. Interestingly, the sensitivity of model variables is robust against parameter variations and against differences between IFNγ induced STAT1 signalling in pancreatic stellate and cancer cells. This provides the basis for a prediction of therapeutic targets that are valid for both cell types.     

A methodology for performing global uncertainty and sensitivity analysis in systems biology

Accuracy of results from mathematical and computer models of biological systems is often complicated by the presence of uncertainties in experimental data that are used to estimate parameter values. Current mathematical modeling approaches typically use either single-parameter or local sensitivity analyses. However, these methods do not accurately assess uncertainty and sensitivity in the system as, by default, they hold all other parameters fixed at baseline values. Using techniques described within we demonstrate how a multi-dimensional parameter space can be studied globally so all uncertainties can be identified. Further, uncertainty and sensitivity analysis techniques can help to identify and ultimately control uncertainties. In this work we develop methods for applying existing analytical tools to perform analyses on a variety of mathematical and computer models. We compare two specific types of global sensitivity analysis indexes that have proven to be among the most robust and efficient. Through familiar and new examples of mathematical and computer models, we provide a complete methodology for performing these analyses, in both deterministic and stochastic settings, and propose novel techniques to handle problems encountered during these types of analyses.     

The effect of connective tissue material uncertainties on knee joint mechanics under isolated loading conditions

Although variability in connective tissue parameters is widely reported and recognized, systematic examination of the effect of such parametric uncertainties on predictions derived from a full anatomical joint model is lacking. As such, a sensitivity analysis was performed to consider the behavior of a three-dimensional, non-linear, finite element knee model with connective tissue material parameters that varied within a given interval. The model included the coupled mechanics of the tibio-femoral and patello-femoral degrees of freedom. Seven primary connective tissues modeled as non-linear continua, articular cartilages described by a linear elastic model, and menisci modeled as transverse isotropic elastic materials were included. In this study, a multi-factorial global sensitivity analysis is proposed, which can detect the contribution of influential material parameters while maintaining the potential effect of parametric interactions. To illustrate the effect of material uncertainties on model predictions, exemplar loading conditions reported in a number of isolated experimental paradigms were used. Our findings illustrated that the inclusion of material uncertainties in a coupled tibio-femoral and patello-femoral model reveals biomechanical interactions that otherwise would remain unknown. For example, our analysis revealed that the effect of anterior cruciate ligament parameter variations on the patello-femoral kinematic and kinetic response sensitivities was significantly larger, over a range of flexion angles, when compared to variations associated with material parameters of tissues intrinsic to the patello-femoral joint. We argue that the systematic sensitivity framework presented herein will help identify key material uncertainties that merit further research and provide insight on those uncertainties that may not be as relative to a given response.     

Evolutionary optimization with data collocation for reverse engineering of biological networks

MOTIVATION: Modern experimental biology is moving away from analyses of single elements to whole-organism measurements. Such measured time-course data contain a wealth of information about the structure and dynamic of the pathway or network. The dynamic modeling of the whole systems is formulated as a reverse problem that requires a well-suited mathematical model and a very efficient computational method to identify the model structure and parameters. Numerical integration for differential equations and finding global parameter values are still two major challenges in this field of the parameter estimation of nonlinear dynamic biological systems. RESULTS: We compare three techniques of parameter estimation for nonlinear dynamic biological systems. In the proposed scheme, the modified collocation method is applied to convert the differential equations to the system of algebraic equations. The observed time-course data are then substituted into the algebraic system equations to decouple system interactions in order to obtain the approximate model profiles. Hybrid differential evolution (HDE) with population size of five is able to find a global solution. The method is not only suited for parameter estimation but also can be applied for structure identification. The solution obtained by HDE is then used as the starting point for a local search method to yield the refined estimates.     

GAGE: generally applicable gene set enrichment for pathway analysis

Background  

Gene set analysis (GSA) is a widely used strategy for gene expression data analysis based on pathway knowledge. GSA focuses on sets of related genes and has established major advantages over individual gene analyses, including greater robustness, sensitivity and biological relevance. However, previous GSA methods have limited usage as they cannot handle datasets of different sample sizes or experimental designs.     

Insights into pathological mechanisms and interventions revealed by analyzing a mathematical model for cone metabolism

This work analyzes a mathematical model for the metabolic dynamics of a cone photoreceptor, which is the first model to account for energy generation from fatty acids oxidation of shed photoreceptor outer segments (POS). Multiple parameter bifurcation analysis shows that joint variations in external glucose, the efficiency of glucose transporter 1 (GLUT1), lipid utilization for POS renewal, and oxidation of fatty acids affect the cone’s metabolic vitality and its capability to adapt under glucose-deficient conditions. The analysis further reveals that when glucose is scarce, cone viability cannot be sustained by only fueling energy production in the mitochondria, but it also requires supporting anabolic processes to create lipids necessary for cell maintenance and repair. In silico experiments are used to investigate how the duration of glucose deprivation impacts the cell without and with a potential GLUT1 or oxidation of fatty acids intervention as well as a dual intervention. The results show that for prolonged duration of glucose deprivation, the cone metabolic system does not recover with higher oxidation of fatty acids and requires greater effectiveness of GLUT1 to recover. Finally, time-varying global sensitivity analysis (GSA) is applied to assess the sensitivity of the model outputs of interest to changes and uncertainty in the parameters at specific times. The results reveal a critical temporal window where there would be more flexibility for interventions to rescue a cone cell from the detrimental consequences of glucose shortage.     

Sensitivity analysis and optimization for a head movement model

A sixth order nonlinear model for horizontal head rotations in humans is analyzed using an extended parameter sensitivity analysis and a global optimization algorithm. The sensitivity analysis is used in both the direct sense, as a model fitting tool, and in the indirect sense, as a guide to experimental design. Resolution is defined in terms of the sensitivity table, and is used to interpret the sensitivity results. Using sensitivity analyses, the head and eye movement systems are compared and contrasted. Controller signal parameters are the most influential. Their variations and effects on head movement trajectories and accelerations are investigated, and the conclusions are compared with clinical neurological findings. The global optimization algorithm, in addition to automating the fitting of various types of data, is combined with time optimality theory to give theoretical time-optimal inputs to the model.On leave from Department of Neurology, University of Hamburg, FRG; supported by Deutsche Forschungsgemeinschaft Bonn, FRG     

The importance of assessing parameter sensitivity when using biophysical models: a case study using plethodontid salamanders

Landscapes are continually changing due to numerous assaults, including habitat alteration, anthropogenic disturbances, and climate change. Understanding how species will respond to these changes is of critical importance for conservation and management. Mechanistic models, such as biophysical models (BPMs), are an increasingly popular tool to predict how local population dynamics or species’ distributions may be altered in response to environmental and climate changes. By mechanistically modeling relationships between environmental conditions, physiology and behavior, it is possible to make accurate predictions about how species may respond. However, BPMs are often difficult to implement due to lack of appropriate, species-specific data that is biologically realistic or relevant. In this study, we present a BPM for the salamander Plethodon jordani and assess how adding more biological realism has potential to alter model predictions about annual energy budgets. Additionally, we conducted local and global sensitivity analyses to evaluate the importance of accurately specifying model parameter values and functional relationships. We found that the addition of biological realism resulted in greater model complexity as well as substantially different estimates of energy balance. Correct parameterization of biophysical models is also critical, as small changes in parameter values can result in disproportionately large changes in downstream model estimates. Our model highlights the overall importance of using ecologically relevant and specific data for input parameters, as well as careful assessment of parameter sensitivity. We encourage researchers to be aware of the data they are using to parameterize BPMs, and urge the collection of system-specific data that is relevant in spatial and temporal scale. We also recommend greater and more transparent use of sensitivity analyses to provide a better understanding of the model, as well as greater confidence in model predictions.     

Meta-models as a straightforward approach to the sensitivity analysis of complex models

Complex simulation models are important tools in applied ecological and conservation research. However sensitivity analysis of this important class of models can be difficult to conduct. High level interactions and non-linear responses are common in complex simulations, and this necessitates a global sensitivity analysis, where each parameter is tested at a range of values, and in combination with changes in many other parameters. We reviewed the literature, searching for population viability analyses that used simulation models. We found only 9 out of the 122 simulation population viability analysis used global sensitivity analysis. This result is typical of other simulation models in applied ecology, where global sensitivity analysis is rare. We then demonstrate how to conduct a meta-modeling sensitivity analysis, where a simpler statistically fit function (the meta-model, also known as the surrogate model or emulator) is used to approximate the behavior of the complicated simulation. This simpler meta-model is interrogated to inform on the behavior of simulation model. We fit two example meta-models, a generalized linear model and a boosted regression tree, to exemplify the approach. Our hope is that by going through these techniques thoroughly they will become more widely adopted.     

Robustness as a measure of plausibility in models of biochemical networks

Theory, experiment, and observation suggest that biochemical networks which are conserved across species are robust to variations in concentrations and kinetic parameters. Here, we exploit this expectation to propose an approach to model building and selection. We represent a model as a mapping from parameter space to behavior space, and utilize bifurcation analysis to study the robustness of each region of steady-state behavior to parameter variations. The hypothesis that potential errors in models will result in parameter sensitivities is tested by analysis of two models of the biochemical oscillator underlying the Xenopus cell cycle. Our analysis successfully identifies known weaknesses in the older model and suggests areas for further investigation in the more recent, more plausible model. It also correctly highlights why the more recent model is more plausible.     

Model parameterization to represent processes at unresolved scales and changing properties of evolving systems

Modeling has become an indispensable tool for scientific research. However, models generate great uncertainty when they are used to predict or forecast ecosystem responses to global change. This uncertainty is partly due to parameterization, which is an essential procedure for model specification via defining parameter values for a model. The classic doctrine of parameterization is that a parameter is constant. However, it is commonly known from modeling practice that a model that is well calibrated for its parameters at one site may not simulate well at another site unless its parameters are tuned again. This common practice implies that parameter values have to vary with sites. Indeed, parameter values that are estimated using a statistically rigorous approach, that is, data assimilation, vary with time, space, and treatments in global change experiments. This paper illustrates that varying parameters is to account for both processes at unresolved scales and changing properties of evolving systems. A model, no matter how complex it is, could not represent all the processes of one system at resolved scales. Interactions of processes at unresolved scales with those at resolved scales should be reflected in model parameters. Meanwhile, it is pervasively observed that properties of ecosystems change over time, space, and environmental conditions. Parameters, which represent properties of a system under study, should change as well. Tuning has been practiced for many decades to change parameter values. Yet this activity, unfortunately, did not contribute to our knowledge on model parameterization at all. Data assimilation makes it possible to rigorously estimate parameter values and, consequently, offers an approach to understand which, how, how much, and why parameters vary. To fully understand those issues, extensive research is required. Nonetheless, it is clear that changes in parameter values lead to different model predictions even if the model structure is the same.     

A mathematical model of the role of aggregation in sonic hedgehog signalling

Effective regulation of the sonic hedgehog (Shh) signalling pathway is essential for normal development in a wide variety of species. Correct Shh signalling requires the formation of Shh aggregates on the surface of producing cells. Shh aggregates subsequently diffuse away and are recognised in receiving cells located elsewhere in the developing embryo. Various mechanisms have been postulated regarding how these aggregates form and what their precise role is in the overall signalling process. To understand the role of these mechanisms in the overall signalling process, we formulate and analyse a mathematical model of Shh aggregation using nonlinear ordinary differential equations. We consider Shh aggregate formation to comprise of multimerisation, association with heparan sulfate proteoglycans (HSPG) and binding with lipoproteins. We show that the size distribution of the Shh aggregates formed on the producing cell surface resembles an exponential distribution, a result in agreement with experimental data. A detailed sensitivity analysis of our model reveals that this exponential distribution is robust to parameter changes, and subsequently, also to variations in the processes by which Shh is recruited by HSPGs and lipoproteins. The work demonstrates the time taken for different sized Shh aggregates to form and the important role this likely plays in Shh diffusion.