Universally sloppy parameter sensitivities in systems biology models

Quantitative computational models play an increasingly important role in modern biology. Such models typically involve many free parameters, and assigning their values is often a substantial obstacle to model development. Directly measuring in vivo biochemical parameters is difficult, and collectively fitting them to other experimental data often yields large parameter uncertainties. Nevertheless, in earlier work we showed in a growth-factor-signaling model that collective fitting could yield well-constrained predictions, even when it left individual parameters very poorly constrained. We also showed that the model had a “sloppy” spectrum of parameter sensitivities, with eigenvalues roughly evenly distributed over many decades. Here we use a collection of models from the literature to test whether such sloppy spectra are common in systems biology. Strikingly, we find that every model we examine has a sloppy spectrum of sensitivities. We also test several consequences of this sloppiness for building predictive models. In particular, sloppiness suggests that collective fits to even large amounts of ideal time-series data will often leave many parameters poorly constrained. Tests over our model collection are consistent with this suggestion. This difficulty with collective fits may seem to argue for direct parameter measurements, but sloppiness also implies that such measurements must be formidably precise and complete to usefully constrain many model predictions. We confirm this implication in our growth-factor-signaling model. Our results suggest that sloppy sensitivity spectra are universal in systems biology models. The prevalence of sloppiness highlights the power of collective fits and suggests that modelers should focus on predictions rather than on parameters.     

Parameter estimation in systems biology models using spline approximation

Background  

Mathematical models for revealing the dynamics and interactions properties of biological systems play an important role in computational systems biology. The inference of model parameter values from time-course data can be considered as a "reverse engineering" process and is still one of the most challenging tasks. Many parameter estimation methods have been developed but none of these methods is effective for all cases and can overwhelm all other approaches. Instead, various methods have their advantages and disadvantages. It is worth to develop parameter estimation methods which are robust against noise, efficient in computation and flexible enough to meet different constraints.     

Steady-State Differential Dose Response in Biological Systems

In pharmacology and systems biology, it is a fundamental problem to determine how biological systems change their dose-response behavior upon perturbations. In particular, it is unclear how topologies, reactions, and parameters (differentially) affect the dose response. Because parameters are often unknown, systematic approaches should directly relate network structure and function. Here, we outline a procedure to compare general non-monotone dose-response curves and subsequently develop a comprehensive theory for differential dose responses of biochemical networks captured by non-equilibrium steady-state linear framework models. Although these models are amenable to analytical derivations of non-equilibrium steady states in principle, their size frequently increases (super) exponentially with model size. We extract general principles of differential responses based on a model’s graph structure and thereby alleviate the combinatorial explosion. This allows us, for example, to determine reactions that affect differential responses, to identify classes of networks with equivalent differential, and to reject hypothetical models reliably without needing to know parameter values. We exemplify such applications for models of insulin signaling.     

Evolutionary optimization with data collocation for reverse engineering of biological networks

MOTIVATION: Modern experimental biology is moving away from analyses of single elements to whole-organism measurements. Such measured time-course data contain a wealth of information about the structure and dynamic of the pathway or network. The dynamic modeling of the whole systems is formulated as a reverse problem that requires a well-suited mathematical model and a very efficient computational method to identify the model structure and parameters. Numerical integration for differential equations and finding global parameter values are still two major challenges in this field of the parameter estimation of nonlinear dynamic biological systems. RESULTS: We compare three techniques of parameter estimation for nonlinear dynamic biological systems. In the proposed scheme, the modified collocation method is applied to convert the differential equations to the system of algebraic equations. The observed time-course data are then substituted into the algebraic system equations to decouple system interactions in order to obtain the approximate model profiles. Hybrid differential evolution (HDE) with population size of five is able to find a global solution. The method is not only suited for parameter estimation but also can be applied for structure identification. The solution obtained by HDE is then used as the starting point for a local search method to yield the refined estimates.     

How localized consumption stabilizes predator-prey systems with finite frequency of mixing

Predator-prey theory began with aspatial models that assumed organisms interacted as if they were "well-mixed" particles that obey the laws of mass action, but it has become clear that both the spatial and individual nature of many organisms can change how the dynamics of such systems function. Here I examine how localized consumption of prey by predators changes the dynamics of predator-prey systems; I use an individual-based simulation of the Rosenzweig-MacArthur model in implicit space and its mean-field approximation. In combination with limited movement, localized consumption makes the predator-prey dynamics more stable than the comparable "well-mixed" Rosenzweig-MacArthur model. Using a spatial correlation, one can directly compare a simplified version of the individual-based model with the Rosenzweig-MacArthur model. While this comparison allows the changes in the dynamics to be captured by the "well-mixed" Rosenzweig-MacArthur model, the parameters of the functional response are now dependent on the movement parameters, and so the functional response must be estimated statistically from the dynamics of the individual-based model. Yet this implies that aspatial models may work in a scale-specific fashion for spatial systems. Unlike many recent spatial models, the localized consumption and limited movement in the model presented here cannot produce coherent spatial patterns and do not depend on a patchy structure, as found in metapopulation models. Instead, the individual nature of the interactions creates a diffusion-limited reaction, which appears closer to a form of ephemeral refuge.     

Modeling and simulation of biological systems from image data

This essay provides an introduction to the terminology, concepts, methods, and challenges of image‐based modeling in biology. Image‐based modeling and simulation aims at using systematic, quantitative image data to build predictive models of biological systems that can be simulated with a computer. This allows one to disentangle molecular mechanisms from effects of shape and geometry. Questions like “what is the functional role of shape” or “how are biological shapes generated and regulated” can be addressed in the framework of image‐based systems biology. The combination of image quantification, model building, and computer simulation is illustrated here using the example of diffusion in the endoplasmic reticulum.     

Probabilistic polynomial dynamical systems for reverse engineering of gene regulatory networks

Elucidating the structure and/or dynamics of gene regulatory networks from experimental data is a major goal of systems biology. Stochastic models have the potential to absorb noise, account for un-certainty, and help avoid data overfitting. Within the frame work of probabilistic polynomial dynamical systems, we present an algorithm for the reverse engineering of any gene regulatory network as a discrete, probabilistic polynomial dynamical system. The resulting stochastic model is assembled from all minimal models in the model space and the probability assignment is based on partitioning the model space according to the likeliness with which a minimal model explains the observed data. We used this method to identify stochastic models for two published synthetic network models. In both cases, the generated model retains the key features of the original model and compares favorably to the resulting models from other algorithms.     

Accelerated maximum likelihood parameter estimation for stochastic biochemical systems

ABSTRACT: BACKGROUND: A prerequisite for the mechanistic simulation of a biochemical system is detailed knowledge of its kinetic parameters. Despite recent experimental advances, the estimation of unknown parameter values from observed data is still a bottleneck for obtaining accurate simulation results. Many methods exist for parameter estimation in deterministic biochemical systems; methods for discrete stochastic systems are less well developed. Given the probabilistic nature of stochastic biochemical models, a natural approach is to choose parameter values that maximize the probability of the observed data with respect to the unknown parameters, a.k.a. the maximum likelihood parameter estimates (MLEs). MLE computation for all but the simplest models requires the simulation of many system trajectories that are consistent with experimental data. For models with unknown parameters, this presents a computational challenge, as the generation of consistent trajectories can be an extremely rare occurrence. RESULTS: We have developed Monte Carlo Expectation-Maximization with Modified Cross-Entropy Method (MCEM2): an accelerated method for calculating MLEs that combines advances in rare event simulation with a computationally efficient version of the Monte Carlo expectation-maximization (MCEM) algorithm. Our method requires no prior knowledge regarding parameter values, and it automatically provides a multivariate parameter uncertainty estimate. We applied the method to five stochastic systems of increasing complexity, progressing from an analytically tractable pure-birth model to a computationally demanding model of yeast-polarization. Our results demonstrate that MCEM2 substantially accelerates MLE computation on all tested models when compared to a stand-alone version of MCEM. Additionally, we show how our method identifies parameter values for certain classes of models more accurately than two recently proposed computationally efficient methods. CONCLUSIONS: This work provides a novel, accelerated version of a likelihood-based parameter estimation method that can be readily applied to stochastic biochemical systems. In addition, our results suggest opportunities for added efficiency improvements that will further enhance our ability to mechanistically simulate biological processes.     

Integrative database management for mouse development: systems and concepts

Cells in the developing embryo must integrate complex signals from the genome and environment to make decisions about their behavior or fate. The ability to understand the fundamental biology of the decision-making process, and how these decisions may go awry during abnormal development, requires a systems biology paradigm. Presently, the ability to build models with predictive capability in birth defects research is constrained by an incomplete understanding of the fundamental parameters underlying embryonic susceptibility, sensitivity, and vulnerability. Key developmental milestones must be parameterized in terms of system structure and dynamics, the relevant control methods, and the overall design logic of metabolic and regulatory networks. High-content data from genome-based studies provide some comprehensive coverage of these operational processes but a key research challenge is data integration. Analysis can be facilitated by data management resources and software to reveal the structure and function of bionetwork motifs potentially associated with an altered developmental phenotype. Borrowing from applied mathematics and artificial intelligence, we conceptualize a system that can help address the new challenges posed by the transformation of birth defects research into a data-driven science.     

Estimating the Stochastic Bifurcation Structure of Cellular Networks

High throughput measurement of gene expression at single-cell resolution, combined with systematic perturbation of environmental or cellular variables, provides information that can be used to generate novel insight into the properties of gene regulatory networks by linking cellular responses to external parameters. In dynamical systems theory, this information is the subject of bifurcation analysis, which establishes how system-level behaviour changes as a function of parameter values within a given deterministic mathematical model. Since cellular networks are inherently noisy, we generalize the traditional bifurcation diagram of deterministic systems theory to stochastic dynamical systems. We demonstrate how statistical methods for density estimation, in particular, mixture density and conditional mixture density estimators, can be employed to establish empirical bifurcation diagrams describing the bistable genetic switch network controlling galactose utilization in yeast Saccharomyces cerevisiae. These approaches allow us to make novel qualitative and quantitative observations about the switching behavior of the galactose network, and provide a framework that might be useful to extract information needed for the development of quantitative network models.     

Using topological equivalence to discover stable control parameters in biodynamic systems

In order to better understand the mechanisms that contribute to low back pain, researchers have developed mathematical models and simulations. A mathematical model including neuromuscular feedback control is developed for a person balancing on an unstable sitting apparatus, the wobble chair. When the application of a direct method fails to discover appropriate controller gain parameters for the wobble chair, we show how topological equivalence can be used to indirectly identify appropriate parameter values. The solution is found by first transforming the wobble chair into the Acrobot, another member of the same family of topologically equivalent dynamical systems. After finding appropriate gain parameters for the Acrobot, a continuous transformation is performed to convert the Acrobot back to the wobble chair, during which the gain parameters are adjusted to maintain stability. Thus, we demonstrate how topological equivalence can be used to indirectly solve a problem that was difficult to solve directly.     

Using topological equivalence to discover stable control parameters in biodynamic systems

In order to better understand the mechanisms that contribute to low back pain, researchers have developed mathematical models and simulations. A mathematical model including neuromuscular feedback control is developed for a person balancing on an unstable sitting apparatus, the wobble chair. When the application of a direct method fails to discover appropriate controller gain parameters for the wobble chair, we show how topological equivalence can be used to indirectly identify appropriate parameter values. The solution is found by first transforming the wobble chair into the Acrobot, another member of the same family of topologically equivalent dynamical systems. After finding appropriate gain parameters for the Acrobot, a continuous transformation is performed to convert the Acrobot back to the wobble chair, during which the gain parameters are adjusted to maintain stability. Thus, we demonstrate how topological equivalence can be used to indirectly solve a problem that was difficult to solve directly.     

Modeling adaptive biological systems

During the evolution of many systems found in nature, both the system composition and the interactions between components will vary. Equating the dimension with the number of different components, a system which adds or deletes components belongs to a class of dynamical systems with a finite dimensional phase space of variable dimension. We present two models of biochemical systems with a variable phase space, a model of autocatalytic reaction networks in the prebiotic soup and a model of the idiotypic network of the immune system. Each model contains characteristic meta-dynamical rules for constructing equations of motion from component properties. The simulation of each model occurs on two levels. On one level, the equations of motion are integrated to determine the state of each component. On a second level, algorithms which approximate physical processes in the real system are employed to change the equations of motion. Models with meta-dynamical rules possess several advantages for the study of evolving systems. First, there are no explicit fitness functions to determine how the components of the model rank in terms of survivability. The success of any component is a function of its relationship to the rest of the system. A second advantage is that since the phase space representation of the system is always finite but continually changing, we can explore a potentially infinite phase space which would otherwise be inaccessible with finite computer resources. Third, the enlarged capacity of systems with meta-dynamics for variation allows us to conduct true evolution experiments. The modeling methods presented here can be applied to many real biological systems. In the two studies we present, we are investigating two apparent properties of adaptive networks. With the simulation of the prebiotic soup, we are most interested in how a chemical reaction network might emerge from an initial state of relative disorder. With the study of the immune system, we study the self-regulation of the network including its ability to distinguish between species which are part of the network and those which are not.     

A computational method for the investigation of multistable systems and its application to genetic switches

Background

Genetic switches exhibit multistability, form the basis of epigenetic memory, and are found in natural decision making systems, such as cell fate determination in developmental pathways. Synthetic genetic switches can be used for recording the presence of different environmental signals, for changing phenotype using synthetic inputs and as building blocks for higher-level sequential logic circuits. Understanding how multistable switches can be constructed and how they function within larger biological systems is therefore key to synthetic biology.

Results

Here we present a new computational tool, called StabilityFinder, that takes advantage of sequential Monte Carlo methods to identify regions of parameter space capable of producing multistable behaviour, while handling uncertainty in biochemical rate constants and initial conditions. The algorithm works by clustering trajectories in phase space, and iteratively minimizing a distance metric. Here we examine a collection of models of genetic switches, ranging from the deterministic Gardner toggle switch to stochastic models containing different positive feedback connections. We uncover the design principles behind making bistable, tristable and quadristable switches, and find that rate of gene expression is a key parameter. We demonstrate the ability of the framework to examine more complex systems and examine the design principles of a three gene switch. Our framework allows us to relax the assumptions that are often used in genetic switch models and we show that more complex abstractions are still capable of multistable behaviour.

Conclusions

Our results suggest many ways in which genetic switches can be enhanced and offer designs for the construction of novel switches. Our analysis also highlights subtle changes in correlation of experimentally tunable parameters that can lead to bifurcations in deterministic and stochastic systems. Overall we demonstrate that StabilityFinder will be a valuable tool in the future design and construction of novel gene networks.

     

Estimating parameters for generalized mass action models using constraint propagation

As modern molecular biology moves towards the analysis of biological systems as opposed to their individual components, the need for appropriate mathematical and computational techniques for understanding the dynamics and structure of such systems is becoming more pressing. For example, the modeling of biochemical systems using ordinary differential equations (ODEs) based on high-throughput, time-dense profiles is becoming more common-place, which is necessitating the development of improved techniques to estimate model parameters from such data. Due to the high dimensionality of this estimation problem, straight-forward optimization strategies rarely produce correct parameter values, and hence current methods tend to utilize genetic/evolutionary algorithms to perform non-linear parameter fitting. Here, we describe a completely deterministic approach, which is based on interval analysis. This allows us to examine entire sets of parameters, and thus to exhaust the global search within a finite number of steps. In particular, we show how our method may be applied to a generic class of ODEs used for modeling biochemical systems called Generalized Mass Action Models (GMAs). In addition, we show that for GMAs our method is amenable to the technique in interval arithmetic called constraint propagation, which allows great improvement of its efficiency. To illustrate the applicability of our method we apply it to some networks of biochemical reactions appearing in the literature, showing in particular that, in addition to estimating system parameters in the absence of noise, our method may also be used to recover the topology of these networks.     

A methodology for performing global uncertainty and sensitivity analysis in systems biology

Accuracy of results from mathematical and computer models of biological systems is often complicated by the presence of uncertainties in experimental data that are used to estimate parameter values. Current mathematical modeling approaches typically use either single-parameter or local sensitivity analyses. However, these methods do not accurately assess uncertainty and sensitivity in the system as, by default, they hold all other parameters fixed at baseline values. Using techniques described within we demonstrate how a multi-dimensional parameter space can be studied globally so all uncertainties can be identified. Further, uncertainty and sensitivity analysis techniques can help to identify and ultimately control uncertainties. In this work we develop methods for applying existing analytical tools to perform analyses on a variety of mathematical and computer models. We compare two specific types of global sensitivity analysis indexes that have proven to be among the most robust and efficient. Through familiar and new examples of mathematical and computer models, we provide a complete methodology for performing these analyses, in both deterministic and stochastic settings, and propose novel techniques to handle problems encountered during these types of analyses.     

Model parameterization to represent processes at unresolved scales and changing properties of evolving systems

Modeling has become an indispensable tool for scientific research. However, models generate great uncertainty when they are used to predict or forecast ecosystem responses to global change. This uncertainty is partly due to parameterization, which is an essential procedure for model specification via defining parameter values for a model. The classic doctrine of parameterization is that a parameter is constant. However, it is commonly known from modeling practice that a model that is well calibrated for its parameters at one site may not simulate well at another site unless its parameters are tuned again. This common practice implies that parameter values have to vary with sites. Indeed, parameter values that are estimated using a statistically rigorous approach, that is, data assimilation, vary with time, space, and treatments in global change experiments. This paper illustrates that varying parameters is to account for both processes at unresolved scales and changing properties of evolving systems. A model, no matter how complex it is, could not represent all the processes of one system at resolved scales. Interactions of processes at unresolved scales with those at resolved scales should be reflected in model parameters. Meanwhile, it is pervasively observed that properties of ecosystems change over time, space, and environmental conditions. Parameters, which represent properties of a system under study, should change as well. Tuning has been practiced for many decades to change parameter values. Yet this activity, unfortunately, did not contribute to our knowledge on model parameterization at all. Data assimilation makes it possible to rigorously estimate parameter values and, consequently, offers an approach to understand which, how, how much, and why parameters vary. To fully understand those issues, extensive research is required. Nonetheless, it is clear that changes in parameter values lead to different model predictions even if the model structure is the same.     

Heuristic approaches to models and modeling in systems biology

Prediction and control sufficient for reliable medical and other interventions are prominent aims of modeling in systems biology. The short-term attainment of these goals has played a strong role in projecting the importance and value of the field. In this paper I identify the standard models must meet to achieve these objectives as predictive robustness—predictive reliability over large domains. Drawing on the results of an ethnographic investigation and various studies in the systems biology literature, I explore four current obstacles to achieving predictive robustness; data constraints, parameter uncertainty, collaborative constraints and system-scale requirements. I use a case study and the commentary of systems biologists themselves to show that current practices in the field, rather than pursuing these goals, frequently use models heuristically to investigate and build understanding of biological systems that do not meet standards of predictive robustness but are nonetheless effective uses of computation. A more heuristic conception of modeling allows us to interpret current practices as ways that manage these obstacles more effectively, particularly collaborative constraints, such that modelers can in the long-run at least work towards prediction and control.