The limitations,dangers, and benefits of simple methods for testing identifiability

In their Commentary paper, Villaverde and Massonis (On testing structural identifiability by a simple scaling method: relying on scaling symmetries can be misleading) have commented on our paper in which we proposed a simple scaling method to test structural identifiability. Our scaling invariance method (SIM) tests for scaling symmetries only, and Villaverde and Massonis correctly show the SIM may fail to detect identifiability problems when a model has other types of symmetries. We agree with the limitations raised by these authors but, also, we emphasize that the method is still valuable for its applicability to a wide variety of models, its simplicity, and even as a tool to introduce the problem of identifiability to investigators with little training in mathematics.

In their Commentary paper, Villaverde and Massonis (On testing structural identifiability by a simple scaling method: relying on scaling symmetries can be misleading [1]) have commented on our paper in which we proposed a simple scaling method to test structural identifiability [2]. Our scaling invariance method (SIM) tests for scaling symmetries only, and Villaverde and Massonis correctly show the SIM may fail to detect identifiability problems when a model has other types of symmetries (we indeed indicated but not investigated the importance of generalizing the method to other symmetries). Thus, we agree that our simple method provides a necessary but not sufficient condition for identifiability, and we appreciate their careful analysis and constructive criticism.We nevertheless think that the simple method remains useful because it is so simple. Even for investigators with little training in mathematics, the method provides a necessary condition for structural identifiability that can be derived in a few minutes with pen and paper. Similarly, we have found its pedagogic strength by teaching the method to our own graduate students and colleagues. More advanced methods (such as STRIKE-GOLDD [3,4], COMBOS [5], or SIAN [6]) are typically intimidating for researchers with a background in Biology or Bioinformatics. This simple method can help those practitioners to familiarize themselves with the identifiability problem and better understand their models.Finally, it is worth noting that if scaling invariance is the only symmetry (as it was in all the cases we analyzed), our SIM remains valuable (albeit uncontrolled), and surprisingly effective for a wide variety of problems (as the extensive list collected in the Supplementary Material our paper [2]). We guess that the SIM especially fails when applied to linear models (as more potential rotations of the variables leave the system invariant), and in non-linear scenarios where some parameters are identical. For instance, the FitzHugh-Nagumo model raised by Villaverde and Massonis, x˙1(t)=c(x1(t)−x13(t)3−x2(t)+d),x˙2(t)=1c(x1(t)+a−b·x2(t)),y(t)=x1(t), could have been written as x˙1(t)=λ1x1(t)−λ2x13(t)3−λ3x2(t)+d,x˙2(t)=λ4x1(t)+a−b·x2(t),y(t)=x1(t) where λ₁ = λ₂ = λ₃ = 1/λ₄ = c. One of the reasons why our method fails, in this case, might be these additional symmetries introduced in this more elaborate notation of the model.Hence, it is worth understanding generic conditions under which the SIM method is expected to be fragile, possibly using STRIKE-GOLDD to test large families of nonlinear models.As a final remark, we appreciate that Villaverde and Massonis have shared their source code, so researchers might have a gold standard to test identifiability.     

Identifiability and online estimation of diagnostic parameters with in the glucose insulin homeostasis

Today, diagnostic decisions about pre-diabetes or diabetes are made using static threshold rules for the measured plasma glucose. In order to develop an alternative diagnostic approach, dynamic models as the Minimal Model may be deployed. We present a novel method to analyze the identifiability of model parameters based on the interpretation of the empirical observability Gramian. This allows a unifying view of both, the observability of the system's states (with dynamics) and the identifiability of the system's parameters (without dynamics). We give an iterative algorithm, in order to find an optimized set of states and parameters to be estimated. For this set, estimation results using an Unscented Kalman Filter (UKF) are presented. Two parameters are of special interest for diagnostic purposes: the glucose effectiveness S(G) characterizes the ability of plasma glucose clearance, and the insulin sensitivity S(I) quantifies the impact from the plasma insulin to the interstitial insulin subsystem. Applying the identifiability analysis to the trajectories of the insulin glucose system during an intravenous glucose tolerance test (IVGTT) shows the following result: (1) if only plasma glucose G(t) is measured, plasma insulin I(t) and S(G) can be estimated, but not S(I). (2) If plasma insulin I(t) is captured additionally, identifiability is improved significantly such that up to four model parameters can be estimated including S(I). (3) The situation of the first case can be improved, if a controlled external dosage of insulin is applied. Then, parameters of the insulin subsystem can be identified approximately from measurement of plasma glucose G(t) only.     

Structural identifiability for a class of non-linear compartmental systems using linear/non-linear splitting and symbolic computation

Under certain controllability and observability restrictions, two different parameterisations for a non-linear compartmental model can only have the same input-output behaviour if they differ by a locally diffeomorphic change of basis for the state space. With further restrictions, it is possible to gain valuable information with respect to identifiability via a linear analysis. Examples are presented where non-linear identifiability analyses are substantially simplified by means of an initial linear analysis. For complex models, with four or more compartments, this linear analysis can prove lengthy to perform by hand and so symbolic computation has been employed to aid this procedure.     

Construction of a large scale integrated map of macrophage pathogen recognition and effector systems

Background

Mathematical models provide abstract representations of the information gained from experimental observations on the structure and function of a particular biological system. Conferring a predictive character on a given mathematical formulation often relies on determining a number of non-measurable parameters that largely condition the model's response. These parameters can be identified by fitting the model to experimental data. However, this fit can only be accomplished when identifiability can be guaranteed.

Results

We propose a novel iterative identification procedure for detecting and dealing with the lack of identifiability. The procedure involves the following steps: 1) performing a structural identifiability analysis to detect identifiable parameters; 2) globally ranking the parameters to assist in the selection of the most relevant parameters; 3) calibrating the model using global optimization methods; 4) conducting a practical identifiability analysis consisting of two (a priori and a posteriori) phases aimed at evaluating the quality of given experimental designs and of the parameter estimates, respectively and 5) optimal experimental design so as to compute the scheme of experiments that maximizes the quality and quantity of information for fitting the model.

Conclusions

The presented procedure was used to iteratively identify a mathematical model that describes the NF-κB regulatory module involving several unknown parameters. We demonstrated the lack of identifiability of the model under typical experimental conditions and computed optimal dynamic experiments that largely improved identifiability properties.     

Parameter Identifiability and Redundancy: Theoretical Considerations

Background

Models for complex biological systems may involve a large number of parameters. It may well be that some of these parameters cannot be derived from observed data via regression techniques. Such parameters are said to be unidentifiable, the remaining parameters being identifiable. Closely related to this idea is that of redundancy, that a set of parameters can be expressed in terms of some smaller set. Before data is analysed it is critical to determine which model parameters are identifiable or redundant to avoid ill-defined and poorly convergent regression.

Methodology/Principal Findings

In this paper we outline general considerations on parameter identifiability, and introduce the notion of weak local identifiability and gradient weak local identifiability. These are based on local properties of the likelihood, in particular the rank of the Hessian matrix. We relate these to the notions of parameter identifiability and redundancy previously introduced by Rothenberg (Econometrica 39 (1971) 577–591) and Catchpole and Morgan (Biometrika 84 (1997) 187–196). Within the widely used exponential family, parameter irredundancy, local identifiability, gradient weak local identifiability and weak local identifiability are shown to be largely equivalent. We consider applications to a recently developed class of cancer models of Little and Wright (Math Biosciences 183 (2003) 111–134) and Little et al. (J Theoret Biol 254 (2008) 229–238) that generalize a large number of other recently used quasi-biological cancer models.

Conclusions/Significance

We have shown that the previously developed concepts of parameter local identifiability and redundancy are closely related to the apparently weaker properties of weak local identifiability and gradient weak local identifiability—within the widely used exponential family these concepts largely coincide.     

Similarity transformation approach to identifiability analysis of nonlinear compartmental models

Through use of the local state isomorphism theorem instead of the algebraic equivalence theorem of linear systems theory, the similarity transformation approach is extended to nonlinear models, resulting in finitely verifiable sufficient and necessary conditions for global and local identifiability. The approach requires testing of certain controllability and observability conditions, but in many practical examples these conditions prove very easy to verify. In principle the method also involves nonlinear state variable transformations, but in all of the examples presented in the paper the transformations turn out to be linear. The method is applied to an unidentifiable nonlinear model and a locally identifiable nonlinear model, and these are the first nonlinear models other than bilinear models where the reason for lack of global identifiability is nontrivial. The method is also applied to two models with Michaelis-Menten elimination kinetics, both of considerable importance in pharmacokinetics, and for both of which the complicated nature of the algebraic equations arising from the Taylor series approach has hitherto defeated attempts to establish identifiability results for specific input functions.     

Competence and determination in the process of in vitro shoot organogenesis

Leaf explants of Convolvulus arvensis produce shoots when cultured on Murashige and Skoog salts, sucrose, vitamins and 0.05 mg/liter IAA plus 7.0 mg/liter 2-isopentenyl adenine. Shoot-inducing, root-inducing, or callus-inducing medium (SIM, RIM, or CIM) will cause small amounts of callus to form at the cut edges of the explant. This first-formed callus is developmentally interchangeable: SIM induces shoots in callus formed on CIM or SIM with equal effect and efficiency. Once induction begins in competent callus, the callus is no longer interchangeable. Under the continued influence of SIM, cells, or groups of cells become determined for shoot formation. This determination is strongly canalized for shoot formation: subsequent transfer to root-inducing medium does not affect the formation of shoots by the explant. The control of organogenesis by the auxin/cytokinin balance must occur between the time the tissue becomes competent and the time it is determined for shoot (or root) development. It is not known whether this control is a single or multiple phenomenon.     

On the identifiability of metabolic network models

A major problem for the identification of metabolic network models is parameter identifiability, that is, the possibility to unambiguously infer the parameter values from the data. Identifiability problems may be due to the structure of the model, in particular implicit dependencies between the parameters, or to limitations in the quantity and quality of the available data. We address the detection and resolution of identifiability problems for a class of pseudo-linear models of metabolism, so-called linlog models. Linlog models have the advantage that parameter estimation reduces to linear or orthogonal regression, which facilitates the analysis of identifiability. We develop precise definitions of structural and practical identifiability, and clarify the fundamental relations between these concepts. In addition, we use singular value decomposition to detect identifiability problems and reduce the model to an identifiable approximation by a principal component analysis approach. The criterion is adapted to real data, which are frequently scarce, incomplete, and noisy. The test of the criterion on a model with simulated data shows that it is capable of correctly identifying the principal components of the data vector. The application to a state-of-the-art dataset on central carbon metabolism in Escherichia coli yields the surprising result that only $4$ out of $31$ reactions, and $37$ out of $100$ parameters, are identifiable. This underlines the practical importance of identifiability analysis and model reduction in the modeling of large-scale metabolic networks. Although our approach has been developed in the context of linlog models, it carries over to other pseudo-linear models, such as generalized mass-action (power-law) models. Moreover, it provides useful hints for the identifiability analysis of more general classes of nonlinear models of metabolism.     

Review: To be or not to be an identifiable model. Is this a relevant question in animal science modelling?

What is a good (useful) mathematical model in animal science? For models constructed for prediction purposes, the question of model adequacy (usefulness) has been traditionally tackled by statistical analysis applied to observed experimental data relative to model-predicted variables. However, little attention has been paid to analytic tools that exploit the mathematical properties of the model equations. For example, in the context of model calibration, before attempting a numerical estimation of the model parameters, we might want to know if we have any chance of success in estimating a unique best value of the model parameters from available measurements. This question of uniqueness is referred to as structural identifiability; a mathematical property that is defined on the sole basis of the model structure within a hypothetical ideal experiment determined by a setting of model inputs (stimuli) and observable variables (measurements). Structural identifiability analysis applied to dynamic models described by ordinary differential equations (ODEs) is a common practice in control engineering and system identification. This analysis demands mathematical technicalities that are beyond the academic background of animal science, which might explain the lack of pervasiveness of identifiability analysis in animal science modelling. To fill this gap, in this paper we address the analysis of structural identifiability from a practitioner perspective by capitalizing on the use of dedicated software tools. Our objectives are (i) to provide a comprehensive explanation of the structural identifiability notion for the community of animal science modelling, (ii) to assess the relevance of identifiability analysis in animal science modelling and (iii) to motivate the community to use identifiability analysis in the modelling practice (when the identifiability question is relevant). We focus our study on ODE models. By using illustrative examples that include published mathematical models describing lactation in cattle, we show how structural identifiability analysis can contribute to advancing mathematical modelling in animal science towards the production of useful models and, moreover, highly informative experiments via optimal experiment design. Rather than attempting to impose a systematic identifiability analysis to the modelling community during model developments, we wish to open a window towards the discovery of a powerful tool for model construction and experiment design.     

Identifiability from parameter bounds. Structural and numerical aspects

Some a priori and a posteriori aspects of the identifiability problem for unidentifiable models are discussed. It is argued that the nation of identifiability from parameter bounds has a minor a priori structural relevance. The parameter bounds rationale may prove a useful a posteriori numerical notion. However, its practical potentiality needs careful evaluation, as the use of point estimates automatically builds into the model some hidden structural constraints. Examples are given.     

Stress-induced mutagenesis and complex adaptation

Because mutations are mostly deleterious, mutation rates should be reduced by natural selection. However, mutations also provide the raw material for adaptation. Therefore, evolutionary theory suggests that the mutation rate must balance between adaptability—the ability to adapt—and adaptedness—the ability to remain adapted. We model an asexual population crossing a fitness valley and analyse the rate of complex adaptation with and without stress-induced mutagenesis (SIM)—the increase of mutation rates in response to stress or maladaptation. We show that SIM increases the rate of complex adaptation without reducing the population mean fitness, thus breaking the evolutionary trade-off between adaptability and adaptedness. Our theoretical results support the hypothesis that SIM promotes adaptation and provide quantitative predictions of the rate of complex adaptation with different mutational strategies.     

S. cerevisiae Mre11 recruits conjugated SUMO moieties to facilitate the assembly and function of the Mre11-Rad50-Xrs2 complex

Double-strand breaks (DSBs) in chromosomes are the most challenging type of DNA damage. The yeast and mammalian Mre11-Rad50-Xrs2/Nbs1 (MRX/N)-Sae2/Ctp1 complex catalyzes the resection of DSBs induced by secondary structures, chemical adducts or covalently-attached proteins. MRX/N also initiates two parallel DNA damage responses—checkpoint phosphorylation and global SUMOylation—to boost a cell''s ability to repair DSBs. However, the molecular mechanism of this SUMO-mediated response is not completely known. In this study, we report that Saccharomyces cerevisiae Mre11 can non-covalently recruit the conjugated SUMO moieties, particularly the poly-SUMO chain. Mre11 has two evolutionarily-conserved SUMO-interacting motifs, Mre11^SIM1 and Mre11^SIM2, which reside on the outermost surface of Mre11. Mre11^SIM1 is indispensable for MRX assembly. Mre11^SIM2 non-covalently links MRX with the SUMO enzymes (E2/Ubc9 and E3/Siz2) to promote global SUMOylation of DNA repair proteins. Mre11^SIM2 acts independently of checkpoint phosphorylation. During meiosis, the mre11^SIM2 mutant, as for mre11S, rad50S and sae2Δ, allows initiation but not processing of Spo11-induced DSBs. Using MRX and DSB repair as a model, our work reveals a general principle in which the conjugated SUMO moieties non-covalently facilitate the assembly and functions of multi-subunit protein complexes.     

Improved response to disasters and outbreaks by tracking population movements with mobile phone network data: a post-earthquake geospatial study in Haiti

Background

Population movements following disasters can cause important increases in morbidity and mortality. Without knowledge of the locations of affected people, relief assistance is compromised. No rapid and accurate method exists to track population movements after disasters. We used position data of subscriber identity module (SIM) cards from the largest mobile phone company in Haiti (Digicel) to estimate the magnitude and trends of population movements following the Haiti 2010 earthquake and cholera outbreak.

Methods and Findings

Geographic positions of SIM cards were determined by the location of the mobile phone tower through which each SIM card connects when calling. We followed daily positions of SIM cards 42 days before the earthquake and 158 days after. To exclude inactivated SIM cards, we included only the 1.9 million SIM cards that made at least one call both pre-earthquake and during the last month of study. In Port-au-Prince there were 3.2 persons per included SIM card. We used this ratio to extrapolate from the number of moving SIM cards to the number of moving persons. Cholera outbreak analyses covered 8 days and tracked 138,560 SIM cards.An estimated 630,000 persons (197,484 Digicel SIM cards), present in Port-au-Prince on the day of the earthquake, had left 19 days post-earthquake. Estimated net outflow of people (outflow minus inflow) corresponded to 20% of the Port-au-Prince pre-earthquake population. Geographic distribution of population movements from Port-au-Prince corresponded well with results from a large retrospective, population-based UN survey. To demonstrate feasibility of rapid estimates and to identify areas at potentially increased risk of outbreaks, we produced reports on SIM card movements from a cholera outbreak area at its immediate onset and within 12 hours of receiving data.

Conclusions

Results suggest that estimates of population movements during disasters and outbreaks can be delivered rapidly and with potentially high validity in areas with high mobile phone use. Please see later in the article for the Editors'' Summary     

Analysis of endogenous process behavior in activated sludge

In this article, an autonomous four-compartment model that describes the endogenous respiration in an aerobic biodegradation process is proposed and analyzed theoretically. First, the multi-time scale of the system's behavior, to be taken into account in subsequent analyses, is emphasized. Then, an identifiability and observability study, given measurements of MLVSS (mixed liquor volatile suspended solids) and respiration rate, is performed for use under practical circumstances, such as in state and parameter estimation. It appears that the process is observable, but not fully identifiable. Hence, for the identification of some of the model parameters, additional measurements or experiments, also indicated here, have to be performed. Furthermore, it is shown that, under quasi-steady state conditions which, in general, appear shortly after initialization of an endogenous respiration experiment, the model can be reduced significantly. Finally, results of parameter estimation from available data are presented and discussed.     

Parameter Identifiability and Redundancy in a General Class of Stochastic Carcinogenesis Models

Background

Heidenreich et al. (Risk Anal 1997 17 391–399) considered parameter identifiability in the context of the two-mutation cancer model and demonstrated that combinations of all but two of the model parameters are identifiable. We consider the problem of identifiability in the recently developed carcinogenesis models of Little and Wright (Math Biosci 2003 183 111–134) and Little et al. (J Theoret Biol 2008 254 229–238). These models, which incorporate genomic instability, generalize a large number of other quasi-biological cancer models, in particular those of Armitage and Doll (Br J Cancer 1954 8 1–12), the two-mutation model (Moolgavkar et al. Math Biosci 1979 47 55–77), the generalized multistage model of Little (Biometrics 1995 51 1278–1291), and a recently developed cancer model of Nowak et al. (PNAS 2002 99 16226–16231).

Methodology/Principal Findings

We show that in the simpler model proposed by Little and Wright (Math Biosci 2003 183 111–134) the number of identifiable combinations of parameters is at most two less than the number of biological parameters, thereby generalizing previous results of Heidenreich et al. (Risk Anal 1997 17 391–399) for the two-mutation model. For the more general model of Little et al. (J Theoret Biol 2008 254 229–238) the number of identifiable combinations of parameters is at most less than the number of biological parameters, where is the number of destabilization types, thereby also generalizing all these results. Numerical evaluations suggest that these bounds are sharp. We also identify particular combinations of identifiable parameters.

Conclusions/Significance

We have shown that the previous results on parameter identifiability can be generalized to much larger classes of quasi-biological carcinogenesis model, and also identify particular combinations of identifiable parameters. These results are of theoretical interest, but also of practical significance to anyone attempting to estimate parameters for this large class of cancer models.     

Assay methods for small ubiquitin-like modifier (SUMO)–SUMO-interacting motif (SIM) interactions in vivo and in vitro using a split-luciferase complementation system

SUMOylation is a posttranslational process that attaches a small ubiquitin-like modifier (SUMO) to its target proteins covalently. SUMOylation controls multiple cellular processes through the recognition of SUMO by a SUMO-interacting motif (SIM). In this study, we developed assay systems for detecting noncovalent interactions between SUMO and SIM in cells using split-luciferase complementation. We applied a version of this assay to the detection of in vitro SUMO–SIM interactions using a bacterial expression system. These novel assays enable screening of inhibitors of SUMO-dependent protein–protein interactions, either in vivo or in vitro, in a high-throughput manner.     

Easy parameter identifiability analysis with COPASI

Background and scope

Differential equation systems modeling biochemical reaction networks can only give quantitative predictions, when they are in accordance with experimental data. However, even if a model can well recapitulate given data, it is often the case that some of its kinetic parameters can be arbitrarily chosen without significantly affecting the simulation results. This indicates a lack of appropriate data to determine those parameters. In this case, the parameter is called to be practically non-identifiable. Well-identified parameters are paramount for reliable quantitative predictions and, therefore, identifiability analysis is an important topic in modeling of biochemical reaction networks. Here, we describe a hidden feature of the free modeling software COPASI, which can be exploited to easily and quickly conduct a parameter identifiability analysis of differential equation systems by calculating likelihood profiles. The proposed combination of an established method for parameter identifiability analysis with the user-friendly features of COPASI offers an easy and rapid access to parameter identifiability analysis even for non-experts.

Availability

COPASI is freely available for academic use at http://www.copasi.org.     

Indistinguishability and identifiability analysis of linear compartmental models.

Two compartmental model structures are said to be indistinguishable if they have the same input-output properties. In cases in which available a priori information is not sufficient to specify a unique compartmental model structure, indistinguishable model structures may have to be generated and their attributes examined for relevance. An algorithm is developed that, for a given compartmental model, investigates the complete set of models with the same number of compartments and the same input-output structure as the original model, applies geometrical rules necessary for indistinguishable models, and test models meeting the geometrical criteria for equality of transfer functions. Identifiability is also checked in the algorithm. The software consists of three programs. Program 1 determines the number of locally identifiable parameters. Program 2 applies several geometrical rules that eliminate many (generally most) of the candidate models. Program 3 checks the equality between system transfer functions of the original model and models being tested. Ranks of Jacobian matrices and submatrices and other criteria are used to check patterns of moment invariants and local identifiability. Structural controllability and structural observability are checked throughout the programs. The approach was successfully used to corroborate results from examples investigated by others.     

Simultaneous immunization against tuberculosis

Background

BCG, the only licensed vaccine against tuberculosis, provides some protection against disseminated disease in infants but has little effect on prevention of adult pulmonary disease. Newer parenteral immunization prime boost regimes may provide improved protection in experimental animal models but are unproven in man so that there remains a need for new and improved immunization strategies.

Methods and Findings

Mice were immunized parenterally, intranasally or simultaneously by both routes with BCG or recombinant mycobacterial antigens plus appropriate adjuvants. They were challenged with Mycobacterium tuberculosis (Mtb) and the kinetics of Mtb growth in the lungs measured. We show that simultaneous immunization (SIM) of mice by the intranasal and parenteral routes is highly effective in increasing protection over parenteral BCG administration alone. Intranasal immunization induces local pulmonary immunity capable of inhibiting the growth of Mtb in the early phase (the first week) of infection, while parenteral immunization has a later effect on Mtb growth. Importantly, these two effects are additive and do not depend on priming and boosting the immune response. The best SIM regimes reduce lung Mtb load by up to 2 logs more than BCG given by either route alone.

Conclusions

These data establish SIM as a novel and highly effective immunization strategy for Mtb that could be carried out at a single clinic visit. The efficacy of SIM does not depend on priming and boosting an immune response, but SIM is complementary to prime boost strategies and might be combined with them.     

Structural Properties of Gene Regulatory Networks: Definitions and Connections

The study of gene regulatory networks is a significant problem in systems biology. Of particular interest is the problem of determining the unknown or hidden higher level regulatory signals by using gene expression data from DNA microarray experiments. Several studies in this area have demonstrated the critical aspect of the network structure in tackling the network modelling problem. Structural analysis of systems has proved useful in a number of contexts, viz., observability, controllability, fault diagnosis, sparse matrix computations etc. In this contribution, we formally define structural properties that are relevant to Gene Regulatory Networks. We explore the structural implications of certain quantitative methods and explain completely the connections between the identifiability conditions and structural criteria of observability and distinguishability. We illustrate these concepts in case studies using representative biologically motivated network examples. The present work bridges the quantitative modelling methods with those based on the structural analysis.