Minimal output sets for identifiability

Ordinary differential equation models in biology often contain a large number of parameters that must be determined from measurements by parameter estimation. For a parameter estimation procedure to be successful, there must be a unique set of parameters that can have produced the measured data. This is not the case if a model is not uniquely structurally identifiable with the given set of outputs selected as measurements. In designing an experiment for the purpose of parameter estimation, given a set of feasible but resource-consuming measurements, it is useful to know which ones must be included in order to obtain an identifiable system, or whether the system is unidentifiable from the feasible measurement set. We have developed an algorithm that, from a user-provided set of variables and parameters or functions of them assumed to be measurable or known, determines all subsets that when used as outputs give a locally structurally identifiable system and are such that any output set for which the system is structurally identifiable must contain at least one of the calculated subsets. The algorithm has been implemented in Mathematica and shown to be feasible and efficient. We have successfully applied it in the analysis of large signalling pathway models from the literature.     

Dynamic Bayesian sensitivity analysis of a myocardial metabolic model

Dynamic compartmentalized metabolic models are identified by a large number of parameters, several of which are either non-physical or extremely difficult to measure. Typically, the available data and prior information is insufficient to fully identify the system. Since the models are used to predict the behavior of unobserved quantities, it is important to understand how sensitive the output of the system is to perturbations in the poorly identifiable parameters. Classically, it is the goal of sensitivity analysis to asses how much the output changes as a function of the parameters. In the case of dynamic models, the output is a function of time and therefore its sensitivity is a time dependent function. If the output is a differentiable function of the parameters, the sensitivity at one time instance can be computed from its partial derivatives with respect to the parameters. The time course of these partial derivatives describes how the sensitivity varies in time.When the model is not uniquely identifiable, or if the solution of the parameter identification problem is known only approximately, we may have not one, but a distribution of possible parameter values. This is always the case when the parameter identification problem is solved in a statistical framework. In that setting, the proper way to perform sensitivity analysis is to not rely on the values of the sensitivity functions corresponding to a single model, but to consider the distributed nature of the sensitivity functions, inherited from the distribution of the vector of the model parameters.In this paper we propose a methodology for analyzing the sensitivity of dynamic metabolic models which takes into account the variability of the sensitivity over time and across a sample. More specifically, we draw a representative sample from the posterior density of the vector of model parameters, viewed as a random variable. To interpret the output of this doubly varying sensitivity analysis, we propose visualization modalities particularly effective at displaying simultaneously variations over time and across a sample. We perform an analysis of the sensitivity of the concentrations of lactate and glycogen in cytosol, and of ATP, ADP, NAD⁺ and NADH in cytosol and mitochondria, to the parameters identifying a three compartment model for myocardial metabolism during ischemia.     

Finding identifiable parameter combinations in nonlinear ODE models and the rational reparameterization of their input-output equations

When examining the structural identifiability properties of dynamic system models, some parameters can take on an infinite number of values and yet yield identical input-output data. These parameters and the model are then said to be unidentifiable. Finding identifiable combinations of parameters with which to reparameterize the model provides a means for quantitatively analyzing the model and computing solutions in terms of the combinations. In this paper, we revisit and explore the properties of an algorithm for finding identifiable parameter combinations using Gröbner Bases and prove useful theoretical properties of these parameter combinations. We prove a set of M algebraically independent identifiable parameter combinations can be found using this algorithm and that there exists a unique rational reparameterization of the input-output equations over these parameter combinations. We also demonstrate application of the procedure to a nonlinear biomodel.     

Dynamic Finite Size Effects in Spiking Neural Networks

We investigate the dynamics of a deterministic finite-sized network of synaptically coupled spiking neurons and present a formalism for computing the network statistics in a perturbative expansion. The small parameter for the expansion is the inverse number of neurons in the network. The network dynamics are fully characterized by a neuron population density that obeys a conservation law analogous to the Klimontovich equation in the kinetic theory of plasmas. The Klimontovich equation does not possess well-behaved solutions but can be recast in terms of a coupled system of well-behaved moment equations, known as a moment hierarchy. The moment hierarchy is impossible to solve but in the mean field limit of an infinite number of neurons, it reduces to a single well-behaved conservation law for the mean neuron density. For a large but finite system, the moment hierarchy can be truncated perturbatively with the inverse system size as a small parameter but the resulting set of reduced moment equations that are still very difficult to solve. However, the entire moment hierarchy can also be re-expressed in terms of a functional probability distribution of the neuron density. The moments can then be computed perturbatively using methods from statistical field theory. Here we derive the complete mean field theory and the lowest order second moment corrections for physiologically relevant quantities. Although we focus on finite-size corrections, our method can be used to compute perturbative expansions in any parameter.     

An algorithm for finding globally identifiable parameter combinations of nonlinear ODE models using Gröbner Bases

The parameter identifiability problem for dynamic system ODE models has been extensively studied. Nevertheless, except for linear ODE models, the question of establishing identifiable combinations of parameters when the model is unidentifiable has not received as much attention and the problem is not fully resolved for nonlinear ODEs. Identifiable combinations are useful, for example, for the reparameterization of an unidentifiable ODE model into an identifiable one. We extend an existing algorithm for finding globally identifiable parameters of nonlinear ODE models to generate the ‘simplest’ globally identifiable parameter combinations using Gröbner Bases. We also provide sufficient conditions for the method to work, demonstrate our algorithm and find associated identifiable reparameterizations for several linear and nonlinear unidentifiable biomodels.     

A simple work flow for biologically inspired model reduction - application to early JAK-STAT signaling

Background  

Modeling of biological pathways is a key issue in systems biology. When constructing a model, it is tempting to incorporate all known interactions of pathway species, which results in models with a large number of unknown parameters. Fortunately, unknown parameters need not necessarily be measured directly, but some parameter values can be estimated indirectly by fitting the model to experimental data. However, parameter fitting, or, more precisely, maximum likelihood parameter estimation, only provides valid results, if the complexity of the model is in balance with the amount and quality of the experimental data. If this is the case the model is said to be identifiable for the given data. If a model turns out to be unidentifiable, two steps can be taken. Either additional experiments need to be conducted, or the model has to be simplified.     

Model‐based identifiable parameter determination applied to a simultaneous saccharification and fermentation process model for bio‐ethanol production

In this work, a methodology for the model‐based identifiable parameter determination (MBIPD) is presented. This systematic approach is proposed to be used for structure and parameter identification of nonlinear models of biological reaction networks. Usually, this kind of problems are over‐parameterized with large correlations between parameters. Hence, the related inverse problems for parameter determination and analysis are mathematically ill‐posed and numerically difficult to solve. The proposed MBIPD methodology comprises several tasks: (i) model selection, (ii) tracking of an adequate initial guess, and (iii) an iterative parameter estimation step which includes an identifiable parameter subset selection (SsS) algorithm and accuracy analysis of the estimated parameters. The SsS algorithm is based on the analysis of the sensitivity matrix by rank revealing factorization methods. Using this, a reduction of the parameter search space to a reasonable subset, which can be reliably and efficiently estimated from available measurements, is achieved. The simultaneous saccharification and fermentation (SSF) process for bio‐ethanol production from cellulosic material is used as case study for testing the methodology. The successful application of MBIPD to the SSF process demonstrates a relatively large reduction in the identified parameter space. It is shown by a cross‐validation that using the identified parameters (even though the reduction of the search space), the model is still able to predict the experimental data properly. Moreover, it is shown that the model is easily and efficiently adapted to new process conditions by solving reduced and well conditioned problems. © 2013 American Institute of Chemical Engineers Biotechnol. Prog., 29:1064–1082, 2013     

Multistationarity in mass action networks with applications to ERK activation

Ordinary Differential Equations (ODEs) are an important tool in many areas of Quantitative Biology. For many ODE systems multistationarity (i.e. the existence of at least two positive steady states) is a desired feature. In general establishing multistationarity is a difficult task as realistic biological models are large in terms of states and (unknown) parameters and in most cases poorly parameterized (because of noisy measurement data of few components, a very small number of data points and only a limited number of repetitions). For mass action networks establishing multistationarity hence is equivalent to establishing the existence of at least two positive solutions of a large polynomial system with unknown coefficients. For mass action networks with certain structural properties, expressed in terms of the stoichiometric matrix and the reaction rate-exponent matrix, we present necessary and sufficient conditions for multistationarity that take the form of linear inequality systems. Solutions of these inequality systems define pairs of steady states and parameter values. We also present a sufficient condition to identify networks where the aforementioned conditions hold. To show the applicability of our results we analyse an ODE system that is defined by the mass action network describing the extracellular signal-regulated kinase (ERK) cascade (i.e. ERK-activation).     

Exploration of the intercellular heterogeneity of topotecan uptake into human breast cancer cells through compartmental modelling

A mathematical multi-cell model for the in vitro kinetics of the anti-cancer agent topotecan (TPT) following administration into a culture medium containing a population of human breast cancer cells (MCF-7 cell line) is described. This non-linear compartmental model is an extension of an earlier single-cell type model and has been validated using experimental data obtained using two-photon laser scanning microscopy (TPLSM). A structural identifiability analysis is performed prior to parameter estimation to test whether the unknown parameters within the model are uniquely determined by the model outputs. The full model has 43 compartments, with 107 unknown parameters, and it was found that the structural identifiability result could not be established even when using the latest version of the symbolic computation software Mathematica. However, by assuming that a priori knowledge is available for certain parameters, it was possible to reduce the number of parameters to 81, and it was found that this (Stage Two) model was globally (uniquely) structurally identifiable. The identifiability analysis demonstrated how valuable symbolic computation is in this context, as the analysis is far too lengthy and difficult to be performed by hand.     

Identifying evolutionary trees and substitution parameters for the general Markov model with invariable sites

The general Markov plus invariable sites (GM+I) model of biological sequence evolution is a two-class model in which an unknown proportion of sites are not allowed to change, while the remainder undergo substitutions according to a Markov process on a tree. For statistical use it is important to know if the model is identifiable; can both the tree topology and the numerical parameters be determined from a joint distribution describing sequences only at the leaves of the tree? We establish that for generic parameters both the tree and all numerical parameter values can be recovered, up to clearly understood issues of 'label swapping'. The method of analysis is algebraic, using phylogenetic invariants to study the variety defined by the model. Simple rational formulas, expressed in terms of determinantal ratios, are found for recovering numerical parameters describing the invariable sites.     

Parameter Identification in a Generalized Time-Harmonic Rayleigh Damping Model for Elastography

The identifiability of the two damping components of a Generalized Rayleigh Damping model is investigated through analysis of the continuum equilibrium equations as well as a simple spring-mass system. Generalized Rayleigh Damping provides a more diversified attenuation model than pure Viscoelasticity, with two parameters to describe attenuation effects and account for the complex damping behavior found in biological tissue. For heterogeneous Rayleigh Damped materials, there is no equivalent Viscoelastic system to describe the observed motions. For homogeneous systems, the inverse problem to determine the two Rayleigh Damping components is seen to be uniquely posed, in the sense that the inverse matrix for parameter identification is full rank, with certain conditions: when either multi-frequency data is available or when both shear and dilatational wave propagation is taken into account. For the multi-frequency case, the frequency dependency of the elastic parameters adds a level of complexity to the reconstruction problem that must be addressed for reasonable solutions. For the dilatational wave case, the accuracy of compressional wave measurement in fluid saturated soft tissues becomes an issue for qualitative parameter identification. These issues can be addressed with reasonable assumptions on the negligible damping levels of dilatational waves in soft tissue. In general, the parameters of a Generalized Rayleigh Damping model are identifiable for the elastography inverse problem, although with more complex conditions than the simpler Viscoelastic damping model. The value of this approach is the additional structural information provided by the Generalized Rayleigh Damping model, which can be linked to tissue composition as well as rheological interpretations.     

Decomposition-based qualitative experiment design algorithms for a class of compartmental models.

Qualitative experiment design, to determine experimental input/output configurations that provide identifiability for specific parameters of interest, can be extremely difficult if the number of unknown parameters and the number of compartments are relatively large. However, the problem can be considerably simplified if the parameters can be divided into several groups for separate identification and the model can be decomposed into smaller submodels for separate experiment design. Model decomposition-based experiment design algorithms are proposed for a practical class of large-scale compartmental models representative of biosystems characterized by multiple input sources and unidirectional interconnectivity among subsystems. The model parameters are divided into three types, each of which is identified consecutively, in three stages, using simpler submodel experiment designs. Several practical examples are presented. Necessary and sufficient conditions for identifiability using the algorithm are also discussed.     

Data-based identifiability analysis of non-linear dynamical models

MOTIVATION: Mathematical modelling of biological systems is becoming a standard approach to investigate complex dynamic, non-linear interaction mechanisms in cellular processes. However, models may comprise non-identifiable parameters which cannot be unambiguously determined. Non-identifiability manifests itself in functionally related parameters, which are difficult to detect. RESULTS: We present the method of mean optimal transformations, a non-parametric bootstrap-based algorithm for identifiability testing, capable of identifying linear and non-linear relations of arbitrarily many parameters, regardless of model size or complexity. This is performed with use of optimal transformations, estimated using the alternating conditional expectation algorithm (ACE). An initial guess or prior knowledge concerning the underlying relation of the parameters is not required. Independent, and hence identifiable parameters are determined as well. The quality of data at disposal is included in our approach, i.e. the non-linear model is fitted to data and estimated parameter values are investigated with respect to functional relations. We exemplify our approach on a realistic dynamical model and demonstrate that the variability of estimated parameter values decreases from 81 to 1% after detection and fixation of structural non-identifiabilities.     

The parameter identification problem for the somatic shunt model

The somatic shunt model, a generalized version of the Rall equivalent cylinder model, is used commonly to describe the passive electrotonic properties of neurons. Procedures for determining the parameters of the somatic shunt model that best describe a given neuron typically rely on the response of the cell to a small step of hyperpolarizing current injected by an intrasomatic recording electrode. In this study it is shown that the problem of estimating model parameters for the somatic shunt model using physiological data is ill-posed, in that very small errors in measured data can lead to large and unpredictable errors in parameter estimates. If the somatic shunt is assumed to be a real property of the intact neuron, the effects of these errors are not severe when predicting EPSP waveshapes resulting from synaptic input at a given location. However, if the somatic shunt is assumed to be a consequence of a leakage pathway around the recording electrode, and a correction for the shunt is applied, then the instability of the inverse problem can introduce large errors in estimates of EPSP waveshape as a function of synaptic location in the intact cell. Morphological constraints can be used to improve the accuracy of the inversion procedure in terms of both parameter estimates and predicted EPSP responses.     

State estimation bias induced by optimization under uncertainty and error cost asymmetry is likely reflected in perception

It is well known from numerous studies that perception can be significantly affected by intended action in many everyday situations, indicating that perception and related decision-making is not a simple, one-way sequence, but a complex iterative cognitive process. However, the underlying functional mechanisms are yet unclear. Based on an optimality approach, a quantitative computational model of one such mechanism has been developed in this study. It is assumed in the model that significant uncertainty about task-related parameters of the environment results in parameter estimation errors and an optimal control system should minimize the cost of such errors in terms of the optimality criterion. It is demonstrated that, if the cost of a parameter estimation error is significantly asymmetrical with respect to error direction, the tendency to minimize error cost creates a systematic deviation of the optimal parameter estimate from its maximum likelihood value. Consequently, optimization of parameter estimate and optimization of control action cannot be performed separately from each other under parameter uncertainty combined with asymmetry of estimation error cost, thus making the certainty equivalence principle non-applicable under those conditions. A hypothesis that not only the action, but also perception itself is biased by the above deviation of parameter estimate is supported by ample experimental evidence. The results provide important insights into the cognitive mechanisms of interaction between sensory perception and planning an action under realistic conditions. Implications for understanding related functional mechanisms of optimal control in the CNS are discussed.     

Structural sensitivity and resilience in a predator–prey model with density-dependent mortality

Numerous formulations with the same mathematical properties can be relevant to model a biological process. Different formulations can predict different model dynamics like equilibrium vs. oscillations even if they are quantitatively close (structural sensitivity). The question we address in this paper is: does the choice of a formulation affect predictions on the number of stable states? We focus on a predator–prey model with predator competition that exhibits multiple stable states. A bifurcation analysis is realized with respect to prey carrying capacity and species body mass ratio within range of values found in food web models. Bifurcation diagrams built for two type-II functional responses are different in two ways. First, the kind of stable state (equilibrium vs. oscillations) is different for 26.0–49.4% of the parameter values, depending on the parameter space investigated. Using generalized modelling, we highlight the role of functional response slope in this difference. Secondly, the number of stable states is higher with Ivlev's functional response for 0.1–14.3% of the parameter values. These two changes interact to create different model predictions if a parameter value or a state variable is altered. In these two examples of disturbance, Holling's disc equation predicts a higher system resilience. Indeed, Ivlev's functional response predicts that disturbance may trap the system into an alternative stable state that can be escaped from only by a larger alteration (hysteresis phenomena). Two questions arise from this work: (i) how much complex ecological models can be affected by this sensitivity to model formulation? and (ii) how to deal with these uncertainties in model predictions?     

Evaluation of four methods for computing parameter sensitivities in dynamic system models

Computation of state sensitivities with respect to parameters can be a difficult and costly numerical problem when the number of states and parameters is large, or when sensitivities must be computed repeatedly, as with many optimization algorithms. Four methods are evaluated in terms of solution accuracy, and computer-time and storage requirements: direct numerical integration of the complete sensitivity-system differential equations, a reduced-order method based on the controllable states of the sensitivity system, a numerical-quadratures technique applied directly to the analytic solution of the original system, and an approach based on the solution of the transition matrix. Three linear system models, with four different types of inputs, were used as test cases, the largest having 6 states and 12 parameters. The reduced-order method was the most time-efficient in a majority of cases, but it was prone to numerical instability problems in certain situations which may be encountered in applications. It also had the largest storage requirements. For the highest-order system, only direct numerical integration and the transition-matrix method produced sufficiently accurate results for most applications, because of matrix-inversion problems with the other methods. For impulse inputs, the transition-matrix and the numerical-quadratures methods overall were the most computationally efficient, but the transition-matrix approach required much more memory storage.     

What life cycle graphs can tell about the evolution of life histories

We analyze long-term evolutionary dynamics in a large class of life history models. The model family is characterized by discrete-time population dynamics and a finite number of individual states such that the life cycle can be described in terms of a population projection matrix. We allow an arbitrary number of demographic parameters to be subject to density-dependent population regulation and two or more demographic parameters to be subject to evolutionary change. Our aim is to identify structural features of life cycles and modes of population regulation that correspond to specific evolutionary dynamics. Our derivations are based on a fitness proxy that is an algebraically simple function of loops within the life cycle. This allows us to phrase the results in terms of properties of such loops which are readily interpreted biologically. The following results could be obtained. First, we give sufficient conditions for the existence of optimisation principles in models with an arbitrary number of evolving traits. These models are then classified with respect to their appropriate optimisation principle. Second, under the assumption of just two evolving traits we identify structural features of the life cycle that determine whether equilibria of the monomorphic adaptive dynamics (evolutionarily singular points) correspond to fitness minima or maxima. Third, for one class of frequency-dependent models, where optimisation is not possible, we present sufficient conditions that allow classifying singular points in terms of the curvature of the trade-off curve. Throughout the article we illustrate the utility of our framework with a variety of examples.     

The Edge-Disjoint Path Problem on Random Graphs by Message-Passing

We present a message-passing algorithm to solve a series of edge-disjoint path problems on graphs based on the zero-temperature cavity equations. Edge-disjoint paths problems are important in the general context of routing, that can be defined by incorporating under a unique framework both traffic optimization and total path length minimization. The computation of the cavity equations can be performed efficiently by exploiting a mapping of a generalized edge-disjoint path problem on a star graph onto a weighted maximum matching problem. We perform extensive numerical simulations on random graphs of various types to test the performance both in terms of path length minimization and maximization of the number of accommodated paths. In addition, we test the performance on benchmark instances on various graphs by comparison with state-of-the-art algorithms and results found in the literature. Our message-passing algorithm always outperforms the others in terms of the number of accommodated paths when considering non trivial instances (otherwise it gives the same trivial results). Remarkably, the largest improvement in performance with respect to the other methods employed is found in the case of benchmarks with meshes, where the validity hypothesis behind message-passing is expected to worsen. In these cases, even though the exact message-passing equations do not converge, by introducing a reinforcement parameter to force convergence towards a sub optimal solution, we were able to always outperform the other algorithms with a peak of 27% performance improvement in terms of accommodated paths. On random graphs, we numerically observe two separated regimes: one in which all paths can be accommodated and one in which this is not possible. We also investigate the behavior of both the number of paths to be accommodated and their minimum total length.     

一类自催化反应扩散模型正解的唯一性与稳定性

主要考虑了一类三分子自催化反应扩散系统．在齐次Dirichlet和Robin边界条件下,当反应率c适当小,系统没有共存态;当c适当大,系统至少有一个共存态;当c充分大,系统有唯一渐近稳定的共存态．特别地,在一维空间上共存态是唯一的．在齐次Neumann边界条件下系统是一个简单系统．