Model‐based identifiable parameter determination applied to a simultaneous saccharification and fermentation process model for bio‐ethanol production

In this work, a methodology for the model‐based identifiable parameter determination (MBIPD) is presented. This systematic approach is proposed to be used for structure and parameter identification of nonlinear models of biological reaction networks. Usually, this kind of problems are over‐parameterized with large correlations between parameters. Hence, the related inverse problems for parameter determination and analysis are mathematically ill‐posed and numerically difficult to solve. The proposed MBIPD methodology comprises several tasks: (i) model selection, (ii) tracking of an adequate initial guess, and (iii) an iterative parameter estimation step which includes an identifiable parameter subset selection (SsS) algorithm and accuracy analysis of the estimated parameters. The SsS algorithm is based on the analysis of the sensitivity matrix by rank revealing factorization methods. Using this, a reduction of the parameter search space to a reasonable subset, which can be reliably and efficiently estimated from available measurements, is achieved. The simultaneous saccharification and fermentation (SSF) process for bio‐ethanol production from cellulosic material is used as case study for testing the methodology. The successful application of MBIPD to the SSF process demonstrates a relatively large reduction in the identified parameter space. It is shown by a cross‐validation that using the identified parameters (even though the reduction of the search space), the model is still able to predict the experimental data properly. Moreover, it is shown that the model is easily and efficiently adapted to new process conditions by solving reduced and well conditioned problems. © 2013 American Institute of Chemical Engineers Biotechnol. Prog., 29:1064–1082, 2013     

A simple work flow for biologically inspired model reduction - application to early JAK-STAT signaling

Background  

Modeling of biological pathways is a key issue in systems biology. When constructing a model, it is tempting to incorporate all known interactions of pathway species, which results in models with a large number of unknown parameters. Fortunately, unknown parameters need not necessarily be measured directly, but some parameter values can be estimated indirectly by fitting the model to experimental data. However, parameter fitting, or, more precisely, maximum likelihood parameter estimation, only provides valid results, if the complexity of the model is in balance with the amount and quality of the experimental data. If this is the case the model is said to be identifiable for the given data. If a model turns out to be unidentifiable, two steps can be taken. Either additional experiments need to be conducted, or the model has to be simplified.     

Efficient parameter estimation for RNA secondary structure prediction

MOTIVATION: Accurate prediction of RNA secondary structure from the base sequence is an unsolved computational challenge. The accuracy of predictions made by free energy minimization is limited by the quality of the energy parameters in the underlying free energy model. The most widely used model, the Turner99 model, has hundreds of parameters, and so a robust parameter estimation scheme should efficiently handle large data sets with thousands of structures. Moreover, the estimation scheme should also be trained using available experimental free energy data in addition to structural data. RESULTS: In this work, we present constraint generation (CG), the first computational approach to RNA free energy parameter estimation that can be efficiently trained on large sets of structural as well as thermodynamic data. Our CG approach employs a novel iterative scheme, whereby the energy values are first computed as the solution to a constrained optimization problem. Then the newly computed energy parameters are used to update the constraints on the optimization function, so as to better optimize the energy parameters in the next iteration. Using our method on biologically sound data, we obtain revised parameters for the Turner99 energy model. We show that by using our new parameters, we obtain significant improvements in prediction accuracy over current state of-the-art methods. AVAILABILITY: Our CG implementation is available at http://www.rnasoft.ca/CG/.     

Estimating a predator-prey dynamical model with the parameter cascades method

Summary .   Ordinary differential equations (ODEs) are widely used in ecology to describe the dynamical behavior of systems of interacting populations. However, systems of ODEs rarely provide quantitative solutions that are close to real field observations or experimental data because natural systems are subject to environmental and demographic noise and ecologists are often uncertain about the correct parameterization. In this article we introduce "parameter cascades" as an improved method to estimate ODE parameters such that the corresponding ODE solutions fit the real data well. This method is based on the modified penalized smoothing with the penalty defined by ODEs and a generalization of profiled estimation, which leads to fast estimation and good precision for ODE parameters from noisy data. This method is applied to a set of ODEs originally developed to describe an experimental predator–prey system that undergoes oscillatory dynamics. The new parameterization considerably improves the fit of the ODE model to the experimental data sets. At the same time, our method reveals that important structural assumptions that underlie the original ODE model are essentially correct. The mathematical formulations of the two nonlinear interaction terms (functional responses) that link the ODEs in the predator–prey model are validated by estimating the functional responses nonparametrically from the real data. We suggest two major applications of "parameter cascades" to ecological modeling: It can be used to estimate parameters when original data are noisy, missing, or when no reliable priori estimates are available; it can help to validate the structural soundness of the mathematical modeling approach.     

A self-organizing state-space-model approach for parameter estimation in hodgkin-huxley-type models of single neurons

Traditional approaches to the problem of parameter estimation in biophysical models of neurons and neural networks usually adopt a global search algorithm (for example, an evolutionary algorithm), often in combination with a local search method (such as gradient descent) in order to minimize the value of a cost function, which measures the discrepancy between various features of the available experimental data and model output. In this study, we approach the problem of parameter estimation in conductance-based models of single neurons from a different perspective. By adopting a hidden-dynamical-systems formalism, we expressed parameter estimation as an inference problem in these systems, which can then be tackled using a range of well-established statistical inference methods. The particular method we used was Kitagawa's self-organizing state-space model, which was applied on a number of Hodgkin-Huxley-type models using simulated or actual electrophysiological data. We showed that the algorithm can be used to estimate a large number of parameters, including maximal conductances, reversal potentials, kinetics of ionic currents, measurement and intrinsic noise, based on low-dimensional experimental data and sufficiently informative priors in the form of pre-defined constraints imposed on model parameters. The algorithm remained operational even when very noisy experimental data were used. Importantly, by combining the self-organizing state-space model with an adaptive sampling algorithm akin to the Covariance Matrix Adaptation Evolution Strategy, we achieved a significant reduction in the variance of parameter estimates. The algorithm did not require the explicit formulation of a cost function and it was straightforward to apply on compartmental models and multiple data sets. Overall, the proposed methodology is particularly suitable for resolving high-dimensional inference problems based on noisy electrophysiological data and, therefore, a potentially useful tool in the construction of biophysical neuron models.     

PEtab—Interoperable specification of parameter estimation problems in systems biology

Reproducibility and reusability of the results of data-based modeling studies are essential. Yet, there has been—so far—no broadly supported format for the specification of parameter estimation problems in systems biology. Here, we introduce PEtab, a format which facilitates the specification of parameter estimation problems using Systems Biology Markup Language (SBML) models and a set of tab-separated value files describing the observation model and experimental data as well as parameters to be estimated. We already implemented PEtab support into eight well-established model simulation and parameter estimation toolboxes with hundreds of users in total. We provide a Python library for validation and modification of a PEtab problem and currently 20 example parameter estimation problems based on recent studies.     

Are Assumptions about the Model Type Necessary in Reaction-Diffusion Modeling? A FRAP Application

At present, fluorescence recovery after photobleaching (FRAP) data are interpreted using various types of reaction-diffusion (RD) models: the model type is usually fixed first, and corresponding model parameters are inferred subsequently. In this article, we describe what we believe to be a novel approach for RD modeling without using any assumptions of model type or parameters. To the best of our knowledge, this is the first attempt to address both model-type and parameter uncertainties in inverting FRAP data. We start from the most general RD model, which accounts for a flexible number of molecular fractions, all mobile, with different diffusion coefficients. The maximal number of possible binding partners is identified and optimal parameter sets for these models are determined in a global search of the parameter-space using the Simulated Annealing strategy. The numerical performance of the described techniques was assessed using artificial and experimental FRAP data. Our general RD model outperformed the standard RD models used previously in modeling FRAP measurements and showed that intracellular molecular mobility can only be described adequately by allowing for multiple RD processes. Therefore, it is important to search not only for the optimal parameter set but also for the optimal model type.     

Steady state analysis of influx and transbilayer distribution of ergosterol in the yeast plasma membrane

Mathematical modeling is now frequently used in outbreak investigations to understand underlying mechanisms of infectious disease dynamics, assess patterns in epidemiological data, and forecast the trajectory of epidemics. However, the successful application of mathematical models to guide public health interventions lies in the ability to reliably estimate model parameters and their corresponding uncertainty. Here, we present and illustrate a simple computational method for assessing parameter identifiability in compartmental epidemic models. We describe a parametric bootstrap approach to generate simulated data from dynamical systems to quantify parameter uncertainty and identifiability. We calculate confidence intervals and mean squared error of estimated parameter distributions to assess parameter identifiability. To demonstrate this approach, we begin with a low-complexity SEIR model and work through examples of increasingly more complex compartmental models that correspond with applications to pandemic influenza, Ebola, and Zika. Overall, parameter identifiability issues are more likely to arise with more complex models (based on number of equations/states and parameters). As the number of parameters being jointly estimated increases, the uncertainty surrounding estimated parameters tends to increase, on average, as well. We found that, in most cases, R0 is often robust to parameter identifiability issues affecting individual parameters in the model. Despite large confidence intervals and higher mean squared error of other individual model parameters, R0 can still be estimated with precision and accuracy. Because public health policies can be influenced by results of mathematical modeling studies, it is important to conduct parameter identifiability analyses prior to fitting the models to available data and to report parameter estimates with quantified uncertainty. The method described is helpful in these regards and enhances the essential toolkit for conducting model-based inferences using compartmental dynamic models.     

Mathematical and kinetic modeling of biofilm reactor based on ant colony optimization

Inverse estimation of model parameters via mathematical modeling route, known as inverse modeling (IM), is an attractive alternative approach to the experimental methods. This approach makes use of efficient optimization techniques in the course of solution of an inverse problem with the aid of measured data. In this study, a novel optimization method based on ant colony optimization (ACO), denoted by ACO-IM, is presented for inverse estimation of kinetic and film thickness parameters of biofilm models that describe an experimental fixed bed anaerobic reactor. The proposed optimization method for parameter estimation emulates the fact that ants are capable of finding the shortest path from a food source to their nest by depositing a trial of pheromone during their walk. The efficacy of the ACO-IM for numerical estimation of bio-kinetic parameters is demonstrated through its application for the anaerobic treatment of industry wastewater in a fixed bed biofilm process. The results explain the rigorousness of mathematical models, the form of kinetic and film thickness models and the type of packing to be used with the biofilm process for accurate determination of kinetic and film thickness parameters so as to ensure reliable predictive performance of the biofilm reactor models.     

Biological networks 101: Computational modeling for molecular biologists

Computational modeling of biological networks permits the comprehensive analysis of cells and tissues to define molecular phenotypes and novel hypotheses. Although a large number of software tools have been developed, the versatility of these tools is limited by mathematical complexities that prevent their broad adoption and effective use by molecular biologists. This study clarifies the basic aspects of molecular modeling, how to convert data into useful input, as well as the number of time points and molecular parameters that should be considered for molecular regulatory models with both explanatory and predictive potential. We illustrate the necessary experimental preconditions for converting data into a computational model of network dynamics. This model requires neither a thorough background in mathematics nor precise data on intracellular concentrations, binding affinities or reaction kinetics. Finally, we show how an interactive model of crosstalk between signal transduction pathways in primary human articular chondrocytes allows insight into processes that regulate gene expression.     

SBML-PET: a Systems Biology Markup Language-based parameter estimation tool

The estimation of model parameters from experimental data remains a bottleneck for a major breakthrough in systems biology. We present a Systems Biology Markup Language (SBML) based Parameter Estimation Tool (SBML-PET). The tool is designed to enable parameter estimation for biological models including signaling pathways, gene regulation networks and metabolic pathways. SBML-PET supports import and export of the models in the SBML format. It can estimate the parameters by fitting a variety of experimental data from different experimental conditions. SBML-PET has a unique feature of supporting event definition in the SMBL model. SBML models can also be simulated in SBML-PET. Stochastic Ranking Evolution Strategy (SRES) is incorporated in SBML-PET for parameter estimation jobs. A classic ODE Solver called ODEPACK is used to solve the Ordinary Differential Equation (ODE) system. AVAILABILITY: http://sysbio.molgen.mpg.de/SBML-PET/. The website also contains detailed documentation for SBML-PET.     

Systematic identifiability testing for unambiguous mechanistic modeling – application to JAK-STAT,MAP kinase,and NF-κB signaling pathway models

Background  

When creating mechanistic mathematical models for biological signaling processes it is tempting to include as many known biochemical interactions into one large model as possible. For the JAK-STAT, MAP kinase, and NF-κB pathways a lot of biological insight is available, and as a consequence, large mathematical models have emerged. For large models the question arises whether unknown model parameters can uniquely be determined by parameter estimation from measured data. Systematic approaches to answering this question are indispensable since the uniqueness of model parameter values is essential for predictive mechanistic modeling.     

Optimal sampling time selection for parameter estimation in dynamic pathway modeling

Systems Biology is an emerging research area, which considers mathematical representations of inter- and intra-cellular dynamics. Among the many research problems that have been addressed, dynamic modeling of signal transduction pathways has received increasing attention. The usual approach to represent intra-cellular dynamics are nonlinear, usually ordinary, differential equations. The purpose of the models is to test and generate hypothesis of specific pathways and it is therefore required to estimate model parameters from experimental data. The experiments to generate data are complex and expensive, as a consequence of which the time series available are usually rather short, with few if any replicates. Almost certainly, not all variables one would like to include in a model can be measured. Parameter estimation is therefore an important research problem in Systems Biology and the focus of this paper. In particular, we are interested in optimizing the sampling time selection in order to minimize the variance of the parameter estimation error. With few sampling time points feasible, their selection is of practical importance in experimental design. Finally, the theoretical results are supported with an application.     

Simultaneous versus sequential optimal experiment design for the identification of multi-parameter microbial growth kinetics as a function of temperature

Optimal experiment design for parameter estimation (OED/PE) has become a popular tool for efficient and accurate estimation of kinetic model parameters. When the kinetic model under study encloses multiple parameters, different optimization strategies can be constructed. The most straightforward approach is to estimate all parameters simultaneously from one optimal experiment (single OED/PE strategy). However, due to the complexity of the optimization problem or the stringent limitations on the system's dynamics, the experimental information can be limited and parameter estimation convergence problems can arise. As an alternative, we propose to reduce the optimization problem to a series of two-parameter estimation problems, i.e., an optimal experiment is designed for a combination of two parameters while presuming the other parameters known. Two different approaches can be followed: (i) all two-parameter optimal experiments are designed based on identical initial parameter estimates and parameters are estimated simultaneously from all resulting experimental data (global OED/PE strategy), and (ii) optimal experiments are calculated and implemented sequentially whereby the parameter values are updated intermediately (sequential OED/PE strategy).This work exploits OED/PE for the identification of the Cardinal Temperature Model with Inflection (CTMI) (Rosso et al., 1993). This kinetic model describes the effect of temperature on the microbial growth rate and encloses four parameters. The three OED/PE strategies are considered and the impact of the OED/PE design strategy on the accuracy of the CTMI parameter estimation is evaluated. Based on a simulation study, it is observed that the parameter values derived from the sequential approach deviate more from the true parameters than the single and global strategy estimates. The single and global OED/PE strategies are further compared based on experimental data obtained from design implementation in a bioreactor. Comparable estimates are obtained, but global OED/PE estimates are, in general, more accurate and reliable.     

Parameter estimation techniques for interaction and redistribution models: a predator-prey example

Summary The use of parameter estimation techniques for partial differential equations is illustrated using a predatorprey model. Whereas ecologists have often estimated parameters in models, they have not previously been able to do so for models that describe interactions in heterogeneous environments. The techniques we describe for partial differential equations will be generally useful for models of interacting species in spatially complex environments and for models that include the movement of organisms. We demonstrate our methods using field data from a ladybird beetle (Coccinella septempunctata) and aphid (Uroleucon nigrotuberculatum) interaction. Our parameter estimation algorithms can be employed to identify models that explain better than 80% of the observed variance in aphid and ladybird densities. Such parameter estimation techniques can bridge the gap between detail-rich experimental studies and abstract mathematical models. By relating the particular bestfit models identified from our experimental data to other information on Coccinella behavior, we conclude that a term describing local taxis of ladybirds towards prey (aphids in this case) is needed in the model.     

Efficient parameter estimation for spatio-temporal models of pattern formation: case study of Drosophila melanogaster

MOTIVATION: Diffusable and non-diffusable gene products play a major role in body plan formation. A quantitative understanding of the spatio-temporal patterns formed in body plan formation, by using simulation models is an important addition to experimental observation. The inverse modelling approach consists of describing the body plan formation by a rule-based model, and fitting the model parameters to real observed data. In body plan formation, the data are usually obtained from fluorescent immunohistochemistry or in situ hybridizations. Inferring model parameters by comparing such data to those from simulation is a major computational bottleneck. An important aspect in this process is the choice of method used for parameter estimation. When no information on parameters is available, parameter estimation is mostly done by means of heuristic algorithms. RESULTS: We show that parameter estimation for pattern formation models can be efficiently performed using an evolution strategy (ES). As a case study we use a quantitative spatio-temporal model of the regulatory network for early development in Drosophila melanogaster. In order to estimate the parameters, the simulated results are compared to a time series of gene products involved in the network obtained with immunohistochemistry. We demonstrate that a (mu,lambda)-ES can be used to find good quality solutions in the parameter estimation. We also show that an ES with multiple populations is 5-140 times as fast as parallel simulated annealing for this case study, and that combining ES with a local search results in an efficient parameter estimation method.     

Identification in an anaerobic batch system: global sensitivity analysis, multi-start strategy and optimization criterion selection

Several mathematical models have been developed in anaerobic digestion systems and a variety of methods have been used for parameter estimation and model validation. However, structural and parametric identifiability questions are relatively seldom addressed in the reported AD modeling studies. This paper presents a 3-step procedure for the reliable estimation of a set of kinetic and stoichiometric parameters in a simplified model of the anaerobic digestion process. This procedure includes the application of global sensitivity analysis, which allows to evaluate the interaction among the identified parameters, multi-start strategy that gives a picture of the possible local minima and the selection of optimization criteria or cost functions. This procedure is applied to the experimental data collected from a lab-scale sequencing batch reactor. Two kinetic parameters and two stoichiometric coefficients are estimated and their accuracy was also determined. The classical least-squares cost function appears to be the best choice in this case study.     

Least-squares estimation of batch culture kinetic parameters

This article concerns the development of a simple and effective least-squares procedure for estimating the kinetic parameters in Monod expressions from batch culture data. The basic approach employed in this work was to translate the problem of parameter estimation to a mathematical model containing a single decision variable. The resulting model was then solved by an efficient one-dimensional search algorithm which can be adapted to any microcomputer or advanced programmable calculator. The procedure was tested on synthetic data (substrate concentrations) with different types and levels of error. The effect of endogeneous respiration on the estimated values of the kinetic parameters was also assessed. From the results of these analyses the least-squares procedure developed was concluded to be very effective.     

Harvest-based Bayesian estimation of sika deer populations using state-space models

We have estimated the number of sika deer, Cervus nippon, in Hokkaido, Japan, with the aim of developing a management program that will reduce the level of agricultural damage caused by these deer. A population index that is defined by the population divided by the population of 1993 is first estimated from the data obtained during a spotlight survey. A generalized linear mixed model (GLMM) with corner point constraints is used in this estimation. We then estimate the population from the index by evaluating the response of index to the known amount of harvest, including hunting. A stage-structured model is used in this harvest-based estimation. It is well-known that estimates of indices suffer from large observation errors when the probability of the observation fluctuates widely; therefore, we apply state-space modeling to the harvest-based estimation to remove the observation errors. We propose the use of Bayesian estimation with uniform prior-distributions as an approximation of the maximum likelihood estimation, without permitting an arbitrary assumption that the parameters fluctuate following prior-distributions. We are able to demonstrate that the harvest-based Bayesian estimation is effective in reducing the observation errors in sika deer populations, but the stage-structured model requires many demographic parameters to be known prior to running the analyses. These parameters cannot be estimated from the observed time-series of the index if there is insufficient data. We then construct a univariate model by simplifying the stage-structured model and show that the simplified model yields estimates that are nearly identical to those obtained from the stage-structured model. This simplification of the model simultaneously clarifies which parameter is important in estimating the population. Electronic supplementary material The online version of this article (doi:) contains supplementary material, which is available to authorized users.