Parameter inversion estimation in photosynthetic models: Impact of different simulation methods

When we apply ecological models in environmental management, we must assess the accuracy of parameter estimation and its impact on model predictions. Parameters estimated by conventional techniques tend to be nonrobust and require excessive computational resources. However, optimization algorithms are highly robust and generally exhibit convergence of parameter estimation by inversion with nonlinear models. They can simultaneously generate a large number of parameter estimates using an entire data set. In this study, we tested four inversion algorithms (simulated annealing, shuffled complex evolution, particle swarm optimization, and the genetic algorithm) to optimize parameters in photosynthetic models depending on different temperatures. We investigated if parameter boundary values and control variables influenced the accuracy and efficiency of the various algorithms and models. We obtained optimal solutions with all of the inversion algorithms tested if the parameter bounds and control variables were constrained properly. However, the efficiency of processing time use varied with the control variables obtained. In addition, we investigated if temperature dependence formalization impacted optimally the parameter estimation process. We found that the model with a peaked temperature response provided the best fit to the data.     

Online nonlinear sequential Bayesian estimation of a biological wastewater treatment process

Online estimation of unknown state variables is a key component in the accurate modelling of biological wastewater treatment processes due to a lack of reliable online measurement systems. The extended Kalman filter (EKF) algorithm has been widely applied for wastewater treatment processes. However, the series approximations in the EKF algorithm are not valid, because biological wastewater treatment processes are highly nonlinear with a time-varying characteristic. This work proposes an alternative online estimation approach using the sequential Monte Carlo (SMC) methods for recursive online state estimation of a biological sequencing batch reactor for wastewater treatment. SMC is an algorithm that makes it possible to recursively construct the posterior probability density of the state variables, with respect to all available measurements, through a random exploration of the states by entities called ‘particle’. In this work, the simplified and modified Activated Sludge Model No. 3 with nonlinear biological kinetic models is used as a process model and formulated in a dynamic state-space model applied to the SMC method. The performance of the SMC method for online state estimation applied to a biological sequencing batch reactor with online and offline measured data is encouraging. The results indicate that the SMC method could emerge as a powerful tool for solving online state and parameter estimation problems without any model linearization or restrictive assumptions pertaining to the type of nonlinear models for biological wastewater treatment processes.     

Simultaneous versus sequential optimal experiment design for the identification of multi-parameter microbial growth kinetics as a function of temperature

Optimal experiment design for parameter estimation (OED/PE) has become a popular tool for efficient and accurate estimation of kinetic model parameters. When the kinetic model under study encloses multiple parameters, different optimization strategies can be constructed. The most straightforward approach is to estimate all parameters simultaneously from one optimal experiment (single OED/PE strategy). However, due to the complexity of the optimization problem or the stringent limitations on the system's dynamics, the experimental information can be limited and parameter estimation convergence problems can arise. As an alternative, we propose to reduce the optimization problem to a series of two-parameter estimation problems, i.e., an optimal experiment is designed for a combination of two parameters while presuming the other parameters known. Two different approaches can be followed: (i) all two-parameter optimal experiments are designed based on identical initial parameter estimates and parameters are estimated simultaneously from all resulting experimental data (global OED/PE strategy), and (ii) optimal experiments are calculated and implemented sequentially whereby the parameter values are updated intermediately (sequential OED/PE strategy).This work exploits OED/PE for the identification of the Cardinal Temperature Model with Inflection (CTMI) (Rosso et al., 1993). This kinetic model describes the effect of temperature on the microbial growth rate and encloses four parameters. The three OED/PE strategies are considered and the impact of the OED/PE design strategy on the accuracy of the CTMI parameter estimation is evaluated. Based on a simulation study, it is observed that the parameter values derived from the sequential approach deviate more from the true parameters than the single and global strategy estimates. The single and global OED/PE strategies are further compared based on experimental data obtained from design implementation in a bioreactor. Comparable estimates are obtained, but global OED/PE estimates are, in general, more accurate and reliable.     

Efficient Characterization of Parametric Uncertainty of Complex (Bio)chemical Networks

Parametric uncertainty is a particularly challenging and relevant aspect of systems analysis in domains such as systems biology where, both for inference and for assessing prediction uncertainties, it is essential to characterize the system behavior globally in the parameter space. However, current methods based on local approximations or on Monte-Carlo sampling cope only insufficiently with high-dimensional parameter spaces associated with complex network models. Here, we propose an alternative deterministic methodology that relies on sparse polynomial approximations. We propose a deterministic computational interpolation scheme which identifies most significant expansion coefficients adaptively. We present its performance in kinetic model equations from computational systems biology with several hundred parameters and state variables, leading to numerical approximations of the parametric solution on the entire parameter space. The scheme is based on adaptive Smolyak interpolation of the parametric solution at judiciously and adaptively chosen points in parameter space. As Monte-Carlo sampling, it is “non-intrusive” and well-suited for massively parallel implementation, but affords higher convergence rates. This opens up new avenues for large-scale dynamic network analysis by enabling scaling for many applications, including parameter estimation, uncertainty quantification, and systems design.     

Structured modeling and state estimation in a fermentation process: Lipase production by Candida rugosa

A simple structured mathematical model coupled with a methodology of state and parameter estimation is developed for lipase production by Candida rugosa in batch fermentation. The model describes the system according to the following qualitative observations and hypothesis: Lipase production is induced by extracellular oleic acid present in the medium. The acid is transported into the cell where it is consumed, transformed, and stored. Lipase is excreted to the medium where it is distributed between the available oil-water interphase and aqueous phase. Cell growth is modulated by the intracellular substrate concentration. Model parameters have been determined and the whole model validated against experiments not used in their determination. The estimation problem consists in the estimation of three state variables (biomass, intra- and extracellular substrate) and two kinetic parameters by using only the on-line measurement provided by exhaust gas analysis. The presented estimation strategy divides the complex problem into three subproblems that can be solved by stable algorithms. The estimation of biomass (X) and the specific growth rate (mu), is achieved by a recursive prediction error algorithm using the on-line measurement of the carbon dioxide evolution rate. mu is then used to perform an estimation of intracellular substrate and the other kinetic parameter related to substrate transport (A) by an adaptive observer. Extracellular substrate is then evaluated by means of the estimated values of intracellular substrate and biomass through the material balance of the reactor. Simulation and experimental tests showed good performance of the developed estimator, which appears suitable to be used for process control and monitoring. (c) 1995 John Wiley & Sons, Inc.     

Isotope-assisted metabolic flux analysis as an equality-constrained nonlinear program for improved scalability and robustness

Stable isotope-assisted metabolic flux analysis (MFA) is a powerful method to estimate carbon flow and partitioning in metabolic networks. At its core, MFA is a parameter estimation problem wherein the fluxes and metabolite pool sizes are model parameters that are estimated, via optimization, to account for measurements of steady-state or isotopically-nonstationary isotope labeling patterns. As MFA problems advance in scale, they require efficient computational methods for fast and robust convergence. The structure of the MFA problem enables it to be cast as an equality-constrained nonlinear program (NLP), where the equality constraints are constructed from the MFA model equations, and the objective function is defined as the sum of squared residuals (SSR) between the model predictions and a set of labeling measurements. This NLP can be solved by using an algebraic modeling language (AML) that offers state-of-the-art optimization solvers for robust parameter estimation and superior scalability to large networks. When implemented in this manner, the optimization is performed with no distinction between state variables and model parameters. During each iteration of such an optimization, the system state is updated instead of being calculated explicitly from scratch, and this occurs concurrently with improvement in the model parameter estimates. This optimization approach starkly contrasts with traditional “shooting” methods where the state variables and model parameters are kept distinct and the system state is computed afresh during each iteration of a stepwise optimization. Our NLP formulation uses the MFA modeling framework of Wiechert et al. [1], which is amenable to incorporation of the model equations into an NLP. The NLP constraints consist of balances on either elementary metabolite units (EMUs) or cumomers. In this formulation, both the steady-state and isotopically-nonstationary MFA (inst-MFA) problems may be solved as an NLP. For the inst-MFA case, the ordinary differential equation (ODE) system describing the labeling dynamics is transcribed into a system of algebraic constraints for the NLP using collocation. This large-scale NLP may be solved efficiently using an NLP solver implemented on an AML. In our implementation, we used the reduced gradient solver CONOPT, implemented in the General Algebraic Modeling System (GAMS). The NLP framework is particularly advantageous for inst-MFA, scaling well to large networks with many free parameters, and having more robust convergence properties compared to the shooting methods that compute the system state and sensitivities at each iteration. Additionally, this NLP approach supports the use of tandem-MS data for both steady-state and inst-MFA when the cumomer framework is used. We assembled a software, eiFlux, written in Python and GAMS that uses the NLP approach and supports both steady-state and inst-MFA. We demonstrate the effectiveness of the NLP formulation on several examples, including a genome-scale inst-MFA model, to highlight the scalability and robustness of this approach. In addition to typical inst-MFA applications, we expect that this framework and our associated software, eiFlux, will be particularly useful for applying inst-MFA to complex MFA models, such as those developed for eukaryotes (e.g. algae) and co-cultures with multiple cell types.     

Fed-batch optimization of alpha-amylase and protease-producing Bacillus subtilis using Markov chain methods

A stoichiometry-based model for the fed-batch culture of the recombinant bacterium Bacillus subtilis ATCC 6051a, producing extracellular alpha-amylase as a desirable product and proteases as undesirable products, was developed and verified. The model was then used for optimizing the feeding schedule in fed-batch culture. To handle higher-order model equations (14 state variables), an optimization methodology for the dual-enzyme system is proposed by integrating Pontryagin's optimum principle with fermentation measurements. Markov chain Monte Carlo (MCMC) procedures were appropriate for model parameter and decision variable estimation by using a priori parameter distributions reflecting the experimental results. Using a simplified Metropolis-Hastings algorithm, the specific productivity of alpha-amylase was maximized and the optimum path was confirmed by experimentation. The optimization process predicted a further 14% improvement of alpha-amylase productivity that could not be realized because of the onset of sporulation. Among the decision variables, the switching time from batch to fed-batch operation (t(s)) was the most sensitive decision variable.     

RankAggreg, an R package for weighted rank aggregation

Background

Determining the parameters of a mathematical model from quantitative measurements is the main bottleneck of modelling biological systems. Parameter values can be estimated from steady-state data or from dynamic data. The nature of suitable data for these two types of estimation is rather different. For instance, estimations of parameter values in pathway models, such as kinetic orders, rate constants, flux control coefficients or elasticities, from steady-state data are generally based on experiments that measure how a biochemical system responds to small perturbations around the steady state. In contrast, parameter estimation from dynamic data requires time series measurements for all dependent variables. Almost no literature has so far discussed the combined use of both steady-state and transient data for estimating parameter values of biochemical systems.

Results

In this study we introduce a constrained optimization method for estimating parameter values of biochemical pathway models using steady-state information and transient measurements. The constraints are derived from the flux connectivity relationships of the system at the steady state. Two case studies demonstrate the estimation results with and without flux connectivity constraints. The unconstrained optimal estimates from dynamic data may fit the experiments well, but they do not necessarily maintain the connectivity relationships. As a consequence, individual fluxes may be misrepresented, which may cause problems in later extrapolations. By contrast, the constrained estimation accounting for flux connectivity information reduces this misrepresentation and thereby yields improved model parameters.

Conclusion

The method combines transient metabolic profiles and steady-state information and leads to the formulation of an inverse parameter estimation task as a constrained optimization problem. Parameter estimation and model selection are simultaneously carried out on the constrained optimization problem and yield realistic model parameters that are more likely to hold up in extrapolations with the model.     

Parameter estimation of a monod-type model. Part I: Theoretical identifiability and sensitivity analysis

Summary Parameter estimation of a Monod-type model based on the study of the theoretical identifiability of the model followed by the sensitivity analysis of the state variables with respect to parameters is presented. Theorerical identifiability allows to establish the unicity of the solution. On the other hand, sensitivity analysis throws light on the conditions that make parameters identifiable. Thus, the introduction of additional parameters, especially substrate maintenance and death constant, increases the estimation difficulty.     

Estimating the hidden learning representations

Successful adaptation relies on the ability to learn the consequence of our actions in different environments. However, understanding the neural bases of this ability still represents one of the great challenges of system neuroscience. In fact, the neuronal plasticity changes occurring during learning cannot be fully controlled experimentally and their evolution is hidden. Our approach is to provide hypotheses about the structure and dynamics of the hidden plasticity changes using behavioral learning theory. In fact, behavioral models of animal learning provide testable predictions about the hidden learning representations by formalizing their relation with the observables of the experiment (stimuli, actions and outcomes). Thus, we can understand whether and how the predicted learning processes are represented at the neural level by estimating their evolution and correlating them with neural data. Here, we present a bayesian model approach to estimate the evolution of the internal learning representations from the observations of the experiment (state estimation), and to identify the set of models' parameters (parameter estimation) and the class of behavioral model (model selection) that are most likely to have generated a given sequence of actions and outcomes. More precisely, we use Sequential Monte Carlo methods for state estimation and the maximum likelihood principle (MLP) for model selection and parameter estimation. We show that the method recovers simulated trajectories of learning sessions on a single-trial basis and provides predictions about the activity of different categories of neurons that should participate in the learning process. By correlating the estimated evolutions of the learning variables, we will be able to test the validity of different models of instrumental learning and possibly identify the neural bases of learning.     

Convergence of EU nitrogen surplus,the RDP indicator of water quality

The EU Water Framework Directive (WFD), EU Nitrate Directive and EU Rural Development Policy (RDP) aim to improve water quality. The nutrient content of water can be decreased by reducing nitrogen emission. In this article a novel approach is applied to the evaluation of the impact of Agri Environmental Measures (AEM), which are part of axis 2 of the EU Rural Development Programme. The spending on AEM is linked to the reduction of nitrogen surplus, and hence, to the improvement of water quality. Reduction of nitrogen surplus is considered as a beta convergence process, in which the nitrogen surplus of EU member states converges to a steady state level. The convergence is tested, applying spatial econometrics on a panel data set of EU member states. The development over time of nitrogen surplus is explained applying the conditional beta convergence methodology. To allow for varying steady state nitrogen surpluses, structural variables are added to the analysis. RDP spending on AEM was added as structural variable to evaluate whether they affect the reduction of nitrogen surplus. The fixed effects panel data specification was tested to be the best model and preferred over spatial econometric specifications. A significantly negative effect is found between AEM expenditures and nitrogen surplus. Based on these estimation results it can be concluded that spending on AEM affects the convergence of nitrogen surplus towards a steady state level. A causal relationship cannot be tested with data on EU Member State level and additional analysis at smaller spatial level is warranted.     

A Two-Stage Algorithm for Origin-Destination Matrices Estimation Considering Dynamic Dispersion Parameter for Route Choice

This paper proposes a two-stage algorithm to simultaneously estimate origin-destination (OD) matrix, link choice proportion, and dispersion parameter using partial traffic counts in a congested network. A non-linear optimization model is developed which incorporates a dynamic dispersion parameter, followed by a two-stage algorithm in which Generalized Least Squares (GLS) estimation and a Stochastic User Equilibrium (SUE) assignment model are iteratively applied until the convergence is reached. To evaluate the performance of the algorithm, the proposed approach is implemented in a hypothetical network using input data with high error, and tested under a range of variation coefficients. The root mean squared error (RMSE) of the estimated OD demand and link flows are used to evaluate the model estimation results. The results indicate that the estimated dispersion parameter theta is insensitive to the choice of variation coefficients. The proposed approach is shown to outperform two established OD estimation methods and produce parameter estimates that are close to the ground truth. In addition, the proposed approach is applied to an empirical network in Seattle, WA to validate the robustness and practicality of this methodology. In summary, this study proposes and evaluates an innovative computational approach to accurately estimate OD matrices using link-level traffic flow data, and provides useful insight for optimal parameter selection in modeling travelers’ route choice behavior.     

Reconstructing Mammalian Sleep Dynamics with Data Assimilation

Data assimilation is a valuable tool in the study of any complex system, where measurements are incomplete, uncertain, or both. It enables the user to take advantage of all available information including experimental measurements and short-term model forecasts of a system. Although data assimilation has been used to study other biological systems, the study of the sleep-wake regulatory network has yet to benefit from this toolset. We present a data assimilation framework based on the unscented Kalman filter (UKF) for combining sparse measurements together with a relatively high-dimensional nonlinear computational model to estimate the state of a model of the sleep-wake regulatory system. We demonstrate with simulation studies that a few noisy variables can be used to accurately reconstruct the remaining hidden variables. We introduce a metric for ranking relative partial observability of computational models, within the UKF framework, that allows us to choose the optimal variables for measurement and also provides a methodology for optimizing framework parameters such as UKF covariance inflation. In addition, we demonstrate a parameter estimation method that allows us to track non-stationary model parameters and accommodate slow dynamics not included in the UKF filter model. Finally, we show that we can even use observed discretized sleep-state, which is not one of the model variables, to reconstruct model state and estimate unknown parameters. Sleep is implicated in many neurological disorders from epilepsy to schizophrenia, but simultaneous observation of the many brain components that regulate this behavior is difficult. We anticipate that this data assimilation framework will enable better understanding of the detailed interactions governing sleep and wake behavior and provide for better, more targeted, therapies.     

On the stock estimation for some fishery systems

In this work we address the stock estimation problem for two fishery models. We show that a tool from nonlinear control theory called “observer” can be helpful to deal with the resource stock estimation in the field of renewable resource management. It is often difficult or expensive to measure all the state variables characterising the evolution of a given population system, therefore the question arises whether from the observation of certain indicators of the considered system, the whole state of the population system can be recovered or at least estimated. The goal of this paper is to show how some techniques of control theory can be applied for the approximate estimation of the unmeasurable state variables using only the observed data together with the dynamical model describing the evolution of the system. More precisely we shall consider two fishery models and we shall show how to built for each model an auxiliary dynamical system (the observer) that uses the available data (the total of caught fish) and which produces a dynamical estimation of the unmeasurable stock state x(t). Moreover the convergence speed of towards x(t) can be chosen.     

A hybrid asymptotic-Kalman observer for bioprocesses

The exponential observers (extended Kalman or Luenberger observers, high gain observers) allow the use of a tuning parameter for managing the rate of convergence of the state estimate towards the true state. But their results are strongly dependent on the model quality (especially the kinetic model in bioprocesses). On the other hand, asymptotic observers (like the observer of Bastin and Dochain) have a rate of convergence which is a function of the experimental conditions (namely the dilution rate). However, this lack of tuning parameter is compensated by the absence of need for any kinetic model. In this paper, a hybrid technique is proposed which allows to jointly estimate the state and identify on-line the confidence on the kinetic model. The two limit cases (100 and 0 confidence) allow to recover rigorously the extended Kalman filter and the asymptotic observer of Bastin and Dochain. A simulation example (a fed-batch bacterial culture) is proposed and exhibits very satisfactory results.     

Asymptotic behavior of the scaled mutation rate estimators

Important aspects of population evolution have been investigated using nucleotide sequences. Under the neutral Wright–Fisher model, the scaled mutation rate represents twice the average number of new mutations per generations and it is one of the key parameters in population genetics. In this study, we present various methods of estimation of this parameter, analytical studies of their asymptotic behavior as well as comparisons of the distribution's behavior of these estimators through simulations. As knowledge of the genealogy is needed to estimate the maximum likelihood estimator (MLE), an application with real data is also presented, using jackknife to correct the bias of the MLE, which can be generated by the estimation of the tree. We proved analytically that the Waterson's estimator and the MLE are asymptotically equivalent with the same rate of convergence to normality. Furthermore, we showed that the MLE has a better rate of convergence than Waterson's estimator for values of the parameter greater than one and this relationship is reversed when the parameter is less than one.     

On parameter estimation for biaxial mechanical behavior of arteries

This article considers the parameter estimation of multi-fiber family models for biaxial mechanical behavior of passive arteries in the presence of the measurement errors. First, the uncertainty propagation due to the errors in variables has been carefully characterized using the constitutive model. Then, the parameter estimation of the artery model has been formulated into nonlinear least squares optimization with an appropriately chosen weight from the uncertainty model. The proposed technique is evaluated using multiple sets of synthesized data with fictitious measurement noises. The results of the estimation are compared with those of the conventional nonlinear least squares optimization without a proper weight factor. The proposed method significantly improves the quality of parameter estimation as the amplitude of the errors in variables becomes larger. We also investigate model selection criteria to decide the optimal number of fiber families in the multi-fiber family model with respect to the experimental data balancing between variance and bias errors.     

Uniform step-by-step observer for aerobic bioreactor based on super-twisting algorithm

This paper describes a fixed-time convergent step-by-step high order sliding mode observer for a certain type of aerobic bioreactor system. The observer was developed using a hierarchical structure based on a modified super-twisting algorithm. The modification included nonlinear gains of the output error that were used to prove uniform convergence of the estimation error. An energetic function similar to a Lyapunov one was used for proving the convergence between the observer and the bioreactor variables. A nonsmooth analysis was proposed to prove the fixed-time convergence of the observer states to the bioreactor variables. The observer was tested to solve the state estimation problem of an aerobic bioreactor described by the time evolution of biomass, substrate and dissolved oxygen. This last variable was used as the output information because it is feasible to measure it online by regular sensors. Numerical simulations showed the superior behavior of this observer compared to the one having linear output error injection terms (high-gain type) and one having an output injection obtaining first-order sliding mode structure. A set of numerical simulations was developed to demonstrate how the proposed observer served to estimate real information obtained from a real aerobic process with substrate inhibition.     

The equilibrium assumption in estimating the parameters of metapopulation models

1.  The construction of a predictive metapopulation model includes three steps: the choice of factors affecting metapopulation dynamics, the choice of model structure, and finally parameter estimation and model testing.
2.  Unless the assumption is made that the metapopulation is at stochastic quasi-equilibrium and unless the method of parameter estimation of model parameters uses that assumption, estimates from a limited amount of data will usually predict a trend in metapopulation size.
3.  This implicit estimation of a trend occurs because extinction-colonization stochasticity, possibly amplified by regional stochasticity, leads to unequal numbers of observed extinction and colonization events during a short study period.
4.  Metapopulation models, such as those based on the logistic regression model, that rely on observed population turnover events in parameter estimation are sensitive to the implicit estimation of a trend.
5.  A new parameter estimation method, based on Monte Carlo inference for statistically implicit models, allows an explicit decision about whether metapopulation quasi-stability is assumed or not.
6. Our confidence in metapopulation model parameter estimates that have been produced from only a few years of data is decreased by the need to know before parameter estimation whether the metapopulation is in quasi-stable state or not.
7. The choice of whether metapopulation stability is assumed or not in parameter estimation should be done consciously. Typical data sets cover only a few years and rarely allow a statistical test of a possible trend. While making the decision about stability one should consider any information about the landscape history and species and metapopulation characteristics.     

A tunable multivariable nonlinear robust observer for biological systems

This paper presents a robust nonlinear asymptotic observer with adjustable convergence rate with a great potential of applicability for biological systems in which the main state variables are difficult and expensive to measure or such measurements do not exist. This observer scheme is based on the classical asymptotic observer, which is modified to allow the tuning of the convergence rate. It is shown that the proposed observer provides fast and satisfactory estimates when facing load disturbances, system failures and parameter uncertainty while maintaining the excellent robustness and stability properties of the classical asymptotic observer. The implementation of the tunable observer is carried out by numerical simulations of a mathematical model of an anaerobic digestion process used for wastewater treatment. The key results are examined and further developed.