Bayesian calibration,process modeling and uncertainty quantification in biotechnology

High-throughput experimentation has revolutionized data-driven experimental sciences and opened the door to the application of machine learning techniques. Nevertheless, the quality of any data analysis strongly depends on the quality of the data and specifically the degree to which random effects in the experimental data-generating process are quantified and accounted for. Accordingly calibration, i.e. the quantitative association between observed quantities and measurement responses, is a core element of many workflows in experimental sciences.Particularly in life sciences, univariate calibration, often involving non-linear saturation effects, must be performed to extract quantitative information from measured data. At the same time, the estimation of uncertainty is inseparably connected to quantitative experimentation. Adequate calibration models that describe not only the input/output relationship in a measurement system but also its inherent measurement noise are required. Due to its mathematical nature, statistically robust calibration modeling remains a challenge for many practitioners, at the same time being extremely beneficial for machine learning applications.In this work, we present a bottom-up conceptual and computational approach that solves many problems of understanding and implementing non-linear, empirical calibration modeling for quantification of analytes and process modeling. The methodology is first applied to the optical measurement of biomass concentrations in a high-throughput cultivation system, then to the quantification of glucose by an automated enzymatic assay. We implemented the conceptual framework in two Python packages, calibr8 and murefi, with which we demonstrate how to make uncertainty quantification for various calibration tasks more accessible. Our software packages enable more reproducible and automatable data analysis routines compared to commonly observed workflows in life sciences.Subsequently, we combine the previously established calibration models with a hierarchical Monod-like ordinary differential equation model of microbial growth to describe multiple replicates of Corynebacterium glutamicum batch cultures. Key process model parameters are learned by both maximum likelihood estimation and Bayesian inference, highlighting the flexibility of the statistical and computational framework.     

Bayesian calibration of a stochastic,multiscale agent-based model for predicting in vitro tumor growth

Hybrid multiscale agent-based models (ABMs) are unique in their ability to simulate individual cell interactions and microenvironmental dynamics. Unfortunately, the high computational cost of modeling individual cells, the inherent stochasticity of cell dynamics, and numerous model parameters are fundamental limitations of applying such models to predict tumor dynamics. To overcome these challenges, we have developed a coarse-grained two-scale ABM (cgABM) with a reduced parameter space that allows for an accurate and efficient calibration using a set of time-resolved microscopy measurements of cancer cells grown with different initial conditions. The multiscale model consists of a reaction-diffusion type model capturing the spatio-temporal evolution of glucose and growth factors in the tumor microenvironment (at tissue scale), coupled with a lattice-free ABM to simulate individual cell dynamics (at cellular scale). The experimental data consists of BT474 human breast carcinoma cells initialized with different glucose concentrations and tumor cell confluences. The confluence of live and dead cells was measured every three hours over four days. Given this model, we perform a time-dependent global sensitivity analysis to identify the relative importance of the model parameters. The subsequent cgABM is calibrated within a Bayesian framework to the experimental data to estimate model parameters, which are then used to predict the temporal evolution of the living and dead cell populations. To this end, a moment-based Bayesian inference is proposed to account for the stochasticity of the cgABM while quantifying uncertainties due to limited temporal observational data. The cgABM reduces the computational time of ABM simulations by 93% to 97% while staying within a 3% difference in prediction compared to ABM. Additionally, the cgABM can reliably predict the temporal evolution of breast cancer cells observed by the microscopy data with an average error and standard deviation for live and dead cells being 7.61±2.01 and 5.78±1.13, respectively.     

Cartogramming uncertainty in species distribution models: A Bayesian approach

Predicting the geographical distribution of a species is a central topic in ecology, conservation and management of natural resources especially for invasive organisms. Invasive species can modify the structure and function of invaded ecosystems, altering their biodiversity, and causing significant economic losses locally and globally. Therefore, measuring and visualizing the uncertainty inherent in species’ potential distributions is fundamental for effective biodiversity monitoring and planning conservation interventions. This paper discusses a new Bayesian approach to mapping this uncertainty using cartograms, previously published knowledge, and presence/absence data.     

Using Bayesian networks to incorporate uncertainty in habitat suitability index models

Habitat suitability index (HSI) models rarely characterize the uncertainty associated with their estimates of habitat quality despite the fact that uncertainty can have important management implications. The purpose of this paper was to explore the use of Bayesian belief networks (BBNs) for representing and propagating 3 types of uncertainty in HSI models—uncertainty in the suitability index relationships, the parameters of the HSI equation, and measurement of habitat variables (i.e., model inputs). I constructed a BBN–HSI model, based on an existing HSI model, using Netica™ software. I parameterized the BBN's conditional probability tables via Monte Carlo methods, and developed a discretization scheme that met specifications for numerical error. I applied the model to both real and dummy sites in order to demonstrate the utility of the BBN–HSI model for 1) determining whether sites with different habitat types had statistically significant differences in HSI, and 2) making decisions based on rules that reflect different attitudes toward risk—maximum expected value, maximin, and maximax. I also examined effects of uncertainty in the habitat variables on the model's output. Some sites with different habitat types had different values for E[HSI], the expected value of HSI, but habitat suitability was not significantly different based on the overlap of 90% confidence intervals for E[HSI]. The different decision rules resulted in different rankings of sites, and hence, different decisions based on risk. As measurement uncertainty in habitat variables increased, sites with significantly different (α = 0.1) E[HSI] became statistically more similar. Incorporating uncertainty in HSI models enables explicit consideration of risk and more robust habitat management decisions. © 2012 The Wildlife Society.     

Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings

Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating.     

Selection of likelihood parameters for complex models determines the effectiveness of Bayesian calibration

Assessing the parameter uncertainty of complex ecosystem models is a key challenge for improving our understanding of real world abstractions, such as those for explaining carbon and nitrogen cycle at ecosystem scale and associated biosphere-atmosphere-hydrosphere exchange processes. The lack of data about the variance of measurements forces scientists to revisit assumptions used in estimating the parameter distribution of complex ecosystem models.An increasingly used tool for assessing parameter uncertainty of complex ecosystem models is Bayesian calibration. In this paper, we generate two data sets which may represent a seasonal temperature curve or the seasonality of soil carbon dioxide flux and a single high peak put on a low background signal as is e.g. typical for soil nitrous oxide emission. Based on these examples we illustrate that commonly used assumptions for measurement uncertainty can lead to a sampling of wrong areas in the parameter space, incorrect parameter dependencies, and an underestimation of parameter uncertainties. This step needs particular attention by modelers as these issues lead to erroneous model simulations a) in present and future domains, b) misinterpretations of process feedback and functioning of the model, and c) to an underestimation of model uncertainty (e.g. for soil greenhouse gas fluxes).We also test the extension of the Bayesian framework with a model error term to compensate the effects caused by the false assumption of a perfect model and show that this approach can alleviate the observed problems in estimating the model parameter distribution.     

Data-driven flow-map models for data-efficient discovery of dynamics and fast uncertainty quantification of biological and biochemical systems

Computational models are increasingly used to investigate and predict the complex dynamics of biological and biochemical systems. Nevertheless, governing equations of a biochemical system may not be (fully) known, which would necessitate learning the system dynamics directly from, often limited and noisy, observed data. On the other hand, when expensive models are available, systematic and efficient quantification of the effects of model uncertainties on quantities of interest can be an arduous task. This paper leverages the notion of flow-map (de)compositions to present a framework that can address both of these challenges via learning data-driven models useful for capturing the dynamical behavior of biochemical systems. Data-driven flow-map models seek to directly learn the integration operators of the governing differential equations in a black-box manner, irrespective of structure of the underlying equations. As such, they can serve as a flexible approach for deriving fast-to-evaluate surrogates for expensive computational models of system dynamics, or, alternatively, for reconstructing the long-term system dynamics via experimental observations. We present a data-efficient approach to data-driven flow-map modeling based on polynomial chaos Kriging. The approach is demonstrated for discovery of the dynamics of various benchmark systems and a coculture bioreactor subject to external forcing, as well as for uncertainty quantification of a microbial electrosynthesis reactor. Such data-driven models and analyses of dynamical systems can be paramount in the design and optimization of bioprocesses and integrated biomanufacturing systems.     

Visualizing uncertainty in habitat suitability models with the hyper‐envelope modeling interface,version 2

Habitat suitability models (HSMs) are popular and used for a wide variety of applications but most do not include analysis of the uncertainty of the model outputs. Additionally, some overfit the data and few allow the ability to fill data gaps with expert opinion. HEMI 1 addressed issues with overfitting data and allowed models to incorporate both occurrence data and expert opinion. HEMI 2 improves on HEMI 1 with a simplified interface and the ability to inject random noise into occurrence locations and environmental variable values to generate uncertainty maps. HEMI 2 uses Monte Carlo methods to perform uncertainty, validation, and sensitivity testing and generates mean and standard deviation habitat suitability maps.     

Model calibration and uncertainty analysis in signaling networks

1. Download : Download high-res image (90KB)
2. Download : Download full-size image

     

Correcting for measurement error in fractional polynomial models using Bayesian modelling and regression calibration,with an application to alcohol and mortality

Exposure measurement error can result in a biased estimate of the association between an exposure and outcome. When the exposure–outcome relationship is linear on the appropriate scale (e.g. linear, logistic) and the measurement error is classical, that is the result of random noise, the result is attenuation of the effect. When the relationship is non‐linear, measurement error distorts the true shape of the association. Regression calibration is a commonly used method for correcting for measurement error, in which each individual's unknown true exposure in the outcome regression model is replaced by its expectation conditional on the error‐prone measure and any fully measured covariates. Regression calibration is simple to execute when the exposure is untransformed in the linear predictor of the outcome regression model, but less straightforward when non‐linear transformations of the exposure are used. We describe a method for applying regression calibration in models in which a non‐linear association is modelled by transforming the exposure using a fractional polynomial model. It is shown that taking a Bayesian estimation approach is advantageous. By use of Markov chain Monte Carlo algorithms, one can sample from the distribution of the true exposure for each individual. Transformations of the sampled values can then be performed directly and used to find the expectation of the transformed exposure required for regression calibration. A simulation study shows that the proposed approach performs well. We apply the method to investigate the relationship between usual alcohol intake and subsequent all‐cause mortality using an error model that adjusts for the episodic nature of alcohol consumption.     

江西千烟洲地区散射辐射的模拟与验证

散射辐射的准确估算对于评价其对陆地生态系统碳交换的影响具有重要意义.基于我国中亚热带江西千烟洲气象观测场2012年3月1日—2013年2月28日散射辐射实际观测数据对目前常用的5个散射辐射分解模型（Reindl-1、Reindl-2、Reindl-3、Boland、BRL）的模拟结果进行验证.结果表明： 在30 min尺度上,虽然5个模型在总体上都可以较好地模拟该地区的散射辐射,但模型模拟效果随着晴空指数（k_t）的升高而显著降低.特别是当k_t>0.75时,各模型已无法模拟该地区散射辐射.从散射辐射季节变化的模拟来看,5个模型能够很好地模拟大多数月份的散射辐射.5个模型年尺度散射辐射模拟值与观测值的相对偏差最高为7.1%（BRL）,最低为0.04%（Reindl-1）,平均为3.6%.在全年辐射最强、温度最高和降水偏少的夏季,5个模型的模拟值均出现了过高估计.以7月为例,散射辐射被高估14.5%～28.2%,平均高估21.2%.这可能与高k_t条件下散射辐射的估算方法有关,这种不确定性需要在模型应用中做进一步深入评价.根据验证结果并考虑模拟精度和输入变量的易获取性,5个模型的模拟效果依次为BRL>Reindl-3>Reindl-2>Reindl-1>Boland.     

Age and growth of the common carp, Cyprinus carpio, in the River Murray, Australia: validation, consistency of age interpretation, and growth models

The present study validates age estimates from a suite of calcified structures (scales, opercular bones and otoliths), assesses the consistency of age interpretations and evaluates growth models in common carp from the lower Murray River, Australia. Marginal increment analysis was used to validate annulus counts, with attention to the 'edge interpretation problem'. The formation of annuli occurred annually after pooling annulus groups, and in carp 4 and 5 years old. The regular alternation of opaque and translucent zones in opercular bones and whole otoliths of younger and older carp was also suggestive of annual periodicity. From systematic comparisons, the use of both opercular bones and whole otoliths in routine age determinations is recommended. Six growth models, including the von Bertalanffy growth function (VBGF) and five polynomial curves were tested to describe growth in length. A log-log quadratic function, by virtue of its precision, and the VBGF, with wider applicability and more biological realism, were chosen.     

Simultaneous in vivo optical quantification of key metabolic and vascular endpoints reveals tumor metabolic diversity in murine breast tumor models

Therapeutically exploiting vascular and metabolic endpoints becomes critical to translational cancer studies because altered vascularity and deregulated metabolism are two important cancer hallmarks. The metabolic and vascular phenotypes of three sibling breast tumor lines with different metastatic potential are investigated in vivo with a newly developed quantitative spectroscopy system. All tumor lines have different metabolic and vascular characteristics compared to normal tissues, and there are strong positive correlations between metabolic (glucose uptake and mitochondrial membrane potential) and vascular (oxygen saturations and hemoglobin concentrations) parameters for metastatic (4T1) tumors but not for micrometastatic (4T07) and nonmetastatic (67NR) tumors. A longitudinal study shows that both vascular and metabolic endpoints of 4T1 tumors increased up to a specific tumor size threshold beyond which these parameters decreased. The synchronous changes between metabolic and vascular parameters, along with the strong positive correlations between these endpoints suggest that 4T1 tumors rely on strong oxidative phosphorylation in addition to glycolysis. This study illustrates the great potential of our optical technique to provide valuable dynamic information about the interplay between the metabolic and vascular status of tumors, with important implications for translational cancer investigations.      

植被界面过程(VIP)模型的改进与验证

为了提高植被界面过程(VIP)模型的预报能力,对VIP模型的一些参数化方案进行了更新,包括基于相对生育期的根深动态、根系分布密度和比叶面积季节变化,使模型能从机理上更合理地描述作物同化物分配及土壤水分运动。将改进后的模型应用于河北栾城冬小麦生长季的叶面积指数、生物量和土壤水分模拟,并与相应时期的试验资料对比验证。结果表明,改进后的VIP模型对土壤水分动态和冬小麦叶面积指数的模拟效果更好,模型各状态变量模拟值与观测值的均方根误差和相关系数都得到明显提高。     

Development,characterization, and validation of a sensitive primate-specific quantification assay for forensic analysis

Accurate human-specific DNA quantification is essential for forensic casework analysis. In this work, we describe a microplate-based quantification assay that utilizes the PCR amplification of human-specific TH01 primers. This method enables the reliable quantification of human DNA samples from 0.2 to 40 ng, even in mixtures with nonhuman DNA. Analysis of samples can be semi-automated using 96-well microplates and a spreadsheet-based concentration calculator for high-throughput demands. We have used this quantification method with more than 15,000 forensic samples.