Bayesian inference for stochastic kinetic models using a diffusion approximation

This article is concerned with the Bayesian estimation of stochastic rate constants in the context of dynamic models of intracellular processes. The underlying discrete stochastic kinetic model is replaced by a diffusion approximation (or stochastic differential equation approach) where a white noise term models stochastic behavior and the model is identified using equispaced time course data. The estimation framework involves the introduction of m- 1 latent data points between every pair of observations. MCMC methods are then used to sample the posterior distribution of the latent process and the model parameters. The methodology is applied to the estimation of parameters in a prokaryotic autoregulatory gene network.     

Inference for discretely observed stochastic kinetic networks with applications to epidemic modeling

We present a new method for Bayesian Markov Chain Monte Carlo-based inference in certain types of stochastic models, suitable for modeling noisy epidemic data. We apply the so-called uniformization representation of a Markov process, in order to efficiently generate appropriate conditional distributions in the Gibbs sampler algorithm. The approach is shown to work well in various data-poor settings, that is, when only partial information about the epidemic process is available, as illustrated on the synthetic data from SIR-type epidemics and the Center for Disease Control and Prevention data from the onset of the H1N1 pandemic in the United States.     

Bayesian sequential inference for stochastic kinetic biochemical network models.

As postgenomic biology becomes more predictive, the ability to infer rate parameters of genetic and biochemical networks will become increasingly important. In this paper, we explore the Bayesian estimation of stochastic kinetic rate constants governing dynamic models of intracellular processes. The underlying model is replaced by a diffusion approximation where a noise term represents intrinsic stochastic behavior and the model is identified using discrete-time (and often incomplete) data that is subject to measurement error. Sequential MCMC methods are then used to sample the model parameters on-line in several data-poor contexts. The methodology is illustrated by applying it to the estimation of parameters in a simple prokaryotic auto-regulatory gene network.     

Moment fitting for parameter inference in repeatedly and partially observed stochastic biological models

The inference of reaction rate parameters in biochemical network models from time series concentration data is a central task in computational systems biology. Under the assumption of well mixed conditions the network dynamics are typically described by the chemical master equation, the Fokker Planck equation, the linear noise approximation or the macroscopic rate equation. The inverse problem of estimating the parameters of the underlying network model can be approached in deterministic and stochastic ways, and available methods often compare individual or mean concentration traces obtained from experiments with theoretical model predictions when maximizing likelihoods, minimizing regularized least squares functionals, approximating posterior distributions or sequentially processing the data. In this article we assume that the biological reaction network can be observed at least partially and repeatedly over time such that sample moments of species molecule numbers for various time points can be calculated from the data. Based on the chemical master equation we furthermore derive closed systems of parameter dependent nonlinear ordinary differential equations that predict the time evolution of the statistical moments. For inferring the reaction rate parameters we suggest to not only compare the sample mean with the theoretical mean prediction but also to take the residual of higher order moments explicitly into account. Cost functions that involve residuals of higher order moments may form landscapes in the parameter space that have more pronounced curvatures at the minimizer and hence may weaken or even overcome parameter sloppiness and uncertainty. As a consequence both deterministic and stochastic parameter inference algorithms may be improved with respect to accuracy and efficiency. We demonstrate the potential of moment fitting for parameter inference by means of illustrative stochastic biological models from the literature and address topics for future research.     

Parameter inference for stochastic biochemical models from perturbation experiments parallelised at the single cell level

Understanding and characterising biochemical processes inside single cells requires experimental platforms that allow one to perturb and observe the dynamics of such processes as well as computational methods to build and parameterise models from the collected data. Recent progress with experimental platforms and optogenetics has made it possible to expose each cell in an experiment to an individualised input and automatically record cellular responses over days with fine time resolution. However, methods to infer parameters of stochastic kinetic models from single-cell longitudinal data have generally been developed under the assumption that experimental data is sparse and that responses of cells to at most a few different input perturbations can be observed. Here, we investigate and compare different approaches for calculating parameter likelihoods of single-cell longitudinal data based on approximations of the chemical master equation (CME) with a particular focus on coupling the linear noise approximation (LNA) or moment closure methods to a Kalman filter. We show that, as long as cells are measured sufficiently frequently, coupling the LNA to a Kalman filter allows one to accurately approximate likelihoods and to infer model parameters from data even in cases where the LNA provides poor approximations of the CME. Furthermore, the computational cost of filtering-based iterative likelihood evaluation scales advantageously in the number of measurement times and different input perturbations and is thus ideally suited for data obtained from modern experimental platforms. To demonstrate the practical usefulness of these results, we perform an experiment in which single cells, equipped with an optogenetic gene expression system, are exposed to various different light-input sequences and measured at several hundred time points and use parameter inference based on iterative likelihood evaluation to parameterise a stochastic model of the system.     

Hyperparameter-tuned batch-updated stochastic gradient descent: Plant species identification by using hybrid deep learning

The accurate identification of plant species is crucial for the conservation of biodiversity. However, traditional methods for identifying plant species are often complicated, time-consuming, and prone to errors. Therefore, it is essential to address these challenges and develop automated identification methods to enhance the efficiency and accuracy of plant species identification. In this study, a step-by-step method was utilized to identify and classify plant species. The dataset was first loaded, and then preprocessing was performed to remove noisy data. Following that, data augmentation was carried out to improve model accuracy. The deep convolutional neural network (CNN) and visual geometry group-16 (VGG-16) were then employed to extract only the relevant features, owing to their efficient learning capabilities. Feature-level fusion was accomplished by utilizing dimensionality reduction, and enhanced Spearman's principal component analysis (ESPCA) was employed to address the overfitting problem, eliminate redundant data, and reduce storage space and training time requirements. For classification, the hyperparameter-tuned batch-updated stochastic gradient descent (HP-BSGD) method was utilized. The Flavia and Swedish datasets were utilized in the experiments. The proposed hybrid classifier yielded excellent results due to its high convergence speed, good computational effectiveness, and high flexibility. To validate the experimental results, performance and comparative analyses were carried out using standard metrics. The analytical results demonstrated the superior efficiency and suitability of the proposed method in the classification of plant species over existing methods. The hybrid method achieved approximately 97% and 98.85% accuracy in the Flavia and Swedish datasets, respectively, when considering combined features. The performance of the proposed method was further enhanced by considering leaves at different stages, such as seedlings, tiny, mature, and dried leaves.     

Parameter estimation for discretely observed linear birth‐and‐death processes

Birth‐and‐death processes are widely used to model the development of biological populations. Although they are relatively simple models, their parameters can be challenging to estimate, as the likelihood can become numerically unstable when data arise from the most common sampling schemes, such as annual population censuses. A further difficulty arises when the discrete observations are not equi‐spaced, for example, when census data are unavailable for some years. We present two approaches to estimating the birth, death, and growth rates of a discretely observed linear birth‐and‐death process: via an embedded Galton‐Watson process and by maximizing a saddlepoint approximation to the likelihood. We study asymptotic properties of the estimators, compare them on numerical examples, and apply the methodology to data on monitored populations.     

Likelihood-based inference for stochastic models of sexual network formation

Sexually-transmitted diseases (STDs) constitute a major public health concern. Mathematical models for the transmission dynamics of STDs indicate that heterogeneity in sexual activity level allow them to persist even when the typical behavior of the population would not support endemicity. This insight focuses attention on the distribution of sexual activity level in a population. In this paper, we develop several stochastic process models for the formation of sexual partnership networks. Using likelihood-based model selection procedures, we assess the fit of the different models to three large distributions of sexual partner counts: (1) Rakai, Uganda, (2) Sweden, and (3) the USA. Five of the six single-sex networks were fit best by the negative binomial model. The American women's network was best fit by a power-law model, the Yule. For most networks, several competing models fit approximately equally well. These results suggest three conclusions: (1) no single unitary process clearly underlies the formation of these sexual networks, (2) behavioral heterogeneity plays an essential role in network structure, (3) substantial model uncertainty exists for sexual network degree distributions. Behavioral research focused on the mechanisms of partnership formation will play an essential role in specifying the best model for empirical degree distributions. We discuss the limitations of inferences from such data, and the utility of degree-based epidemiological models more generally.     

Bayesian inference for stochastic epidemic models with time-inhomogeneous removal rates

Stochastic compartmental models of the SEIR type are often used to make inferences on epidemic processes from partially observed data in which only removal times are available. For many epidemics, the assumption of constant removal rates is not plausible. We develop methods for models in which these rates are a time-dependent step function. A reversible jump MCMC algorithm is described that permits Bayesian inferences to be made on model parameters, particularly those associated with the step function. The method is applied to two datasets on outbreaks of smallpox and a respiratory disease. The analyses highlight the importance of allowing for time dependence by contrasting the predictive distributions for the removal times and comparing them with the observed data.      

Scalable and flexible inference framework for stochastic dynamic single-cell models

Understanding the inherited nature of how biological processes dynamically change over time and exhibit intra- and inter-individual variability, due to the different responses to environmental stimuli and when interacting with other processes, has been a major focus of systems biology. The rise of single-cell fluorescent microscopy has enabled the study of those phenomena. The analysis of single-cell data with mechanistic models offers an invaluable tool to describe dynamic cellular processes and to rationalise cell-to-cell variability within the population. However, extracting mechanistic information from single-cell data has proven difficult. This requires statistical methods to infer unknown model parameters from dynamic, multi-individual data accounting for heterogeneity caused by both intrinsic (e.g. variations in chemical reactions) and extrinsic (e.g. variability in protein concentrations) noise. Although several inference methods exist, the availability of efficient, general and accessible methods that facilitate modelling of single-cell data, remains lacking. Here we present a scalable and flexible framework for Bayesian inference in state-space mixed-effects single-cell models with stochastic dynamic. Our approach infers model parameters when intrinsic noise is modelled by either exact or approximate stochastic simulators, and when extrinsic noise is modelled by either time-varying, or time-constant parameters that vary between cells. We demonstrate the relevance of our approach by studying how cell-to-cell variation in carbon source utilisation affects heterogeneity in the budding yeast Saccharomyces cerevisiae SNF1 nutrient sensing pathway. We identify hexokinase activity as a source of extrinsic noise and deduce that sugar availability dictates cell-to-cell variability.     

Parameter optimization and sensitivity analysis for large kinetic models using a real-coded genetic algorithm

Dynamic modeling is a powerful tool for predicting changes in metabolic regulation. However, a large number of input parameters, including kinetic constants and initial metabolite concentrations, are required to construct a kinetic model. Therefore, it is important not only to optimize the kinetic parameters, but also to investigate the effects of their perturbations on the overall system. We investigated the efficiency of the use of a real-coded genetic algorithm (RCGA) for parameter optimization and sensitivity analysis in the case of a large kinetic model involving glycolysis and the pentose phosphate pathway in Escherichia coli K-12. Sensitivity analysis of the kinetic model using an RCGA demonstrated that the input parameter values had different effects on model outputs. The results showed highly influential parameters in the model and their allowable ranges for maintaining metabolite-level stability. Furthermore, it was revealed that changes in these influential parameters may complement one another. This study presents an efficient approach based on the use of an RCGA for optimizing and analyzing parameters in large kinetic models.     

A tutorial introduction to Bayesian inference for stochastic epidemic models using Markov chain Monte Carlo methods

Recent Bayesian methods for the analysis of infectious disease outbreak data using stochastic epidemic models are reviewed. These methods rely on Markov chain Monte Carlo methods. Both temporal and non-temporal data are considered. The methods are illustrated with a number of examples featuring different models and datasets.     

Simulation and inference for stochastic volatility models driven by Levy processes

We study Ornstein-Uhlenbeck stochastic processes driven by Lévyprocesses, and extend them to more general non-Ornstein-Uhlenbeckmodels. In particular, we investigate the means of making thecorrelation structure in the volatility process more flexible.For one model, we implement a method for introducing quasi long-memoryinto the volatility model. We demonstrate that the models canbe fitted to real share price returns data.     

On relationship inference using gamete identity by descent data.

Related individuals are identical by descent (IBD) at a genetic locus if they share the same DNA material from a common ancestor. Continuous gamete IBD data consist of the lengths of (in order) IBD and non-IBD regions along the genomes for gametes segregating from two related individuals and can be used to distinguish different relationships. Under the assumption that the crossovers follow a Poisson process, we show that the exact calculation of the likelihood of a particular relationship for a given gamete IBD datum is tractable. Greatgrandparent--greatgrandchild and cousin relationships are used as examples to illustrate our methods.     

Moment closure approximations for stochastic kinetic models with rational rate laws

Stochastic models are often used when modelling chemical species that have low numbers of molecules. However, as these models become large, it can become computationally expensive to simulate even a single realisation of the system since even efficient simulation techniques have a high computational cost. One possible technique to approximate the stochastic system is moment closure. The moment closure approximation is used to provide analytic approximations to non-linear stochastic models. Until now, this approximation has only been applied to models with polynomial rate laws. In this paper we extend the moment closure method to cover models with rational rate laws.     

Parameter estimation and inference for stochastic reaction-diffusion systems: application to morphogenesis in D. melanogaster

Background  

Reaction-diffusion systems are frequently used in systems biology to model developmental and signalling processes. In many applications, count numbers of the diffusing molecular species are very low, leading to the need to explicitly model the inherent variability using stochastic methods. Despite their importance and frequent use, parameter estimation for both deterministic and stochastic reaction-diffusion systems is still a challenging problem.