DeepCME: A deep learning framework for computing solution statistics of the chemical master equation

Stochastic models of biomolecular reaction networks are commonly employed in systems and synthetic biology to study the effects of stochastic fluctuations emanating from reactions involving species with low copy-numbers. For such models, the Kolmogorov’s forward equation is called the chemical master equation (CME), and it is a fundamental system of linear ordinary differential equations (ODEs) that describes the evolution of the probability distribution of the random state-vector representing the copy-numbers of all the reacting species. The size of this system is given by the number of states that are accessible by the chemical system, and for most examples of interest this number is either very large or infinite. Moreover, approximations that reduce the size of the system by retaining only a finite number of important chemical states (e.g. those with non-negligible probability) result in high-dimensional ODE systems, even when the number of reacting species is small. Consequently, accurate numerical solution of the CME is very challenging, despite the linear nature of the underlying ODEs. One often resorts to estimating the solutions via computationally intensive stochastic simulations. The goal of the present paper is to develop a novel deep-learning approach for computing solution statistics of high-dimensional CMEs by reformulating the stochastic dynamics using Kolmogorov’s backward equation. The proposed method leverages superior approximation properties of Deep Neural Networks (DNNs) to reliably estimate expectations under the CME solution for several user-defined functions of the state-vector. This method is algorithmically based on reinforcement learning and it only requires a moderate number of stochastic simulations (in comparison to typical simulation-based approaches) to train the “policy function”. This allows not just the numerical approximation of various expectations for the CME solution but also of its sensitivities with respect to all the reaction network parameters (e.g. rate constants). We provide four examples to illustrate our methodology and provide several directions for future research.     

A Dynamical Low-Rank Approach to the Chemical Master Equation

Stochastic reaction kinetics have increasingly been used to study cellular systems, with applications ranging from viral replication to gene regulatory networks and to signaling pathways. The underlying evolution equation, known as the chemical master equation (CME), can rarely be solved with traditional methods due to the huge number of degrees of freedom. We present a new approach to directly solve the CME by a dynamical low-rank approximation based on the Dirac–Frenkel–McLachlan variational principle. The new approach has the capability to substantially reduce the number of degrees of freedom, and to turn the CME into a computationally tractable problem. We illustrate the accuracy and efficiency of our methods in application to two examples of biological interest.     

Time-dependent propagators for stochastic models of gene expression: an analytical method

The inherent stochasticity of gene expression in the context of regulatory networks profoundly influences the dynamics of the involved species. Mathematically speaking, the propagators which describe the evolution of such networks in time are typically defined as solutions of the corresponding chemical master equation (CME). However, it is not possible in general to obtain exact solutions to the CME in closed form, which is due largely to its high dimensionality. In the present article, we propose an analytical method for the efficient approximation of these propagators. We illustrate our method on the basis of two categories of stochastic models for gene expression that have been discussed in the literature. The requisite procedure consists of three steps: a probability-generating function is introduced which transforms the CME into (a system of) partial differential equations (PDEs); application of the method of characteristics then yields (a system of) ordinary differential equations (ODEs) which can be solved using dynamical systems techniques, giving closed-form expressions for the generating function; finally, propagator probabilities can be reconstructed numerically from these expressions via the Cauchy integral formula. The resulting ‘library’ of propagators lends itself naturally to implementation in a Bayesian parameter inference scheme, and can be generalised systematically to related categories of stochastic models beyond the ones considered here.

     

Solving the chemical master equation for monomolecular reaction systems analytically

The stochastic dynamics of a well-stirred mixture of molecular species interacting through different biochemical reactions can be accurately modelled by the chemical master equation (CME). Research in the biology and scientific computing community has concentrated mostly on the development of numerical techniques to approximate the solution of the CME via many realizations of the associated Markov jump process. The domain of exact and/or efficient methods for directly solving the CME is still widely open, which is due to its large dimension that grows exponentially with the number of molecular species involved. In this article, we present an exact solution formula of the CME for arbitrary initial conditions in the case where the underlying system is governed by monomolecular reactions. The solution can be expressed in terms of the convolution of multinomial and product Poisson distributions with time-dependent parameters evolving according to the traditional reaction-rate equations. This very structured representation allows to deduce easily many properties of the solution. The model class includes many interesting examples. For more complex reaction systems, our results can be seen as a first step towards the construction of new numerical integrators, because solutions to the monomolecular case provide promising ansatz functions for Galerkin-type methods.     

Solving the chemical master equation using sliding windows

Background  

The chemical master equation (CME) is a system of ordinary differential equations that describes the evolution of a network of chemical reactions as a stochastic process. Its solution yields the probability density vector of the system at each point in time. Solving the CME numerically is in many cases computationally expensive or even infeasible as the number of reachable states can be very large or infinite. We introduce the sliding window method, which computes an approximate solution of the CME by performing a sequence of local analysis steps. In each step, only a manageable subset of states is considered, representing a "window" into the state space. In subsequent steps, the window follows the direction in which the probability mass moves, until the time period of interest has elapsed. We construct the window based on a deterministic approximation of the future behavior of the system by estimating upper and lower bounds on the populations of the chemical species.     

Decoy selection for protein structure prediction via extreme gradient boosting and ranking

Numerical solutions of the chemical master equation (CME) are important for understanding the stochasticity of biochemical systems. However, solving CMEs is a formidable task. This task is complicated due to the nonlinear nature of the reactions and the size of the networks which result in different realizations. Most importantly, the exponential growth of the size of the state-space, with respect to the number of different species in the system makes this a challenging assignment. When the biochemical system has a large number of variables, the CME solution becomes intractable. We introduce the intelligent state projection (ISP) method to use in the stochastic analysis of these systems. For any biochemical reaction network, it is important to capture more than one moment: this allows one to describe the system’s dynamic behaviour. ISP is based on a state-space search and the data structure standards of artificial intelligence (AI). It can be used to explore and update the states of a biochemical system. To support the expansion in ISP, we also develop a Bayesian likelihood node projection (BLNP) function to predict the likelihood of the states. To demonstrate the acceptability and effectiveness of our method, we apply the ISP method to several biological models discussed in prior literature. The results of our computational experiments reveal that the ISP method is effective both in terms of the speed and accuracy of the expansion, and the accuracy of the solution. This method also provides a better understanding of the state-space of the system in terms of blueprint patterns. The ISP is the de-novo method which addresses both accuracy and performance problems for CME solutions. It systematically expands the projection space based on predefined inputs. This ensures accuracy in the approximation and an exact analytical solution for the time of interest. The ISP was more effective both in predicting the behavior of the state-space of the system and in performance management, which is a vital step towards modeling large biochemical systems.     

Investigating the two-moment characterisation of subcellular biochemical networks

While ordinary differential equations (ODEs) form the conceptual framework for modelling many cellular processes, specific situations demand stochastic models to capture the influence of noise. The most common formulation of stochastic models for biochemical networks is the chemical master equation (CME). While stochastic simulations are a practical way to realise the CME, analytical approximations offer more insight into the influence of noise. Towards that end, the two-moment approximation (2MA) is a promising addition to the established analytical approaches including the chemical Langevin equation (CLE) and the related linear noise approximation (LNA). The 2MA approach directly tracks the mean and (co)variance which are coupled in general. This coupling is not obvious in CME and CLE and ignored by LNA and conventional ODE models. We extend previous derivations of 2MA by allowing (a) non-elementary reactions and (b) relative concentrations. Often, several elementary reactions are approximated by a single step. Furthermore, practical situations often require the use of relative concentrations. We investigate the applicability of the 2MA approach to the well-established fission yeast cell cycle model. Our analytical model reproduces the clustering of cycle times observed in experiments. This is explained through multiple resettings of M-phase promoting factor (MPF), caused by the coupling between mean and (co)variance, near the G2/M transition.     

A Hybrid of the Chemical Master Equation and the Gillespie Algorithm for Efficient Stochastic Simulations of Sub-Networks

Modeling stochastic behavior of chemical reaction networks is an important endeavor in many aspects of chemistry and systems biology. The chemical master equation (CME) and the Gillespie algorithm (GA) are the two most fundamental approaches to such modeling; however, each of them has its own limitations: the GA may require long computing times, while the CME may demand unrealistic memory storage capacity. We propose a method that combines the CME and the GA that allows one to simulate stochastically a part of a reaction network. First, a reaction network is divided into two parts. The first part is simulated via the GA, while the solution of the CME for the second part is fed into the GA in order to update its propensities. The advantage of this method is that it avoids the need to solve the CME or stochastically simulate the entire network, which makes it highly efficient. One of its drawbacks, however, is that most of the information about the second part of the network is lost in the process. Therefore, this method is most useful when only partial information about a reaction network is needed. We tested this method against the GA on two systems of interest in biology - the gene switch and the Griffith model of a genetic oscillator—and have shown it to be highly accurate. Comparing this method to four different stochastic algorithms revealed it to be at least an order of magnitude faster than the fastest among them.     

The mammalian endocytic cytoskeleton

Clathrin-mediated endocytosis (CME) is the major route through which cells internalise various substances and recycle membrane components. Via the coordinated action of many proteins, the membrane bends and invaginates to form a vesicle that buds off—along with its contents—into the cell. The contribution of the actin cytoskeleton to this highly dynamic process in mammalian cells is not well understood. Unlike in yeast, where there is a strict requirement for actin in CME, the significance of the actin cytoskeleton to mammalian CME is variable. However, a growing number of studies have established the actin cytoskeleton as a core component of mammalian CME, and our understanding of its contribution has been increasing at a rapid pace. In this review, we summarise the state-of-the-art regarding our understanding of the endocytic cytoskeleton, its physiological significance, and the questions that remain to be answered.     

Numerical studies of unreactive contractile networks.

We present a finite difference algorithm for integrating the reactive flow model of contractile biological polymer networks on a fixed Eulerian mesh. We discuss the accuracy and limits of the algorithm. To illustrate the application of the algorithm, we carry out a family of computations involving an unreactive contractile network contained in a two-dimensional square reaction vessel. By numerical experiments using different values of the physical parameters of the model, we find that for this simple sort of system two major dynamical modes of contraction are predicted to occur. There is the squeezing type contraction in which the network contracts to a single small clump with gradual expulsion of solution material, and the rending type contraction in which the network tears itself into a number of separate pieces. We find that to a good approximation the transition between the squeezing mode and the rending mode is controlled by a single nondimensional number (the rending number). We discuss the relevance of these results for the analysis of various experimental observations.     

A new current expression for selective ion permeation across membranes

Classical Nernst-Planck electrodiffusion approach to selective ion permeation through membranes, remains a currently popular approach to the problem. Some attempts have, however, been made to use Eyring's theory to explain selective ion permeation through the trans-membrane channels. The common problem with either approach is that the final expression for ionic current does not have selectivity inbuilt, rather selectivity is introduced in the current equation as an arbitrarily chosen parameter. In addition, the negative conductance region is non-reproduceable. To the best of our knowledge, we describe, for the first time in the literature, a selective ion current equation which, in addition, also has inbuilt negative conductance property within the framework of a kinematic approach—to a level of first degree approximation.     

Direct Solution of the Chemical Master Equation Using Quantized Tensor Trains

The Chemical Master Equation (CME) is a cornerstone of stochastic analysis and simulation of models of biochemical reaction networks. Yet direct solutions of the CME have remained elusive. Although several approaches overcome the infinite dimensional nature of the CME through projections or other means, a common feature of proposed approaches is their susceptibility to the curse of dimensionality, i.e. the exponential growth in memory and computational requirements in the number of problem dimensions. We present a novel approach that has the potential to “lift” this curse of dimensionality. The approach is based on the use of the recently proposed Quantized Tensor Train (QTT) formatted numerical linear algebra for the low parametric, numerical representation of tensors. The QTT decomposition admits both, algorithms for basic tensor arithmetics with complexity scaling linearly in the dimension (number of species) and sub-linearly in the mode size (maximum copy number), and a numerical tensor rounding procedure which is stable and quasi-optimal. We show how the CME can be represented in QTT format, then use the exponentially-converging -discontinuous Galerkin discretization in time to reduce the CME evolution problem to a set of QTT-structured linear equations to be solved at each time step using an algorithm based on Density Matrix Renormalization Group (DMRG) methods from quantum chemistry. Our method automatically adapts the “basis” of the solution at every time step guaranteeing that it is large enough to capture the dynamics of interest but no larger than necessary, as this would increase the computational complexity. Our approach is demonstrated by applying it to three different examples from systems biology: independent birth-death process, an example of enzymatic futile cycle, and a stochastic switch model. The numerical results on these examples demonstrate that the proposed QTT method achieves dramatic speedups and several orders of magnitude storage savings over direct approaches.     

Clathrin-mediated endocytosis at synapses

Neurons are communication specialists that convert electrical into chemical signals at specialized cell-cell junctions termed synapses. Arrival of an action potential triggers calcium-regulated exocytosis of neurotransmitter (NT) from small synaptic vesicles (SVs), which then diffuses across the synaptic cleft and binds to postsynaptic receptors to elicit specific changes within the postsynaptic cell. Endocytosis of pre- and postsynaptic membrane proteins including SV components and postsynaptic NT receptors is essential for the proper functioning of the synapse. During the past several years, we have witnessed enormous progress in our understanding of the mechanics of clathrin-mediated endocytosis (CME) and its role in regulating exo-endocytic vesicle cycling at synapses. Here we summarize the molecular machinery used for recognition of synaptic membrane protein cargo and its clathrin-dependent internalization, and describe the inventory of tools that can be used to monitor vesicle cycling at synapses or to inhibit CME in a stage-specific manner.     

Moment fitting for parameter inference in repeatedly and partially observed stochastic biological models

The inference of reaction rate parameters in biochemical network models from time series concentration data is a central task in computational systems biology. Under the assumption of well mixed conditions the network dynamics are typically described by the chemical master equation, the Fokker Planck equation, the linear noise approximation or the macroscopic rate equation. The inverse problem of estimating the parameters of the underlying network model can be approached in deterministic and stochastic ways, and available methods often compare individual or mean concentration traces obtained from experiments with theoretical model predictions when maximizing likelihoods, minimizing regularized least squares functionals, approximating posterior distributions or sequentially processing the data. In this article we assume that the biological reaction network can be observed at least partially and repeatedly over time such that sample moments of species molecule numbers for various time points can be calculated from the data. Based on the chemical master equation we furthermore derive closed systems of parameter dependent nonlinear ordinary differential equations that predict the time evolution of the statistical moments. For inferring the reaction rate parameters we suggest to not only compare the sample mean with the theoretical mean prediction but also to take the residual of higher order moments explicitly into account. Cost functions that involve residuals of higher order moments may form landscapes in the parameter space that have more pronounced curvatures at the minimizer and hence may weaken or even overcome parameter sloppiness and uncertainty. As a consequence both deterministic and stochastic parameter inference algorithms may be improved with respect to accuracy and efficiency. We demonstrate the potential of moment fitting for parameter inference by means of illustrative stochastic biological models from the literature and address topics for future research.     

Measurement of grouped intracellular solute osmotic virial coefficients

Models of cellular osmotic behaviour depend on thermodynamic solution theories to calculate chemical potentials in the solutions inside and outside the cell. These solutions are generally thermodynamically non-ideal under cryobiological conditions. The molality-based Elliott et al. form of the multi-solute osmotic virial equation is a solution theory which has been demonstrated to provide accurate predictions in cryobiological solutions, accounting for the non-ideality of these solutions using solute-specific thermodynamic parameters called osmotic virial coefficients. However, this solution theory requires as inputs the exact concentration of every solute in the solution being modeled, which poses a problem for the cytoplasm, where such detailed information is rarely available. This problem can be overcome by using a grouped solute approach for modeling the cytoplasm, where all the non-permeating intracellular solutes are treated as a single non-permeating “grouped” intracellular solute. We have recently shown (Zielinski et al., J Physical Chemistry B, 2017) that such a grouped solute approach is theoretically valid when used with the Elliott et al. model, and Ross-Rodriguez et al. (Biopreservation and Biobanking, 2012) have previously developed a method for measuring the cell type-specific osmotic virial coefficients of the grouped intracellular solute. However, the Ross-Rodriguez et al. method suffers from a lack of precision, which—as we demonstrate in this work—can severely impact the accuracy of osmotic model predictions under certain conditions. Thus, we herein develop a novel method for measuring grouped intracellular solute osmotic virial coefficients which yields more precise values than the existing method and then apply this new method to measure these coefficients for human umbilical vein endothelial cells.     

General characteristics of the sigmoidal model equation representing quasi-static pulmonary P-V curves.

A pulmonary pressure-volume (P-V) curve represented by a sigmoidal model equation with four parameters, V(P) = a + b[1 + exp[-(P - c)/d]](-1), has been demonstrated to fit inflation and deflation data obtained under a variety of conditions extremely well. In the present report, a differential equation on V(P) is identified, thus relating the fourth parameter, d, to the difference between the upper and the lower asymptotes of the volume, b, through a proportionality constant, alpha, with its order of magnitude of 10(-4) to 10(-5) (in ml(-1). cmH(2)O(-1)). When the model equation is normalized using a nondimensional volume, (-1 < < 1), and a nondimensional pressure, (=(p/c) - 1), the resulting - curve depends on a single nondimensional parameter, Lambda = alphabc. A nondimensional work of expansion/compression, (1-2), is also obtained along the quasi-static sigmoidal P-V curve between an initial volume (at 1) and a final volume (at 2). Six sets of P-V data available in the literature are used to show the changes that occur in these two parameters (Lambda defining the shape of the sigmoidal curve and (1-2) accounting for the range of clinical data) with different conditions of the total respiratory system. The clinical usefulness of these parameters requires further study.