Maximum Likelihood Estimation of Long‐Term HIV Dynamic Models and Antiviral Response

Summary HIV dynamics studies, based on differential equations, have significantly improved the knowledge on HIV infection. While first studies used simplified short‐term dynamic models, recent works considered more complex long‐term models combined with a global analysis of whole patient data based on nonlinear mixed models, increasing the accuracy of the HIV dynamic analysis. However statistical issues remain, given the complexity of the problem. We proposed to use the SAEM (stochastic approximation expectation‐maximization) algorithm, a powerful maximum likelihood estimation algorithm, to analyze simultaneously the HIV viral load decrease and the CD4 increase in patients using a long‐term HIV dynamic system. We applied the proposed methodology to the prospective COPHAR2–ANRS 111 trial. Very satisfactory results were obtained with a model with latent CD4 cells defined with five differential equations. One parameter was fixed, the 10 remaining parameters (eight with between‐patient variability) of this model were well estimated. We showed that the efficacy of nelfinavir was reduced compared to indinavir and lopinavir.     

Design of ant-inspired stochastic control policies for collective transport by robotic swarms

In this paper, we present an approach to designing decentralized robot control policies that mimic certain microscopic and macroscopic behaviors of ants performing collective transport tasks. In prior work, we used a stochastic hybrid system model to characterize the observed team dynamics of ant group retrieval of a rigid load. We have also used macroscopic population dynamic models to design enzyme-inspired stochastic control policies that allocate a robotic swarm around multiple boundaries in a way that is robust to environmental variations. Here, we build on this prior work to synthesize stochastic robot attachment–detachment policies for tasks in which a robotic swarm must achieve non-uniform spatial distributions around multiple loads and transport them at a constant velocity. Three methods are presented for designing robot control policies that replicate the steady-state distributions, transient dynamics, and fluxes between states that we have observed in ant populations during group retrieval. The equilibrium population matching method can be used to achieve a desired transport team composition as quickly as possible; the transient matching method can control the transient population dynamics of the team while driving it to the desired composition; and the rate matching method regulates the rates at which robots join and leave a load during transport. We validate our model predictions in an agent-based simulation, verify that each controller design method produces successful transport of a load at a regulated velocity, and compare the advantages and disadvantages of each method.     

Coupled map lattice approximations for spatially explicit individual-based models of ecology

Spatially explicit individual-based models are widely used in ecology but they are often difficult to treat analytically. Despite their intractability they often exhibit clear temporal and spatial patterning. We demonstrate how a spatially explicit individual-based model of scramble competition with local dispersal can be approximated by a stochastic coupled map lattice. The approximation disentangles the deterministic and stochastic element of local interaction and dispersal. We are thus able to understand the individual-based model through a simplified set of equations. In particular, we demonstrate that demographic noise leads to increased stability in the dynamics of locally dispersing single-species populations. The coupled map lattice approximation has general application to a range of spatially explicit individual-based models. It provides a new alternative to current approximation techniques, such as the method of moments and reaction-diffusion approximation, that captures both stochastic effects and large-scale patterning arising in individual-based models.     

A stochastic metapopulation model accounting for habitat dynamics

A stochastic metapopulation model accounting for habitat dynamics is presented. This is the stochastic SIS logistic model with the novel aspect that it incorporates varying carrying capacity. We present results of Kurtz and Barbour, that provide deterministic and diffusion approximations for a wide class of stochastic models, in a form that most easily allows their direct application to population models. These results are used to show that a suitably scaled version of the metapopulation model converges, uniformly in probability over finite time intervals, to a deterministic model previously studied in the ecological literature. Additionally, they allow us to establish a bivariate normal approximation to the quasi-stationary distribution of the process. This allows us to consider the effects of habitat dynamics on metapopulation modelling through a comparison with the stochastic SIS logistic model and provides an effective means for modelling metapopulations inhabiting dynamic landscapes.     

Pair-edge approximation for heterogeneous lattice population models

To increase the analytical tractability of lattice stochastic spatial population models, several approximations have been developed. The pair-edge approximation is a moment-closure method that is effective in predicting persistence criteria and invasion speeds on a homogeneous lattice. Here we evaluate the effectiveness of the pair-edge approximation on a spatially heterogeneous lattice in which some sites are unoccupiable, or "dead". This model has several possible interpretations, including a spatial SIS epidemic model, in which some sites are occupied by immobile host-species individuals while others are empty. We find that, as in the homogeneous model, the pair-edge approximation is significantly more accurate than the ordinary pair approximation in determining conditions for persistence. However, habitat heterogeneity decreases invasion speed more than is predicted by the pair-edge approximation, and the discrepancy increases with greater clustering of "dead" sites. The accuracy of the approximation validates the underlying heuristic picture of population spread and therefore provides qualitative insight into the dynamics of lattice models. Conversely, the situations where the approximation is less accurate reveals limitations of pair approximation in the presence of spatial heterogeneity.     

Moment fitting for parameter inference in repeatedly and partially observed stochastic biological models

The inference of reaction rate parameters in biochemical network models from time series concentration data is a central task in computational systems biology. Under the assumption of well mixed conditions the network dynamics are typically described by the chemical master equation, the Fokker Planck equation, the linear noise approximation or the macroscopic rate equation. The inverse problem of estimating the parameters of the underlying network model can be approached in deterministic and stochastic ways, and available methods often compare individual or mean concentration traces obtained from experiments with theoretical model predictions when maximizing likelihoods, minimizing regularized least squares functionals, approximating posterior distributions or sequentially processing the data. In this article we assume that the biological reaction network can be observed at least partially and repeatedly over time such that sample moments of species molecule numbers for various time points can be calculated from the data. Based on the chemical master equation we furthermore derive closed systems of parameter dependent nonlinear ordinary differential equations that predict the time evolution of the statistical moments. For inferring the reaction rate parameters we suggest to not only compare the sample mean with the theoretical mean prediction but also to take the residual of higher order moments explicitly into account. Cost functions that involve residuals of higher order moments may form landscapes in the parameter space that have more pronounced curvatures at the minimizer and hence may weaken or even overcome parameter sloppiness and uncertainty. As a consequence both deterministic and stochastic parameter inference algorithms may be improved with respect to accuracy and efficiency. We demonstrate the potential of moment fitting for parameter inference by means of illustrative stochastic biological models from the literature and address topics for future research.     

Modeling salt-mediated electrostatics of macromolecules: the discrete surface charge optimization algorithm and its application to the nucleosome

Much progress has been achieved on quantitative assessment of electrostatic interactions on the all-atom level by molecular mechanics and dynamics, as well as on the macroscopic level by models of continuum solvation. Bridging of the two representations-an area of active research-is necessary for studying integrated functions of large systems of biological importance. Following perspectives of both discrete (N-body) interaction and continuum solvation, we present a new algorithm, DiSCO (Discrete Surface Charge Optimization), for economically describing the electrostatic field predicted by Poisson-Boltzmann theory using a discrete set of Debye-Hückel charges distributed on a virtual surface enclosing the macromolecule. The procedure in DiSCO relies on the linear behavior of the Poisson-Boltzmann equation in the far zone; thus contributions from a number of molecules may be superimposed, and the electrostatic potential, or equivalently the electrostatic field, may be quickly and efficiently approximated by the summation of contributions from the set of charges. The desired accuracy of this approximation is achieved by minimizing the difference between the Poisson-Boltzmann electrostatic field and that produced by the linearized Debye-Hückel approximation using our truncated Newton optimization package. DiSCO is applied here to describe the salt-dependent electrostatic environment of the nucleosome core particle in terms of several hundred surface charges. This representation forms the basis for modeling-by dynamic simulations (or Monte Carlo)-the folding of chromatin. DiSCO can be applied more generally to many macromolecular systems whose size and complexity warrant a model resolution between the all-atom and macroscopic levels.     

On learning dynamics underlying the evolution of learning rules

In order to understand the development of non-genetically encoded actions during an animal’s lifespan, it is necessary to analyze the dynamics and evolution of learning rules producing behavior. Owing to the intrinsic stochastic and frequency-dependent nature of learning dynamics, these rules are often studied in evolutionary biology via agent-based computer simulations. In this paper, we show that stochastic approximation theory can help to qualitatively understand learning dynamics and formulate analytical models for the evolution of learning rules. We consider a population of individuals repeatedly interacting during their lifespan, and where the stage game faced by the individuals fluctuates according to an environmental stochastic process. Individuals adjust their behavioral actions according to learning rules belonging to the class of experience-weighted attraction learning mechanisms, which includes standard reinforcement and Bayesian learning as special cases. We use stochastic approximation theory in order to derive differential equations governing action play probabilities, which turn out to have qualitative features of mutator-selection equations. We then perform agent-based simulations to find the conditions where the deterministic approximation is closest to the original stochastic learning process for standard 2-action 2-player fluctuating games, where interaction between learning rules and preference reversal may occur. Finally, we analyze a simplified model for the evolution of learning in a producer–scrounger game, which shows that the exploration rate can interact in a non-intuitive way with other features of co-evolving learning rules. Overall, our analyses illustrate the usefulness of applying stochastic approximation theory in the study of animal learning.     

Phylodynamic Inference for Structured Epidemiological Models

Coalescent theory is routinely used to estimate past population dynamics and demographic parameters from genealogies. While early work in coalescent theory only considered simple demographic models, advances in theory have allowed for increasingly complex demographic scenarios to be considered. The success of this approach has lead to coalescent-based inference methods being applied to populations with rapidly changing population dynamics, including pathogens like RNA viruses. However, fitting epidemiological models to genealogies via coalescent models remains a challenging task, because pathogen populations often exhibit complex, nonlinear dynamics and are structured by multiple factors. Moreover, it often becomes necessary to consider stochastic variation in population dynamics when fitting such complex models to real data. Using recently developed structured coalescent models that accommodate complex population dynamics and population structure, we develop a statistical framework for fitting stochastic epidemiological models to genealogies. By combining particle filtering methods with Bayesian Markov chain Monte Carlo methods, we are able to fit a wide class of stochastic, nonlinear epidemiological models with different forms of population structure to genealogies. We demonstrate our framework using two structured epidemiological models: a model with disease progression between multiple stages of infection and a two-population model reflecting spatial structure. We apply the multi-stage model to HIV genealogies and show that the proposed method can be used to estimate the stage-specific transmission rates and prevalence of HIV. Finally, using the two-population model we explore how much information about population structure is contained in genealogies and what sample sizes are necessary to reliably infer parameters like migration rates.     

Maximum entropy modeling of short sequence motifs with applications to RNA splicing signals.

We propose a framework for modeling sequence motifs based on the maximum entropy principle (MEP). We recommend approximating short sequence motif distributions with the maximum entropy distribution (MED) consistent with low-order marginal constraints estimated from available data, which may include dependencies between nonadjacent as well as adjacent positions. Many maximum entropy models (MEMs) are specified by simply changing the set of constraints. Such models can be utilized to discriminate between signals and decoys. Classification performance using different MEMs gives insight into the relative importance of dependencies between different positions. We apply our framework to large datasets of RNA splicing signals. Our best models out-perform previous probabilistic models in the discrimination of human 5' (donor) and 3' (acceptor) splice sites from decoys. Finally, we discuss mechanistically motivated ways of comparing models.     

Dynamics of infection with multiple transmission mechanisms in unmanaged/managed animal populations

Deterministic and stochastic models motivated by Salmonella transmission in unmanaged/managed populations are studied. The SIRS models incorporate three routes of transmission (direct, vertical and indirect via free-living infectious units in the environment). With deterministic models we are able to understand the effects of different routes of transmission and other epidemiological factors on infection dynamics. In particular, vertical transmission has little influence on this dynamics, whereas the higher the indirect (direct) transmission rate the greater the tendency to persistent oscillation (stable endemic states). We show that the sustained cycles are also prone to demographic effect, i.e., persistent oscillation becomes impossible in the managed case (in the sense of balanced recruitment and death rates) by comparing with results in unmanaged populations (exponential population dynamics). Further, approximations of quasi-stationary distributions are derived for stochastic versions of the proposed models based on a diffusion approximation to the infection process. The effect of transmission parameters on the ratio of mean to standard deviation of the approximating distribution, used to judge the validity of the approximations and the expected time until fade out of infection, is further discussed. We conclude that strengthening any route of transmission may or may not reduce the expected time to fade out of infection, depending on the population dynamics.     

Dynamics of Epidemiological Models

We study the SIS and SIRI epidemic models discussing different approaches to compute the thresholds that determine the appearance of an epidemic disease. The stochastic SIS model is a well known mathematical model, studied in several contexts. Here, we present recursively derivations of the dynamic equations for all the moments and we derive the stationary states of the state variables using the moment closure method. We observe that the steady states give a good approximation of the quasi-stationary states of the SIS model. We present the relation between the SIS stochastic model and the contact process introducing creation and annihilation operators. For the spatial stochastic epidemic reinfection model SIRI, where susceptibles S can become infected I, then recover and remain only partial immune against reinfection R, we present the phase transition lines using the mean field and the pair approximation for the moments. We use a scaling argument that allow us to determine analytically an explicit formula for the phase transition lines in pair approximation.     

Demographic noise and resilience in a semi-arid ecosystem model

The scarcity of water characterising drylands forces vegetation to adopt appropriate survival strategies. Some of these generate water–vegetation feedback mechanisms that can lead to spatial self-organisation of vegetation, as it has been shown with models representing plants by a density of biomass, varying continuously in time and space. However, although plants are usually quite plastic they also display discrete qualities and stochastic behaviour. These features may give rise to demographic noise, which in certain cases can influence the qualitative dynamics of ecosystem models. In the present work we explore the effects of demographic noise on the resilience of a model semi-arid ecosystem. We introduce a spatial stochastic eco-hydrological hybrid model in which plants are modelled as discrete entities subject to stochastic dynamical rules, while the dynamics of surface and soil water are described by continuous variables. The model has a deterministic approximation very similar to previous continuous models of arid and semi-arid ecosystems. By means of numerical simulations we show that demographic noise can have important effects on the extinction and recovery dynamics of the system. In particular we find that the stochastic model escapes extinction under a wide range of conditions for which the corresponding deterministic approximation predicts absorption into desert states.