Weak convergence of a sequence of stochastic difference equations to a stochastic ordinary differential equation

We consider a sequence of discrete parameter stochastic processes defined by solutions to stochastic difference equations. A condition is given that this sequence converges weakly to a continuous parameter process defined by solutions to a stochastic ordinary differential equation. Applying this result, two limit theorems related to population biology are proved. Random parameters in stochastic difference equations are autocorrelated stationary Gaussian processes in the first case. They are jump-type Markov processes in the second case. We discuss a problem of continuous time approximations for discrete time models in random environments.     

On the stationary state analysis of reaction-diffusion mechanisms for biological pattern formation

We present a biologically plausible two-variable reaction-diffusion model for the developing vertebrate limb, for which we postulate the existence of a stationary solution. A consequence of this assumption is that the stationary state depends on only a single concentration-variable. Under these circumstances, features of potential biological significance, such as the dependence of the steady-state concentration profile of this variable on parameters such as tissue size and shape, can be studied without detailed information about the rate functions. As the existence and stability of stationary solutions, which must be assumed for any biochemical system governing morphogenesis, cannot be investigated without such information, an analysis is made of the minimal requirements for stable, stationary non-uniform solutions in a general class of reaction-diffusion systems. We discuss the strategy of studying stationary-state properties of systems that are incompletely specified. Where abrupt transitions between successive compartment-sizes occur, as in the developing limb, we argue that it is reasonable to model pattern reorganization as a sequence of independent stationary states.     

Coexistence of competitors in spatially and temporally varying environments: A look at the combined effects of different sorts of variability

A stochastic model is developed for competition among organisms living in a patchy and varying environment. The model is designed to be suitable for species with sedentary adults and widely dispersing larvae or propagules, and applies best to marine systems but may also be adequate for some terrestrial systems. Three kinds of environmental variation are incorporated simultaneously in the model. These are pure spatial variation, pure temporal variation, and the space × time interaction. All three kinds of variation can promote coexistence, and when variation is restricted to immigration rates, all three kinds act very similarly. Moreover, for long-lived organisms their action is nearly identical, and their effects, when present together, combine equivalently. For short-lived organisms, however, pure temporal variation is a less effective promoter of coexistence. Variation in death rates acts quite differently from variation in birth rates for it may demote coexistence in some circumstances, while promoting coexistence in other circumstances. Furthermore, pure spatial variation in death rates has quite different effects than other kinds of death-rate variation. In addition to conditions for coexistence, information is given on population fluctuations, convergence to stationary distributions, and asymptotic distributions for long-lived organisms. While the model is presented as an ecological model, a genetical interpretation is also possible. This leads to new suggested mechanisms for the maintenance of polymorphisms in populations.     

Stochastic disease dynamics of a hospital infection model

A stochastic model for hospital infection incorporating both direct transmission and indirect transmission via free-living bacteria in the environment is investigated. We examine the long term behavior of the model by calculating a stationary distribution and normal approximation of the distribution. The quasi-stationary distribution of the model is studied to investigate the models’ behavior before extinction and the time to extinction. Numerical results show agreement between the calculated distributions and results of event-driven simulations. Hand hygiene of volunteers is more effective in terms of reducing the mean (or standard deviation) of the stationary distribution of colonized patients and the expected time to extinction compared to hand hygiene of health care workers (HCWs), on the basis of our parameter values. However, the indirect (or direct) transmission rate can lead to either increase or decrease in the standard deviation of the stationary distribution, but the impact of the indirect transmission is much greater than that of the direct transmission. The findings suggest that isolation of new admitted colonized patients is most effective in reducing both the mean and standard deviation of the stationary distribution and measures related to indirect transmission are secondary in their effects compared to other interventions.     

Approximating the long-term behaviour of a model for parasitic infection

In a companion paper two stochastic models, useful for the initial behaviour of a parasitic infection, were introduced. Now we analyse the long term behaviour. First a law of large numbers is proved which allows us to analyse the deterministic analogues of the stochastic models. The behaviour of the deterministic models is analogous to the stochastic models in that again three basic reproduction ratios are necessary to fully describe the information needed to separate growth from extinction. The existence of stationary solutions is shown in the deterministic models, which can be used as a justification for simulation of quasi-equilibria in the stochastic models. Host-mortality is included in all models. The proofs involve martingale and coupling methods.     

Dynamics of a physiologically structured population in a time-varying environment

Physiologically structured population models have become a valuable tool to model the dynamics of populations. In a stationary environment such models can exhibit equilibrium solutions as well as periodic solutions. However, for many organisms the environment is not stationary, but varies more or less regularly. In order to understand the interaction between an external environmental forcing and the internal dynamics in a population, we examine the response of a physiologically structured population model to a periodic variation in the food resource. We explore the addition of forcing in two cases: (A) where the population dynamics is in equilibrium in a stationary environment, and (B) where the population dynamics exhibits a periodic solution in a stationary environment. When forcing is applied in case A, the solutions are mainly periodic. In case B the forcing signal interacts with the oscillations of the unforced system, and both periodic and irregular (quasi-periodic or chaotic) solutions occur. In both cases the periodic solutions include one and multiple period cycles, and each cycle can have several reproduction pulses.     

Novel bivariate moment-closure approximations

Nonlinear stochastic models are typically intractable to analytic solutions and hence, moment-closure schemes are used to provide approximations to these models. Existing closure approximations are often unable to describe transient aspects caused by extinction behaviour in a stochastic process. Recent work has tackled this problem in the univariate case. In this study, we address this problem by introducing novel bivariate moment-closure methods based on mixture distributions. Novel closure approximations are developed, based on the beta-binomial, zero-modified distributions and the log-Normal, designed to capture the behaviour of the stochastic SIS model with varying population size, around the threshold between persistence and extinction of disease. The idea of conditional dependence between variables of interest underlies these mixture approximations. In the first approximation, we assume that the distribution of infectives (I) conditional on population size (N) is governed by the beta-binomial and for the second form, we assume that I is governed by zero-modified beta-binomial distribution where in either case N follows a log-Normal distribution. We analyse the impact of coupling and inter-dependency between population variables on the behaviour of the approximations developed. Thus, the approximations are applied in two situations in the case of the SIS model where: (1) the death rate is independent of disease status; and (2) the death rate is disease-dependent. Comparison with simulation shows that these mixture approximations are able to predict disease extinction behaviour and describe transient aspects of the process.     

Influence of stochastic perturbation on prey-predator systems

We analyse the influence of various stochastic perturbations on prey-predator systems. The prey-predator model is described by stochastic versions of a deterministic Lotka-Volterra system. We study long-time behaviour of both trajectories and distributions of the solutions. We indicate the differences between the deterministic and stochastic models.     

On the stability of separable solutions of a sexual age-structured population dynamics model

The Sharpe-Lotka-McKendrick-von Foerster equations for non-dispersing age-sex-structured populations with a harmonic mean type mating law are considered and their separable solutions are analysed. For certain forms of the demographic rates the underlying evolution equations are reduced to systems of ODEs, the long time behavior of their solutions is studied, and the stability of separable solutions is discussed. It is found that for the constant death rates and constant sex ratio of newborns with stationary birth rates this model admits one one-parameter class of separable solutions, two such classes (repeated or different) or no such ones. In the case of special forms of age-dependent birth rates, solutions of one of these two different classes corresponding to the greater root of the characteristic equation are locally stable, solutions of the other one corresponding to the smaller root are unstable, and the population dies out if the model does not admit separable solutions or if initial densities of newborns are small enough in the case of the existence of separable solutions. In the case of constant vital rates, the model has no separable solutions or admits only one class of such ones that are globally stable.     

Size-Independent Differences between the Mean of Discrete Stochastic Systems and the Corresponding Continuous Deterministic Systems

In this paper, it is shown that for a class of reaction networks, the discrete stochastic nature of the reacting species and reactions results in qualitative and quantitative differences between the mean of exact stochastic simulations and the prediction of the corresponding deterministic system. The differences are independent of the number of molecules of each species in the system under consideration. These reaction networks are open systems of chemical reactions with no zero-order reaction rates. They are characterized by at least two stationary points, one of which is a nonzero stable point, and one unstable trivial solution (stability based on a linear stability analysis of the deterministic system). Starting from a nonzero initial condition, the deterministic system never reaches the zero stationary point due to its unstable nature. In contrast, the result presented here proves that this zero-state is a stable stationary state for the discrete stochastic system, and other finite states have zero probability of existence at large times. This result generalizes previous theoretical studies and simulations of specific systems and provides a theoretical basis for analyzing a class of systems that exhibit such inconsistent behavior. This result has implications in the simulation of infection, apoptosis, and population kinetics, as it can be shown that for certain models the stochastic simulations will always yield different predictions for the mean behavior than the deterministic simulations.     

Quasi-stationary and ratio of expectations distributions: A comparative study

Many stochastic systems, including biological applications, use Markov chains in which there is a set of absorbing states. It is then needed to consider analogs of the stationary distribution of an irreducible chain. In this context, quasi-stationary distributions play a fundamental role to describe the long-term behavior of the system. The rationale for using quasi-stationary distribution is well established in the abundant existing literature. The aim of this study is to reformulate the ratio of means approach ( [Darroch and Seneta, 1965] and [Darroch and Seneta, 1967]) which provides a simple alternative. We have a two-fold objective. The first objective is viewing quasi-stationarity and ratio of expectations as two different approaches for understanding the dynamics of the system before absorption. At this point, we remark that the quasi-stationary distribution and a ratio of means distribution may give or not give similar information. In this way, we arrive to the second objective; namely, to investigate the possibility of using the ratio of expectations distribution as an approximation to the quasi-stationary distribution. This second objective is explored by comparing both distributions in some selected scenarios, which are mainly inspired in stochastic epidemic models. Previously, the rate of convergence to the quasi-stationary regime is taking into account in order to make meaningful the comparison.     

Density functions of residence times for deterministic and stochastic compartmental systems

A significant consideration in modeling systems with stages is to obtain models for the individual stages that have probability density functions (pdfs) of residence times that are close to those of the real system. Consequently, the theory of residence time distributions is important for modeling. Here I show first that linear deterministic compartmental systems with constant coefficients and their corresponding stochastic analogs (stochastic compartmental systems with linear rate laws) have the same pdfs of residence times for the same initial distributions of inputs. Furthermore, these are independent of inflows. Then I show that does not hold for non-linear deterministic systems and their stochastic analogs (stochastic compartmental systems with non-linear rate laws). In fact, for given initial distributions of inputs, the pdfs of non-linear determistic systems without inflows and of their stochastic analogs, are functions of the initial amounts injected. For systems with inflows, the pdfs change as the inflows influence the occupancies of the compartments of the system; they are state-dependent pdfs.     

Relative phase dynamics in perturbed interlimb coordination: stability and stochasticity

 Various stability features of bimanual rhythmic coordination, including phase transitions, have been modeled successfully by means of a one-dimensional equation of motion for relative phase obeying a gradient dynamics, the Haken-Kelso-Bunz model. The present study aimed at assessing pattern stability for stationary performance and estimating the model parameters (a, b, and Q) for the stochastic extension of this model. Estimates of a and b allowed for reconstruction of the potential defining the gradient dynamics. Two coordination patterns between the forearms (in-phase, anti-phase) were performed at seven different frequencies. Model parameters were estimated on the basis of an exponential decay parameter describing the relaxation behavior of continuous relative phase following a mechanical perturbation. Variability of relative phase and relaxation time provided measures of pattern stability. Although the predicted inverse relation between pattern stability and movement frequency was observed for the lower tempo conditions, it was absent for the higher tempos, reflecting the influence of task constraints. No statistically significant differences in stability were observed between the two coordination modes, indicating the influence of intention. The reconstructed potential reflected the observed stability features, underscoring the adequacy of the parameter estimations. The relaxation process could not be captured adequately by means of a simple exponential decay function but required an additional oscillatory term. In accordance with previous assumptions, noise strength Q did not vary as a function of movement frequency. However, systematic differences in Q were observed between the two coordination modes. The advantages and (potential) pitfalls of using stationary performance of single patterns to examine the stability features of a bistable potential were discussed. Received: 12 July 1999 / Accepted in revised form: 14 April 2000     

Solving the chemical master equation for monomolecular reaction systems analytically

The stochastic dynamics of a well-stirred mixture of molecular species interacting through different biochemical reactions can be accurately modelled by the chemical master equation (CME). Research in the biology and scientific computing community has concentrated mostly on the development of numerical techniques to approximate the solution of the CME via many realizations of the associated Markov jump process. The domain of exact and/or efficient methods for directly solving the CME is still widely open, which is due to its large dimension that grows exponentially with the number of molecular species involved. In this article, we present an exact solution formula of the CME for arbitrary initial conditions in the case where the underlying system is governed by monomolecular reactions. The solution can be expressed in terms of the convolution of multinomial and product Poisson distributions with time-dependent parameters evolving according to the traditional reaction-rate equations. This very structured representation allows to deduce easily many properties of the solution. The model class includes many interesting examples. For more complex reaction systems, our results can be seen as a first step towards the construction of new numerical integrators, because solutions to the monomolecular case provide promising ansatz functions for Galerkin-type methods.     

Analyzing Short-Term Noise Dependencies of Spike-Counts in Macaque Prefrontal Cortex Using Copulas and the Flashlight Transformation

Simultaneous spike-counts of neural populations are typically modeled by a Gaussian distribution. On short time scales, however, this distribution is too restrictive to describe and analyze multivariate distributions of discrete spike-counts. We present an alternative that is based on copulas and can account for arbitrary marginal distributions, including Poisson and negative binomial distributions as well as second and higher-order interactions. We describe maximum likelihood-based procedures for fitting copula-based models to spike-count data, and we derive a so-called flashlight transformation which makes it possible to move the tail dependence of an arbitrary copula into an arbitrary orthant of the multivariate probability distribution. Mixtures of copulas that combine different dependence structures and thereby model different driving processes simultaneously are also introduced. First, we apply copula-based models to populations of integrate-and-fire neurons receiving partially correlated input and show that the best fitting copulas provide information about the functional connectivity of coupled neurons which can be extracted using the flashlight transformation. We then apply the new method to data which were recorded from macaque prefrontal cortex using a multi-tetrode array. We find that copula-based distributions with negative binomial marginals provide an appropriate stochastic model for the multivariate spike-count distributions rather than the multivariate Poisson latent variables distribution and the often used multivariate normal distribution. The dependence structure of these distributions provides evidence for common inhibitory input to all recorded stimulus encoding neurons. Finally, we show that copula-based models can be successfully used to evaluate neural codes, e.g., to characterize stimulus-dependent spike-count distributions with information measures. This demonstrates that copula-based models are not only a versatile class of models for multivariate distributions of spike-counts, but that those models can be exploited to understand functional dependencies.     

Expected Shannon Entropy and Shannon Differentiation between Subpopulations for Neutral Genes under the Finite Island Model

Shannon entropy H and related measures are increasingly used in molecular ecology and population genetics because (1) unlike measures based on heterozygosity or allele number, these measures weigh alleles in proportion to their population fraction, thus capturing a previously-ignored aspect of allele frequency distributions that may be important in many applications; (2) these measures connect directly to the rich predictive mathematics of information theory; (3) Shannon entropy is completely additive and has an explicitly hierarchical nature; and (4) Shannon entropy-based differentiation measures obey strong monotonicity properties that heterozygosity-based measures lack. We derive simple new expressions for the expected values of the Shannon entropy of the equilibrium allele distribution at a neutral locus in a single isolated population under two models of mutation: the infinite allele model and the stepwise mutation model. Surprisingly, this complex stochastic system for each model has an entropy expressable as a simple combination of well-known mathematical functions. Moreover, entropy- and heterozygosity-based measures for each model are linked by simple relationships that are shown by simulations to be approximately valid even far from equilibrium. We also identify a bridge between the two models of mutation. We apply our approach to subdivided populations which follow the finite island model, obtaining the Shannon entropy of the equilibrium allele distributions of the subpopulations and of the total population. We also derive the expected mutual information and normalized mutual information (“Shannon differentiation”) between subpopulations at equilibrium, and identify the model parameters that determine them. We apply our measures to data from the common starling (Sturnus vulgaris) in Australia. Our measures provide a test for neutrality that is robust to violations of equilibrium assumptions, as verified on real world data from starlings.     

Stochastic Computations in Cortical Microcircuit Models

Experimental data from neuroscience suggest that a substantial amount of knowledge is stored in the brain in the form of probability distributions over network states and trajectories of network states. We provide a theoretical foundation for this hypothesis by showing that even very detailed models for cortical microcircuits, with data-based diverse nonlinear neurons and synapses, have a stationary distribution of network states and trajectories of network states to which they converge exponentially fast from any initial state. We demonstrate that this convergence holds in spite of the non-reversibility of the stochastic dynamics of cortical microcircuits. We further show that, in the presence of background network oscillations, separate stationary distributions emerge for different phases of the oscillation, in accordance with experimentally reported phase-specific codes. We complement these theoretical results by computer simulations that investigate resulting computation times for typical probabilistic inference tasks on these internally stored distributions, such as marginalization or marginal maximum-a-posteriori estimation. Furthermore, we show that the inherent stochastic dynamics of generic cortical microcircuits enables them to quickly generate approximate solutions to difficult constraint satisfaction problems, where stored knowledge and current inputs jointly constrain possible solutions. This provides a powerful new computing paradigm for networks of spiking neurons, that also throws new light on how networks of neurons in the brain could carry out complex computational tasks such as prediction, imagination, memory recall and problem solving.     

Stochastic resonance in psychophysics and in animal behavior

 A recent analysis of the energy detector model in sensory psychophysics concluded that stochastic resonance does not occur in a measure of signal detectability (d′), but can occur in a percent-correct measure of performance as an epiphenomenon of nonoptimal criterion placement [Tougaard (2000) Biol Cybern 83: 471–480]. When generalized to signal detection in sensory systems in general, this conclusion is a serious challenge to the idea that stochastic resonance could play a significant role in sensory processing in humans and other animals. It also seems to be inconsistent with recent demonstrations of stochastic resonance in sensory systems of both nonhuman animals and humans using measures of system performance such as signal-to-noise ratio of power spectral densities and percent-correct detections in a two-interval forced-choice paradigm, both closely related to d′. In this paper we address this apparent dilemma by discussing several models of how stochastic resonance can arise in signal detection systems, including especially those that implement a “soft threshold” at the input transform stage. One example involves redefining d′ for energy increments in terms of parameters of the spike-count distribution of FitzHugh–Nagumo neurons. Another involves a Poisson spike generator that receives an exponentially transformed noisy periodic signal. In this case it can be shown that the signal-to-noise ratio of the power spectral density at the signal frequency, which exhibits stochastic resonance, is proportional to d′. Finally, a variant of d′ is shown to exhibit stochastic resonance when calculated directly from the distributions of power spectral densities at the signal frequency resulting from transformation of noise alone and a noisy signal by a sufficiently steep nonlinear response function. All of these examples, and others from the literature, imply that stochastic resonance is more than an epiphenomenon, although significant limitations to the extent to which adding noise can aid detection do exist. Received: 22 January 2001 / Accepted in revised form: 8 March 2002