Restricted transition probabilities and their applications to some problems in the dynamics of biological populations

In this paper a theory of a class of restricted transition probabilities is developed and applied to a problem in the dynamics of biological populations under the assumption that the underlying stochastic process is a continuous time parameter Markov chain with stationary transition probabilities. The paper is divided into three parts. Part one contains sufficient background from the theory of Markov processes to define restricted transition probabilities in a rigorous manner. In addition, some basic concepts in the theory of stochastic processes are interpreted from the biological point of view. Part two is concerned with the problem of finding representations for restricted transition probabilities. Finally, in part three the theory of restricted transition probabilities is applied to the problem of finding and analyzing some properties of the distribution function of the maximum size attained by the population in a finite time interval for a rather wide class of Markov processes. Some other applications of restricted transition probabilities to other problems in the dynamics of biological populations are also suggested. These applications will be discussed more fully in a companion paper. The research reported in this paper was supported by the United States Atomic Energy Commission, Division of Biology and Medicine Project AT(45-1)-1729.     

A neural network model for generating complex birdsong syntax

The singing behavior of songbirds has been investigated as a model of sequence learning and production. The song of the Bengalese finch, Lonchura striata var. domestica, is well described by a finite state automaton including a stochastic transition of the note sequence, which can be regarded as a higher-order Markov process. Focusing on the neural structure of songbirds, we propose a neural network model that generates higher-order Markov processes. The neurons in the robust nucleus of the archistriatum (RA) encode each note; they are activated by RA-projecting neurons in the HVC (used as a proper name). We hypothesize that the same note included in different chunks is encoded by distinct RA-projecting neuron groups. From this assumption, the output sequence of RA is a higher-order Markov process, even though the RA-projecting neurons in the HVC fire on first-order Markov processes. We developed a neural network model of the local circuits in the HVC that explains the mechanism by which RA-projecting neurons transit stochastically on first-order Markov processes. Numerical simulation showed that this model can generate first-order Markov process song sequences.     

A unifying framework for chaos and stochastic stability in discrete population models

 In this paper we propose a general framework for discrete time one-dimensional Markov population models which is based on two fundamental premises in population dynamics. We show that this framework incorporates both earlier population models, like the Ricker and Hassell models, and experimental observations concerning the structure of density dependence. The two fundamental premises of population dynamics are sufficient to guarantee that the model will exhibit chaotic behaviour for high values of the natural growth and the density-dependent feedback, and this observation is independent of the particular structure of the model. We also study these models when the environment of the population varies stochastically and address the question under what conditions we can find an invariant probability distribution for the population under consideration. The sufficient conditions for this stochastic stability that we derive are of some interest, since studying certain statistical characteristics of these stochastic population processes may only be possible if the process converges to such an invariant distribution. Received 15 May 1995; received in revised form 17 April 1996     

Non-homogeneous Markov processes for biomedical data analysis

Some time ago, the Markov processes were introduced in biomedical sciences in order to study disease history events. Homogeneous and Non-homogeneous Markov processes are an important field of research into stochastic processes, especially when exact transition times are unknown and interval-censored observations are present in the analysis. Non-homogeneous Markov process should be used when the homogeneous assumption is too strong. However these sorts of models increase the complexity of the analysis and standard software is limited. In this paper, some methods for fitting non-homogeneous Markov models are reviewed and an algorithm is proposed for biomedical data analysis. The method has been applied to analyse breast cancer data. Specific software for this purpose has been implemented.     

A Multi-stage Representation of Cell Proliferation as a Markov Process

The stochastic simulation algorithm commonly known as Gillespie’s algorithm (originally derived for modelling well-mixed systems of chemical reactions) is now used ubiquitously in the modelling of biological processes in which stochastic effects play an important role. In well-mixed scenarios at the sub-cellular level it is often reasonable to assume that times between successive reaction/interaction events are exponentially distributed and can be appropriately modelled as a Markov process and hence simulated by the Gillespie algorithm. However, Gillespie’s algorithm is routinely applied to model biological systems for which it was never intended. In particular, processes in which cell proliferation is important (e.g. embryonic development, cancer formation) should not be simulated naively using the Gillespie algorithm since the history-dependent nature of the cell cycle breaks the Markov process. The variance in experimentally measured cell cycle times is far less than in an exponential cell cycle time distribution with the same mean.Here we suggest a method of modelling the cell cycle that restores the memoryless property to the system and is therefore consistent with simulation via the Gillespie algorithm. By breaking the cell cycle into a number of independent exponentially distributed stages, we can restore the Markov property at the same time as more accurately approximating the appropriate cell cycle time distributions. The consequences of our revised mathematical model are explored analytically as far as possible. We demonstrate the importance of employing the correct cell cycle time distribution by recapitulating the results from two models incorporating cellular proliferation (one spatial and one non-spatial) and demonstrating that changing the cell cycle time distribution makes quantitative and qualitative differences to the outcome of the models. Our adaptation will allow modellers and experimentalists alike to appropriately represent cellular proliferation—vital to the accurate modelling of many biological processes—whilst still being able to take advantage of the power and efficiency of the popular Gillespie algorithm.     

T helper cell polarization and the generation of varied antigen-dependent immune responses: a Markov chain model.

BACKGROUND: Adaptive immune responses are deterministically classified into humoral or cell-mediated depending on the pattern of Th cell polarization into Th1 or Th2. Evidence suggests that the process of Th polarization is stochastic, however, the presence of some deterministic components has not been ruled out. Here, a Markov chain model that accounts for Th-mediated immune responses was developed based on the assumption that Th polarization and consequent transition events are stochastic. RESULTS: Using assumed probability values, model analysis suggests that there is a rapid convergence to produce an immune response once the Th cell is stimulated by an antigen which is amplified as the number of transitions increases. The expected number of visits between Th and itself, B and itself and Tc and itself is about one whereas it is zero, less than one or degrees in the rest of the transition events depending on the interacting states. CONCLUSIONS: Based on model analysis and validation, modeling Th-mediated immune responses as a Markov chain process seems to be plausible. The large degree of flexibility inherent in such a view of adaptive immunity can be helpful in addressing questions pertinent to Th function and behavior.     

Analysing grouping of nucleotides in DNA sequences using lumped processes constructed from Markov chains

The most commonly used models for analysing local dependencies in DNA sequences are (high-order) Markov chains. Incorporating knowledge relative to the possible grouping of the nucleotides enables to define dedicated sub-classes of Markov chains. The problem of formulating lumpability hypotheses for a Markov chain is therefore addressed. In the classical approach to lumpability, this problem can be formulated as the determination of an appropriate state space (smaller than the original state space) such that the lumped chain defined on this state space retains the Markov property. We propose a different perspective on lumpability where the state space is fixed and the partitioning of this state space is represented by a one-to-many probabilistic function within a two-level stochastic process. Three nested classes of lumped processes can be defined in this way as sub-classes of first-order Markov chains. These lumped processes enable parsimonious reparameterizations of Markov chains that help to reveal relevant partitions of the state space. Characterizations of the lumped processes on the original transition probability matrix are derived. Different model selection methods relying either on hypothesis testing or on penalized log-likelihood criteria are presented as well as extensions to lumped processes constructed from high-order Markov chains. The relevance of the proposed approach to lumpability is illustrated by the analysis of DNA sequences. In particular, the use of lumped processes enables to highlight differences between intronic sequences and gene untranslated region sequences.     

Properties of membrane stationary states

Summary The flux of permeant species through a membrane is examined using discrete state stochastic models for the transport process within the membrane. While a membrane flux is maintained due to a concentration gradient between bathing solutions, the distribution of species within the membrane evolves to a time invariant configuration which can differ significantly from the equilibrium configuration. Some special properties of these stationary states are examined using linear, microcanonical models for the membrane transport process. Analysis of these models reveals properties which are masked by the phenomenological analysis of irreversible thermodynamics. For example, the models can be used to study the nature of multi-state relaxation within the membrane by observation of the time dependence of the net membrane flux when the membrane is perturbed from its stationary state distribution. Under some conditions, multi-state models will produce relaxation similar to that observed for single-state processes. The symmetry within the membrane is a critical factor for monitoring relaxation processes within the membrane. Because of the stationary nature of the membrane configuration, statistical thermodynamic variables can be defined for the membrane configuration. The total system is not in equilibrium since the baths must still be described by dissipation functions. In the stationary state, the configurational entropy of the membrane is lowered relative to equilibrium and is shown to depend quadratically on the time independent parameter (j/p) wherej is the membrane flux andp is a characteristic transition probability for intra-membrane transitions. The basic membrane system serves as a quantitative example of the entropy reduction possible in a stationary state system. An allosteric transition mediated by the stationary state configuration is examined as a means of utilizing this negentropy production.     

Timing of Pathogen Adaptation to a Multicomponent Treatment

The sustainable use of multicomponent treatments such as combination therapies, combination vaccines/chemicals, and plants carrying multigenic resistance requires an understanding of how their population-wide deployment affects the speed of the pathogen adaptation. Here, we develop a stochastic model describing the emergence of a mutant pathogen and its dynamics in a heterogeneous host population split into various types by the management strategy. Based on a multi-type Markov birth and death process, the model can be used to provide a basic understanding of how the life-cycle parameters of the pathogen population, and the controllable parameters of a management strategy affect the speed at which a pathogen adapts to a multicomponent treatment. Our results reveal the importance of coupling stochastic mutation and migration processes, and illustrate how their stochasticity can alter our view of the principles of managing pathogen adaptive dynamics at the population level. In particular, we identify the growth and migration rates that allow pathogens to adapt to a multicomponent treatment even if it is deployed on only small proportions of the host. In contrast to the accepted view, our model suggests that treatment durability should not systematically be identified with mutation cost. We show also that associating a multicomponent treatment with defeated monocomponent treatments can be more durable than associating it with intermediate treatments including only some of the components. We conclude that the explicit modelling of stochastic processes underlying evolutionary dynamics could help to elucidate the principles of the sustainable use of multicomponent treatments in population-wide management strategies intended to impede the evolution of harmful populations.     

Genetic drift in an infinite population. The pseudohitchhiking model

Selected substitutions at one locus can induce stochastic dynamics that resemble genetic drift at a closely linked neutral locus. The pseudohitchhiking model is a one-locus model that approximates these effects and can be used to describe the major consequences of linked selection. As the changes in neutral allele frequencies when hitchhiking are rapid, diffusion theory is not appropriate for studying neutral dynamics. A stationary distribution and some results on substitution processes are presented that use the theory of continuous-time Markov processes with discontinuous sample paths. The coalescent of the pseudohitchhiking model is shown to have a random number of branches at each node, which leads to a frequency spectrum that is different from that of the equilibrium neutral model. If genetic draft, the name given to these induced stochastic effects, is a more important stochastic force than genetic drift, then a number of paradoxes that have plagued population genetics disappear.     

Time-varying Markov regression random-effect model with Bayesian estimation procedures: Application to dynamics of functional recovery in patients with stroke

The rates of functional recovery after stroke tend to decrease with time. Time-varying Markov processes (TVMP) may be more biologically plausible than time-invariant Markov process for modeling such data. However, analysis of such stochastic processes, particularly tackling reversible transitions and the incorporation of random effects into models, can be analytically intractable. We make use of ordinary differential equations to solve continuous-time TVMP with reversible transitions. The proportional hazard form was used to assess the effects of an individual’s covariates on multi-state transitions with the incorporation of random effects that capture the residual variation after being explained by measured covariates under the concept of generalized linear model. We further built up Bayesian directed acyclic graphic model to obtain full joint posterior distribution. Markov chain Monte Carlo (MCMC) with Gibbs sampling was applied to estimate parameters based on posterior marginal distributions with multiple integrands. The proposed method was illustrated with empirical data from a study on the functional recovery after stroke.     

Weak convergence of a sequence of stochastic difference equations to a stochastic ordinary differential equation

We consider a sequence of discrete parameter stochastic processes defined by solutions to stochastic difference equations. A condition is given that this sequence converges weakly to a continuous parameter process defined by solutions to a stochastic ordinary differential equation. Applying this result, two limit theorems related to population biology are proved. Random parameters in stochastic difference equations are autocorrelated stationary Gaussian processes in the first case. They are jump-type Markov processes in the second case. We discuss a problem of continuous time approximations for discrete time models in random environments.     

Rare Dissipative Transitions Punctuate the Initiation of Chemical Denaturation in Proteins

Protein unfolding dynamics are bound by their degree of entropy production, a quantity that relates the amount of heat dissipated by a nonequilibrium process to a system’s forward and time-reversed trajectories. We here explore the statistics of heat dissipation that emerge in protein molecules subjected to a chemical denaturant. Coupling large molecular dynamics datasets and Markov state models with the theory of entropy production, we demonstrate that dissipative processes can be rigorously characterized over the course of the urea-induced unfolding of the protein chymotrypsin inhibitor 2. By enumerating full entropy production probability distributions as a function of time, we first illustrate that distinct passive and dissipative regimes are present in the denaturation dynamics. Within the dissipative dynamical region, we next find that chymotrypsin inhibitor 2 is strongly driven into unfolded states in which the protein’s hydrophobic core has been penetrated by urea molecules and disintegrated. Detailed analyses reveal that urea’s interruption of key hydrophobic contacts between core residues causes many of the protein’s native structural features to dissolve.     

Markov process of maintained impulse activity in central single neurons

To clarify the stochast properties of the maintained impulse activity of the central nervous system, we proposed a measure of statistical dependency on the basis of Shannon's entropy. This measure could provide the Markov properties of the neural impulse sequences, representing the necessary and sufficient condition for the statistical dependence. The order of Markov process of the sequence is determined by the conditional entropy which is derived from the joint entropy. Here the joint entropy in the case of Gaussian process is directly related with the covariance matrix which is substituted for the matrix of the serial correlation coefficients. Therefore the condition to determine the order of Markov process is obtained by the equation of the matrices of the serial correlation coefficients. The order of Markov process of the neural impulse sequences recorded from the mesencephalic reticular formation (MRF), red nucleus (RN), and lateral geniculate nucleus (LGN) neurons has been estimated. The maintained impulse activity of the MRF and RN neurons had from the 2-nd to 4-th order Markov property, while that of the LGN had no Markov property, in the consecutive impulse sequences.     

Graphs, random sums, and sojourn time distributions, with application to ion-channel modeling.

This paper considers the distribution of a sojourn time in a class of states of a stochastic process having finite discrete state space where sojourn times in any individual state are independent and identically distributed, and transitions between states follow a Markov chain. The state space and possible transitions of the process are represented by a graph. Class sojourn time distributions are derived by modifying this graph using 'composition' of states, defining a new Markov chain on the modified graph, and expressing the sojourn time in a composition state as a random sum. Appropriate compositions are chosen according to the possible "cores" of sojourns in the particular class, where a core describes the structure of a sojourn in terms of a single state or a chain in the original graph. Graph methods provide an algorithmic basis for the derivation, which can be simplified by using symmetry results. Models of ion-channel kinetics are used throughout for illustration; class sojourn time distributions are important in such models because individual states are often indistinguishable experimentally. Markov processes are the special case where sojourn times in individual states are exponentially distributed. In this case kinetic parameter estimation based on the observed class sojourn time distribution is briefly discussed; explicit estimating equations applicable to sequential models of nicotinic receptor kinetics are given.     

Bayesian selection of markov models for symbol sequences: application to microsaccadic eye movements

Complex biological dynamics often generate sequences of discrete events which can be described as a Markov process. The order of the underlying Markovian stochastic process is fundamental for characterizing statistical dependencies within sequences. As an example for this class of biological systems, we investigate the Markov order of sequences of microsaccadic eye movements from human observers. We calculate the integrated likelihood of a given sequence for various orders of the Markov process and use this in a Bayesian framework for statistical inference on the Markov order. Our analysis shows that data from most participants are best explained by a first-order Markov process. This is compatible with recent findings of a statistical coupling of subsequent microsaccade orientations. Our method might prove to be useful for a broad class of biological systems.     

Bayesian restoration of a hidden Markov chain with applications to DNA sequencing.

Hidden Markov models (HMMs) are a class of stochastic models that have proven to be powerful tools for the analysis of molecular sequence data. A hidden Markov model can be viewed as a black box that generates sequences of observations. The unobservable internal state of the box is stochastic and is determined by a finite state Markov chain. The observable output is stochastic with distribution determined by the state of the hidden Markov chain. We present a Bayesian solution to the problem of restoring the sequence of states visited by the hidden Markov chain from a given sequence of observed outputs. Our approach is based on a Monte Carlo Markov chain algorithm that allows us to draw samples from the full posterior distribution of the hidden Markov chain paths. The problem of estimating the probability of individual paths and the associated Monte Carlo error of these estimates is addressed. The method is illustrated by considering a problem of DNA sequence multiple alignment. The special structure for the hidden Markov model used in the sequence alignment problem is considered in detail. In conclusion, we discuss certain interesting aspects of biological sequence alignments that become accessible through the Bayesian approach to HMM restoration.     

Bringing consistency to simulation of population models--Poisson simulation as a bridge between micro and macro simulation

Population models concern collections of discrete entities such as atoms, cells, humans, animals, etc., where the focus is on the number of entities in a population. Because of the complexity of such models, simulation is usually needed to reproduce their complete dynamic and stochastic behaviour. Two main types of simulation models are used for different purposes, namely micro-simulation models, where each individual is described with its particular attributes and behaviour, and macro-simulation models based on stochastic differential equations, where the population is described in aggregated terms by the number of individuals in different states. Consistency between micro- and macro-models is a crucial but often neglected aspect. This paper demonstrates how the Poisson Simulation technique can be used to produce a population macro-model consistent with the corresponding micro-model. This is accomplished by defining Poisson Simulation in strictly mathematical terms as a series of Poisson processes that generate sequences of Poisson distributions with dynamically varying parameters. The method can be applied to any population model. It provides the unique stochastic and dynamic macro-model consistent with a correct micro-model. The paper also presents a general macro form for stochastic and dynamic population models. In an appendix Poisson Simulation is compared with Markov Simulation showing a number of advantages. Especially aggregation into state variables and aggregation of many events per time-step makes Poisson Simulation orders of magnitude faster than Markov Simulation. Furthermore, you can build and execute much larger and more complicated models with Poisson Simulation than is possible with the Markov approach.     

DNA damage in non-proliferating cells subjected to ionizing irradiation at high or low dose rates

Ionizing radiation damage to the genome of a non-cycling mammalian cell is analyzed using continuous time Markov chains. Immediate damage induced by the radiation is modeled as a batch Poisson arrival process of DNA double strand breaks (DSBs). Different kinds of radiation, for example gamma rays or alpha particles, have different batch probabilities. Enzymatic modulation of the immediate damage is modeled as a Markov process similar to the processes described by the master equation of stochastic chemical kinetics. An illustrative example is the restitution/complete exchange model, which postulates that radiation induced DSBs can subsequently either undergo enzymatically mediated repair (restitution) or can participate pairwise in chromosome exchanges, some of which make irremediable lesions such as dicentric chromosome aberrations. One may have rapid irradiation followed by enzymatic DSB processing or have prolonged irradiation with both DSB arrival and enzymatic DSB processing continuing throughout the irradiation period. A complete solution of the Markov chain is known for the case that the exchange rate constant is negligible so that no irremediable chromosome lesions are produced and DSBs are the only damage to the genome. Using PDEs for generating functions, a perturbation calculation is made assuming the exchange rate constant is small compared to the repair rate constant. Some non-perturbative results applicable to very prolonged irradiation are also obtained using matrix methods: Perron-Frobenius theory, variational methods and numerical approximations of eigenvalues. Applications to experimental results on expected values, variances and statistical distributions of DNA lesions are briefly outlined.Continuous time Markov chain models are the most systematic of those current radiation damage models which treat DSB-DSB interactions within the cell nucleus as homogeneous (e.g. ignore diffusion limitations). They contain most other homogeneous models as special cases, limiting cases or approximations. However, applying the continuous time Markov chain models to studying spatial dependence of DSB interactions, which is generally believed to be very important in some situations, presents difficulties.     

Entropy and convergence in dynamics and demography

Demographic dynamics is formally equivalent to the dynamics of a Markov chain, as is true of some nonlinear dynamical systems. Convergence to demographic equilibrium can be studied in terms of convergence in the Markov chain. Tuljapurkar (1982) showed that population entropy (Kolmogorov-Sinai entropy) provides information on the rate of this convergence. This paper begins by considering finite state Markov chains, providing elementary proofs of the relationship between convergence rate and entropy, and discusses in detail the uses and limitations of entropy as a convergence measure; these results also apply to Markovian dynamical systems. Next, new qualitative and quantitative arguments are used to discuss the demographic meaning of entropy. An exact relationship is established giving population entropy in terms of the eigenvalues of the Leslie matrix characteristic equation. Finally, the significance of imprimitive and periodic limits is discussed in relation to population entropy.