Semi‐Markov Models with Phase‐Type Sojourn Distributions

Summary Continuous‐time multistate models are widely used for categorical response data, particularly in the modeling of chronic diseases. However, inference is difficult when the process is only observed at discrete time points, with no information about the times or types of events between observation times, unless a Markov assumption is made. This assumption can be limiting as rates of transition between disease states might instead depend on the time since entry into the current state. Such a formulation results in a semi‐Markov model. We show that the computational problems associated with fitting semi‐Markov models to panel‐observed data can be alleviated by considering a class of semi‐Markov models with phase‐type sojourn distributions. This allows methods for hidden Markov models to be applied. In addition, extensions to models where observed states are subject to classification error are given. The methodology is demonstrated on a dataset relating to development of bronchiolitis obliterans syndrome in post‐lung‐transplantation patients.     

Applications of hidden hybrid Markov/semi‐Markov models: from stopover duration to breeding success dynamics

Usually in capture–recapture, a model parameter is time or time since first capture dependent. However, the case where the probability of staying in one state depends on the time spent in that particular state is not rare. Hidden Markov models are not appropriate to manage these situations. A more convenient approach would be to consider models that incorporate semi‐Markovian states which explicitly define the waiting time distribution and have been used in previous biologic studies as a convenient framework for modeling the time spent in a given physiological state. Here, we propose hidden Markovian models that combine several nonhomogeneous Markovian states with one semi‐Markovian state and which (i) are well adapted to imperfect and variable detection and (ii) allow us to consider time, time since first capture, and time spent in one state effects. Implementation details depending on the number of semi‐Markovian states are discussed. From a user's perspective, the present approach enhances the toolbox for analyzing capture–recapture data. We then show the potential of this framework by means of two ecological examples: (i) stopover duration and (ii) breeding success dynamics.     

Continuous time Markov models for binary longitudinal data

Longitudinal data usually consist of a number of short time series. A group of subjects or groups of subjects are followed over time and observations are often taken at unequally spaced time points, and may be at different times for different subjects. When the errors and random effects are Gaussian, the likelihood of these unbalanced linear mixed models can be directly calculated, and nonlinear optimization used to obtain maximum likelihood estimates of the fixed regression coefficients and parameters in the variance components. For binary longitudinal data, a two state, non-homogeneous continuous time Markov process approach is used to model serial correlation within subjects. Formulating the model as a continuous time Markov process allows the observations to be equally or unequally spaced. Fixed and time varying covariates can be included in the model, and the continuous time model allows the estimation of the odds ratio for an exposure variable based on the steady state distribution. Exact likelihoods can be calculated. The initial probability distribution on the first observation on each subject is estimated using logistic regression that can involve covariates, and this estimation is embedded in the overall estimation. These models are applied to an intervention study designed to reduce children's sun exposure.     

Time-varying Markov regression random-effect model with Bayesian estimation procedures: Application to dynamics of functional recovery in patients with stroke

The rates of functional recovery after stroke tend to decrease with time. Time-varying Markov processes (TVMP) may be more biologically plausible than time-invariant Markov process for modeling such data. However, analysis of such stochastic processes, particularly tackling reversible transitions and the incorporation of random effects into models, can be analytically intractable. We make use of ordinary differential equations to solve continuous-time TVMP with reversible transitions. The proportional hazard form was used to assess the effects of an individual’s covariates on multi-state transitions with the incorporation of random effects that capture the residual variation after being explained by measured covariates under the concept of generalized linear model. We further built up Bayesian directed acyclic graphic model to obtain full joint posterior distribution. Markov chain Monte Carlo (MCMC) with Gibbs sampling was applied to estimate parameters based on posterior marginal distributions with multiple integrands. The proposed method was illustrated with empirical data from a study on the functional recovery after stroke.     

Deciphering the developmental plasticity of walnut saplings in relation to climatic factors and light environment

Developmental plasticity, the acclimation of plants to their local environment, is known to be crucial for the fitness of perennial organisms such as trees. However, deciphering the many possible developmental and environmental influences involved in such plasticity in natural conditions requires dedicated statistical models integrating developmental phases, environmental factors, and interindividual heterogeneity. These models should be able to analyse retrospective data (number of leaves or length of annual shoots along the main stem in the present case). In this study Markov switching linear mixed models were applied to the analysis of the developmental plasticity of walnut saplings during the establishment phase in a mixed Mediterranean forest. In the Markov switching linear mixed models estimated from walnut data sets, the underlying Markov chain represents both the succession and lengths of growth phases, while the linear mixed models represent both the influence of climatic factors and interindividual heterogeneity within each growth phase. On the basis of these integrative statistical models, it is shown that walnut saplings have an opportunistic mode of development that is primarily driven by the changing light environment. In particular, light availability explains the ability of a tree to reach a phase of strong growth where the first branches can appear. It is also shown that growth fluctuation amplitudes in response to climatic factors increased while interindividual heterogeneity decreased during tree development.     

Identifying ontogenetic, environmental and individual components of forest tree growth

Background and Aims

This study aimed to identify and characterize the ontogenetic, environmental and individual components of forest tree growth. In the proposed approach, the tree growth data typically correspond to the retrospective measurement of annual shoot characteristics (e.g. length) along the trunk.

Methods

Dedicated statistical models (semi-Markov switching linear mixed models) were applied to data sets of Corsican pine and sessile oak. In the semi-Markov switching linear mixed models estimated from these data sets, the underlying semi-Markov chain represents both the succession of growth phases and their lengths, while the linear mixed models represent both the influence of climatic factors and the inter-individual heterogeneity within each growth phase.

Key Results

On the basis of these integrative statistical models, it is shown that growth phases are not only defined by average growth level but also by growth fluctuation amplitudes in response to climatic factors and inter-individual heterogeneity and that the individual tree status within the population may change between phases. Species plasticity affected the response to climatic factors while tree origin, sampling strategy and silvicultural interventions impacted inter-individual heterogeneity.

Conclusions

The transposition of the proposed integrative statistical modelling approach to cambial growth in relation to climatic factors and the study of the relationship between apical growth and cambial growth constitute the next steps in this research.     

A conditional Markov model for clustered progressive multistate processes under incomplete observation

Clustered progressive chronic disease processes arise when interest lies in modeling damage in paired organ systems (e.g., kidneys, eyes), in diseases manifest in different organ systems, or in systemic conditions for which damage may occur in several locations of the body. Multistate Markov models have considerable appeal for modeling damage in such settings, particularly when patients are only under intermittent observation. Generalizations are necessary, however, to deal with the fact that processes within subjects may not be independent. We describe a conditional Markov model in which the clustering in processes within subjects is addressed by the use of multiplicative random effects for each transition intensity. The random effects for the different transition intensities may be correlated within subjects, but are assumed to be independent for different subjects. We apply the mixed Markov model to a motivating data set of patients with psoriatic arthritis, and characterize the progressive course of damage in joints of the hand. A generalization to accommodate a subpopulation of "stayers" and extensions which facilitate regression are indicated and illustrated.     

A semi-Markov model based on generalized Weibull distribution with an illustration for HIV disease

Multi-state stochastic models are useful tools for studying complex dynamics such as chronic diseases. Semi-Markov models explicitly define distributions of waiting times, giving an extension of continuous time and homogeneous Markov models based implicitly on exponential distributions. This paper develops a parametric model adapted to complex medical processes. (i) We introduced a hazard function of waiting times with a U or inverse U shape. (ii) These distributions were specifically selected for each transition. (iii) The vector of covariates was also selected for each transition. We applied this method to the evolution of HIV infected patients. We used a sample of 1244 patients followed up at the hospital in Nice, France.     

Spatio-temporal variation in Markov chain models of subtidal community succession

In this paper we ask whether succession in a rocky subtidal community varies in space and time, and if so how much affect that variation has on predictions of community dynamics and structure. We describe succession by Markov chain models based on observed frequencies of species replacements. We use loglinear analysis to detect and quantify spatio‐temporal variation in the transition matrices describing succession. The analysis shows that space and time, but not their interaction, have highly significant effects on transition probabilities. To explore the ecological importance of the spatio‐temporal variability detected in this analysis, we compare the equilibria and the transient dynamics among three Markov chain models: a time‐averaged model that includes the effects of space on succession, a spatially averaged model that include the effects of time, and a constant matrix that averages over the effects of space and time. All three models predicted similar equilibrium composition and similar rates of convergence to equilibrium, as measured by the damping ratio or the subdominant Lyapunov exponent. The predicted equilibria from all three models were very similar to the observed community structure. Thus, although spatial and temporal variation is statistically significant, at least in this system this variation does not prevent homogeneous models from predicting community structure.     

Likelihood-ratio tests for hidden Markov models

We consider hidden Markov models as a versatile class of models for weakly dependent random phenomena. The topic of the present paper is likelihood-ratio testing for hidden Markov models, and we show that, under appropriate conditions, the standard asymptotic theory of likelihood-ratio tests is valid. Such tests are crucial in the specification of multivariate Gaussian hidden Markov models, which we use to illustrate the applicability of our general results. Finally, the methodology is illustrated by means of a real data set.     

Markov modelling of immunological and virological states in HIV-1 infected patients

The purpose of this study was to evaluate the evolution of HIV infected patients and to bring out some significant factors associated with this pathology. The main criteria revealing the State of illness is viral load measurement (VL). However the CD4 lymphocytes also represent an important marker as these reflect the State of the immune reservoir. Many studies have been carried out in this field and different models have been proposed with a view to a better understanding of this disease. Multi State Markov models defined in terms of CD4 counts, or in terms of viral load, have proved to be very useful tools for modelling HIV disease progression. The model we have developed in this study is based on both the CD4 lymphocytes counts and VL. Markov models are characterized by transition intensities. In this paper we explored several structures in succession. First, we used a homogeneous continuous time Markov process with four states defined by crossed values of CD4 and VL in a given patient at a given time. Then, the effect of certain covariates on the infection process was introduced into the model via the transition intensity functions, as with a Cox regression model. Since the hypothesis of homogeneity may be unrealistic in certain cases, we also considered piecewise homogeneous Markov models. Finally, the effects of covariates and time were combined in a piecewise homogeneous model with a covariate. We applied these methods to data from 1313 HIV-infected patients included in the NADIS cohort.     

Hierarchical Bayesian modeling of random and residual variance–covariance matrices in bivariate mixed effects models

Bivariate mixed effects models are often used to jointly infer upon covariance matrices for both random effects ( u ) and residuals ( e ) between two different phenotypes in order to investigate the architecture of their relationship. However, these (co)variances themselves may additionally depend upon covariates as well as additional sets of exchangeable random effects that facilitate borrowing of strength across a large number of clusters. We propose a hierarchical Bayesian extension of the classical bivariate mixed effects model by embedding additional levels of mixed effects modeling of reparameterizations of u‐ level and e ‐level (co)variances between two traits. These parameters are based upon a recently popularized square‐root‐free Cholesky decomposition and are readily interpretable, each conveniently facilitating a generalized linear model characterization. Using Markov Chain Monte Carlo methods, we validate our model based on a simulation study and apply it to a joint analysis of milk yield and calving interval phenotypes in Michigan dairy cows. This analysis indicates that the e ‐level relationship between the two traits is highly heterogeneous across herds and depends upon systematic herd management factors.     

Analysis of Longitudinal Data of Epileptic Seizure Counts – A Two‐State Hidden Markov Regression Approach

This paper discusses a two‐state hidden Markov Poisson regression (MPR) model for analyzing longitudinal data of epileptic seizure counts, which allows for the rate of the Poisson process to depend on covariates through an exponential link function and to change according to the states of a two‐state Markov chain with its transition probabilities associated with covariates through a logit link function. This paper also considers a two‐state hidden Markov negative binomial regression (MNBR) model, as an alternative, by using the negative binomial instead of Poisson distribution in the proposed MPR model when there exists extra‐Poisson variation conditional on the states of the Markov chain. The two proposed models in this paper relax the stationary requirement of the Markov chain, allow for overdispersion relative to the usual Poisson regression model and for correlation between repeated observations. The proposed methodology provides a plausible analysis for the longitudinal data of epileptic seizure counts, and the MNBR model fits the data much better than the MPR model. Maximum likelihood estimation using the EM and quasi‐Newton algorithms is discussed. A Monte Carlo study for the proposed MPR model investigates the reliability of the estimation method, the choice of probabilities for the initial states of the Markov chain, and some finite sample behaviors of the maximum likelihood estimates, suggesting that (1) the estimation method is accurate and reliable as long as the total number of observations is reasonably large, and (2) the choice of probabilities for the initial states of the Markov process has little impact on the parameter estimates.     

Markov models for covariate dependence of binary sequences

Suppose that a heterogeneous group of individuals is followed over time and that each individual can be in state 0 or state 1 at each time point. The sequence of states is assumed to follow a binary Markov chain. In this paper we model the transition probabilities for the 0 to 0 and 1 to 0 transitions by two logistic regressions, thus showing how the covariates relate to changes in state. With p covariates, there are 2(p + 1) parameters including intercepts, which we estimate by maximum likelihood. We show how to use transition probability estimates to test hypotheses about the probability of occupying state 0 at time i (i = 2, ..., T) and the equilibrium probability of state 0. These probabilities depend on the covariates. A recursive algorithm is suggested to estimate regression coefficients when some responses are missing. Extensions of the basic model which allow time-dependent covariates and nonstationary or second-order Markov chains are presented. An example shows the model applied to a study of the psychological impact of breast cancer in which women did or did not manifest distress at four time points in the year following surgery.     

In silico exploration of the effects of host genotype and nutrition on the genetic parameters of lambs challenged with gastrointestinal parasites

An in silico mathematical model was used to explore the effect of, and the interaction between, (i) nutrition, (ii) genotype for growth and (iii) genotype for resistance, on the estimates of genetic parameters for resistance and performance in a population of lambs trickle-challenged daily with 3,000 L3s of Teladorsagia circumcincta. A previously published model for nematode infections in sheep was developed to include heritable variation in sheep growth traits, as well as in immunologically controlled traits such as establishment of incoming larvae, mortality of the adult worms and fecundity of the adult female worms. The simulated population comprised 10,000 lambs, these being the offspring of 250 sires mated to 5,000 dams. The model assumed the lambs to be parasitologically naïve at weaning (2 months of age), at which point the trickle challenge commenced and the model was updated daily until slaughter (at 6 months of age). Dietary treatments included a good and a poor quality feed, offered ad libitum. Two genotypes for growth were assumed: (i) fast and (ii) slow growing. Three genotypes for resistance were used: (i) benchmark, (ii) susceptible and (iii) resistant, differing in their ability to cope with nematode infections. Genetic parameters for output traits, including growth rate, food intake, worm burden and faecal egg count were estimated using a linear mixed model, fitting sire as a random effect to capture genetic effects. Heritabilities and correlations were found to change over time. In general, the heritabilities of immunity traits increased over time, whereas genetic correlations between production and immunity traits became weaker. Diet had a significant effect on the means and the estimated correlations of output traits, while genotypes for growth and resistance had smaller effects. These results suggest that discrepancies between published genetic parameters for nematode resistance may be a function of environmental factors rather than differences in host genotype.     

Seed banks of temperate deciduous forests during secondary succession

Question: (i) How does former land use and land use intensity affect seed bank development during post‐agricultural succession? (ii) How does time since the last clear‐cut change seed bank composition during post‐clear‐cut succession? Methods: One data set was compiled per succession type using the following selection criteria: (i) the data set included a successional series, (ii) plots were located in mesotrophic forest plant communities and (iii) vegetation data were available. The post‐agricultural succession data set comprised 76 recent forest plots (eight studies); the post‐clear‐cut succession data set comprised 218 ancient forest plots (three studies). Each data set was analysed separately using either linear mixed models or generalized linear models, controlling for both environmental heterogeneity and variation between study locations. Results: In the post‐agricultural succession data set, land use and time significantly affected nearly all the studied seed bank characteristics. Seed banks on former arable land recovered poorly even after 150 year of restored forest cover, whereas moderate land use intensities (grasslands, heathlands) yielded more rapid seed bank recovery. Time was a significant determinant of all but two soil seed bank characteristics during post‐clear‐cut succession. Seed banks in managed ancient forest differed strongly in their characteristics compared to primary forest seed banks. Conclusions: Forest seed banks bear the marks of former land use and/or forest management and continue to do so for at least 150 years. Nevertheless, time since the last major disturbance, being either former land use or clear‐cutting, remains a significant determinant of the seed bank.     

Multivariate Markov process models for the transmission of methicillin-resistant Staphylococcus aureus in a hospital ward

Summary .   Methicillin-resistant Staphylococcus Aureus (MRSA) is a pathogen that continues to be of major concern in hospitals. We develop models and computational schemes based on observed weekly incidence data to estimate MRSA transmission parameters. We extend the deterministic model of McBryde, Pettitt, and McElwain (2007, Journal of Theoretical Biology 245, 470–481) involving an underlying population of MRSA colonized patients and health-care workers that describes, among other processes, transmission between uncolonized patients and colonized health-care workers and vice versa. We develop new bivariate and trivariate Markov models to include incidence so that estimated transmission rates can be based directly on new colonizations rather than indirectly on prevalence. Imperfect sensitivity of pathogen detection is modeled using a hidden Markov process. The advantages of our approach include (i) a discrete valued assumption for the number of colonized health-care workers, (ii) two transmission parameters can be incorporated into the likelihood, (iii) the likelihood depends on the number of new cases to improve precision of inference, (iv) individual patient records are not required, and (v) the possibility of imperfect detection of colonization is incorporated. We compare our approach with that used by McBryde et al. (2007) based on an approximation that eliminates the health-care workers from the model, uses Markov chain Monte Carlo and individual patient data. We apply these models to MRSA colonization data collected in a small intensive care unit at the Princess Alexandra Hospital, Brisbane, Australia.     

Robust Linear Mixed Models with Normal/Independent Distributions and Bayesian MCMC Implementation

Linear mixed effects models have been widely used in analysis of data where responses are clustered around some random effects, so it is not reasonable to assume independence between observations in the same cluster. In most biological applications, it is assumed that the distributions of the random effects and of the residuals are Gaussian. This makes inferences vulnerable to the presence of outliers. Here, linear mixed effects models with normal/independent residual distributions for robust inferences are described. Specific distributions examined include univariate and multivariate versions of the Student‐ t, the slash and the contaminated normal. A Bayesian framework is adopted and Markov chain Monte Carlo is used to carry out the posterior analysis. The procedures are illustrated using birth weight data on rats in a toxicological experiment. Results from the Gaussian and robust models are contrasted, and it is shown how the implementation can be used for outlier detection. The thick‐tailed distributions provide an appealing robust alternative to the Gaussian process in linear mixed models, and they are easily implemented using data augmentation and MCMC techniques.     

Markov chain approach to analyze the dynamics of pathogen fecal shedding--example of Listeria monocytogenes shedding in a herd of dairy cattle

Fecal shedding is an important mechanism of spreading of a number of human and animal pathogens. Understanding of the dynamics of pathogen fecal shedding is critical to be able to control or prevent the spread of diseases caused by these pathogens. The objective of this study was to develop a model for analysis of the dynamics of pathogen fecal shedding. Fecal shedding of Listeria monocytogenes in dairy cattle was used as a model system. A Markov chain model (MCM) with two states, shedding and non-shedding, has been developed for overall L. monocytogenes fecal shedding (all L. monocytogenes subtypes) and fecal shedding of three L. monocytogenes subtypes (ribotypes 1058A, 1039E and 1042B) using data from one study farm. The matrices of conditional probabilities of transition between shedding and non-shedding states for different sets of covariates have been estimated by application of logistic regression. The covariate-specific matrices of conditional probabilities, describing the presence of different risk factors, were used to estimate (i) the stationary prevalence of dairy cows that shed any L. monocytogenes subtype or ribotypes 1058A, 1039E, and 1042B, (ii) the duration of overall and subtype specific fecal shedding, and (iii) the duration of periods without shedding. A non-homogeneous MCM was constructed to study how the prevalence of fecal shedders changes over time. The model was validated with data from the study farm and published literature. The results of our modeling work indicated that (i) the prevalence of L. monocytogenes fecal shedders varies over time and can be higher than 90%, (ii) L. monocytogenes subtypes exhibit different dynamics of fecal shedding, (iii) the dynamics of L. monocytogenes fecal shedding are highly associated with contamination of silage (fermented feed) and cows' exposure to stress, and (iv) the developed approach can be readily used to study the dynamics of fecal shedding in other pathogen-host-environment systems.