Semi‐Markov Models with Phase‐Type Sojourn Distributions

Summary Continuous‐time multistate models are widely used for categorical response data, particularly in the modeling of chronic diseases. However, inference is difficult when the process is only observed at discrete time points, with no information about the times or types of events between observation times, unless a Markov assumption is made. This assumption can be limiting as rates of transition between disease states might instead depend on the time since entry into the current state. Such a formulation results in a semi‐Markov model. We show that the computational problems associated with fitting semi‐Markov models to panel‐observed data can be alleviated by considering a class of semi‐Markov models with phase‐type sojourn distributions. This allows methods for hidden Markov models to be applied. In addition, extensions to models where observed states are subject to classification error are given. The methodology is demonstrated on a dataset relating to development of bronchiolitis obliterans syndrome in post‐lung‐transplantation patients.     

Non-homogeneous Markov processes for biomedical data analysis

Some time ago, the Markov processes were introduced in biomedical sciences in order to study disease history events. Homogeneous and Non-homogeneous Markov processes are an important field of research into stochastic processes, especially when exact transition times are unknown and interval-censored observations are present in the analysis. Non-homogeneous Markov process should be used when the homogeneous assumption is too strong. However these sorts of models increase the complexity of the analysis and standard software is limited. In this paper, some methods for fitting non-homogeneous Markov models are reviewed and an algorithm is proposed for biomedical data analysis. The method has been applied to analyse breast cancer data. Specific software for this purpose has been implemented.     

Combining cross-sectional and prospective data methods to improve transition parameter estimation for characterizing the accumulation of HIV-1 drug resistance mutations

The order and rate of acquisition of HIV drug resistance mutations have been estimated from longitudinal and cross-sectional data using Markov models and branching trees, respectively. This article proposes methods that make use of both longitudinal and cross-sectional data simultaneously by employing link functions between the two parameter sets. Most functions that link the two parameter sets also depend on the time on treatment before the start of the study-information that may not be available. Nonetheless, under certain assumptions, some link functions eliminate the dependence on time. Using such functions, the two sources of information can be combined to improve the precision of parameter estimation. The method also accommodates error in the link functions from uncertainty in the assumptions required for the links or other reasons. These methods are applied to data from AIDS Clinical Trial Group protocol 398, a randomized comparison of mono- versus dual-protease inhibitor use in heavily treatment experienced HIV patients. Combining the two sources of information allows detection of differences between rates of transition that are not detectable using prospective data alone.     

Non-homogeneous Markov process models with informative observations with an application to Alzheimer's disease

Identifying risk factors for transition rates among normal cognition, mildly cognitive impairment, dementia and death in an Alzheimer's disease study is very important. It is known that transition rates among these states are strongly time dependent. While Markov process models are often used to describe these disease progressions, the literature mainly focuses on time homogeneous processes, and limited tools are available for dealing with non-homogeneity. Further, patients may choose when they want to visit the clinics, which creates informative observations. In this paper, we develop methods to deal with non-homogeneous Markov processes through time scale transformation when observation times are pre-planned with some observations missing. Maximum likelihood estimation via the EM algorithm is derived for parameter estimation. Simulation studies demonstrate that the proposed method works well under a variety of situations. An application to the Alzheimer's disease study identifies that there is a significant increase in transition rates as a function of time. Furthermore, our models reveal that the non-ignorable missing mechanism is perhaps reasonable.     

Estimating Behavioral Transition Rates: Problems and Solutions

Continuous time Markov chain (CTMC) models offer ethologists a powerful tool. The methods are based on well-established procedures for estimating the rates at which one state (e.g. resting) changes to some other set of states (e.g. feeding, fighting, etc.). Unfortunately, ethological data typically differ in a very critical manner from the type of data to which these methods are usually applied: ethological data are usually heavily censored in the sense that each behavioral state shows frequent transitions to several other possible states. This occurs when several competing processes can each end a bout.
We used computer simulation of various behavioral models with known transition rates to investigate the unknown performance of four of the most popular statistical tests for screening data prior to application of CTMC models; this included a modification of one of these tests derived under the assumption of random censoring. Two of the four tests failed completely and would result in rejection of nearly all data even if the model did fit the assumptions of the CTMC methods. Only Barlow's total-time-on test performed with an acceptable α error rate under all conditions. None of the tests were particularly effective at detecting certain types of departures from the CTMC assumptions.
Guidelines are given as to how much confidence should be attached to apparent changes in transition rates.     

Nonparametric estimation of a Markov 'illness-death' process from interval-censored observations, with application to diabetes survival data

The nonparametric estimation of the cumulative transition intensityfunctions in a threestate time-nonhomogeneous Markov processwith irreversible transitions, an ‘illness-death’model, is considered when times of the intermediate transition,e.g. onset of a disease, are interval-censored. The times of‘death’ are assumed to be known exactly or to beright-censored. In addition the observed process may be left-truncated.Data of this type arise when the process is sampled periodically.For example, when the patients are monitored through periodicexaminations the observations on times of change in their diseasestatus will be interval-censored. Under the sampling schemeconsidered here the Nelson–Aalen estimator (Aalen, 1978)for a cumulative transition intensity is not applicable. Inthe proposed method the maximum likelihood estimators of someof the transition intensities are derived from the estimatorsof the corresponding subdistribution functions. The maximumlikelihood estimators are shown to have a self-consistency property.The self-consistency algorithm is developed for the computationof the estimators. This approach generalises the results fromTurnbull (1976) and Frydman (1992). The methods are illustratedwith diabetes survival data.     

Markov modelling of immunological and virological states in HIV-1 infected patients

The purpose of this study was to evaluate the evolution of HIV infected patients and to bring out some significant factors associated with this pathology. The main criteria revealing the State of illness is viral load measurement (VL). However the CD4 lymphocytes also represent an important marker as these reflect the State of the immune reservoir. Many studies have been carried out in this field and different models have been proposed with a view to a better understanding of this disease. Multi State Markov models defined in terms of CD4 counts, or in terms of viral load, have proved to be very useful tools for modelling HIV disease progression. The model we have developed in this study is based on both the CD4 lymphocytes counts and VL. Markov models are characterized by transition intensities. In this paper we explored several structures in succession. First, we used a homogeneous continuous time Markov process with four states defined by crossed values of CD4 and VL in a given patient at a given time. Then, the effect of certain covariates on the infection process was introduced into the model via the transition intensity functions, as with a Cox regression model. Since the hypothesis of homogeneity may be unrealistic in certain cases, we also considered piecewise homogeneous Markov models. Finally, the effects of covariates and time were combined in a piecewise homogeneous model with a covariate. We applied these methods to data from 1313 HIV-infected patients included in the NADIS cohort.     

A multistate model for bivariate interval-censored failure time data

SUMMARY: Interval-censored life-history data arise when the events of interest are only detectable at periodic assessments. When interest lies in the occurrence of two such events, bivariate-interval censored event time data are obtained. We describe how to fit a four-state Markov model useful for characterizing the association between two interval-censored event times when the assessment times for the two events may be generated by different inspection processes. The approach treats the two events symmetrically and enables one to fit multiplicative intensity models that give estimates of covariate effects as well as relative risks characterizing the association between the two events. An expectation-maximization (EM) algorithm is described for estimation in which the maximization step can be carried out with standard software. The method is illustrated by application to data from a trial of HIV patients where the events are the onset of viral shedding in the blood and urine among individuals infected with cytomegalovirus.     

Modeling nonhomogeneous Markov processes via time transformation

Summary .   Longitudinal studies are a powerful tool for characterizing the course of chronic disease. These studies are usually carried out with subjects observed at periodic visits giving rise to panel data. Under this observation scheme the exact times of disease state transitions and sequence of disease states visited are unknown and Markov process models are often used to describe disease progression. Most applications of Markov process models rely on the assumption of time homogeneity, that is, that the transition rates are constant over time. This assumption is not satisfied when transition rates depend on time from the process origin. However, limited statistical tools are available for dealing with nonhomogeneity. We propose models in which the time scale of a nonhomogeneous Markov process is transformed to an operational time scale on which the process is homogeneous. We develop a method for jointly estimating the time transformation and the transition intensity matrix for the time transformed homogeneous process. We assess maximum likelihood estimation using the Fisher scoring algorithm via simulation studies and compare performance of our method to homogeneous and piecewise homogeneous models. We apply our methodology to a study of delirium progression in a cohort of stem cell transplantation recipients and show that our method identifies temporal trends in delirium incidence and recovery.     

Generalizing HMMs to Continuous Time for Fast Kinetics: Hidden Markov Jump Processes

The hidden Markov model (HMM) is a framework for time series analysis widely applied to single-molecule experiments. Although initially developed for applications outside the natural sciences, the HMM has traditionally been used to interpret signals generated by physical systems, such as single molecules, evolving in a discrete state space observed at discrete time levels dictated by the data acquisition rate. Within the HMM framework, transitions between states are modeled as occurring at the end of each data acquisition period and are described using transition probabilities. Yet, whereas measurements are often performed at discrete time levels in the natural sciences, physical systems evolve in continuous time according to transition rates. It then follows that the modeling assumptions underlying the HMM are justified if the transition rates of a physical process from state to state are small as compared to the data acquisition rate. In other words, HMMs apply to slow kinetics. The problem is, because the transition rates are unknown in principle, it is unclear, a priori, whether the HMM applies to a particular system. For this reason, we must generalize HMMs for physical systems, such as single molecules, because these switch between discrete states in “continuous time”. We do so by exploiting recent mathematical tools developed in the context of inferring Markov jump processes and propose the hidden Markov jump process. We explicitly show in what limit the hidden Markov jump process reduces to the HMM. Resolving the discrete time discrepancy of the HMM has clear implications: we no longer need to assume that processes, such as molecular events, must occur on timescales slower than data acquisition and can learn transition rates even if these are on the same timescale or otherwise exceed data acquisition rates.     

Stochastic modelling of a single ion channel: an alternating renewal approach with application to limited time resolution

Stochastic models of ion channels have been based largely on Markov theory where individual states and transition rates must be specified, and sojourn-time densities for each state are constrained to be exponential. This study presents an approach based on random-sum methods and alternating-renewal theory, allowing individual states to be grouped into classes provided the successive sojourn times in a given class are independent and identically distributed. Under these conditions Markov models form a special case. The utility of the approach is illustrated by considering the effects of limited time resolution (modelled by using a discrete detection limit, xi) on the properties of observable events, with emphasis on the observed open-time (xi-open-time). The cumulants and Laplace transform for a xi-open-time are derived for a range of Markov and non-Markov models; several useful approximations to the xi-open-time density function are presented. Numerical studies show that the effects of limited time resolution can be extreme, and also highlight the relative importance of the various model parameters. The theory could form a basis for future inferential studies in which parameter estimation takes account of limited time resolution in single channel records. Appendixes include relevant results concerning random sums and a discussion of the role of exponential distributions in Markov models.     

Sieve Estimation for the Cox Model with Clustered Interval-Censored Failure Time Data

Clustered interval-censored failure time data occur when the failure times of interest are clustered into small groups and known only to lie in certain intervals. A number of methods have been proposed for regression analysis of clustered failure time data, but most of them apply only to clustered right-censored data. In this paper, a sieve estimation procedure is proposed for fitting a Cox frailty model to clustered interval-censored failure time data. In particular, a two-step algorithm for parameter estimation is developed and the asymptotic properties of the resulting sieve maximum likelihood estimators are established. The finite sample properties of the proposed estimators are investigated through a simulation study and the method is illustrated by the data arising from a lymphatic filariasis study.     

Fitting semi-Markov models to interval-censored data with unknown initiation times

In a semi-Markov model, the hazard of making a transition between stages depends on the time spent in the current stage but is independent of time spent in other stages. If the initiation time (time of entry into the network) is not known for some persons and if transition time data are interval censored (i.e., if transition times are not known exactly but are known only to have occurred in some interval), then the length of time these persons spent in any stage is not known. We show how a semi-Markov model can still be fit to interval-censored data with missing initiation times. For the special case of models in which all persons enter the network at the same initial stage and proceed through the same succession of stages to a unique absorbing stage, we present discrete-time nonparametric maximum likelihood estimators of the waiting-time distributions for this type of data.     

A Modeling Study of T-Type Ca Channel Gating and Modulation by L-Cysteine in Rat Nociceptors

L-cysteine (L-cys) increases the amplitude of T-type Ca²⁺ currents in rat T-rich nociceptor-like dorsal root ganglia neurons. The modulation of T-type Ca²⁺ channel gating by L-cys was studied by fitting Markov state models to whole-cell currents recorded from T-rich neurons. The best fitting model tested included three resting states and inactivation from the second resting state and the open state. Inactivation and the final opening step were voltage-independent, whereas transitions between the resting states and deactivation were voltage-dependent. The transition rates between the first two resting states were an order of magnitude faster than those between the second and third resting states, and the voltage-dependency of forward transitions through resting states was two to three times greater than for analogous backward transitions. Analysis with the best fitting model suggested that L-cys increases current amplitude mainly by increasing the transition rate from resting to open and decreasing the transition rate from open to inactivated. An additional model was developed that could account for the bi-exponential time course of recovery from inactivation of the currents and the high frequency of blank sweeps in single channel recordings. This model detected basically the same effects of L-cys on channel gating as the best fitting model.     

Estimating transition rates in aggregated Markov models of ion channel gating with loops and with nearly equal dwell times

A typical task in the application of aggregated Markov models to ion channel data is the estimation of the transition rates between the states. Realistic models for ion channel data often have one or more loops. We show that the transition rates of a model with loops are not identifiable if the model has either equal open or closed dwell times. This non-identifiability of the transition rates also has an effect on the estimation of the transition rates for models which are not subject to the constraint of either equal open or closed dwell times. If a model with loops has nearly equal dwell times, the Hessian matrix of its likelihood function will be ill-conditioned and the standard deviations of the estimated transition rates become extraordinarily large for a number of data points which are typically recorded in experiments.     

Modeling markers of disease progression by a hidden Markov process: application to characterizing CD4 cell decline

Multistate models have been increasingly used to model natural history of many diseases as well as to characterize the follow-up of patients under varied clinical protocols. This modeling allows describing disease evolution, estimating the transition rates, and evaluating the therapy effects on progression. In many cases, the staging is defined on the basis of a discretization of the values of continuous markers (CD4 cell count for HIV application) that are subject to great variability due mainly to short time-scale noise (intraindividual variability) and measurement errors. This led us to formulate a Bayesian hierarchical model where, at a first level, a disease process (Markov model on the true states, which are unobserved) is introduced and, at a second level, the measurement process making the link between the true states and the observed marker values is modeled. This hierarchical formulation allows joint estimation of the parameters of both processes. Estimation of the quantities of interest is performed via stochastic algorithms of the family of Markov chain Monte Carlo methods. The flexibility of this approach is illustrated by analyzing the CD4 data on HIV patients of the Concorde clinical trial.     

A Mixture Model for Bivariate Interval-Censored Failure Times with Dependent Susceptibility

Interval-censored failure times arise when the status with respect to an event of interest is only determined at intermittent examination times. In settings where there exists a sub-population of individuals who are not susceptible to the event of interest, latent variable models accommodating a mixture of susceptible and nonsusceptible individuals are useful. We consider such models for the analysis of bivariate interval-censored failure time data with a model for bivariate binary susceptibility indicators and a copula model for correlated failure times given joint susceptibility. We develop likelihood, composite likelihood, and estimating function methods for model fitting and inference, and assess asymptotic-relative efficiency and finite sample performance. Extensions dealing with higher-dimensional responses and current status data are also described.

     

Sampling errors create bias in Markov models for community dynamics: the problem and a method for its solution

Repeated, spatially explicit sampling is widely used to characterize the dynamics of sessile communities in both terrestrial and aquatic systems, yet our understanding of the consequences of errors made in such sampling is limited. In particular, when Markov transition probabilities are calculated by tracking individual points over time, misidentification of the same spatial locations will result in biased estimates of transition probabilities, successional rates, and community trajectories. Nonetheless, to date, all published studies that use such data have implicitly assumed that resampling occurs without error when making estimates of transition rates. Here, we develop and test a straightforward maximum likelihood approach, based on simple field estimates of resampling errors, to arrive at corrected estimates of transition rates between species in a rocky intertidal community. We compare community Markov models based on raw and corrected transition estimates using data from Endocladia muricata-dominated plots in a California intertidal assemblage, finding that uncorrected predictions of succession consistently overestimate recovery time. We tested the precision and accuracy of the approach using simulated datasets and found good performance of our estimation method over a range of realistic sample sizes and error rates.     

A proportional hazards model for multivariate interval-censored failure time data

This paper focuses on the methodology developed for analyzing a multivariate interval-censored data set from an AIDS observational study. A purpose of the study was to determine the natural history of the opportunistic infection cytomeglovirus (CMV) in an HIV-infected individual. For this observational study, laboratory tests were performed at scheduled clinic visits to test for the presence of the CMV virus in the blood and in the urine (called CMV shedding in the blood and urine). The study investigators were interested in determining whether the stage of HIV disease at study entry was predictive of an increased risk for CMV shedding in either the blood or the urine. If all patients had made each clinic visit, the data would be multivariate grouped failure time data and published methods could be used. However, many patients missed several visits, and when they returned, their lab tests indicated a change in their blood and/or urine CMV shedding status, resulting in interval-censored failure time data. This paper outlines a method for applying the proportional hazards model to the analysis of multivariate interval-censored failure time data from a study of CMV in HIV-infected patients.     

Exploring Migration Data Using Interval-Censored Time-to-Event Models

Abstract: Ecologists and wildlife biologists rely on periodic observation of radiocollared animals to study habitat use, survival, movement, and migration, resulting in response times (e.g., mortality and migration) known only to occur within an interval of time. We illustrate methods for analyzing interval-censored data using data on the timing of fall migration (from spring-summer-fall to winter ranges) for white-tailed deer (Odocoileus virginianus) in northern Minnesota, USA, during years 1991–1992 to 2005–2006. We compare both nonparametric and parametric methods for estimating the cumulative distribution function of migration times, and we suggest a parametric (cure rate) model that accounts for conditional (facultative) migrators as a potential alternative to traditional parametric models. Lastly, we illustrate methods for exploring the effect of environmental covariates on migration timing. Models with time-dependent covariates (snow depth, temp) were sensitive to the treatment of the data (as interval-censored or known event times), suggesting the need to account for interval-censoring when modeling the effect of these covariates.