The analysis of disease biomarker data using a mixed hidden Markov model (Open Access publication)

A mixed hidden Markov model (HMM) was developed for predicting breeding values of a biomarker (here, somatic cell score) and the individual probabilities of health and disease (here, mastitis) based upon the measurements of the biomarker. At a first level, the unobserved disease process (Markov model) was introduced and at a second level, the measurement process was modeled, making the link between the unobserved disease states and the observed biomarker values. This hierarchical formulation allows joint estimation of the parameters of both processes. The flexibility of this approach is illustrated on the simulated data. Firstly, lactation curves for the biomarker were generated based upon published parameters (mean, variance, and probabilities of infection) for cows with known clinical conditions (health or mastitis due to Escherichia coli or Staphylococcus aureus). Next, estimation of the parameters was performed via Gibbs sampling, assuming the health status was unknown. Results from the simulations and mathematics show that the mixed HMM is appropriate to estimate the quantities of interest although the accuracy of the estimates is moderate when the prevalence of the disease is low. The paper ends with some indications for further developments of the methodology.     

Markov modelling of immunological and virological states in HIV-1 infected patients

The purpose of this study was to evaluate the evolution of HIV infected patients and to bring out some significant factors associated with this pathology. The main criteria revealing the State of illness is viral load measurement (VL). However the CD4 lymphocytes also represent an important marker as these reflect the State of the immune reservoir. Many studies have been carried out in this field and different models have been proposed with a view to a better understanding of this disease. Multi State Markov models defined in terms of CD4 counts, or in terms of viral load, have proved to be very useful tools for modelling HIV disease progression. The model we have developed in this study is based on both the CD4 lymphocytes counts and VL. Markov models are characterized by transition intensities. In this paper we explored several structures in succession. First, we used a homogeneous continuous time Markov process with four states defined by crossed values of CD4 and VL in a given patient at a given time. Then, the effect of certain covariates on the infection process was introduced into the model via the transition intensity functions, as with a Cox regression model. Since the hypothesis of homogeneity may be unrealistic in certain cases, we also considered piecewise homogeneous Markov models. Finally, the effects of covariates and time were combined in a piecewise homogeneous model with a covariate. We applied these methods to data from 1313 HIV-infected patients included in the NADIS cohort.     

Semi‐Markov Models with Phase‐Type Sojourn Distributions

Summary Continuous‐time multistate models are widely used for categorical response data, particularly in the modeling of chronic diseases. However, inference is difficult when the process is only observed at discrete time points, with no information about the times or types of events between observation times, unless a Markov assumption is made. This assumption can be limiting as rates of transition between disease states might instead depend on the time since entry into the current state. Such a formulation results in a semi‐Markov model. We show that the computational problems associated with fitting semi‐Markov models to panel‐observed data can be alleviated by considering a class of semi‐Markov models with phase‐type sojourn distributions. This allows methods for hidden Markov models to be applied. In addition, extensions to models where observed states are subject to classification error are given. The methodology is demonstrated on a dataset relating to development of bronchiolitis obliterans syndrome in post‐lung‐transplantation patients.     

A queueing model for chronic recurrent conditions under panel observation

In many chronic conditions, subjects alternate between an active and an inactive state, and sojourns into the active state may involve multiple lesions, infections, or other recurrences with different times of onset and resolution. We present a biologically interpretable model of such chronic recurrent conditions based on a queueing process. The model has a birth-death process describing recurrences and a semi-Markov process describing the alternation between active and inactive states, and can be fit to panel data that provide only a binary assessment of the active or inactive state at a series of discrete time points using a hidden Markov approach. We accommodate individual heterogeneity and covariates using a random effects model, and simulate the posterior distribution of unknowns using a Markov chain Monte Carlo algorithm. Application to a clinical trial of genital herpes shows how the method can characterize the biology of the disease and estimate treatment efficacy.     

Hidden Markov models for settings with interval-censored transition times and uncertain time origin: application to HIV genetic analyses

The simplicity and flexibility of Markov models make them appealing for investigations of the acquisition of HIV drug-resistance mutations, whose presence can define specific Markov states. Because the exact time of acquiring a mutation is not observed during clinical research studies on HIV infection, it is important that methods for fitting such models accommodate interval-censored transition times. Furthermore, many such studies include patients with extensive treatment experience prior to the onset of the studies. Therefore, the virus in these patients may have already acquired resistance mutations by study entry. Retrospective data regarding the time on treatment, which is often known or known with error, provide information about the acquisition rates before the start of a study. Finally, variability in the genetic sequences of circulating HIV creates uncertainty in the Markov states. This paper considers approaches to fitting Markov models to data with interval-censored transition times when the time origin and the Markov states are known with error. The methods were applied to AIDS Clinical Trial Group protocol 398, a randomized comparison of mono- versus dual-protease inhibitor use in heavily pretreated patients. We found that the estimated rates of acquiring certain classes of mutations depended on the presence of others, and that the precision of these estimates can be considerably improved by inclusion of retrospective data.     

Analysis of Longitudinal Data of Epileptic Seizure Counts – A Two‐State Hidden Markov Regression Approach

This paper discusses a two‐state hidden Markov Poisson regression (MPR) model for analyzing longitudinal data of epileptic seizure counts, which allows for the rate of the Poisson process to depend on covariates through an exponential link function and to change according to the states of a two‐state Markov chain with its transition probabilities associated with covariates through a logit link function. This paper also considers a two‐state hidden Markov negative binomial regression (MNBR) model, as an alternative, by using the negative binomial instead of Poisson distribution in the proposed MPR model when there exists extra‐Poisson variation conditional on the states of the Markov chain. The two proposed models in this paper relax the stationary requirement of the Markov chain, allow for overdispersion relative to the usual Poisson regression model and for correlation between repeated observations. The proposed methodology provides a plausible analysis for the longitudinal data of epileptic seizure counts, and the MNBR model fits the data much better than the MPR model. Maximum likelihood estimation using the EM and quasi‐Newton algorithms is discussed. A Monte Carlo study for the proposed MPR model investigates the reliability of the estimation method, the choice of probabilities for the initial states of the Markov chain, and some finite sample behaviors of the maximum likelihood estimates, suggesting that (1) the estimation method is accurate and reliable as long as the total number of observations is reasonably large, and (2) the choice of probabilities for the initial states of the Markov process has little impact on the parameter estimates.     

Joint modeling of progression of HIV resistance mutations measured with uncertainty and failure time data

Development of HIV resistance mutations is a major cause for failure of antiretroviral treatment. This article proposes a method for jointly modeling the processes of viral genetic changes and treatment failure. Because the viral genome is measured with uncertainty, a hidden Markov model is used to fit the viral genetic process. The uncertain viral genotype is included as a time-dependent covariate in a Cox model for failure time, and an expectation-maximization algorithm is used to estimate the model parameters. This model allows simultaneous evaluation of the sequencing uncertainty and the effect of resistance mutation on the risk of virological and immunological failures. Various model checking tests are provided to assess the appropriateness of the model. Simulation studies are performed to investigate the finite-sample properties of the proposed methods, which are then applied to data collected in three phase II clinical trials testing antiretroviral treatments containing the drug efavirenz.     

A bayesian random-effects markov model for tumor progression in women with a family history of breast cancer

SUMMARY: It makes intuitive sense to model the natural history of breast cancer, tumor progression from preclinical screen-detectable phase (PCDP) to clinical disease, as a multistate process, but the multilevel structure of the available data, which generally comes from cluster (family)-based service screening programs, makes the required parameter estimation intractable because there is a correlation between screening rounds in the same individual, and between subjects within clusters (families). There is also residual heterogeneity after adjusting for covariates. We therefore proposed a Bayesian hierarchical multistate Markov model with fixed and random effects and applied it to data from a high-risk group (women with a family history of breast cancer) participating in a family-based screening program for breast cancer. A total of 4867 women attended (representing 4464 families) and by the end of 2002, a total of 130 breast cancer cases were identified. Parameter estimation was carried out using Markov chain Monte Carlo (MCMC) simulation and the Bayesian software package WinBUGS. Models with and without random effects were considered. Our preferred model included a random-effect term for the transition rate from preclinical to clinical disease (sigma(2)(2f)), which was estimated to be 0.50 (95% credible interval = 0.22-1.49). Failure to account for this random effect was shown to lead to bias. The incorporation of covariates into multistate models with random effect not only reduced residual heterogeneity but also improved the convergence of stationary distribution. Our proposed Bayesian hierarchical multistate model is a valuable tool for estimating the rate of transitions between disease states in the natural history of breast cancer (and possibly other conditions). Unlike existing models, it can cope with the correlation structure of multilevel screening data, covariates, and residual (unexplained) variation.     

Modeling Inter‐Subject Variability in fMRI Activation Location: A Bayesian Hierarchical Spatial Model

Summary The aim of this article is to develop a spatial model for multi‐subject fMRI data. There has been extensive work on univariate modeling of each voxel for single and multi‐subject data, some work on spatial modeling of single‐subject data, and some recent work on spatial modeling of multi‐subject data. However, there has been no work on spatial models that explicitly account for inter‐subject variability in activation locations. In this article, we use the idea of activation centers and model the inter‐subject variability in activation locations directly. Our model is specified in a Bayesian hierarchical framework which allows us to draw inferences at all levels: the population level, the individual level, and the voxel level. We use Gaussian mixtures for the probability that an individual has a particular activation. This helps answer an important question that is not addressed by any of the previous methods: What proportion of subjects had a significant activity in a given region. Our approach incorporates the unknown number of mixture components into the model as a parameter whose posterior distribution is estimated by reversible jump Markov chain Monte Carlo. We demonstrate our method with a fMRI study of resolving proactive interference and show dramatically better precision of localization with our method relative to the standard mass‐univariate method. Although we are motivated by fMRI data, this model could easily be modified to handle other types of imaging data.     

Spatial multistate transitional models for longitudinal event data

Summary .   Follow-up medical studies often collect longitudinal data on patients. Multistate transitional models are useful for analysis in such studies where at any point in time, individuals may be said to occupy one of a discrete set of states and interest centers on the transition process between states. For example, states may refer to the number of recurrences of an event, or the stage of a disease. We develop a hierarchical modeling framework for the analysis of such longitudinal data when the processes corresponding to different subjects may be correlated spatially over a region. Continuous-time Markov chains incorporating spatially correlated random effects are introduced. Here, joint modeling of both spatial dependence as well as dependence between different transition rates is required and a multivariate spatial approach is employed. A proportional intensities frailty model is developed where baseline intensity functions are modeled using parametric Weibull forms, piecewise-exponential formulations, and flexible representations based on cubic B-splines. The methodology is developed within the context of a study examining invasive cardiac procedures in Quebec. We consider patients admitted for acute coronary syndrome throughout the 139 local health units of the province and examine readmission and mortality rates over a 4-year period.     

Micronutrients in HIV: A Bayesian Meta-Analysis

BackgroundApproximately 28.5 million people living with HIV are eligible for treatment (CD4<500), but currently have no access to antiretroviral therapy. Reduced serum level of micronutrients is common in HIV disease. Micronutrient supplementation (MNS) may mitigate disease progression and mortality.ObjectivesWe synthesized evidence on the effect of micronutrient supplementation on mortality and rate of disease progression in HIV disease.MethodsWe searched MEDLINE, EMBASE, the Cochrane Central, AMED and CINAHL databases through December 2014, without language restriction, for studies of greater than 3 micronutrients versus any or no comparator. We built a hierarchical Bayesian random effects model to synthesize results. Inferences are based on the posterior distribution of the population effects; posterior distributions were approximated by Markov chain Monte Carlo in OpenBugs.ConclusionsMNS significantly and substantially slows disease progression in HIV+ adults not on ARV, and possibly reduces mortality. Micronutrient supplements are effective in reducing progression with a posterior probability of 97.9%. Considering MNS low cost and lack of adverse effects, MNS should be standard of care for HIV+ adults not yet on ARV.     

Multi-Stage Markov Analysis of Progressive Disease Applied to Melanoma

A fundamental research goal in clinical studies of progressive, multi-stage disease is to understand its natural history and its relationship with prognostic factors. Our current understanding of this topic is based on the use of two-stage methods for event-time analysis which neglect intermediate transition information. In contrast, a multi-stage model utilizes all available data and provides more accurate insight into disease progression. We specify a forward-flowing multi-stage Markov model based on the discrete clinical stages of disease. By assuming the process to be Markovian, we avoid unnecessary complications to our numerical estimation procedure. Due to noncontinuous patient monitoring and the chronic nature of progressive disease, heavy right- and interval-censoring exists in the transition data. We develop a modified ECM algorithm to numerically carry out the otherwise complicated parameter estimation for this process. We also identify significant prognostic factors relevant to each transition, along with the relative importance of each prognostic factor. The numerical estimation is stable, and the parameter estimates are maximum likelihood estimates (Meng, 1990). In general our forward-flowing multi-stage models provide a flexible framework for the study of the effects of prognostic factors on progression among several stages. We apply our Markov model to a dataset of malignant melanoma patients, and present an inferential discussion. Results from our multi-stage Markov model provide an improved understanding of melanoma progression.     

Joint Inference on HIV Viral Dynamics and Immune Suppression in Presence of Measurement Errors

Summary : In an attempt to provide a tool to assess antiretroviral therapy and to monitor disease progression, this article studies association of human immunodeficiency virus (HIV) viral suppression and immune restoration. The data from a recent acquired immune deficiency syndrome (AIDS) study are used for illustration. We jointly model HIV viral dynamics and time to decrease in CD4/CD8 ratio in the presence of CD4 process with measurement errors, and estimate the model parameters simultaneously via a method based on a Laplace approximation and the commonly used Monte Carlo EM algorithm. The approaches and many of the points presented apply generally.     

Goodness-of-fit diagnostics for Bayesian hierarchical models

This article proposes methodology for assessing goodness of fit in Bayesian hierarchical models. The methodology is based on comparing values of pivotal discrepancy measures (PDMs), computed using parameter values drawn from the posterior distribution, to known reference distributions. Because the resulting diagnostics can be calculated from standard output of Markov chain Monte Carlo algorithms, their computational costs are minimal. Several simulation studies are provided, each of which suggests that diagnostics based on PDMs have higher statistical power than comparable posterior-predictive diagnostic checks in detecting model departures. The proposed methodology is illustrated in a clinical application; an application to discrete data is described in supplementary material.     

An Inhomogeneous Markov Model to Fit the Migration Process

In this paper we study the migration process considering an inhomogeneous Markov model. This is a certain condition to investigate age-dependent population distributions, where the transition probabilities are not constant. We consider also a death process for a population alive in a region at age t and, as a result of this, combined transition probabilities between the states of the concerning Markov chain. The model has non-stationary distribution for t →∞, because the condition of ergodicity does not hold.     

Covariate Adjustment of Event Histories Estimated from Markov Chains: The Additive Approach

Markov chain models are frequently used for studying event histories that include transitions between several states. An empirical transition matrix for nonhomogeneous Markov chains has previously been developed, including a detailed statistical theory based on counting processes and martingales. In this article, we show how to estimate transition probabilities dependent on covariates. This technique may, e.g., be used for making estimates of individual prognosis in epidemiological or clinical studies. The covariates are included through nonparametric additive models on the transition intensities of the Markov chain. The additive model allows for estimation of covariate-dependent transition intensities, and again a detailed theory exists based on counting processes. The martingale setting now allows for a very natural combination of the empirical transition matrix and the additive model, resulting in estimates that can be expressed as stochastic integrals, and hence their properties are easily evaluated. Two medical examples will be given. In the first example, we study how the lung cancer mortality of uranium miners depends on smoking and radon exposure. In the second example, we study how the probability of being in response depends on patient group and prophylactic treatment for leukemia patients who have had a bone marrow transplantation. A program in R and S-PLUS that can carry out the analyses described here has been developed and is freely available on the Internet.     

Empirical and hierarchical Bayesian estimation of ancestral states

Several methods have been proposed to infer the states at the ancestral nodes on a phylogeny. These methods assume a specific tree and set of branch lengths when estimating the ancestral character state. Inferences of the ancestral states, then, are conditioned on the tree and branch lengths being true. We develop a hierarchical Bayes method for inferring the ancestral states on a tree. The method integrates over uncertainty in the tree, branch lengths, and substitution model parameters by using Markov chain Monte Carlo. We compare the hierarchical Bayes inferences of ancestral states with inferences of ancestral states made under the assumption that a specific tree is correct. We find that the methods are correlated, but that accommodating uncertainty in parameters of the phylogenetic model can make inferences of ancestral states even more uncertain than they would be in an empirical Bayes analysis.     

Generalized hierarchical markov models for the discovery of length-constrained sequence features from genome tiling arrays

A generalized hierarchical Markov model for sequences that contain length-restricted features is introduced. This model is motivated by the recent development of high-density tiling array data for determining genomic elements of functional importance. Due to length constraints on certain features of interest, as well as variability in probe behavior, usual hidden Markov-type models are not always applicable. A robust Bayesian framework that can incorporate length constraints, probe variability, and bias is developed. Moreover, a novel recursion-based Monte Carlo algorithm is proposed to estimate the parameters and impute hidden states under length constraints. Application of this methodology to yeast chromosomal arrays demonstrate substantial improvement over currently existing methods in terms of sensitivity as well as biological interpretability.     

Exploratory Bayesian model selection for serial genetics data

Characterizing the process by which molecular and cellular level changes occur over time will have broad implications for clinical decision making and help further our knowledge of disease etiology across many complex diseases. However, this presents an analytic challenge due to the large number of potentially relevant biomarkers and the complex, uncharacterized relationships among them. We propose an exploratory Bayesian model selection procedure that searches for model simplicity through independence testing of multiple discrete biomarkers measured over time. Bayes factor calculations are used to identify and compare models that are best supported by the data. For large model spaces, i.e., a large number of multi-leveled biomarkers, we propose a Markov chain Monte Carlo (MCMC) stochastic search algorithm for finding promising models. We apply our procedure to explore the extent to which HIV-1 genetic changes occur independently over time.