Analysis of Longitudinal Data of Epileptic Seizure Counts – A Two‐State Hidden Markov Regression Approach

This paper discusses a two‐state hidden Markov Poisson regression (MPR) model for analyzing longitudinal data of epileptic seizure counts, which allows for the rate of the Poisson process to depend on covariates through an exponential link function and to change according to the states of a two‐state Markov chain with its transition probabilities associated with covariates through a logit link function. This paper also considers a two‐state hidden Markov negative binomial regression (MNBR) model, as an alternative, by using the negative binomial instead of Poisson distribution in the proposed MPR model when there exists extra‐Poisson variation conditional on the states of the Markov chain. The two proposed models in this paper relax the stationary requirement of the Markov chain, allow for overdispersion relative to the usual Poisson regression model and for correlation between repeated observations. The proposed methodology provides a plausible analysis for the longitudinal data of epileptic seizure counts, and the MNBR model fits the data much better than the MPR model. Maximum likelihood estimation using the EM and quasi‐Newton algorithms is discussed. A Monte Carlo study for the proposed MPR model investigates the reliability of the estimation method, the choice of probabilities for the initial states of the Markov chain, and some finite sample behaviors of the maximum likelihood estimates, suggesting that (1) the estimation method is accurate and reliable as long as the total number of observations is reasonably large, and (2) the choice of probabilities for the initial states of the Markov process has little impact on the parameter estimates.     

Estimating transition rates in aggregated Markov models of ion channel gating with loops and with nearly equal dwell times

A typical task in the application of aggregated Markov models to ion channel data is the estimation of the transition rates between the states. Realistic models for ion channel data often have one or more loops. We show that the transition rates of a model with loops are not identifiable if the model has either equal open or closed dwell times. This non-identifiability of the transition rates also has an effect on the estimation of the transition rates for models which are not subject to the constraint of either equal open or closed dwell times. If a model with loops has nearly equal dwell times, the Hessian matrix of its likelihood function will be ill-conditioned and the standard deviations of the estimated transition rates become extraordinarily large for a number of data points which are typically recorded in experiments.     

transition priors for protein hidden Markov models: an empirical study towards maximum discrimination.

Insertions and deletions in a profile hidden Markov model (HMM) are modeled by transition probabilities between insert, delete and match states. These are estimated by combining observed data and prior probabilities. The transition prior probabilities can be defined either ad hoc or by maximum likelihood (ML) estimation. We show that the choice of transition prior greatly affects the HMM's ability to discriminate between true and false hits. HMM discrimination was measured using the HMMER 2.2 package applied to 373 families from Pfam. We measured the discrimination between true members and noise sequences employing various ML transition priors and also systematically scanned the parameter space of ad hoc transition priors. Our results indicate that ML priors produce far from optimal discrimination, and we present an empirically derived prior that considerably decreases the number of misclassifications compared to ML. Most of the difference stems from the probabilities for exiting a delete state. The ML prior, which is unaware of noise sequences, estimates a delete-to-delete probability that is relatively high and does not penalize noise sequences enough for optimal discrimination.     

A compact statistical model of the song syntax in Bengalese finch

Songs of many songbird species consist of variable sequences of a finite number of syllables. A common approach for characterizing the syntax of these complex syllable sequences is to use transition probabilities between the syllables. This is equivalent to the Markov model, in which each syllable is associated with one state, and the transition probabilities between the states do not depend on the state transition history. Here we analyze the song syntax in Bengalese finch. We show that the Markov model fails to capture the statistical properties of the syllable sequences. Instead, a state transition model that accurately describes the statistics of the syllable sequences includes adaptation of the self-transition probabilities when states are revisited consecutively, and allows associations of more than one state to a given syllable. Such a model does not increase the model complexity significantly. Mathematically, the model is a partially observable Markov model with adaptation (POMMA). The success of the POMMA supports the branching chain network model of how syntax is controlled within the premotor song nucleus HVC, but also suggests that adaptation and many-to-one mapping from the syllable-encoding chain networks in HVC to syllables should be included in the network model.     

A hidden Markov model approach to neuron firing patterns.

Analysis and characterization of neuronal discharge patterns are of interest to neurophysiologists and neuropharmacologists. In this paper we present a hidden Markov model approach to modeling single neuron electrical activity. Basically the model assumes that each interspike interval corresponds to one of several possible states of the neuron. Fitting the model to experimental series of interspike intervals by maximum likelihood allows estimation of the number of possible underlying neuron states, the probability density functions of interspike intervals corresponding to each state, and the transition probabilities between states. We present an application to the analysis of recordings of a locus coeruleus neuron under three pharmacological conditions. The model distinguishes two states during halothane anesthesia and during recovery from halothane anesthesia, and four states after administration of clonidine. The transition probabilities yield additional insights into the mechanisms of neuron firing.     

Adaptive processing techniques based on hidden Markov models for characterizing very small channel currents buried in noise and deterministic interferences.

Techniques for characterizing very small single-channel currents buried in background noise are described and tested on simulated data to give confidence when applied to real data. Single channel currents are represented as a discrete-time, finite-state, homogeneous, Markov process, and the noise that obscures the signal is assumed to be white and Gaussian. The various signal model parameters, such as the Markov state levels and transition probabilities, are unknown. In addition to white Gaussian noise, the signal can be corrupted by deterministic interferences of known form but unknown parameters, such as the sinusoidal disturbance stemming from AC interference and a drift of the base line owing to a slow development of liquid-junction potentials. To characterize the signal buried in such stochastic and deterministic interferences, the problem is first formulated in the framework of a Hidden Markov Model and then the Expectation Maximization algorithm is applied to obtain the maximum likelihood estimates of the model parameters (state levels, transition probabilities), signals, and the parameters of the deterministic disturbances. Using fictitious channel currents embedded in the idealized noise, we first show that the signal processing technique is capable of characterizing the signal characteristics quite accurately even when the amplitude of currents is as small as 5-10 fA. The statistics of the signal estimated from the processing technique include the amplitude, mean open and closed duration, open-time and closed-time histograms, probability of dwell-time and the transition probability matrix. With a periodic interference composed, for example, of 50 Hz and 100 Hz components, or a linear drift of the baseline added to the segment containing channel currents and white noise, the parameters of the deterministic interference, such as the amplitude and phase of the sinusoidal wave, or the rate of linear drift, as well as all the relevant statistics of the signal, are accurately estimated with the algorithm we propose. Also, if the frequencies of the periodic interference are unknown, they can be accurately estimated. Finally, we provide a technique by which channel currents originating from the sum of two or more independent single channels are decomposed so that each process can be separately characterized. This process is also formulated as a Hidden Markov Model problem and solved by applying the Expectation Maximization algorithm. The scheme relies on the fact that the transition matrix of the summed Markov process can be construed as a tensor product of the transition matrices of individual processes.     

Bias in Markov models of disease

We examine bias in Markov models of diseases, including both chronic and infectious diseases. We consider two common types of Markov disease models: ones where disease progression changes by severity of disease, and ones where progression of disease changes in time or by age. We find sufficient conditions for bias to exist in models with aggregated transition probabilities when compared to models with state/time dependent transition probabilities. We also find that when aggregating data to compute transition probabilities, bias increases with the degree of data aggregation. We illustrate by examining bias in Markov models of Hepatitis C, Alzheimer’s disease, and lung cancer using medical data and find that the bias is significant depending on the method used to aggregate the data. A key implication is that by not incorporating state/time dependent transition probabilities, studies that use Markov models of diseases may be significantly overestimating or underestimating disease progression. This could potentially result in incorrect recommendations from cost-effectiveness studies and incorrect disease burden forecasts.     

Modeling subunit cooperativity in opening of tetrameric ion channels

Most potassium channels are tetramers of four homologous polypeptides (subunits). During channel gating, each subunit undergoes several conformational changes independent of the state of other subunits before reaching a permissive state, from which the channel can open. However, transition from the permissive states to the open state involves a concerted movement of all subunits. This cooperative transition must be included in Markov models of channel gating. Previously, it was implemented by considering all possible combinations of four subunit states in a much larger expanded model of channel states (e.g., 27,405 channel states versus 64 subunit states), which complicates modeling and is computationally intense, especially when accurate modeling requires a large number of subunit states. To overcome these complexities and retain the tetrameric molecular structure, a modeling approach was developed to incorporate the cooperative transition directly from the subunit models. In this approach, the open state is separated from the subunit models and represented by the net flux between the open state and the permissive states. Dynamic variations of the probability of state residencies computed using this direct approach and the expanded model were identical. Implementation of the direct approach is simple and its computational time is orders-of-magnitude shorter than the equivalent expanded model.     

Estimating kinetic constants from single channel data.

The process underlying the opening and closing of ionic channels in biological or artificial lipid membranes can be modeled kinetically as a time-homogeneous Markov chain. The elements of the chain are kinetic states that can be either open or closed. A maximum likelihood procedure is described for estimating the transition rates between these states from single channel data. The method has been implemented for linear kinetic schemes of fewer than six states, and is suitable for nonstationary data in which one or more independent channels are functioning simultaneously. It also provides standard errors for all estimates of rate constants and permits testing of smoothly parameterized subhypotheses of a general model. We have illustrated our approach by analysis of single channel data simulated on a computer and have described a procedure for analysis of experimental data.     

From complex spatial dynamics to simple Markov chain models: do predators and prey leave footprints?

In this paper we present a concept for using presence–absence data to recover information on the population dynamics of predator–prey systems. We use a highly complex and spatially explicit simulation model of a predator–prey mite system to generate simple presence–absence data: the number of patches with both prey and predators, with prey only, with predators only, and with neither species, along with the number of patches that change from one state to another in each time step. The average number of patches in the four states, as well as the average transition probabilities from one state to another, are then depicted in a state transition diagram, constituting the "footprints" of the underlying population dynamics. We investigate to what extent changes in the population processes modeled in the complex simulation (i.e. the predator's functional response and the dispersal rates of both species) are reflected by different footprints
The transition probabilities can be used to forecast the expected fate of a system given its current state. However, the transition probabilities in the modeled system depend on the number of patches in each state. We develop a model for the dependence of transition probabilities on state variables, and combine this information in a Markov chain transition matrix model. Finally, we use this extended model to predict the long-term dynamics of the system and to reveal its asymptotic steady state properties.     

Markov models for covariate dependence of binary sequences

Suppose that a heterogeneous group of individuals is followed over time and that each individual can be in state 0 or state 1 at each time point. The sequence of states is assumed to follow a binary Markov chain. In this paper we model the transition probabilities for the 0 to 0 and 1 to 0 transitions by two logistic regressions, thus showing how the covariates relate to changes in state. With p covariates, there are 2(p + 1) parameters including intercepts, which we estimate by maximum likelihood. We show how to use transition probability estimates to test hypotheses about the probability of occupying state 0 at time i (i = 2, ..., T) and the equilibrium probability of state 0. These probabilities depend on the covariates. A recursive algorithm is suggested to estimate regression coefficients when some responses are missing. Extensions of the basic model which allow time-dependent covariates and nonstationary or second-order Markov chains are presented. An example shows the model applied to a study of the psychological impact of breast cancer in which women did or did not manifest distress at four time points in the year following surgery.     

A new method for inferring hidden markov models from noisy time sequences

We present a new method for inferring hidden Markov models from noisy time sequences without the necessity of assuming a model architecture, thus allowing for the detection of degenerate states. This is based on the statistical prediction techniques developed by Crutchfield et al. and generates so called causal state models, equivalent in structure to hidden Markov models. The new method is applicable to any continuous data which clusters around discrete values and exhibits multiple transitions between these values such as tethered particle motion data or Fluorescence Resonance Energy Transfer (FRET) spectra. The algorithms developed have been shown to perform well on simulated data, demonstrating the ability to recover the model used to generate the data under high noise, sparse data conditions and the ability to infer the existence of degenerate states. They have also been applied to new experimental FRET data of Holliday Junction dynamics, extracting the expected two state model and providing values for the transition rates in good agreement with previous results and with results obtained using existing maximum likelihood based methods. The method differs markedly from previous Markov-model reconstructions in being able to uncover truly hidden states.     

Characterization of single channel currents using digital signal processing techniques based on Hidden Markov Models.

Techniques for extracting small, single channel ion currents from background noise are described and tested. It is assumed that single channel currents are generated by a first-order, finite-state, discrete-time, Markov process to which is added 'white' background noise from the recording apparatus (electrode, amplifiers, etc). Given the observations and the statistics of the background noise, the techniques described here yield a posteriori estimates of the most likely signal statistics, including the Markov model state transition probabilities, duration (open- and closed-time) probabilities, histograms, signal levels, and the most likely state sequence. Using variations of several algorithms previously developed for solving digital estimation problems, we have demonstrated that: (1) artificial, small, first-order, finite-state, Markov model signals embedded in simulated noise can be extracted with a high degree of accuracy, (2) processing can detect signals that do not conform to a first-order Markov model but the method is less accurate when the background noise is not white, and (3) the techniques can be used to extract from the baseline noise single channel currents in neuronal membranes. Some studies have been included to test the validity of assuming a first-order Markov model for biological signals. This method can be used to obtain directly from digitized data, channel characteristics such as amplitude distributions, transition matrices and open- and closed-time durations.     

Statistical properties of single sodium channels

Single channel currents were obtained from voltage-activated sodium channels in outside-out patches of tissue-cultured GH3 cells, a clonal line from rat pituitary gland. In membrane patches where the probability of overlapping openings was low, the open time histograms were well fit by a single exponential. Most analysis was done on a patch with exactly one channel. We found no evidence for multiple open states at -25 and -40 mV, since open times, burst durations, and autocorrelation functions were time independent. Amplitude histograms showed no evidence of multiple conductance levels. We fit the gating with 25 different time-homogeneous Markov chain models having up to five states, using a maximum likelihood procedure to estimate the rate constants. For selected models, this procedure yielded excellent predictions for open time, closed time, and first latency density functions, as well as the probability of the channel being open after a step depolarization, the burst duration distribution, autocorrelation, and the distribution of number of openings per record. The models were compared statistically using likelihood ratio tests and Akaike's information criterion. Acceptable models allowed inactivation from closed states, as well as from the open state. Among the models eliminated as unacceptable by this survey were the Hodgkin-Huxley model and any model requiring a channel to open before inactivating.     

Model selection in non-nested hidden Markov models for ion channel gating

An important task in the application of Markov models to the analysis of ion channel data is the determination of the correct gating scheme of the ion channel under investigation. Some prior knowledge from other experiments can reduce significantly the number of possible models. If these models are standard statistical procedures nested like likelihood ratio testing, provide reliable selection methods. In the case of non-nested models, information criteria like AIC, BIC, etc., are used. However, it is not known if any of these criteria provide a reliable selection method and which is the best one in the context of ion channel gating. We provide an alternative approach to model selection in the case of non-nested models with an equal number of open and closed states. The models to choose from are embedded in a properly defined general model. Therefore, we circumvent the problems of model selection in the non-nested case and can apply model selection procedures for nested models.     

Estimation of kinetic rate constants from multi-channel recordings by a direct fit of the time series.

The maximum-likelihood technique for the direct estimation of rate constants from the measured patch clamp current is extended to the analysis of multi-channel recordings, including channels with subconductance levels. The algorithm utilizes a simplified approach for the calculation of the matrix exponentials of the probability matrix from the rate constants of the Markov model of the involved channel(s) by making use of the Kronecker sum and product. The extension to multi-channel analysis is tested by the application to simulated data. For these tests, three different channel models were selected: a two-state model, a three-state model with two open states of different conductance, and a three-state model with two closed states. For the simulations, time series of these models were calculated from the related first-order, finite-state, continuous-time Markov processes. Blue background noise was added, and the signals were filtered by a digital filter similar to the anti-aliasing low-pass. The tests showed that the fit algorithm revealed good estimates of the original rate constants from time series of simulated records with up to four independent and identical channels even in the case of signal-to-noise ratios being as low as 2. The number of channels in a record can be determined from the dependence of the likelihood on channel number. For large enough data sets, it takes on a maximum when the assumed channel number is equal to the "true" channel number.     

A generalized mover-stayer model for panel data

A generalized mover-stayer model is described for conditionally Markov processes under panel observation. Marginally the model represents a mixture of nested continuous-time Markov processes in which sub-models are defined by constraining some transition intensities to zero between two or more states of a full model. A Fisher scoring algorithm is described which facilitates maximum likelihood estimation based only on the first derivatives of the transition probability matrices. The model is fit to data from a smoking prevention study and is shown to provide a significant improvement in fit over a time-homogeneous Markov model. Extensions are developed which facilitate examination of covariate effects on both the transition intensities and the mover-stayer probabilities.     

Equivalence of aggregated Markov models of ion-channel gating

One cannot always distinguish different Markov models of ion-channel kinetics solely on the basis of steady-state kinetic data. If two generator (or transition) matrices are related by a similarity transformation that does not combine states with different conductances, then the models described by these generator matrices have the same observable steady-state statistics. This result suggests a procedure for expressing the model in a unique form, and sometimes reducing the number of parameters in a model. I apply the similarity transformation procedure to a number of simple models. When a model specifies the dependence of the rates of transition on an experimentally variable parameter such as the concentration of a ligand or the membrane potential, the class of equivalent models may be further restricted, but a model is not always uniquely determined even under these conditions. Voltage-step experiments produce non-stationary data that can also be used to distinguish models.     

Generalizing HMMs to Continuous Time for Fast Kinetics: Hidden Markov Jump Processes

The hidden Markov model (HMM) is a framework for time series analysis widely applied to single-molecule experiments. Although initially developed for applications outside the natural sciences, the HMM has traditionally been used to interpret signals generated by physical systems, such as single molecules, evolving in a discrete state space observed at discrete time levels dictated by the data acquisition rate. Within the HMM framework, transitions between states are modeled as occurring at the end of each data acquisition period and are described using transition probabilities. Yet, whereas measurements are often performed at discrete time levels in the natural sciences, physical systems evolve in continuous time according to transition rates. It then follows that the modeling assumptions underlying the HMM are justified if the transition rates of a physical process from state to state are small as compared to the data acquisition rate. In other words, HMMs apply to slow kinetics. The problem is, because the transition rates are unknown in principle, it is unclear, a priori, whether the HMM applies to a particular system. For this reason, we must generalize HMMs for physical systems, such as single molecules, because these switch between discrete states in “continuous time”. We do so by exploiting recent mathematical tools developed in the context of inferring Markov jump processes and propose the hidden Markov jump process. We explicitly show in what limit the hidden Markov jump process reduces to the HMM. Resolving the discrete time discrepancy of the HMM has clear implications: we no longer need to assume that processes, such as molecular events, must occur on timescales slower than data acquisition and can learn transition rates even if these are on the same timescale or otherwise exceed data acquisition rates.     

Flexible nonhomogeneous markov models for panel observed data

Methods for fitting nonhomogeneous Markov models to panel-observed data using direct numerical solution to the Kolmogorov Forward equations are developed. Nonhomogeneous Markov models occur most commonly when baseline transition intensities depend on calendar time, but may also occur with deterministic time-dependent covariates such as age. We propose transition intensities based on B-splines as a smooth alternative to piecewise constant intensities and also as a generalization of time transformation models. An expansion of the system of differential equations allows first derivatives of the likelihood to be obtained, which can be used in a Fisher scoring algorithm for maximum likelihood estimation. The method is evaluated through a small simulation study and demonstrated on data relating to the development of cardiac allograft vasculopathy in posttransplantation patients.