Maximum likelihood with multiparameter models of substitution

Maximum-likelihood approaches to phylogenetic estimation have the potential of great flexibility, even though current implementations are highly constrained. One such constraint has been the limitation to one-parameter models of substitution. A general implementation of Newton's maximization procedure was developed that allows the maximum likelihood method to be used with multiparameter models. The Estimate and Maximize (EM) algorithm was also used to obtain a good approximation to the maximum likelihood for a certain class of multiparameter models. The condition for which a multiparameter model will only have a single maximum on the likelihood surface was identified. Two-and three-parameter models of substitution in base-paired regions of RNA sequences were used as examples for computer simulations to show that these implementations of the maximum likelihood method are not substantially slower than one-parameter models. Newton's method is much faster than the EM method but may be subject to divergence in some circumstances. In these cases the EM method can be used to restore convergence.     

Bivariate Mixed Effects Analysis of Clustered Data with Large Cluster Sizes

Linear mixed effects models are widely used to analyze a clustered response variable. Motivated by a recent study to examine and compare the hospital length of stay (LOS) between patients undertaking percutaneous coronary intervention (PCI) and coronary artery bypass graft (CABG) from several international clinical trials, we proposed a bivariate linear mixed effects model for the joint modeling of clustered PCI and CABG LOSs where each clinical trial is considered a cluster. Due to the large number of patients in some trials, commonly used commercial statistical software for fitting (bivariate) linear mixed models failed to run since it could not allocate enough memory to invert large dimensional matrices during the optimization process. We consider ways to circumvent the computational problem in the maximum likelihood (ML) inference and restricted maximum likelihood (REML) inference. Particularly, we developed an expected and maximization (EM) algorithm for the REML inference and presented an ML implementation using existing software. The new REML EM algorithm is easy to implement and computationally stable and efficient. With this REML EM algorithm, we could analyze the LOS data and obtained meaningful results.     

Maximum-likelihood estimation of molecular haplotype frequencies in a diploid population

Molecular techniques allow the survey of a large number of linked polymorphic loci in random samples from diploid populations. However, the gametic phase of haplotypes is usually unknown when diploid individuals are heterozygous at more than one locus. To overcome this difficulty, we implement an expectation-maximization (EM) algorithm leading to maximum-likelihood estimates of molecular haplotype frequencies under the assumption of Hardy-Weinberg proportions. The performance of the algorithm is evaluated for simulated data representing both DNA sequences and highly polymorphic loci with different levels of recombination. As expected, the EM algorithm is found to perform best for large samples, regardless of recombination rates among loci. To ensure finding the global maximum likelihood estimate, the EM algorithm should be started from several initial conditions. The present approach appears to be useful for the analysis of nuclear DNA sequences or highly variable loci. Although the algorithm, in principle, can accommodate an arbitrary number of loci, there are practical limitations because the computing time grows exponentially with the number of polymorphic loci. Although the algorithm, in principle, can accommodate an arbitrary number of loci, there are practical limitations because the computing time grows exponentially with the number of polymorphic loci.      

A mixture model approach to the mapping of quantitative trait loci in complex populations with an application to multiple cattle families.

A mixture model approach is presented for the mapping of one or more quantitative trait loci (QTLs) in complex populations. In order to exploit the full power of complete linkage maps the simultaneous likelihood of phenotype and a multilocus (all markers and putative QTLs) genotype is computed. Maximum likelihood estimation in our mixture models is implemented via an Expectation-Maximization algorithm: exact, stochastic or Monte Carlo EM by using a simple and flexible Gibbs sampler. Parameters include allele frequencies of markers and QTLs, discrete or normal effects of biallelic or multiallelic QTLs, and homogeneous or heterogeneous residual variances. As an illustration a dairy cattle data set consisting of twenty half-sib families has been reanalyzed. We discuss the potential which our and other approaches have for realistic multiple-QTL analyses in complex populations.     

A structural EM algorithm for phylogenetic inference.

A central task in the study of molecular evolution is the reconstruction of a phylogenetic tree from sequences of current-day taxa. The most established approach to tree reconstruction is maximum likelihood (ML) analysis. Unfortunately, searching for the maximum likelihood phylogenetic tree is computationally prohibitive for large data sets. In this paper, we describe a new algorithm that uses Structural Expectation Maximization (EM) for learning maximum likelihood phylogenetic trees. This algorithm is similar to the standard EM method for edge-length estimation, except that during iterations of the Structural EM algorithm the topology is improved as well as the edge length. Our algorithm performs iterations of two steps. In the E-step, we use the current tree topology and edge lengths to compute expected sufficient statistics, which summarize the data. In the M-Step, we search for a topology that maximizes the likelihood with respect to these expected sufficient statistics. We show that searching for better topologies inside the M-step can be done efficiently, as opposed to standard methods for topology search. We prove that each iteration of this procedure increases the likelihood of the topology, and thus the procedure must converge. This convergence point, however, can be a suboptimal one. To escape from such "local optima," we further enhance our basic EM procedure by incorporating moves in the flavor of simulated annealing. We evaluate these new algorithms on both synthetic and real sequence data and show that for protein sequences even our basic algorithm finds more plausible trees than existing methods for searching maximum likelihood phylogenies. Furthermore, our algorithms are dramatically faster than such methods, enabling, for the first time, phylogenetic analysis of large protein data sets in the maximum likelihood framework.     

Standard errors for EM estimates in generalized linear models with random effects

A procedure is derived for computing standard errors of EM estimates in generalized linear models with random effects. Quadrature formulas are used to approximate the integrals in the EM algorithm, where two different approaches are pursued, i.e., Gauss-Hermite quadrature in the case of Gaussian random effects and nonparametric maximum likelihood estimation for an unspecified random effect distribution. An approximation of the expected Fisher information matrix is derived from an expansion of the EM estimating equations. This allows for inferential arguments based on EM estimates, as demonstrated by an example and simulations.     

The EIM algorithm in the joint segregation analysis of quantitative traits

In this article, a new algorithm for obtaining the maximum likelihood estimators (MLEs) of parameters in the joint segregation analysis (JSA) of multiple generations of P1, F1, P2, F2 and F2:3 (MG5) for quantitative traits was set up. Firstly, owing to the fact that the component variance of the heterogeneous genotype in F2:3 included both the first-order genetic parameters (denoted by the means of distributions) and the second-order parameters, a simple closed form for the MLEs of the means of component distributions did not exist while the expectation and maximization (EM) algorithm was used. To simplify the estimation of parameters, the first partial derivative of the above variance on the mean in the sample log-likelihood function was omitted. However, this would be remedied by the iterated method. Then, variances of component distributions for segregating populations were partitioned into major-gene, polygenic and environmental variances so that the generally iterated formulae for estimating the means as well as polygenic and environmental variances of component distributions in the maximization step (M-step) of the EM algorithm were obtained. Therefore, the EM algorithm for estimating parameters in the JSA model for the MG5 was simplified. This is called the expectation and iterated maximization (EIM) algorithm. Finally, an example of the inheritance of the resistance of soybean to beanfly showed that the results of mixed inheritance analysis in this paper coincided with those in both Wang & Gai (2001) and Wei et al. (1989), so the EIM algorithm was appropriate.     

A classical likelihood based approach for admixture mapping using EM algorithm

Several disease-mapping methods have been proposed recently, which use the information generated by recent admixture of populations from historically distinct geographic origins. These methods include both classic likelihood and Bayesian approaches. In this study we directly maximize the likelihood function from the hidden Markov Model for admixture mapping using the EM algorithm, allowing for uncertainty in model parameters, such as the allele frequencies in the parental populations. We determined the robustness of the proposed method by examining the ancestral allele frequency estimate and individual marker-location specific ancestry when the data were generated by different population admixture models and no learning sample was used. The proposed method outperforms a widely used Bayesian MCMC strategy for data generated from various population admixture models. The multipoint information content for ancestry was derived based on the map provided by Smith et al. (2004) and the associated statistical power was calculated. We examined the distribution of admixture LD across the genome for both real and simulated data and established a threshold for genome wide significance applicable to admixture mapping studies. The software ADMIXPROGRAM for performing admixture mapping is available from authors.     

Genetic analysis of growth curves using the SAEM algorithm

The analysis of nonlinear function-valued characters is very important in genetic studies, especially for growth traits of agricultural and laboratory species. Inference in nonlinear mixed effects models is, however, quite complex and is usually based on likelihood approximations or Bayesian methods. The aim of this paper was to present an efficient stochastic EM procedure, namely the SAEM algorithm, which is much faster to converge than the classical Monte Carlo EM algorithm and Bayesian estimation procedures, does not require specification of prior distributions and is quite robust to the choice of starting values. The key idea is to recycle the simulated values from one iteration to the next in the EM algorithm, which considerably accelerates the convergence. A simulation study is presented which confirms the advantages of this estimation procedure in the case of a genetic analysis. The SAEM algorithm was applied to real data sets on growth measurements in beef cattle and in chickens. The proposed estimation procedure, as the classical Monte Carlo EM algorithm, provides significance tests on the parameters and likelihood based model comparison criteria to compare the nonlinear models with other longitudinal methods.     

Multilocus genomics of outcrossing plant populations

The structure and organization of natural plant populations can be understood by estimating the genetic parameters related to mating behavior, recombination frequency, and gene associations with DNA-based markers typed throughout the genome. We developed a statistical and computational model for estimating and testing these parameters from multilocus data collected in a natural population. This model, constructed by a maximum likelihood approach and implemented within the EM algorithm, is shown to be robust for simultaneously estimating the outcrossing rate, recombination frequencies and linkage disequilibria. The algorithm built with three or more markers allows the characterization of crossover interference in meiosis and high-order disequilibria among different genes, thus providing a powerful tool for illustrating a detailed picture of genetic diversity and organization in natural populations. Computer simulations demonstrate the statistical properties of the proposed model. This multilocus model will be useful for studying the pattern and amount of genetic variation within and among populations to further infer the evolutionary history of a plant species.     

A simple EM algorithm for capture-recapture data with categorical covariates

A simple EM algorithm is proposed for obtaining maximum likelihood estimates when fitting a loglinear model to data from k capture-recapture samples with categorical covariates. The method is used to analyze data on screening for the early detection of breast cancer.     

Estimation of dominance components in noninbred populations by using additive animal model residuals

In the case of noninbred and unselected populations with linkage equilibrium, the additive and dominance genetic effects are uncorrelated and the variance-covariance matrix of the second component is simply a product of its variance by a matrix that can be computed from the numerator relationship matrix A. The aim of this study is to present a new approach to estimate the dominance part with a reduced set of equations and hence a lower computing cost. The method proposed is based on the processing of the residual terms resulting from the BLUP methodology applied to an additive animal model. Best linear unbiased prediction of the dominance component d is almost identical to the one given by the full mixed model equations. Based on this approach, an algorithm for restricted maximum likelihood (REML) estimation of the variance components is also presented. By way of illustration, two numerical examples are given and a comparison between the parameters estimated with the expectation maximization (EM) algorithm and those obtained by the proposed algorithm is made. The proposed algorithm is iterative and yields estimates that are close to those obtained by EM, which is also iterative.     

High diversity of alpha-globin haplotypes in a Senegalese population, including many previously unreported variants.

RFLP haplotypes at the alpha-globin gene complex have been examined in 190 individuals from the Niokolo Mandenka population of Senegal: haplotypes were assigned unambiguously for 210 chromosomes. The Mandenka share with other African populations a sample size-independent haplotype diversity that is much greater than that in any non-African population: the number of haplotypes observed in the Mandenka is typically twice that seen in the non-African populations sampled to date. Of these haplotypes, 17.3% had not been observed in any previous surveys, and a further 19.1% have previously been reported only in African populations. The haplotype distribution shows clear differences between African and non-African peoples, but this is on the basis of population-specific haplotypes combined with haplotypes common to all. The relationship of the newly reported haplotypes to those previously recorded suggests that several mutation processes, particularly recombination as homologous exchange or gene conversion, have been involved in their production. A computer program based on the expectation-maximization (EM) algorithm was used to obtain maximum-likelihood estimates of haplotype frequencies for the entire data set: good concordance between the unambiguous and EM-derived sets was seen for the overall haplotype frequencies. Some of the low-frequency haplotypes reported by the estimation algorithm differ greatly, in structure, from those haplotypes known to be present in human populations, and they may not represent haplotypes actually present in the sample.     

EM Algorithm for Segregation Analysis

We consider the problem of estimating segregation ratios in families based on ascertainment through affected children, formulate it as an incomplete problem and work out the EM algorithm for maximum likelihood estimation of segregation ratios. We treat both the cases of known and unknown ascertainment probability. We also derive expressions for the covariance matrix of the estimators suitable for computing along with the EM algorithm. We illustrate the method with an example, compare the computational effort with that required in using the scoring method and argue that the EM algorithm is simpler.     

The EM algorithm for maximum likelihood estimation in the mover-stayer model

The discrete-time mover-stayer model (Blumen, Kogan, and McCarthy, 1955, The Industrial Mobility of Labor as a Probability Process, Ithaca, New York: Cornell University Press) is a useful model for studying changes over time in heterogeneous populations. Using the EM algorithm, we present an alternative method for obtaining maximum likelihood estimates of the parameters of the mover-stayer model, and consider an extension of the basic model to the problem of incomplete follow-up in panel studies. The models and the methods are illustrated with data from a community-based survey of changes in mental health status over a 1-year period.     

Semiparametric maximum likelihood for measurement error model regression

This paper presents an EM algorithm for semiparametric likelihood analysis of linear, generalized linear, and nonlinear regression models with measurement errors in explanatory variables. A structural model is used in which probability distributions are specified for (a) the response and (b) the measurement error. A distribution is also assumed for the true explanatory variable but is left unspecified and is estimated by nonparametric maximum likelihood. For various types of extra information about the measurement error distribution, the proposed algorithm makes use of available routines that would be appropriate for likelihood analysis of (a) and (b) if the true x were available. Simulations suggest that the semiparametric maximum likelihood estimator retains a high degree of efficiency relative to the structural maximum likelihood estimator based on correct distributional assumptions and can outperform maximum likelihood based on an incorrect distributional assumption. The approach is illustrated on three examples with a variety of structures and types of extra information about the measurement error distribution.     

Maximum likelihood methods for cure rate models with missing covariates

We propose maximum likelihood methods for parameter estimation for a novel class of semiparametric survival models with a cure fraction, in which the covariates are allowed to be missing. We allow the covariates to be either categorical or continuous and specify a parametric distribution for the covariates that is written as a sequence of one-dimensional conditional distributions. We propose a novel EM algorithm for maximum likelihood estimation and derive standard errors by using Louis's formula (Louis, 1982, Journal of the Royal Statistical Society, Series B 44, 226-233). Computational techniques using the Monte Carlo EM algorithm are discussed and implemented. A real data set involving a melanoma cancer clinical trial is examined in detail to demonstrate the methodology.     

A general probabilistic model for group independent component analysis and its estimation methods

Independent component analysis (ICA) has become an important tool for analyzing data from functional magnetic resonance imaging (fMRI) studies. ICA has been successfully applied to single-subject fMRI data. The extension of ICA to group inferences in neuroimaging studies, however, is challenging due to the unavailability of a prespecified group design matrix and the uncertainty in between-subjects variability in fMRI data. We present a general probabilistic ICA (PICA) model that can accommodate varying group structures of multisubject spatiotemporal processes. An advantage of the proposed model is that it can flexibly model various types of group structures in different underlying neural source signals and under different experimental conditions in fMRI studies. A maximum likelihood (ML) method is used for estimating this general group ICA model. We propose two expectation-maximization (EM) algorithms to obtain the ML estimates. The first method is an exact EM algorithm, which provides an exact E-step and an explicit noniterative M-step. The second method is a variational approximation EM algorithm, which is computationally more efficient than the exact EM. In simulation studies, we first compare the performance of the proposed general group PICA model and the existing probabilistic group ICA approach. We then compare the two proposed EM algorithms and show the variational approximation EM achieves comparable accuracy to the exact EM with significantly less computation time. An fMRI data example is used to illustrate application of the proposed methods.     

基于EM算法的遗传重组率估计方法

EM算法是在不完全信息资料下实现参数极大似然估计的一种通用方法．本文导出了双位点不同标记类型,包括共显性-共显性,共显性-显性和显性-显性3种模式下,估计遗传重组率的EM算法,以及获得重组率抽样方差的Bootstrap方法;并将之推广到部分个体缺失标记基因型(未检测到电泳谱带)下的重组率估计．通过大量Monte Carlo模拟研究发现: (1)连锁紧密时,样本容量对重组率的估计影响不大;连锁松散时,需要较大样本容量才可检测到连锁以及实现重组率的较精确估计．(2)用包含缺失标记的所有个体估计重组率比仅用其中的非缺失标记个体估计更准确,且可显著提高连锁检测的统计功效．     

Mixture models for continuous data in dose-response studies when some animals are unaffected by treatment.

A mixture model is described for dose-response studies where measurements on a continuous variable suggest that some animals are not affected by treatment. The model combines a logistic regression on dose for the probability an animal will "respond" to treatment with a linear regression on dose for the mean of the responders. Maximum likelihood estimation via the EM algorithm is described and likelihood ratio tests are used to distinguish between the full model and meaningful reduced-parameter versions. Use of the model is illustrated with three real-data examples.