Obtaining Critical Values for Simultaneous Confidence Intervals and Multiple Testing

There are many situations where it is desired to make simultaneous tests or give simultaneous confidence intervals for linear combinations (contrasts) of population or treatment means. Somerville (1997, 1999) developed algorithms for calculating the critical values for a large class of simultaneous tests and simultaneous confidence intervals. Fortran 90 and SAS‐IML batch programs and interactive programs were developed. These programs calculate the critical values for 15 different simultaneous confidence interval procedures (and the corresponding simultaneous tests) and for arbitrary procedures where the user specifies a combination of one and two sided contrasts. The programs can also be used to obtain the constants for “step‐down” testing of multiple hypotheses. This paper gives examples of the use of the algorithms and programs and illustrates their versatility and generality. The designs need not be balanced, multiple covariates may be present and there may be many missing values. The use of multiple regression and dummy variables to obtain the required variance covariance matrix is illustrated. Under weak normality assumptions the methods are “exact” and make the use of approximate methods or “simulation” unnecessary.     

Conditional Inference Following Group Sequential Testing

The work of Fisher (1959) and Buehler (1959) discuss the importance of conditioning on recognizable subsets of the sample space. The stopping time yields an easily identifiable partition of the sample space when considering group sequential testing. We first present confidence intervals that are correct when conditioning on the subset of data for which a trial stopped at a particular analysis. These intervals have very desirable properties for observations that are highly unusual (given any value of the mean). In addition, they provide insight into how information about the mean is distributed between the two sufficient statistics. We then use conditional coverage probabilities to compare the sample mean, stagewise, and repeated confidence intervals. We find that none of these intervals outperforms the others when conditioning on stopping time, and no interval is a uniformly acceptable performer.     

Reliable confidence intervals in quantitative genetics: narrow-sense heritability

Many quantitative genetic statistics are functions of variance components, for which a large number of replicates is needed for precise estimates and reliable measures of uncertainty, on which sound interpretation depends. Moreover, in large experiments the deaths of some individuals can occur, so methods for analysing such data need to be robust to missing values. We show how confidence intervals for narrow-sense heritability can be calculated in a nested full-sib/half-sib breeding design (males crossed with several females) in the presence of missing values. Simulations indicate that the method provides accurate results, and that estimator uncertainty is lowest for sampling designs with many males relative to the number of females per male, and with more females per male than progenies per female. Missing data generally had little influence on estimator accuracy, thus suggesting that the overall number of observations should be increased even if this results in unbalanced data. We also suggest the use of parametrically simulated data for prior investigation of the accuracy of planned experiments. Together with the proposed confidence intervals an informed decision on the optimal sampling design is possible, which allows efficient allocation of resources.     

Model selection of generalized estimating equations with multiply imputed longitudinal data

Longitudinal data often encounter missingness with monotone and/or intermittent missing patterns. Multiple imputation (MI) has been popularly employed for analysis of missing longitudinal data. In particular, the MI‐GEE method has been proposed for inference of generalized estimating equations (GEE) when missing data are imputed via MI. However, little is known about how to perform model selection with multiply imputed longitudinal data. In this work, we extend the existing GEE model selection criteria, including the “quasi‐likelihood under the independence model criterion” (QIC) and the “missing longitudinal information criterion” (MLIC), to accommodate multiple imputed datasets for selection of the MI‐GEE mean model. According to real data analyses from a schizophrenia study and an AIDS study, as well as simulations under nonmonotone missingness with moderate proportion of missing observations, we conclude that: (i) more than a few imputed datasets are required for stable and reliable model selection in MI‐GEE analysis; (ii) the MI‐based GEE model selection methods with a suitable number of imputations generally perform well, while the naive application of existing model selection methods by simply ignoring missing observations may lead to very poor performance; (iii) the model selection criteria based on improper (frequentist) multiple imputation generally performs better than their analogies based on proper (Bayesian) multiple imputation.     

A solution to the problem of monotone likelihood in Cox regression

The phenomenon of monotone likelihood is observed in the fitting process of a Cox model if the likelihood converges to a finite value while at least one parameter estimate diverges to +/- infinity. Monotone likelihood primarily occurs in small samples with substantial censoring of survival times and several highly predictive covariates. Previous options to deal with monotone likelihood have been unsatisfactory. The solution we suggest is an adaptation of a procedure by Firth (1993, Biometrika 80, 27-38) originally developed to reduce the bias of maximum likelihood estimates. This procedure produces finite parameter estimates by means of penalized maximum likelihood estimation. Corresponding Wald-type tests and confidence intervals are available, but it is shown that penalized likelihood ratio tests and profile penalized likelihood confidence intervals are often preferable. An empirical study of the suggested procedures confirms satisfactory performance of both estimation and inference. The advantage of the procedure over previous options of analysis is finally exemplified in the analysis of a breast cancer study.     

A Note on Testing Equality of Means when Sample Sizes are Random with Application

The problem of testing the equality of means of two normal populations is considered when independent random samples of random sizes are given with the total number of observations from both populations being a fixed number. An application in forestry is discussed.     

Personality traits in captive lion-tailed macaques (Macaca silenus)

Personality influences an individual's perception of a situation and orchestrates behavioral responses. It is an important factor in elucidating variation in behavior both within and between species. The major focus of this research was to test a method that differs from those used in most previous personality studies, while investigating the personality traits of 52 captive lion-tailed macaques from four zoos. In this study, data from behavioral observations, a P-type principal components analysis (PCA), and bootstrapped confidence intervals as criteria for judging the significance of factor loadings were used rather than subjective ratings, R-type factor analyses, and arbitrary rules of thumb to determine significance. We investigated the relationships among individual component scores and sex, hormonal status, and dominance rank (controlling for age and social group) using a multiple regression analysis with bootstrapped confidence intervals. Three personality dimensions emerged from this analysis: Component 1 contained Extraversion-like behaviors related to sociability and affiliativeness. The higher mean Component score for females suggests that they are more "extraverted" than males. Only agonistic behaviors were significantly related to component 2. High-ranking individuals exhibited higher mean Component 2 scores than mid- or low-ranked individuals. Bold and cautious behaviors both loaded positively on Component 3, suggesting a dimension related to curiosity. The mean Component 3 score for females was higher than the mean score for males. The method used in this study should facilitate intraspecific and general interspecific comparisons. Developing a standardized trait term list that is applicable to many species, and collecting trait term data in the same manner and concurrent with behavioral observations (and physiologic measures when feasible) could prove useful in primate research and should be explored.     

A multivariate two-sample mean test for small sample size and missing data

We develop a new statistic for testing the equality of two multivariate mean vectors. A scaled chi-squared distribution is proposed as an approximating null distribution. Because the test statistic is based on componentwise statistics, it has the advantage over Hotelling's T2 test of being applicable to the case where the dimension of an observation exceeds the number of observations. An appealing feature of the new test is its ability to handle missing data by relying on only componentwise sample moments. Monte Carlo studies indicate good power compared to Hotelling's T2 and a recently proposed test by Srivastava (2004, Technical Report, University of Toronto). The test is applied to drug discovery data.     

Nonparametric methods for analyzing incomplete nondecreasing repeated measurements

In comparing the effectiveness of two treatments, suppose that nondecreasing repeated measurements of the same characteristic are scheduled to be taken over a common set of time points for each study subject. A class of univariate one-sided global asymptotically distribution-free tests is proposed to test the equality of the two treatments. The test procedures allow different patterns of missing observations in the two groups to be compared, although the missing data mechanism is required to be independent of the observations in each treatment group. Test-based point and interval estimators of the global treatment difference are given. Multiple inference procedures are also provided to examine the time trend of treatment differences over the entire study. The proposed methods are illustrated by an example from a bladder cancer study.     

Model Selection for Generalized Estimating Equations Accommodating Dropout Missingness

Summary The generalized estimating equation (GEE) has been a popular tool for marginal regression analysis with longitudinal data, and its extension, the weighted GEE approach, can further accommodate data that are missing at random (MAR). Model selection methodologies for GEE, however, have not been systematically developed to allow for missing data. We propose the missing longitudinal information criterion (MLIC) for selection of the mean model, and the MLIC for correlation (MLICC) for selection of the correlation structure in GEE when the outcome data are subject to dropout/monotone missingness and are MAR. Our simulation results reveal that the MLIC and MLICC are effective for variable selection in the mean model and selecting the correlation structure, respectively. We also demonstrate the remarkable drawbacks of naively treating incomplete data as if they were complete and applying the existing GEE model selection method. The utility of proposed method is further illustrated by two real applications involving missing longitudinal outcome data.     

Estimation of competing risks with general missing pattern in failure types

In competing risks data, missing failure types (causes) is a very common phenomenon. In this work, we consider a general missing pattern in which, if a failure type is not observed, one observes a set of possible types containing the true type, along with the failure time. We first consider maximum likelihood estimation with missing-at-random assumption via the expectation maximization (EM) algorithm. We then propose a Nelson-Aalen type estimator for situations when certain information on the conditional probability of the true type given a set of possible failure types is available from the experimentalists. This is based on a least-squares type method using the relationships between hazards for different types and hazards for different combinations of missing types. We conduct a simulation study to investigate the performance of this method, which indicates that bias may be small, even for high proportion of missing data, for sufficiently large number of observations. The estimates are somewhat sensitive to misspecification of the conditional probabilities of the true types when the missing proportion is high. We also consider an example from an animal experiment to illustrate our methodology.     

On Comparing Correlated Means in the Presence of Incomplete Data

Two statistics are proposed for testing the hypothesis of equality of the means of a bivariate normal distribution with unknown common variance and correlation coefficient when observations are missing on both variates. One of the statistics reduces to the one proposed by Bhoj (1978, 1984) when the unpaired observations on the variates are equal. The distributions of the statistics are approximated by well known distributions under the null hypothesis. The empirical powers of the tests are computed and compared with those of some known statistics. The comparison supports the use of one of the statistics proposed in this paper.     

Youden Index and optimal cut-point estimated from observations affected by a lower limit of detection

The receiver operating characteristic (ROC) curve is used to evaluate a biomarker's ability for classifying disease status. The Youden Index (J), the maximum potential effectiveness of a biomarker, is a common summary measure of the ROC curve. In biomarker development, levels may be unquantifiable below a limit of detection (LOD) and missing from the overall dataset. Disregarding these observations may negatively bias the ROC curve and thus J. Several correction methods have been suggested for mean estimation and testing; however, little has been written about the ROC curve or its summary measures. We adapt non-parametric (empirical) and semi-parametric (ROC-GLM [generalized linear model]) methods and propose parametric methods (maximum likelihood (ML)) to estimate J and the optimal cut-point (c *) for a biomarker affected by a LOD. We develop unbiased estimators of J and c * via ML for normally and gamma distributed biomarkers. Alpha level confidence intervals are proposed using delta and bootstrap methods for the ML, semi-parametric, and non-parametric approaches respectively. Simulation studies are conducted over a range of distributional scenarios and sample sizes evaluating estimators' bias, root-mean square error, and coverage probability; the average bias was less than one percent for ML and GLM methods across scenarios and decreases with increased sample size. An example using polychlorinated biphenyl levels to classify women with and without endometriosis illustrates the potential benefits of these methods. We address the limitations and usefulness of each method in order to give researchers guidance in constructing appropriate estimates of biomarkers' true discriminating capabilities.     

Methods for comparison of parameters from longitudinal rhythmometric models with multiple components

Multiple components linear least-squares methods have been proposed for the detection of periodic components in nonsinusoidal longitudinal time series. However, a proper test for comparison of parameters obtained from this method for two or more time series is not yet available. Accordingly, we propose two methods, one parametric and one nonparametric, to compare parameters from rhythmometric models with multiple components. The parametric method is based on techniques commonly and generally employed in linear regression analysis. The comparison of parameters among two or more time series is accomplished by the use of so-called dummy variables. The nonparametric method is based on bootstrap techniques. This approach basically tests if the difference in any given parameter obtained by fitting a model with the same periods to two different longitudinal time series differs from zero. This method calculates a confidence interval for the difference in the tested parameter. If this interval does not contain zero, it can be concluded that the parameters obtained from the two time series are different with high probability. An estimation of the p-value for the corresponding test can also be calculated. By the use of similar bootstrap techniques, confidence intervals can also be obtained for any parameter derived from the multiple component fit of several periods to nonsinusoidal longitudinal time series, including the orthophase (peak time), bathyphase (trough time), and global amplitude (difference between the maximum and the minimum) of the fitted model waveform. These methods represent a valuable tool for the comparison of rhythm parameters obtained by multiple component analysis, and they render this approach as a generally applicable one for waveform representation and detection of periodicities in nonsinusoidal, sparse, and noisy longitudinal time series sampled with either equidistant or unequidistant observations.     

Vagaries in the delimitation of character states in quantitative variation--an experimental study

An experimental study on the delimitation of character states in continuous variation indicates that (1) the way data are presented influences the assignment of character states and (2) states in the same data set are delimited in various ways by different individuals. Forty-nine individuals were given a set of graphs denoting variation of 10 characters in the genus Kalmia (Ericaceae) and outgroups, all identification having been removed from the graphs. The variation was represented in one of three ways: as 95% confidence intervals on a linear scale, as 95% confidence intervals on a log10 scale, or with bars showing SD x 2 on a linear scale. No two individuals scored a set of graphs in the same way, and only one character in one representation was scored identically by all individuals; the scoring for this character was completely different when the ordinate was changed from linear to logarithmic. Together, the 49 individuals delimited states within each character between 9 and 16 different ways. In general, variation represented by 2 x SD bars elicited the largest numbers of different scorings, yet with a relatively low number of states; the complexity of the patterns in the graphs in this representation was greatest. Expert knowledge appears to be of dubious value in delimiting states in such variation, and if such characters are to be used in phylogenetic analyses, states could be delimited by people who know nothing of the details of the study being scored; in any case, presentation of data and an explicit protocol to follow when delimiting states are essential. In converting data of this type into character states, psychological factors are particularly likely to come into play. Other implications of our experiments include the severe underdetermination of some phylogenetic hypotheses by observation and the heterogeneous nature of morphological data.     

Confidence intervals for heritability for two-factor mating design single environment linear models

Summary Precision measurement is an essential part of heritability estimate interpretation. Approximate standard errors are commonly used as measures of precision for heritability on a progeny mean basis (H). Their derivation, however, is not inferred from the distribution theory for H. F-distribution based exact confidence intervals have been derived for some onefactor mating design H estimators. Extension of the confidence interval results from one-factor to twofactor mating designs is reported in this paper. Functions of heritability on a full-sib or half-sib progeny mean basis from nested or factorial mating design parameters were distributed according to the F-distribution. Exact confidence intervals were derived for heritability on a full-sib progeny mean basis. Exact confidence intervals for heritability on a half-sib progeny basis were adapted from previous results. Maize (Zea mays L.) data were used to estimate confidence intervals. Complete equations were given for interpolation in F-tables.Oregon Agricultural Experiment Station Technical Paper No. 7659     

A Two Sample CRAMÉR-VON MISES Test for Randomly Censored Data

A CRAMÉR-VON MISES type statistic is introduced for testing the equality of the underlying survival distributions of two populations when observations are subject to arbitrary right censorship. The statistic is appropriate in testing problems where a two-sided alternative is of interest. The asymptotic distribution of the statistic is found; under certain circumstances, the limiting distribution coincides with that of a one sample CRAMÉR-VON MISES type statistic for randomly censored data investigated previously. Approximations to the asymptotic distribution are discussed; an example is given.     

Exact inference for growth curves with intraclass correlation structure

We consider repeated observations taken over time for each of several subjects. For example, one might consider the growth curve of a cohort of babies over time. We assume a simple linear growth curve model. Exact results based on sufficient statistics (exact tests of the null hypothesis that a coefficient is zero, or exact confidence intervals for coefficients) are not available to make inference on regression coefficients when an intraclass correlation structure is assumed. This paper will demonstrate that such exact inference is possible using generalized inference.     

ROC curve inference for best linear combination of two biomarkers subject to limits of detection

The receiver operating characteristic (ROC) curve is a tool commonly used to evaluate biomarker utility in clinical diagnosis of disease. Often, multiple biomarkers are developed to evaluate the discrimination for the same outcome. Levels of multiple biomarkers can be combined via best linear combination (BLC) such that their overall discriminatory ability is greater than any of them individually. Biomarker measurements frequently have undetectable levels below a detection limit sometimes denoted as limit of detection (LOD). Ignoring observations below the LOD or substituting some replacement value as a method of correction has been shown to lead to negatively biased estimates of the area under the ROC curve for some distributions of single biomarkers. In this paper, we develop asymptotically unbiased estimators, via the maximum likelihood technique, of the area under the ROC curve of BLC of two bivariate normally distributed biomarkers affected by LODs. We also propose confidence intervals for this area under curve. Point and confidence interval estimates are scrutinized by simulation study, recording bias and root mean square error and coverage probability, respectively. An example using polychlorinated biphenyl (PCB) levels to classify women with and without endometriosis illustrates the potential benefits of our methods.     

Regression imputation of missing values in longitudinal data sets

A stand-alone, menu-driven PC program, written in GAUSS, which can be used to estimate missing observations in longitudinal data sets is described and made available to interested readers. The program is limited to the situation in which we have complete data on N cases at each of the planned times of measurement t₁, t₂,…, t_T; and we wish to use this information, together with the non-missing values for n additional cases, to estimate the missing values for those cases. The augmented data matrix may be saved in an ASCII file and subsequently imported into programs requiring complete data. The use of the program is illustrated. Ten percent of the observations in a data set consisting of mandibular ramus height measurements for N = 12 young male rhesus monkeys measured at T = 5 time points are randomly discarded. The augmented data matrix is used to determine the lowest degree polynomial adequate to fit the average growth curve (AGC); the regression coefficients are estimated and confidence intervals for them are determined; and confidence bands for the AGC are constructed. The results are compared with those obtained when the original complete data set is used.