A comparison of test statistics for assessing the effects of concomitant variables in survival analysis

In data analysis involving the proportional-hazards regression model due to Cox (1972, Journal of the Royal Statistical Society, Series B 34, 187-220), the test criteria commonly used for assessing the partial contribution to survival of subsets of concomitant variables are the classical likelihood ratio (LR) and Wald statistics. This paper presents an investigation of three other test criteria with potentially major computational advantages over the classical tests, especially for stepwise variable selection in moderate to large data sets. The alternative criteria considered are Rao's efficient score statistic and two other score statistics. Under the Cox model, the performance of these tests is examined empirically and compared with the performance of the LR and Wald statistics. Rao's test performs comparably to the LR test in all the cases considered. The performance of the other criteria is competitive in many cases. The use of these statistics is illustrated in a study of coronary artery disease.     

Exploring the information in p-values for the analysis and planning of multiple-test experiments

Summary A new methodology is proposed for estimating the proportion of true null hypotheses in a large collection of tests. Each test concerns a single parameter δ whose value is specified by the null hypothesis. We combine a parametric model for the conditional cumulative distribution function (CDF) of the p‐value given δ with a nonparametric spline model for the density g(δ) of δ under the alternative hypothesis. The proportion of true null hypotheses and the coefficients in the spline model are estimated by penalized least squares subject to constraints that guarantee that the spline is a density. The estimator is computed efficiently using quadratic programming. Our methodology produces an estimate of the density of δ when the null is false and can address such questions as “when the null is false, is the parameter usually close to the null or far away?” This leads us to define a falsely interesting discovery rate (FIDR), a generalization of the false discovery rate. We contrast the FIDR approach to Efron's (2004, Journal of the American Statistical Association 99, 96–104) empirical null hypothesis technique. We discuss the use of in sample size calculations based on the expected discovery rate (EDR). Our recommended estimator of the proportion of true nulls has less bias compared to estimators based upon the marginal density of the p‐values at 1. In a simulation study, we compare our estimators to the convex, decreasing estimator of Langaas, Lindqvist, and Ferkingstad (2005, Journal of the Royal Statistical Society, Series B 67, 555–572). The most biased of our estimators is very similar in performance to the convex, decreasing estimator. As an illustration, we analyze differences in gene expression between resistant and susceptible strains of barley.     

A Bayesian Chi‐Squared Goodness‐of‐Fit Test for Censored Data Models

Summary We propose a Bayesian chi‐squared model diagnostic for analysis of data subject to censoring. The test statistic has the form of Pearson's chi‐squared test statistic and is easy to calculate from standard output of Markov chain Monte Carlo algorithms. The key innovation of this diagnostic is that it is based only on observed failure times. Because it does not rely on the imputation of failure times for observations that have been censored, we show that under heavy censoring it can have higher power for detecting model departures than a comparable test based on the complete data. In a simulation study, we show that tests based on this diagnostic exhibit comparable power and better nominal Type I error rates than a commonly used alternative test proposed by Akritas (1988, Journal of the American Statistical Association 83, 222–230). An important advantage of the proposed diagnostic is that it can be applied to a broad class of censored data models, including generalized linear models and other models with nonidentically distributed and nonadditive error structures. We illustrate the proposed model diagnostic for testing the adequacy of two parametric survival models for Space Shuttle main engine failures.     

A class of multiplicity adjusted tests for spatial clustering based on case-control point data

A class of tests with quadratic forms for detecting spatial clustering of health events based on case-control point data is proposed. It includes Cuzick and Edwards's test statistic (1990, Journal of the Royal Statistical Society, Series B 52, 73-104). Although they used the property of asymptotic normality of the test statistic, we show that such an approximation is generally poor for moderately large sample sizes. Instead, we suggest a central chi-square distribution as a better approximation to the asymptotic distribution of the test statistic. Furthermore, not only to estimate the optimal value of the unknown parameter on the scale of cluster but also to adjust for multiple testing due to repeating the procedure by changing the parameter value, we propose the minimum of the profile p-value of the test statistic for the parameter as an integrated test statistic. We also provide a statistic to estimate the areas or cases which make large contributions to significant clustering. The proposed methods are illustrated with a data set concerning the locations of cases of childhood leukemia and lymphoma and another on early medieval grave site locations consisting of affected and nonaffected grave sites.     

Estimating the null distribution to adjust observed confidence levels for genome-scale screening

In a novel approach to the multiple testing problem, Efron (2004, Journal of the American Statistical Association 99, 96-104; 2007a Journal of the American Statistical Association 102, 93-103; 2007b, Annals of Statistics 35, 1351-1377) formulated estimators of the distribution of test statistics or nominal p-values under a null distribution suitable for modeling the data of thousands of unaffected genes, nonassociated single-nucleotide polymorphisms, or other biological features. Estimators of the null distribution can improve not only the empirical Bayes procedure for which it was originally intended, but also many other multiple-comparison procedures. Such estimators in some cases improve the proposed multiple-comparison procedure (MCP) based on a recent non-Bayesian framework of minimizing expected loss with respect to a confidence posterior, a probability distribution of confidence levels. The flexibility of that MCP is illustrated with a nonadditive loss function designed for genomic screening rather than for validation. The merit of estimating the null distribution is examined from the vantage point of the confidence-posterior MCP (CPMCP). In a generic simulation study of genome-scale multiple testing, conditioning the observed confidence level on the estimated null distribution as an approximate ancillary statistic markedly improved conditional inference. Specifically simulating gene expression data, however, indicates that estimation of the null distribution tends to exacerbate the conservative bias that results from modeling heavy-tailed data distributions with the normal family. To enable researchers to determine whether to rely on a particular estimated null distribution for inference or decision making, an information-theoretic score is provided. As the sum of the degree of ancillarity and the degree of inferential relevance, the score reflects the balance conditioning would strike between the two conflicting terms. The CPMCP and other methods introduced are applied to gene expression microarray data.     

A simple test of association for contingency tables with multiple column responses

Loughin and Scherer (1998, Biometrics 54, 630-637) investigated tests of association in two-way tables when one of the categorical variables allows for multiple-category responses from individual respondents. Standard chi-squared tests are invalid in this case, and they developed a bootstrap test procedure that provides good control of test levels under the null hypothesis. This procedure and some others that have been proposed are computationally involved and are based on techniques that are relatively unfamiliar to many practitioners. In this paper, the methods introduced by Rao and Scott (1981, Journal of the American Statistical Association 76, 221-230) for analyzing complex survey data are used to develop a simple test based on a corrected chi-squared statistic.     

The Reliability of Tests of Hypotheses when Overdispersed Logistic-Linear Models are Fitted by Maximum Quasi-Likelihood

When overdispersed logistic-linear models are fitted by maximum quasi-likelihood hypotheses can be tested by comparing either the Wald statistic, or the quasi-likelihood score statistic, or the quasi likelihood-ratio statistic, with the approximating null X² distribution. This paper reports a simulation study of the reliability of these tests. Some factors affecting their relative reliabilities are identified. An extended quasi-likelihood ratio test is also considered.     

Testing approaches for overdispersion in poisson regression versus the generalized poisson model

Overdispersion is a common phenomenon in Poisson modeling, and the negative binomial (NB) model is frequently used to account for overdispersion. Testing approaches (Wald test, likelihood ratio test (LRT), and score test) for overdispersion in the Poisson regression versus the NB model are available. Because the generalized Poisson (GP) model is similar to the NB model, we consider the former as an alternate model for overdispersed count data. The score test has an advantage over the LRT and the Wald test in that the score test only requires that the parameter of interest be estimated under the null hypothesis. This paper proposes a score test for overdispersion based on the GP model and compares the power of the test with the LRT and Wald tests. A simulation study indicates the score test based on asymptotic standard Normal distribution is more appropriate in practical application for higher empirical power, however, it underestimates the nominal significance level, especially in small sample situations, and examples illustrate the results of comparing the candidate tests between the Poisson and GP models. A bootstrap test is also proposed to adjust the underestimation of nominal level in the score statistic when the sample size is small. The simulation study indicates the bootstrap test has significance level closer to nominal size and has uniformly greater power than the score test based on asymptotic standard Normal distribution. From a practical perspective, we suggest that, if the score test gives even a weak indication that the Poisson model is inappropriate, say at the 0.10 significance level, we advise the more accurate bootstrap procedure as a better test for comparing whether the GP model is more appropriate than Poisson model. Finally, the Vuong test is illustrated to choose between GP and NB2 models for the same dataset.     

A Space–Time Scan Statistic for Detecting Emerging Outbreaks

Summary As a major analytical method for outbreak detection, Kulldorff's space–time scan statistic (2001, Journal of the Royal Statistical Society, Series A 164, 61–72) has been implemented in many syndromic surveillance systems. Since, however, it is based on circular windows in space, it has difficulty correctly detecting actual noncircular clusters. Takahashi et al. (2008, International Journal of Health Geographics 7 , 14) proposed a flexible space–time scan statistic with the capability of detecting noncircular areas. It seems to us, however, that the detection of the most likely cluster defined in these space–time scan statistics is not the same as the detection of localized emerging disease outbreaks because the former compares the observed number of cases with the conditional expected number of cases. In this article, we propose a new space–time scan statistic which compares the observed number of cases with the unconditional expected number of cases, takes a time‐to‐time variation of Poisson mean into account, and implements an outbreak model to capture localized emerging disease outbreaks more timely and correctly. The proposed models are illustrated with data from weekly surveillance of the number of absentees in primary schools in Kitakyushu‐shi, Japan, 2006.     

Power of the classical twin design revisited.

Statistical power of the classical twin design was revisited. The approximate sampling variances of a least-squares estimate of the heritability in a univariate analysis and estimate of the genetic correlation coefficient in a bivariate analysis were derived analytically for the ACE model. Statistical power to detect additive genetic variation under the ACE model was derived analytically for least-squares, goodness-of-fit and maximum likelihood-based test statistics. The noncentrality parameter for the likelihood ratio test statistic is shown to be a simple function of the MZ and DZ intraclass correlation coefficients and the proportion of MZ and DZ twin pairs in the sample. All theoretical results were validated using simulation. The derived expressions can be used to calculate power of the classical twin design in a simple and rapid manner.     

Small-sample confidence regions in exponential families

This article presents an algorithm for small-sample conditional confidence regions for two or more parameters for any discrete regression model in the generalized linear interactive model family. Regions are constructed by careful inversion of conditional hypothesis tests. This method presupposes the use of approximate or exact techniques for enumerating the sample space for some components of the vector of sufficient statistics conditional on other components. Such enumeration may be performed exactly or by exact or approximate Monte Carlo, including the algorithms of Kolassa and Tanner (1994, Journal of the American Statistical Association 89, 697-702; 1999, Biometrics 55, 246-251). This method also assumes that one can compute certain conditional probabilities for a fixed value of the parameter vector. Because of a property of exponential families, one can use this set of conditional probabilities to directly compute the conditional probabilities associated with any other value of the vector of the parameters of interest. This observation dramatically reduces the computational effort required to invert the hypothesis test to obtain the confidence region. To construct a region with confidence level 1 - alpha, the algorithm begins with a grid of values for the parameters of interest. For each parameter vector on the grid (corresponding to the current null hypothesis), one transforms the initial set of conditional probabilities using exponential tilting and then calculates the p value for this current null hypothesis. The confidence region is the set of parameter values for which the p value is at least alpha.     

Survival analysis in clinical trials: past developments and future directions

The field of survival analysis emerged in the 20th century and experienced tremendous growth during the latter half of the century. The developments in this field that have had the most profound impact on clinical trials are the Kaplan-Meier (1958, Journal of the American Statistical Association 53, 457-481) method for estimating the survival function, the log-rank statistic (Mantel, 1966, Cancer Chemotherapy Report 50, 163-170) for comparing two survival distributions, and the Cox (1972, Journal of the Royal Statistical Society, Series B 34, 187-220) proportional hazards model for quantifying the effects of covariates on the survival time. The counting-process martingale theory pioneered by Aalen (1975, Statistical inference for a family of counting processes, Ph.D. dissertation, University of California, Berkeley) provides a unified framework for studying the small- and large-sample properties of survival analysis statistics. Significant progress has been achieved and further developments are expected in many other areas, including the accelerated failure time model, multivariate failure time data, interval-censored data, dependent censoring, dynamic treatment regimes and causal inference, joint modeling of failure time and longitudinal data, and Baysian methods.     

Median Tests for Censored Survival Data; a Contingency Table Approach

Summary The median failure time is often utilized to summarize survival data because it has a more straightforward interpretation for investigators in practice than the popular hazard function. However, existing methods for comparing median failure times for censored survival data either require estimation of the probability density function or involve complicated formulas to calculate the variance of the estimates. In this article, we modify a K ‐sample median test for censored survival data ( Brookmeyer and Crowley, 1982 , Journal of the American Statistical Association 77, 433–440) through a simple contingency table approach where each cell counts the number of observations in each sample that are greater than the pooled median or vice versa. Under censoring, this approach would generate noninteger entries for the cells in the contingency table. We propose to construct a weighted asymptotic test statistic that aggregates dependent χ² ‐statistics formed at the nearest integer points to the original noninteger entries. We show that this statistic follows approximately a χ² ‐distribution with k? 1 degrees of freedom. For a small sample case, we propose a test statistic based on combined p ‐values from Fisher’s exact tests, which follows a χ² ‐distribution with 2 degrees of freedom. Simulation studies are performed to show that the proposed method provides reasonable type I error probabilities and powers. The proposed method is illustrated with two real datasets from phase III breast cancer clinical trials.     

Testing the Equality of Dependent Variances

The likelihood ratio test for testing equality of vgE;2 correlated variables is developed. In general, evaluation of the test statistic involves the iterative optimization of a likelihood function with 1 + v(v – 1)/2 parameters. The explicit form of the test statistic is derived in the bivariate case, and an iterative algorithm for determining the maximum likelihood estimates is suggested. A limited Monte Carlo study determines the behavior of the proposed procedure under the null hypothesis and variety of parameter values.     

Robustness of the latent variable model for correlated binary data

The marginal regression model offers a useful alternative to conditional approaches to analyzing binary data (Liang, Zeger, and Qaqish, 1992, Journal of the Royal Statistical Society, Series B 54, 3-40). Instead of modelling the binary data directly as do Liang and Zeger (1986, Biometrika 73, 13-22), the parametric marginal regression model developed by Qu et al. (1992, Biometrics 48, 1095-1102) assumes that there is an underlying multivariate normal vector that gives rise to the observed correlated binary outcomes. Although this parametric approach provides a flexible way to model different within-cluster correlation structures and does not restrict the parameter space, it is of interest to know how robust the parameter estimates are with respect to choices of the latent distribution. We first extend the latent modelling to include multivariate t-distributed latent vectors and assess the robustness in this class of distributions. Then we show through a simulation that the parameter estimates are robust with respect to the latent distribution even if latent distribution is skewed. In addtion to this empirical evidence for robustness, we show through the iterative algorithm that the robustness of the regression coefficents with respect to misspecifications of covariance structure in Liang and Zeger's model in fact indicates robustness with respect to underlying distributional assumptions of the latent vector in the latent variable model.     

Permutation tests for random effects in linear mixed models

Inference regarding the inclusion or exclusion of random effects in linear mixed models is challenging because the variance components are located on the boundary of their parameter space under the usual null hypothesis. As a result, the asymptotic null distribution of the Wald, score, and likelihood ratio tests will not have the typical χ(2) distribution. Although it has been proved that the correct asymptotic distribution is a mixture of χ(2) distributions, the appropriate mixture distribution is rather cumbersome and nonintuitive when the null and alternative hypotheses differ by more than one random effect. As alternatives, we present two permutation tests, one that is based on the best linear unbiased predictors and one that is based on the restricted likelihood ratio test statistic. Both methods involve weighted residuals, with the weights determined by the among- and within-subject variance components. The null permutation distributions of our statistics are computed by permuting the residuals both within and among subjects and are valid both asymptotically and in small samples. We examine the size and power of our tests via simulation under a variety of settings and apply our test to a published data set of chronic myelogenous leukemia patients.     

A Combined Invariant Test for a Null Variance Ratio

This paper investigates the properties of a test that combines the Wald and LMPI (locally most powerful invariant) tests of variance components in unbalanced mixed models. The combined test statistic is easily computed and its null distribution may be approximated by a central F distribution with the degrees of freedom of the numerator adjusted in accordance with the degree of imbalance of the design. Numerical methods are used to show that the approximation is accurate over a wide range of conditions and that the efficiency of the combined test, relative to the power envelope, is satisfactorily high overall.     

On the exact null distribution of the generalised likelihood ratio test for analysing unreplicated factorial designs

Al-Shiha and Yang (1999) proposed a multistage procedure for analysing unreplicated factorial experiments, which is based on the statistic that is derived from the generalised likelihood ratio test statistic under the assumption of normality. It was shown by their simulation study that the method is quite competitive with Lenth's (1989) method. In their paper, because of the difficulty of determining the null distribution analytically, the quantiles of the null distribution were empirically simulated. In this paper, we give the exact null distribution of their test statistic, which makes it possible to calculate the critical values of the test.     

Confidence Interval for Rate Ratio in a 2 × 2 Table with Structural Zero: An Application in Assessing False-Negative Rate Ratio When Combining Two Diagnostic Tests

Summary . In this article, we consider problems with correlated data that can be summarized in a 2 × 2 table with structural zero in one of the off‐diagonal cells. Data of this kind sometimes appear in infectious disease studies and two‐step procedure studies. Lui (1998, Biometrics 54, 706–711) considered confidence interval estimation of rate ratio based on Fieller‐type, Wald‐type, and logarithmic transformation statistics. We reexamine the same problem under the context of confidence interval construction on false‐negative rate ratio in diagnostic performance when combining two diagnostic tests. We propose a score statistic for testing the null hypothesis of nonunity false‐negative rate ratio. Score test–based confidence interval construction for false‐negative rate ratio will also be discussed. Simulation studies are conducted to compare the performance of the new derived score test statistic and existing statistics for small to moderate sample sizes. In terms of confidence interval construction, our asymptotic score test–based confidence interval estimator possesses significantly shorter expected width with coverage probability being close to the anticipated confidence level. In terms of hypothesis testing, our asymptotic score test procedure has actual type I error rate close to the pre‐assigned nominal level. We illustrate our methodologies with real examples from a clinical laboratory study and a cancer study.