Optimized group sequential study designs for tests of genetic linkage and association in complex diseases

The study of genetic linkage or association in complex traits requires large sample sizes, as the expected effect sizes are small and extremely low significance levels need to be adopted. One possible way to reduce the numbers of phenotypings and genotypings is the use of a sequential study design. Here, average sample sizes are decreased by conducting interim analyses with the possibility to stop the investigation early if the result is significant. We applied optimized group sequential study designs to the analysis of genetic linkage (one-sided mean test) and association (two-sided transmission/disequilibrium test). For designs with two and three stages at overall significance levels of.05 and.0001 and a power of.8, we calculated necessary sample sizes, time points, and critical boundaries for interim and final analyses. Monte Carlo simulation analyses were performed to confirm the validity of the asymptotic approximation. Furthermore, we calculated average sample sizes required under the null and alternative hypotheses in the different study designs. It was shown that the application of a group sequential design led to a maximal increase in sample size of 8% under the null hypothesis, compared with the fixed-sample design. This was contrasted by savings of up to 20% in average sample sizes under the alternative hypothesis, depending on the applied design. These savings affect the amounts of genotyping and phenotyping required for a study and therefore lead to a significant decrease in cost and time.     

Sequential tests for noninferiority and superiority

The problem of simultaneous sequential tests for noninferiority and superiority of a treatment, as compared to an active control, is considered in terms of continuous hierarchical families of one-sided null hypotheses, in the framework of group sequential and adaptive two-stage designs. The crucial point is that the decision boundaries for the individual null hypotheses may vary over the parameter space. This allows one to construct designs where, e.g., a rigid stopping criterion is chosen, rejecting or accepting all individual null hypotheses simultaneously. Another possibility is to use monitoring type stopping boundaries, which leave some flexibility to the experimenter: he can decide, at the interim analysis, whether he is satisfied with the noninferiority margin achieved at this stage, or wants to go for more at the second stage. In the case where he proceeds to the second stage, he may perform midtrial design modifications (e.g., reassess the sample size). The proposed approach allows one to "spend," e.g., less of alpha for an early proof of noninferiority than for an early proof of superiority, and is illustrated by typical examples.     

Confidence intervals for the mean of diagnostic test charge data containing zeros

In this paper, we consider the problem of interval estimation for the mean of diagnostic test charges. Diagnostic test charge data may contain zero values, and the nonzero values can often be modeled by a log-normal distribution. Under such a model, we propose three different interval estimation procedures: a percentile-t bootstrap interval based on sufficient statistics and two likelihood-based confidence intervals. For theoretical properties, we show that the two likelihood-based one-sided confidence intervals are only first-order accurate and that the bootstrap-based one-sided confidence interval is second-order accurate. For two-sided confidence intervals, all three proposed methods are second-order accurate. A simulation study in finite-sample sizes suggests all three proposed intervals outperform a widely used minimum variance unbiased estimator (MVUE)-based interval except for the case of one-sided lower end-point intervals when the skewness is very small. Among the proposed one-sided intervals, the bootstrap interval has the best coverage accuracy. For the two-sided intervals, when the sample size is small, the bootstrap method still yields the best coverage accuracy unless the skewness is very small, in which case the bias-corrected ML method has the best accuracy. When the sample size is large, all three proposed intervals have similar coverage accuracy. Finally, we analyze with the proposed methods one real example assessing diagnostic test charges among older adults with depression.     

The use of score tests for inference on variance components

Whenever inference for variance components is required, the choice between one-sided and two-sided tests is crucial. This choice is usually driven by whether or not negative variance components are permitted. For two-sided tests, classical inferential procedures can be followed, based on likelihood ratios, score statistics, or Wald statistics. For one-sided tests, however, one-sided test statistics need to be developed, and their null distribution derived. While this has received considerable attention in the context of the likelihood ratio test, there appears to be much confusion about the related problem for the score test. The aim of this paper is to illustrate that classical (two-sided) score test statistics, frequently advocated in practice, cannot be used in this context, but that well-chosen one-sided counterparts could be used instead. The relation with likelihood ratio tests will be established, and all results are illustrated in an analysis of continuous longitudinal data using linear mixed models.     

Two-stage designs for experiments with a large number of hypotheses

MOTIVATION: When a large number of hypotheses are investigated the false discovery rate (FDR) is commonly applied in gene expression analysis or gene association studies. Conventional single-stage designs may lack power due to low sample sizes for the individual hypotheses. We propose two-stage designs where the first stage is used to screen the 'promising' hypotheses which are further investigated at the second stage with an increased sample size. A multiple test procedure based on sequential individual P-values is proposed to control the FDR for the case of independent normal distributions with known variance. RESULTS: The power of optimal two-stage designs is impressively larger than the power of the corresponding single-stage design with equal costs. Extensions to the case of unknown variances and correlated test statistics are investigated by simulations. Moreover, it is shown that the simple multiple test procedure using first stage data for screening purposes and deriving the test decisions only from second stage data is a very powerful option.     

Adaptive sample size calculations in group sequential trials

A method for group sequential trials that is based on the inverse normal method for combining the results of the separate stages is proposed. Without exaggerating the Type I error rate, this method enables data-driven sample size reassessments during the course of the study. It uses the stopping boundaries of the classical group sequential tests. Furthermore, exact test procedures may be derived for a wide range of applications. The procedure is compared with the classical designs in terms of power and expected sample size.     

Predictive probability early termination plans for phase II clinical trials

A phase II clinical trial is designed to gather data to help decide whether an experimental treatment has sufficient effectiveness to justify further study. In a one-arm trial with dichotomous outcome, we wish to test a simple null hypothesis on the Bernoulli parameter against a one-sided alternative in a sample of N patients. It is advisable to have a rule to terminate the trial early when evidence accumulates that the treatment is ineffective. Predictive probabilities based on the binomial distribution and beta and uniform prior distributions for the binomial parameter are found to be useful as the basis of group sequential designs. Size, power and average sample size for these designs are discussed. A process for the specification of an early termination plan, advice on the quantification of prior beliefs, and illustrative examples are included.     

Significance analysis of groups of genes in expression profiling studies

MOTIVATION: Gene class testing (GCT) is a statistical approach to determine whether some functionally predefined classes of genes express differently under two experimental conditions. GCT computes the P-value of each gene class based on the null distribution and the gene classes are ranked for importance in accordance with their P-values. Currently, two null hypotheses have been considered: the Q1 hypothesis tests the relative strength of association with the phenotypes among the gene classes, and the Q2 hypothesis assesses the statistical significance. These two hypotheses are related but not equivalent. METHOD: We investigate three one-sided and two two-sided test statistics under Q1 and Q2. The null distributions of gene classes under Q1 are generated by permuting gene labels and the null distributions under Q2 are generated by permuting samples. RESULTS: We applied the five statistics to a diabetes dataset with 143 gene classes and to a breast cancer dataset with 508 GO (Gene Ontology) terms. In each statistic, the null distributions of the gene classes under Q1 are different from those under Q2 in both datasets, and their rankings can be different too. We clarify the one-sided and two-sided hypotheses, and discuss some issues regarding the Q1 and Q2 hypotheses for gene class ranking in the GCT. Because Q1 does not deal with correlations among genes, we prefer test based on Q2. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.     

Point estimation following group sequential tests

Kim and DeMets (1987, Biometrics 43, 857-864) described an exact procedure for constructing confidence intervals for a normal mean following group sequential tests when the boundaries were generated based on the notion of use functions proposed by Lan and DeMets (1983, Biometrika 70, 659-663). In this article, three point estimators for a normal mean following group sequential tests are considered, and their properties are investigated by Monte Carlo simulation. Based on the simulation results, some suggestions are made as to the choice of group sequential designs and use function.     

Estimation of a parameter and its exact confidence interval following sequential sample size reestimation trials

For confirmatory trials of regulatory decision making, it is important that adaptive designs under consideration provide inference with the correct nominal level, as well as unbiased estimates, and confidence intervals for the treatment comparisons in the actual trials. However, naive point estimate and its confidence interval are often biased in adaptive sequential designs. We develop a new procedure for estimation following a test from a sample size reestimation design. The method for obtaining an exact confidence interval and point estimate is based on a general distribution property of a pivot function of the Self-designing group sequential clinical trial by Shen and Fisher (1999, Biometrics55, 190-197). A modified estimate is proposed to explicitly account for futility stopping boundary with reduced bias when block sizes are small. The proposed estimates are shown to be consistent. The computation of the estimates is straightforward. We also provide a modified weight function to improve the power of the test. Extensive simulation studies show that the exact confidence intervals have accurate nominal probability of coverage, and the proposed point estimates are nearly unbiased with practical sample sizes.     

On testing dependence between time to failure and cause of failure when causes of failure are missing

The hypothesis of independence between the failure time and the cause of failure is studied by using the conditional probabilities of failure due to a specific cause given that there is no failure up to certain fixed time. In practice, there are situations when the failure times are available for all units but the causes of failures might be missing for some units. We propose tests based on U-statistics to test for independence of the failure time and the cause of failure in the competing risks model when all the causes of failure cannot be observed. The asymptotic distribution is normal in each case. Simulation studies look at power comparisons for the proposed tests for two families of distributions. The one-sided and the two-sided tests based on Kendall type statistic perform exceedingly well in detecting departures from independence.     

A Shift Algorithm for Exact Computation in Sequential Tests with Binary Outcome

A simple shift algorithm is described enabling the exact determination of power functions and sample size distributions for a large variety of closed sequential two‐sample designs with a binary outcome variable. The test statistics are assumed to be based on relative frequencies of successes or failures, but the number of interim analyses, the monitoring times, and the continuation regions may be specified as desired. To give examples, exact properties of designs proposed by the program package EaSt (Cytel , 1992) are determined, and plans with interim analyses are considered where decisions are based on the conditional power given the observations obtained so far.     

Robust estimation of the false discovery rate

MOTIVATION: Presently available methods that use p-values to estimate or control the false discovery rate (FDR) implicitly assume that p-values are continuously distributed and based on two-sided tests. Therefore, it is difficult to reliably estimate the FDR when p-values are discrete or based on one-sided tests. RESULTS: A simple and robust method to estimate the FDR is proposed. The proposed method does not rely on implicit assumptions that tests are two-sided or yield continuously distributed p-values. The proposed method is proven to be conservative and have desirable large-sample properties. In addition, the proposed method was among the best performers across a series of 'real data simulations' comparing the performance of five currently available methods. AVAILABILITY: Libraries of S-plus and R routines to implement the method are freely available from www.stjuderesearch.org/depts/biostats.     

A note on P-values under group sequential testing and nonproportional hazards

Summary .   Group sequential designs are often used for periodically assessing treatment efficacy during the course of a clinical trial. Following a group sequential test, P -values computed under the assumption that the data were gathered according to a fixed sample design are no longer uniformly distributed under the null hypothesis of no treatment effect. Various sample space orderings have been proposed for computing proper P -values following a group sequential test. Although many of the proposed orderings have been compared in the setting of time-invariant treatment effects, little attention has been given to their performance when the effect of treatment within an individual varies over time. Our interest here is to compare two of the most commonly used methods for computing proper P -values following a group sequential test, based upon the analysis time (AT) and Z -statistic orderings, with respect to resulting power functions when treatment effects on survival are delayed. Power under the AT ordering is shown to be heavily influenced by the presence of a delayed treatment effect, while power functions corresponding to the Z -statistic ordering remain robust under time-varying treatment effects.     

Testing Equivalence Simultaneously for Location and Dispersion of two Normally Distributed Populations

In clinical trials with an active control usually therapeutical equivalence of a new treatment is investigated by looking at a location parameter of the distributions of the primary efficacy variable. But even if the location parameters are close to each other existing differences in variability may be connected with different risks for under or over treatment in an individual patient. Assuming normally distributed responses a multiple test procedure applying two shifted one-sided t-tests for the mean and accordingly two one-sided F-tests for the variances is proposed. Equivalence in location and variability is established if all four tests lead to a rejection at the (one-sided) level α. A conservative procedure “correcting” the t-tests for heteroscedasticity is derived. The choice of a design in terms of the global level α, the global power, the relevant deviations in the population means and variances, as well as the sample size is outlined. Numerical calculations of the actual level and power for the proposed designs show, that for balanced sample sizes the classical uncorrected one-sided t-tests can be used safely without exaggerating the global type I error probability. Finally an example is given.     

Confidence intervals following group sequential tests in clinical trials

Tsiatis, Rosner, and Mehta (1984, Biometrics 40, 797-803) proposed a procedure for constructing confidence intervals following group sequential tests of a normal mean. This method is first extended for group sequential tests for which the sample sizes between interim analyses are not identical or the times are not equally spaced. Then properties of this confidence interval estimation procedure are studied by simulation. The extension accommodates the flexible procedure by Lan and DeMets (1983, Biometrika 70, 659-663) for constructing discrete group sequential boundaries to form a structure for monitoring and estimation following a class of group sequential tests. Finally, it is demonstrated how to combine the procedures by Lan and DeMets and by Tsiatis, Rosner, and Mehta using a FORTRAN program.     

Nonparametric and semiparametric group sequential methods for comparing accuracy of diagnostic tests

SUMMARY: Comparison of the accuracy of two diagnostic tests using the receiver operating characteristic (ROC) curves from two diagnostic tests has been typically conducted using fixed sample designs. On the other hand, the human experimentation inherent in a comparison of diagnostic modalities argues for periodic monitoring of the accruing data to address many issues related to the ethics and efficiency of the medical study. To date, very little research has been done on the use of sequential sampling plans for comparative ROC studies, even when these studies may use expensive and unsafe diagnostic procedures. In this article we propose a nonparametric group sequential design plan. The nonparametric sequential method adapts a nonparametric family of weighted area under the ROC curve statistics (Wieand et al., 1989, Biometrika 76, 585-592) and a group sequential sampling plan. We illustrate the implementation of this nonparametric approach for sequentially comparing ROC curves in the context of diagnostic screening for nonsmall-cell lung cancer. We also describe a semiparametric sequential method based on proportional hazard models. We compare the statistical properties of the nonparametric approach with alternative semiparametric and parametric analyses in simulation studies. The results show the nonparametric approach is robust to model misspecification and has excellent finite-sample performance.     

Group Sequential Monitoring with Arbitrary Inspection Times

The classical group sequential test procedures that were proposed by Pocock (1977) and O'Brien and Fleming (1979) rest on the assumption of equal sample sizes between the interim analyses. Regarding this it is well known that for most situations there is not a great amount of additional Type I error if monitoring is performed for unequal sample sizes between the stages. In some cases, however, problems can arise resulting in an unacceptable liberal behavior of the test procedure. In this article worst case scenarios in sample size imbalancements between the inspection times are considered. Exact critical values for the Pocock and the O'Brien and Fleming group sequential designs are derived for arbitrary and for varying but bounded sample sizes. The approach represents a reasonable alternative to the flexible method that is based on the Type I error rate spending function. The SAS syntax for performing the calculations is provided. Using these procedures, the inspection times or the sample sizes in the consecutive stages need to be chosen independently of the data observed so far.     

Group sequential study designs in genetic-epidemiological case-control studies

OBJECTIVE: In past years, the focus of genetic-epidemiological studies has shifted to analyzing complex diseases. Here, single genes often contribute only little to the manifestation of traits so that many probands have to be included in a study to reliably detect small effects. To reduce the number of required phenotypings and genotypings and thus facilitate analyzing complex traits, sequential study designs can be applied. METHODS: For sequential analyses of complex diseases in association studies, we compare the procedure by Sobell et al. (Am J Med Genet 1993;48:28-35) with the adaptation of formal group sequential study designs by Pampallona and Tsiatis (J Stat Plan Inf 1994;42:19-35). Error rates and average sample sizes are investigated by Monte-Carlo simulations. RESULTS: Formal sequential designs have a higher power regardless of underlying genetic effects. In addition, compared with conventional designs with fixed samples, average sample sizes are reduced considerably; under the null hypothesis of no association, up to 50% of the required sample size can be spared. CONCLUSIONS: To increase the efficiency of genetic-epidemiological case-control studies, we recommend using formal group sequential study designs. The tremendous savings in average sample sizes are expected to affect both cost and time spent on large-scale studies.     

Optimal spending functions for asymmetric group sequential designs

We present optimized group sequential designs where testing of a single parameter theta is of interest. We require specification of a loss function and of a prior distribution for theta. For the examples presented, we pre-specify Type I and II error rates and minimize the expected sample size over the prior distribution for theta. Minimizing the square of sample size rather than the sample size is found to produce designs with slightly less aggressive interim stopping rules and smaller maximum sample sizes with essentially identical expected sample size. We compare optimal designs using Hwang-Shih-DeCani and Kim-DeMets spending functions to fully optimized designs not restricted by a spending function family. In the examples selected, we also examine when there might be substantial benefit gained by adding an interim analysis. Finally, we provide specific optimal asymmetric spending function designs that should be generally useful and simply applied when a design with minimal expected sample size is desired.