Statistical inference for self-designing clinical trials with a one-sided hypothesis

In the process of monitoring clinical trials, it seems appealing to use the interim findings to determine whether the sample size originally planned will provide adequate power when the alternative hypothesis is true, and to adjust the sample size if necessary. In the present paper, we propose a flexible sequential monitoring method following the work of Fisher (1998), in which the maximum sample size does not have to be specified in advance. The final test statistic is constructed based on a weighted average of the sequentially collected data, where the weight function at each stage is determined by the observed data prior to that stage. Such a weight function is used to maintain the integrity of the variance of the final test statistic so that the overall type I error rate is preserved. Moreover, the weight function plays an implicit role in termination of a trial when a treatment difference exists. Finally, the design allows the trial to be stopped early when the efficacy result is sufficiently negative. Simulation studies confirm the performance of the method.     

A nonparametric approach to the analysis of longitudinal data via a set of level crossing problems with application to the analysis of microarray time course experiments

Here we develop a completely nonparametric method for comparing two groups on a set of longitudinal measurements. No assumptions are made about the form of the mean response function, the covariance structure or the distributional form of disturbances around the mean response function. The solution proposed here is based on the realization that every longitudinal data set can also be thought of as a collection of survival data sets where the events of interest are level crossings. The method for testing for differences in the longitudinal measurements then is as follows: for an arbitrarily large set of levels, for each subject determine the first time the subject has an upcrossing and a downcrossing for each level. For each level one then computes the log rank statistic and uses the maximum in absolute value of all these statistics as the test statistic. By permuting group labels we obtain a permutation test of the hypothesis that the joint distribution of the measurements over time does not depend on group membership. Simulations are performed to investigate the power and it is applied to the area that motivated the method-the analysis of microarrays. In this area small sample sizes, few time points and far too many genes to consider genuine gene level longitudinal modeling have created a need for a simple, model free test to screen for interesting features in the data.     

Testing substitution models within a phylogenetic tree

Phylogenetic tree reconstruction frequently assumes the homogeneity of the substitution process over the whole tree. To test this assumption statistically, we propose a test based on the sample covariance matrix of the set of substitution rate matrices estimated from pairwise sequence comparison. The sample covariance matrix is condensed into a one-dimensional test statistic Delta = sum ln(1 + delta(i)), where delta(i) are the eigenvalues of the sample covariance matrix. The test does not assume a specific mutational model. It analyses the variation in the estimated rate matrices. The distribution of this test statistic is determined by simulations based on the phylogeny estimated from the data. We study the power of the test under various scenarios and apply the test to X chromosome and mtDNA primate sequence data. Finally, we demonstrate how to include rate variation in the test.     

Group sequential methods for an ordinal logistic random-effects model under misspecification

Interim analyses in clinical trials are planned for ethical as well as economic reasons. General results have been published in the literature that allow the use of standard group sequential methodology if one uses an efficient test statistic, e.g., when Wald-type statistics are used in random-effects models for ordinal longitudinal data. These models often assume that the random effects are normally distributed. However, this is not always the case. We will show that, when the random-effects distribution is misspecified in ordinal regression models, the joint distribution of the test statistics over the different interim analyses is still a multivariate normal distribution, but a sandwich-type correction to the covariance matrix is needed in order to obtain the correct covariance matrix. The independent increment structure is also investigated. A bias in estimation will occur due to the misspecification. However, we will also show that the treatment effect estimate will be unbiased under the null hypothesis, thus maintaining the type I error. Extensive simulations based on a toenail dermatophyte onychomycosis trial are used to illustrate our results.     

Self-designing two-stage trials to minimize expected costs

In the design of clinical trials, the sample size for the trial is traditionally calculated from estimates of parameters of interest, such as the mean treatment effect, which can often be inaccurate. However, recalculation of the sample size based on an estimate of the parameter of interest that uses accumulating data from the trial can lead to inflation of the overall Type I error rate of the trial. The self-designing method of Fisher, also known as the variance-spending method, allows the use of all accumulating data in a sequential trial (including the estimated treatment effect) in determining the sample size for the next stage of the trial without inflating the Type I error rate. We propose a self-designing group sequential procedure to minimize the expected total cost of a trial. Cost is an important parameter to consider in the statistical design of clinical trials due to limited financial resources. Using Bayesian decision theory on the accumulating data, the design specifies sequentially the optimal sample size and proportion of the test statistic's variance needed for each stage of a trial to minimize the expected cost of the trial. The optimality is with respect to a prior distribution on the parameter of interest. Results are presented for a simple two-stage trial. This method can extend to nonmonetary costs, such as ethical costs or quality-adjusted life years.     

Sample Size Reassessment and Hypothesis Testing in Adaptive Survival Trials

Mid-study design modifications are becoming increasingly accepted in confirmatory clinical trials, so long as appropriate methods are applied such that error rates are controlled. It is therefore unfortunate that the important case of time-to-event endpoints is not easily handled by the standard theory. We analyze current methods that allow design modifications to be based on the full interim data, i.e., not only the observed event times but also secondary endpoint and safety data from patients who are yet to have an event. We show that the final test statistic may ignore a substantial subset of the observed event times. An alternative test incorporating all event times is found, where a conservative assumption must be made in order to guarantee type I error control. We examine the power of this approach using the example of a clinical trial comparing two cancer therapies.     

A comparison of likelihood ratio tests and Rao's score test for three separable covariance matrix structures

The problem of testing the separability of a covariance matrix against an unstructured variance‐covariance matrix is studied in the context of multivariate repeated measures data using Rao's score test (RST). The RST statistic is developed with the first component of the separable structure as a first‐order autoregressive (AR(1)) correlation matrix or an unstructured (UN) covariance matrix under the assumption of multivariate normality. It is shown that the distribution of the RST statistic under the null hypothesis of any separability does not depend on the true values of the mean or the unstructured components of the separable structure. A significant advantage of the RST is that it can be performed for small samples, even smaller than the dimension of the data, where the likelihood ratio test (LRT) cannot be used, and it outperforms the standard LRT in a number of contexts. Monte Carlo simulations are then used to study the comparative behavior of the null distribution of the RST statistic, as well as that of the LRT statistic, in terms of sample size considerations, and for the estimation of the empirical percentiles. Our findings are compared with existing results where the first component of the separable structure is a compound symmetry (CS) correlation matrix. It is also shown by simulations that the empirical null distribution of the RST statistic converges faster than the empirical null distribution of the LRT statistic to the limiting χ² distribution. The tests are implemented on a real dataset from medical studies.     

Nonparametric rank-based methods for group sequential monitoring of paired censored survival data

This research gives methods for nonparametric sequential monitoring of paired censored survival data in the two-sample problem using paired weighted log-rank statistics with adjustments for dependence in survival and censoring outcomes. The joint asymptotic closed-form distribution of these sequentially monitored statistics has a dependent increments structure. Simulations validating operating characteristics of the proposed methods highlight power and size consequences of ignoring even mildly correlated data. A motivating example is presented via the Early Treatment Diabetic Retinopathy Study.     

Bayesian adaptive trial design for a newly validated surrogate endpoint

The evaluation of surrogate endpoints for primary use in future clinical trials is an increasingly important research area, due to demands for more efficient trials coupled with recent regulatory acceptance of some surrogates as 'valid.' However, little consideration has been given to how a trial that utilizes a newly validated surrogate endpoint as its primary endpoint might be appropriately designed. We propose a novel Bayesian adaptive trial design that allows the new surrogate endpoint to play a dominant role in assessing the effect of an intervention, while remaining realistically cautious about its use. By incorporating multitrial historical information on the validated relationship between the surrogate and clinical endpoints, then subsequently evaluating accumulating data against this relationship as the new trial progresses, we adaptively guard against an erroneous assessment of treatment based upon a truly invalid surrogate. When the joint outcomes in the new trial seem plausible given similar historical trials, we proceed with the surrogate endpoint as the primary endpoint, and do so adaptively-perhaps stopping the trial for early success or inferiority of the experimental treatment, or for futility. Otherwise, we discard the surrogate and switch adaptive determinations to the original primary endpoint. We use simulation to test the operating characteristics of this new design compared to a standard O'Brien-Fleming approach, as well as the ability of our design to discriminate trustworthy from untrustworthy surrogates in hypothetical future trials. Furthermore, we investigate possible benefits using patient-level data from 18 adjuvant therapy trials in colon cancer, where disease-free survival is considered a newly validated surrogate endpoint for overall survival.     

Accurate critical constants for the one-sided approximate likelihood ratio test of a normal mean vector when the covariance matrix is estimated

Tang, Gnecco, and Geller (1989, Biometrika 76, 577-583) proposed an approximate likelihood ratio (ALR) test of the null hypothesis that a normal mean vector equals a null vector against the alternative that all of its components are nonnegative with at least one strictly positive. This test is useful for comparing a treatment group with a control group on multiple endpoints, and the data from the two groups are assumed to follow multivariate normal distributions with different mean vectors and a common covariance matrix (the homoscedastic case). Tang et al. derived the test statistic and its null distribution assuming a known covariance matrix. In practice, when the covariance matrix is estimated, the critical constants tabulated by Tang et al. result in a highly liberal test. To deal with this problem, we derive an accurate small-sample approximation to the null distribution of the ALR test statistic by using the moment matching method. The proposed approximation is then extended to the heteroscedastic case. The accuracy of both the approximations is verified by simulations. A real data example is given to illustrate the use of the approximations.     

Novel application of a quantitative spatial comparison tool to species distribution data

Comparing geographically referenced maps has become an important aspect of spatial ecology (e.g. assessing change in distribution over time). Whilst humans are adept at recognising and extracting structure from maps (i.e. identifying spatial patterns), quantifying these structures can be difficult. Here, we show how the Structural Similarity (SSIM) index, a spatial comparison method adapted from techniques developed in computer science to determine the quality of image compression, can be used to extract additional information from spatial ecological data. We enhance the SSIM index to incorporate uncertainty from the underlying spatial models, and provide a software algorithm to correct for internal edge effects so that loss of spatial information from the map comparison is limited. The SSIM index uses a spatially-local window to calculate statistics based on local mean, variance, and covariance between the maps being compared. A number of statistics can be calculated using the SSIM index, ranging from a single summary statistic to quantify similarities between two maps, to maps of similarities in mean, variance, and covariance that can provide additional insight into underlying biological processes. We demonstrate the applicability of the SSIM approach using a case study of sperm whales in the Mediterranean Sea and identify areas where local-scale differences in space-use between groups and singleton whales occur. We show how novel insights into spatial structure can be extracted, which could not be obtained by visual inspection or cell-by-cell subtraction. As an approach, SSIM is applicable to a broad range of spatial ecological data, providing a novel, implementable tool for map comparison.     

Use of the score test as a goodness-of-fit measure of the covariance structure in genetic analysis of longitudinal data

Model selection is an essential issue in longitudinal data analysis since many different models have been proposed to fit the covariance structure. The likelihood criterion is commonly used and allows to compare the fit of alternative models. Its value does not reflect, however, the potential improvement that can still be reached in fitting the data unless a reference model with the actual covariance structure is available. The score test approach does not require the knowledge of a reference model, and the score statistic has a meaningful interpretation in itself as a goodness-of-fit measure. The aim of this paper was to show how the score statistic may be separated into the genetic and environmental parts, which is difficult with the likelihood criterion, and how it can be used to check parametric assumptions made on variance and correlation parameters. Selection of models for genetic analysis was applied to a dairy cattle example for milk production.     

A Bootstrap Procedure for Adaptive Selection of the Test Statistic in Flexible Two‐Stage Designs

Adaptive two‐stage designs allow a data‐driven change of design characteristics during the ongoing trial. One of the available options is an adaptive choice of the test statistic for the second stage of the trial based on the results of the interim analysis. Since there is often only a vague knowledge of the distribution shape of the primary endpoint in the planning phase of a study, a change of the test statistic may then be considered if the data indicate that the assumptions underlying the initial choice of the test are not correct. Collings and Hamilton proposed a bootstrap method for the estimation of the power of the two‐sample Wilcoxon test for shift alternatives. We use this approach for the selection of the test statistic. By means of a simulation study, we show that the gain in terms of power may be considerable when the initial assumption about the underlying distribution was wrong, whereas the loss is relatively small when in the first instance the optimal test statistic was chosen. The results also hold true for comparison with a one‐stage design. Application of the method is illustrated by a clinical trial example.     

Resampling-based methods in single and multiple testing for equality of covariance/correlation matrices

Traditional resampling-based tests for homogeneity in covariance matrices across multiple groups resample residuals, that is, data centered by group means. These residuals do not share the same second moments when the null hypothesis is false, which makes them difficult to use in the setting of multiple testing. An alternative approach is to resample standardized residuals, data centered by group sample means and standardized by group sample covariance matrices. This approach, however, has been observed to inflate type I error when sample size is small or data are generated from heavy-tailed distributions. We propose to improve this approach by using robust estimation for the first and second moments. We discuss two statistics: the Bartlett statistic and a statistic based on eigen-decomposition of sample covariance matrices. Both statistics can be expressed in terms of standardized errors under the null hypothesis. These methods are extended to test homogeneity in correlation matrices. Using simulation studies, we demonstrate that the robust resampling approach provides comparable or superior performance, relative to traditional approaches, for single testing and reasonable performance for multiple testing. The proposed methods are applied to data collected in an HIV vaccine trial to investigate possible determinants, including vaccine status, vaccine-induced immune response level and viral genotype, of unusual correlation pattern between HIV viral load and CD4 count in newly infected patients.     

Weighted Concordance Analysis

Using the variance stabilizing technique, a product multinomial model is introduced to generate a new statistic to test observers' uncertainty in a weighted concordance analysis. Distance matrices which follow some specific rules are obtained by linear combinations of hierarchical distance matrices whose elements are equal to 0 or 1 and unit diagonal. The new statistic is compared with the kappa statistic interpreted by considering the covariance matrix generated by the data. By rewriting the test statistic in a barycentric form, one demonstrates how to modify the barycentric coefficients to derive an adequate measure of the interobserver agreement. The methods are illustrated using two examples.     

Planning and analyzing adaptive group sequential survival trials

The use of adaptive methods has experienced increasing interest in the current literature on group sequential designs. Group sequential analysis in survival trials usually apply the error spending function approach due to the unpredictable amount of information available in an interim analysis. An alternative way is to apply adaptive methods where additionally the maximum amount of information and other designing parameters can be changed based on the information available at the interim stage. In this paper, it is shown how the inverse normal method can be used within a survival design using the log-rank test for comparing two survival functions. This method allows for many kinds of design modifications. In case of no modifications, the inverse normal method coincides with the commonly used analysis strategy. It is straightforward to specify effect estimates. Confidence intervals for the hazard ratio that can be calculated at each stage of the trial and intervals that can only be computed by the end of the trial are proposed. The latter also enables the calculation of median unbiased estimates. Overall p-values can be defined analogously. Properties of the analyses techniques are investigated and compared with alternative approaches. It is shown that the proposed analysis technique might help to rescue an underpowered study and opens the way to other types of changes in design. The proposed technique is implemented in the software ADDPLAN Adaptive Design, Plans and Analyses (http://www.addplan.com).     

Nonparametric Tests in the Unbalanced Multivariate One‐Way Design

A nonparametric model for the multivariate one‐way design is discussed which entails continuous as well as discontinuous distributions and, therefore, allows for ordinal data. Nonparametric hypotheses are formulated by the normalized version of the marginal distribution functions as well as the common distribution functions. The differences between the distribution functions are described by means of the so‐called relative treatment effects, for which unbiased and consistent estimators are derived. The asymptotic distribution of the vector of the effect estimators is derived and under the marignal hypothesis a consistent estimator for the asymptotic covariance matrix is given. Nonparametric versions of the Wald‐type statistic, the ANOVA‐type statistic and the Lawley‐Hotelling statistic are considered and compared by means of a simulation study. Finally, these tests are applied to a psychiatric clinical trial.     

Nonparametric Location Tests Based on the Empirical Distribution

We present a unified approach to the nonparametric comparison of locations in the one- and two sample case using the empirical distribution functions. This approach reveals how to define a simple test statistic for this comparison in the case of large sample sizes which is equivalent to the known Wilcoxon-Mann-Whitney statistics and is based on the grouping of the data. In the particular case of equal group sizes we find an extremely simple form for this location test.     

Simultaneous group sequential analysis of rank-based and weighted Kaplan-Meier tests for paired censored survival data

This research sequentially monitors paired survival differences using a new class of nonparametric tests based on functionals of standardized paired weighted log-rank (PWLR) and standardized paired weighted Kaplan-Meier (PWKM) tests. During a trial, these tests may alternately assume the role of the more extreme statistic. By monitoring PEMAX, the maximum between the absolute values of the standardized PWLR and PWKM, one combines advantages of rank-based (RB) and non-RB paired testing paradigms. Simulations show that monitoring treatment differences using PEMAX maintains type I error and is nearly as powerful as using the more advantageous of the two tests in proportional hazards (PH) as well as non-PH situations. Hence, PEMAX preserves power more robustly than individually monitored PWLR and PWKM, while maintaining a reasonably simple approach to design and analysis of results. An example from the Early Treatment Diabetic Retinopathy Study (ETDRS) is given.