Superiority inferences on individual endpoints following noninferiority testing in clinical trials

We consider the problem of drawing superiority inferences on individual endpoints following non-inferiority testing. R?hmel et al. (2006) pointed out this as an important problem which had not been addressed by the previous procedures that only tested for global superiority. R?hmel et al. objected to incorporating the non-inferiority tests in the assessment of the global superiority test by exploiting the relationship between the two, since the results of the latter test then depend on the non-inferiority margins specified for the former test. We argue that this is justified, besides the fact that it enhances the power of the global superiority test. We provide a closed testing formulation which generalizes the three-step procedure proposed by R?hmel et al. for two endpoints. For the global superiority test, R?hmel et al. suggest using the L?uter (1996) test which is modified to make it monotone. The resulting test not only is complicated to use, but the modification does not readily extend to more than two endpoints, and it is less powerful in general than several of its competitors. This is verified in a simulation study. Instead, we suggest applying the one-sided likelihood ratio test used by Perlman and Wu (2004) or the union-intersection t(max) test used by Tamhane and Logan (2004).     

A Multivariate Goodness-of-Fit Test for Stochastically Ordered Distributions

There are a number of nonparametric procedures known for testing goodness-of-fit in the univariate case. Similar procedures can be derived for testing goodness-of-fit in the multivariate case through an application of the theory of statistically equivalent blocks (SEB). The SEB transforms the data into coverages which are distributed as spacings from a uniform distribution on [0,1], under the null hypothesis. In this paper, we present a multivariate nonparametric test of goodness-of-fit based on the SEB when the multivariate distributions under the null hypothesis and the alternative hypothesis are “weakly” ordered. Empirical results are given on the performance of the proposed test in an application to the problem of assessing the reliability of a p-component system.     

The use of score tests for inference on variance components

Whenever inference for variance components is required, the choice between one-sided and two-sided tests is crucial. This choice is usually driven by whether or not negative variance components are permitted. For two-sided tests, classical inferential procedures can be followed, based on likelihood ratios, score statistics, or Wald statistics. For one-sided tests, however, one-sided test statistics need to be developed, and their null distribution derived. While this has received considerable attention in the context of the likelihood ratio test, there appears to be much confusion about the related problem for the score test. The aim of this paper is to illustrate that classical (two-sided) score test statistics, frequently advocated in practice, cannot be used in this context, but that well-chosen one-sided counterparts could be used instead. The relation with likelihood ratio tests will be established, and all results are illustrated in an analysis of continuous longitudinal data using linear mixed models.     

Multiple imputation methods for multivariate one-sided tests with missing data

Multivariate one-sided hypotheses testing problems arise frequently in practice. Various tests have been developed. In practice, there are often missing values in multivariate data. In this case, standard testing procedures based on complete data may not be applicable or may perform poorly if the missing data are discarded. In this article, we propose several multiple imputation methods for multivariate one-sided testing problem with missing data. Some theoretical results are presented. The proposed methods are evaluated using simulations. A real data example is presented to illustrate the methods.     

The analysis of multiple endpoints in clinical trials

Treatment comparisons in randomized clinical trials usually involve several endpoints such that conventional significance testing can seriously inflate the overall Type I error rate. One option is to select a single primary endpoint for formal statistical inference, but this is not always feasible. Another approach is to apply Bonferroni correction (i.e., multiply each P-value by the total number of endpoints). Its conservatism for correlated endpoints is examined for multivariate normal data. A third approach is to derive an appropriate global test statistic and this paper explores one such test applicable to any set of asymptotically normal test statistics. Quantitative, binary, and survival endpoints are all considered within this general framework. Two examples are presented and the relative merits of the proposed strategies are discussed.     

A Note on the Bias of O'Brien's OLS Test

O'Brien's Ordinary Least Squares (OLS) test is a well known procedure for testing the multivariate one-sided hypothesis when the covariance matrix is unknown. Simulation results have been reported where the actual test level exceeds the nominal one. Here it is shown analytically that the actual level is always larger than the nominal one if the covariance matrix is nonnegative.     

One-sided significance tests for generalized linear models under dichotomous response

Dichotomous response models are common in many experimental settings. Often, concomitant explanatory variables are recorded, and a generalized linear model, such as a logit model, is fit. In some cases, interest in specific model parameters is directed only at one-sided departures from some null effect. In these cases, procedures can be developed for testing the null effect against a one-sided alternative. These include Bonferroni-type adjustments of univariate Wald tests, and likelihood ratio tests that employ inequality-constrained multivariate theory. This paper examines such tests of significance. Monte Carlo evaluations are undertaken to examine the small-sample properties of the various procedures. The procedures are seen to perform fairly well, generally achieving their nominal sizes at total sample sizes near 100 experimental units. Extensions to the problem of one-sided tests against a control or standard are also considered.     

Tests for Equality of Several Binomial Populations Against an Order Restricted Alternative and Model Selection for One-Dimensional Multinomials

The problem of testing the equality of several binomial population against an order restricted alternative and model selection for one-dimensional multinomials is studied. Test procedures are proposed. The asymptotic distribution of the test statistics are obtained. Comparisons are made with other test statistics including the likelihood ratio test for stochastic ordering. Also alternatives which does not depend on the distribution of test statistic is proposed.     

Combining Global and Marginal Tests to Compare Two Treatments on Multiple Endpoints

We consider the problem of comparing two treatments on multiple endpoints where the goal is to identify the endpoints that have treatment effects, while controlling the familywise error rate. Two current approaches for this are (i) applying a global test within a closed testing procedure, and (ii) adjusting individual endpoint p‐values for multiplicity. We propose combining the two current methods. We compare the combined method with several competing methods in a simulation study. It is concluded that the combined approach maintains higher power under a variety of treatment effect configurations than the other methods and is thus more power‐robust.     

Closed testing procedures for group sequential clinical trials with multiple endpoints

A simple approach is given for conducting closed testing in clinical trials with multiple endpoints in which group sequential monitoring is planned. The approach allows a flexible stopping time; the earliest and latest stopping times are described. The paradigm is applicable both to clinical trials with multiple endpoints and to the one-sided multiple comparison problem of several treatments versus a control. The approach leads to enhancements of previous methods and suggestions for new methods. An example of a respiratory disease trial with four endpoints is given.     

An alternative model for bivariate random-effects meta-analysis when the within-study correlations are unknown

Multivariate meta-analysis models can be used to synthesize multiple, correlated endpoints such as overall and disease-free survival. A hierarchical framework for multivariate random-effects meta-analysis includes both within-study and between-study correlation. The within-study correlations are assumed known, but they are usually unavailable, which limits the multivariate approach in practice. In this paper, we consider synthesis of 2 correlated endpoints and propose an alternative model for bivariate random-effects meta-analysis (BRMA). This model maintains the individual weighting of each study in the analysis but includes only one overall correlation parameter, rho, which removes the need to know the within-study correlations. Further, the only data needed to fit the model are those required for a separate univariate random-effects meta-analysis (URMA) of each endpoint, currently the common approach in practice. This makes the alternative model immediately applicable to a wide variety of evidence synthesis situations, including studies of prognosis and surrogate outcomes. We examine the performance of the alternative model through analytic assessment, a realistic simulation study, and application to data sets from the literature. Our results show that, unless rho is very close to 1 or -1, the alternative model produces appropriate pooled estimates with little bias that (i) are very similar to those from a fully hierarchical BRMA model where the within-study correlations are known and (ii) have better statistical properties than those from separate URMAs, especially given missing data. The alternative model is also less prone to estimation at parameter space boundaries than the fully hierarchical model and thus may be preferred even when the within-study correlations are known. It also suitably estimates a function of the pooled estimates and their correlation; however, it only provides an approximate indication of the between-study variation. The alternative model greatly facilitates the utilization of correlation in meta-analysis and should allow an increased application of BRMA in practice.     

Sample size determination in clinical trials with multiple co‐primary endpoints including mixed continuous and binary variables

In the field of pharmaceutical drug development, there have been extensive discussions on the establishment of statistically significant results that demonstrate the efficacy of a new treatment with multiple co‐primary endpoints. When designing a clinical trial with such multiple co‐primary endpoints, it is critical to determine the appropriate sample size for indicating the statistical significance of all the co‐primary endpoints with preserving the desired overall power because the type II error rate increases with the number of co‐primary endpoints. We consider overall power functions and sample size determinations with multiple co‐primary endpoints that consist of mixed continuous and binary variables, and provide numerical examples to illustrate the behavior of the overall power functions and sample sizes. In formulating the problem, we assume that response variables follow a multivariate normal distribution, where binary variables are observed in a dichotomized normal distribution with a certain point of dichotomy. Numerical examples show that the sample size decreases as the correlation increases when the individual powers of each endpoint are approximately and mutually equal.     

Test for increasing convex order in multivariate response

Trend test based on cross-classified data in dose-response has been a central problem in medicine. Most of existing test methods are known to only fit to binary response variables. However, the approaches for binary response tables may suffer from the lack of a clear choice for dichotomization. For multivariate response with ordered categories, some studies have been done for simple stochastic order, likelihood ratio order and so on. However, methods of statistical inference on increasing convex order for more than two multinomial populations have not been fully developed. For testing the increasing convex order alternative, this article provides a model-free test method which can be used in the case of two-way tables and stratified data. Two real examples will be used to illustrate how to apply our test method.     

Screening for partial conjunction hypotheses

SUMMARY: We consider the problem of testing for partial conjunction of hypothesis, which argues that at least u out of n tested hypotheses are false. It offers an in-between approach to the testing of the conjunction of null hypotheses against the alternative that at least one is not, and the testing of the disjunction of null hypotheses against the alternative that all hypotheses are not null. We suggest powerful test statistics for testing such a partial conjunction hypothesis that are valid under dependence between the test statistics as well as under independence. We then address the problem of testing many partial conjunction hypotheses simultaneously using the false discovery rate (FDR) approach. We prove that if the FDR controlling procedure in Benjamini and Hochberg (1995, Journal of the Royal Statistical Society, Series B 57, 289-300) is used for this purpose the FDR is controlled under various dependency structures. Moreover, we can screen at all levels simultaneously in order to display the findings on a superimposed map and still control an appropriate FDR measure. We apply the method to examples from microarray analysis and functional magnetic resonance imaging (fMRI), two application areas where the need for partial conjunction analysis has been identified.     

Exact Tests for Comparing Two Paired Proportions with Incomplete Data

Various asymptotic test procedures have been developed previously for testing the equality of two binomial proportions with partially incomplete paired data. Test procedures that discard incomplete observations have been shown to be less powerful than those procedures that utilize all available observations. On the other hand, asymptotic test procedures that utilize all available observations may not be reliable in small‐sample problems or sparse data structures. In this article, unconditional exact test procedures are proposed for testing the equality of two paired binomial proportions with partially incomplete paired data under a random mechanism. The proposed unconditional exact test methods are illustrated with real data from a neurological study. Empirical studies are conducted to investigate the performance of these and other test procedures with respect to size and power. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

An Adaptive Location‐Scale Test

Increasing locations are often accompanied by an increase in variability. In this case apparent heteroscedasticity can indicate that there are treatment effects and it is appropriate to consider an alternative involving differences in location as well as in scale. As a location‐scale test the sum of a location and a scale test statistic can be used. However, the power can be raised through weighting the sum. In order to select values for this weighting an adaptive design with an interim analysis is proposed: The data of the first stage are used to calculate the weights and with the second stage's data a weighted location‐scale test is carried out. The p‐values of the two stages are combined through Fisher's combination test. With a Lepage‐type location‐scale test it is illustrated that the resultant adaptive test can be more powerful than the ‘optimum’ test with no interim analysis. The principle to calculate weights, which cannot be reasonably chosen a priori, with the data of the first stage may be useful for other tests which utilize weighted statistics, too. Furthermore, the proposed test is illustrated with an example from experimental ecology.     

Simultaneous confidence intervals from randomization tests with application in testing bioequivalence with multiple endpoints

We propose a method to construct simultaneous confidence intervals for a parameter vector from inverting a series of randomization tests (RT). The randomization tests are facilitated by an efficient multivariate Robbins–Monro procedure that takes the correlation information of all components into account. The estimation method does not require any distributional assumption of the population other than the existence of the second moments. The resulting simultaneous confidence intervals are not necessarily symmetric about the point estimate of the parameter vector but possess the property of equal tails in all dimensions. In particular, we present the constructing the mean vector of one population and the difference between two mean vectors of two populations. Extensive simulation is conducted to show numerical comparison with four methods. We illustrate the application of the proposed method to test bioequivalence with multiple endpoints on some real data.     

Accurate critical constants for the one-sided approximate likelihood ratio test of a normal mean vector when the covariance matrix is estimated

Tang, Gnecco, and Geller (1989, Biometrika 76, 577-583) proposed an approximate likelihood ratio (ALR) test of the null hypothesis that a normal mean vector equals a null vector against the alternative that all of its components are nonnegative with at least one strictly positive. This test is useful for comparing a treatment group with a control group on multiple endpoints, and the data from the two groups are assumed to follow multivariate normal distributions with different mean vectors and a common covariance matrix (the homoscedastic case). Tang et al. derived the test statistic and its null distribution assuming a known covariance matrix. In practice, when the covariance matrix is estimated, the critical constants tabulated by Tang et al. result in a highly liberal test. To deal with this problem, we derive an accurate small-sample approximation to the null distribution of the ALR test statistic by using the moment matching method. The proposed approximation is then extended to the heteroscedastic case. The accuracy of both the approximations is verified by simulations. A real data example is given to illustrate the use of the approximations.     

Trendtests with adaptive scoring

The concept of adaptive two‐stage designs is applied to the problem of testing the equality of several normal means against an ordered (monotone) alternative. The likelihood‐ratio‐test proposed by Bartholomew is known to have favorable power properties when testing against a monotonic trend. Tests based on contrasts provide a flexible way to incorporate available information regarding the pattern of the unknown true means through appropriate specification of the scores. The basic idea of the presented concept is the combination of Bartholomew 's test (first stage) with an “adaptive score test” (second stage) which utilizes the information resulting from isotonic regression estimation at the first stage. In a Monte Carlo simulation study the adaptive scoring procedure is compared to the non‐adaptive two‐stage procedure using the Bartholomew test at both stages. We found that adaptive scoring may improve the power of the two stage design, in particular if the sample size at the first stage is considerably larger than at the second stage.     

Blinded sample size recalculation in multiple composite population designs with normal data and baseline adjustments

The increasing interest in subpopulation analysis has led to the development of various new trial designs and analysis methods in the fields of personalized medicine and targeted therapies. In this paper, subpopulations are defined in terms of an accumulation of disjoint population subsets and will therefore be called composite populations. The proposed trial design is applicable to any set of composite populations, considering normally distributed endpoints and random baseline covariates. Treatment effects for composite populations are tested by combining p-values, calculated on the subset levels, using the inverse normal combination function to generate test statistics for those composite populations while the closed testing procedure accounts for multiple testing. Critical boundaries for intersection hypothesis tests are derived using multivariate normal distributions, reflecting the joint distribution of composite population test statistics given no treatment effect exists. For sample size calculation and sample size, recalculation multivariate normal distributions are derived which describe the joint distribution of composite population test statistics under an assumed alternative hypothesis. Simulations demonstrate the absence of any practical relevant inflation of the type I error rate. The target power after sample size recalculation is typically met or close to being met.