Development of gatekeeping strategies in confirmatory clinical trials

This paper discusses multiplicity issues arising in confirmatory clinical trials with hierarchically ordered multiple objectives. In order to protect the overall type I error rate, multiple objectives are analyzed using multiple testing procedures. When the objectives are ordered and grouped in multiple families (e.g. families of primary and secondary endpoints), gatekeeping procedures are employed to account for this hierarchical structure. We discuss considerations arising in the process of building gatekeeping procedures, including proper use of relevant trial-specific information and criteria for selecting gatekeeping procedures. The methods and principles discussed in this paper are illustrated using a clinical trial in patients with type II diabetes mellitus.     

Gatekeeping testing via adaptive alpha allocation

In a typical clinical trial, there are one or two primary endpoints, and a few secondary endpoints. When at least one primary endpoint achieves statistical significance, there is considerable interest in using results for the secondary endpoints to enhance characterization of the treatment effect. Because multiple endpoints are involved, regulators may require that the familywise type I error rate be controlled at a pre-set level. This requirement can be achieved by using "gatekeeping" methods. However, existing methods suffer from logical oddities such as allowing results for secondary endpoint(s) to impact the likelihood of success for the primary endpoint(s). We propose a novel and easy-to-implement gatekeeping procedure that is devoid of such deficiencies. A real data example and simulation results are used to illustrate efficiency gains of our method relative to existing methods.     

Stepwise gatekeeping procedures in clinical trial applications

This paper discusses multiple testing problems in which families of null hypotheses are tested in a sequential manner and each family serves as a gatekeeper for the subsequent families. Gatekeeping testing strategies of this type arise frequently in clinical trials with multiple objectives, e.g., multiple endpoints and/or multiple dose-control comparisons. It is demonstrated in this paper that the parallel gatekeeping procedure of Dmitrienko, Offen and Westfall (2003) admits a simple stepwise representation (n null hypotheses can be tested in n steps rather than 2n steps required in the closed procedure). The stepwise representation considerably simplifies the implementation of gatekeeping procedures in practice and provides an important insight into the nature of gatekeeping inferences. The derived stepwise gatekeeping procedure is illustrated using clinical trial examples.     

A note on familywise error rate for a primary and secondary endpoint

Hung et al. (2007) considered the problem of controlling the type I error rate for a primary and secondary endpoint in a clinical trial using a gatekeeping approach in which the secondary endpoint is tested only if the primary endpoint crosses its monitoring boundary. They considered a two-look trial and showed by simulation that the naive method of testing the secondary endpoint at full level α at the time the primary endpoint reaches statistical significance does not control the familywise error rate at level α. Tamhane et al. (2010) derived analytic expressions for familywise error rate and power and confirmed the inflated error rate of the naive approach. Nonetheless, many people mistakenly believe that the closure principle can be used to prove that the naive procedure controls the familywise error rate. The purpose of this note is to explain in greater detail why there is a problem with the naive approach and show that the degree of alpha inflation can be as high as that of unadjusted monitoring of a single endpoint.     

Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests

The confirmatory analysis of pre-specified multiple hypotheses has become common in pivotal clinical trials. In the recent past multiple test procedures have been developed that reflect the relative importance of different study objectives, such as fixed sequence, fallback, and gatekeeping procedures. In addition, graphical approaches have been proposed that facilitate the visualization and communication of Bonferroni-based closed test procedures for common multiple test problems, such as comparing several treatments with a control, assessing the benefit of a new drug for more than one endpoint, combined non-inferiority and superiority testing, or testing a treatment at different dose levels in an overall and a subpopulation. In this paper, we focus on extended graphical approaches by dissociating the underlying weighting strategy from the employed test procedure. This allows one to first derive suitable weighting strategies that reflect the given study objectives and subsequently apply appropriate test procedures, such as weighted Bonferroni tests, weighted parametric tests accounting for the correlation between the test statistics, or weighted Simes tests. We illustrate the extended graphical approaches with several examples. In addition, we describe briefly the gMCP package in R, which implements some of the methods described in this paper.     

General multistage gatekeeping procedures

A general multistage (stepwise) procedure is proposed for dealing with arbitrary gatekeeping problems including parallel and serial gatekeeping. The procedure is very simple to implement since it does not require the application of the closed testing principle and the consequent need to test all nonempty intersections of hypotheses. It is based on the idea of carrying forward the Type I error rate for any rejected hypotheses to test hypotheses in the next ordered family. This requires the use of a so-called separable multiple test procedure (MTP) in the earlier family. The Bonferroni MTP is separable, but other standard MTPs such as Holm, Hochberg, Fallback and Dunnett are not. Their truncated versions are proposed which are separable and more powerful than the Bonferroni MTP. The proposed procedure is illustrated by a clinical trial example.     

Sample size calculation for the combination test under nonproportional hazards

For sample size calculation in clinical trials with survival endpoints, the logrank test, which is the optimal method under the proportional hazard (PH) assumption, is predominantly used. In reality, the PH assumption may not hold. For example, in immuno-oncology trials, delayed treatment effects are often expected. The sample size without considering the potential violation of the PH assumption may lead to an underpowered study. In recent years, combination tests such as the maximum weighted logrank test have received great attention because of their robust performance in various hazards scenarios. In this paper, we propose a flexible simulation-free procedure to calculate the sample size using combination tests. The procedure extends the Lakatos' Markov model and allows for complex situations encountered in a clinical trial, like staggered entry, dropouts, etc. We evaluate the procedure using two maximum weighted logrank tests, one projection-type test, and three other commonly used tests under various hazards scenarios. The simulation studies show that the proposed method can achieve the target power for all compared tests in most scenarios. The combination tests exhibit robust performance under correct specification and misspecification scenarios and are highly recommended when the hazard-changing patterns are unknown beforehand. Finally, we demonstrate our method using two clinical trial examples and provide suggestions about the sample size calculations under nonproportional hazards.     

Improved two‐stage group sequential procedures for testing a secondary endpoint after the primary endpoint achieves significance

In two‐stage group sequential trials with a primary and a secondary endpoint, the overall type I error rate for the primary endpoint is often controlled by an α‐level boundary, such as an O'Brien‐Fleming or Pocock boundary. Following a hierarchical testing sequence, the secondary endpoint is tested only if the primary endpoint achieves statistical significance either at an interim analysis or at the final analysis. To control the type I error rate for the secondary endpoint, this is tested using a Bonferroni procedure or any α‐level group sequential method. In comparison with marginal testing, there is an overall power loss for the test of the secondary endpoint since a claim of a positive result depends on the significance of the primary endpoint in the hierarchical testing sequence. We propose two group sequential testing procedures with improved secondary power: the improved Bonferroni procedure and the improved Pocock procedure. The proposed procedures use the correlation between the interim and final statistics for the secondary endpoint while applying graphical approaches to transfer the significance level from the primary endpoint to the secondary endpoint. The procedures control the familywise error rate (FWER) strongly by construction and this is confirmed via simulation. We also compare the proposed procedures with other commonly used group sequential procedures in terms of control of the FWER and the power of rejecting the secondary hypothesis. An example is provided to illustrate the procedures.     

Testing and estimation of proportion (or risk) ratio under the matched‐pair design with multiple binary endpoints

The proportion ratio (PR) of responses between an experimental treatment and a control treatment is one of the most commonly used indices to measure the relative treatment effect in a randomized clinical trial. We develop asymptotic and permutation‐based procedures for testing equality of treatment effects as well as derive confidence intervals of PRs for multivariate binary matched‐pair data under a mixed‐effects exponential risk model. To evaluate and compare the performance of these test procedures and interval estimators, we employ Monte Carlo simulation. When the number of matched pairs is large, we find that all test procedures presented here can perform well with respect to Type I error. When the number of matched pairs is small, the permutation‐based test procedures developed in this paper is of use. Furthermore, using test procedures (or interval estimators) based on a weighted linear average estimator of treatment effects can improve power (or gain precision) when the treatment effects on all response variables of interest are known to fall in the same direction. Finally, we apply the data taken from a crossover clinical trial that monitored several adverse events of an antidepressive drug to illustrate the practical use of test procedures and interval estimators considered here.     

Using secondary outcome to sharpen bounds for treatment harm rate in characterizing heterogeneity

In a clinical trial, statistical reports are typically concerned about the mean difference in two groups. Now there is increasing interest in the heterogeneity of the treatment effect, which has important implications in treatment evaluation and selection. The treatment harm rate (THR), which is defined by the proportion of people who has a worse outcome on the treatment compared to the control, was used to characterize the heterogeneity. Since THR involves the joint distribution of the two potential outcomes, it cannot be identified without further assumptions even in the randomized trials. We can only derive the simple bounds with the observed data. But the simple bounds are usually too wide. In this paper, we use a secondary outcome that satisfies the monotonicity assumption to tighten the bounds. It is shown that the bounds we derive cannot be wider than the simple bounds. We also construct some simulation studies to assess the performance of our bounds in finite sample. The results show that a secondary outcome, which is more closely related to the primary outcome, can lead to narrower bounds. Finally, we illustrate the application of the proposed bounds in a randomized clinical trial of determining whether the intensive glycemia could reduce the risk of development or progression of diabetic retinopathy.     

Test Procedures in Configural Frequency Analysis (CFA) Controlling the Local and Multiple Level

The test statistics used until now in the CFA have been developed under the assumption of the overall hypothesis of total independence. Therefore, the multiple test procedures based on these statistics are really only different tests of the overall hypothesis. If one likes to test a special cell hypothesis, one should only assume that this hypothesis is true and not the whole overall hypothesis. Such cell tests can then be used as elements of a multiple test procedure. In this paper it is shown that the usual test procedures can be very anticonservative (except of the two-dimensional, and, for some procedures, the three-dimensional case), and corrected test procedures are developed. Furthermore, for the construction of multiple tests controlling the multiple level, modifications of Holm's (1979) procedure are proposed which lead to sharper results than his general procedure and can also be performed very easily.     

Comparison of design strategies for a three‐arm clinical trial with time‐to‐event endpoint: Power,time‐to‐analysis,and operational aspects

To optimize resources, randomized clinical trials with multiple arms can be an attractive option to simultaneously test various treatment regimens in pharmaceutical drug development. The motivation for this work was the successful conduct and positive final outcome of a three‐arm randomized clinical trial primarily assessing whether obinutuzumab plus chlorambucil in patients with chronic lympocytic lymphoma and coexisting conditions is superior to chlorambucil alone based on a time‐to‐event endpoint. The inference strategy of this trial was based on a closed testing procedure. We compare this strategy to three potential alternatives to run a three‐arm clinical trial with a time‐to‐event endpoint. The primary goal is to quantify the differences between these strategies in terms of the time it takes until the first analysis and thus potential approval of a new drug, number of required events, and power. Operational aspects of implementing the various strategies are discussed. In conclusion, using a closed testing procedure results in the shortest time to the first analysis with a minimal loss in power. Therefore, closed testing procedures should be part of the statistician's standard clinical trials toolbox when planning multiarm clinical trials.     

Joint models for a primary endpoint and multiple longitudinal covariate processes

Summary .   Joint models are formulated to investigate the association between a primary endpoint and features of multiple longitudinal processes. In particular, the subject-specific random effects in a multivariate linear random-effects model for multiple longitudinal processes are predictors in a generalized linear model for primary endpoints. Li, Zhang, and Davidian (2004, Biometrics 60 , 1–7) proposed an estimation procedure that makes no distributional assumption on the random effects but assumes independent within-subject measurement errors in the longitudinal covariate process. Based on an asymptotic bias analysis, we found that their estimators can be biased when random effects do not fully explain the within-subject correlations among longitudinal covariate measurements. Specifically, the existing procedure is fairly sensitive to the independent measurement error assumption. To overcome this limitation, we propose new estimation procedures that require neither a distributional or covariance structural assumption on covariate random effects nor an independence assumption on within-subject measurement errors. These new procedures are more flexible, readily cover scenarios that have multivariate longitudinal covariate processes, and can be implemented using available software. Through simulations and an analysis of data from a hypertension study, we evaluate and illustrate the numerical performances of the new estimators.     

On testing simultaneously non-inferiority in two multiple primary endpoints and superiority in at least one of them

In a clinical trial with an active treatment and a placebo the situation may occur that two (or even more) primary endpoints may be necessary to describe the active treatment's benefit. The focus of our interest is a more specific situation with two primary endpoints in which superiority in one of them would suffice given that non-inferiority is observed in the other. Several proposals exist in the literature for dealing with this or similar problems, but prove insufficient or inadequate at a closer look (e.g. Bloch et al. (2001, 2006) or Tamhane and Logan (2002, 2004)). For example, we were unable to find a good reason why a bootstrap p-value for superiority should depend on the initially selected non-inferiority margins or on the initially selected type I error alpha. We propose a hierarchical three step procedure, where non-inferiority in both variables must be proven in the first step, superiority has to be shown by a bivariate test (e.g. Holm (1979), O'Brien (1984), Hochberg (1988), a bootstrap (Wang (1998)), or L?uter (1996)) in the second step, and then superiority in at least one variable has to be verified in the third step by a corresponding univariate test. All statistical tests are performed at the same one-sided significance level alpha. From the above mentioned bivariate superiority tests we preferred L?uter's SS test and the Holm procedure for the reason that these have been proven to control the type I error strictly, irrespective of the correlation structure among the primary variables and the sample size applied. A simulation study reveals that the performance regarding power of the bivariate test depends to a considerable degree on the correlation and on the magnitude of the expected effects of the two primary endpoints. Therefore, the recommendation of which test to choose depends on knowledge of the possible correlation between the two primary endpoints. In general, L?uter's SS procedure in step 2 shows the best overall properties, whereas Holm's procedure shows an advantage if both a positive correlation between the two variables and a considerable difference between their standardized effect sizes can be expected.     

Testing a Primary and a Secondary Endpoint in a Group Sequential Design

Summary We consider a clinical trial with a primary and a secondary endpoint where the secondary endpoint is tested only if the primary endpoint is significant. The trial uses a group sequential procedure with two stages. The familywise error rate (FWER) of falsely concluding significance on either endpoint is to be controlled at a nominal level α. The type I error rate for the primary endpoint is controlled by choosing any α‐level stopping boundary, e.g., the standard O'Brien–Fleming or the Pocock boundary. Given any particular α‐level boundary for the primary endpoint, we study the problem of determining the boundary for the secondary endpoint to control the FWER. We study this FWER analytically and numerically and find that it is maximized when the correlation coefficient ρ between the two endpoints equals 1. For the four combinations consisting of O'Brien–Fleming and Pocock boundaries for the primary and secondary endpoints, the critical constants required to control the FWER are computed for different values of ρ. An ad hoc boundary is proposed for the secondary endpoint to address a practical concern that may be at issue in some applications. Numerical studies indicate that the O'Brien–Fleming boundary for the primary endpoint and the Pocock boundary for the secondary endpoint generally gives the best primary as well as secondary power performance. The Pocock boundary may be replaced by the ad hoc boundary for the secondary endpoint with a very little loss of secondary power if the practical concern is at issue. A clinical trial example is given to illustrate the methods.     

Quadratic inference functions for varying-coefficient models with longitudinal data

Nonparametric smoothing methods are used to model longitudinal data, but the challenge remains to incorporate correlation into nonparametric estimation procedures. In this article, we propose an efficient estimation procedure for varying-coefficient models for longitudinal data. The proposed procedure can easily take into account correlation within subjects and deal directly with both continuous and discrete response longitudinal data under the framework of generalized linear models. The proposed approach yields a more efficient estimator than the generalized estimation equation approach when the working correlation is misspecified. For varying-coefficient models, it is often of interest to test whether coefficient functions are time varying or time invariant. We propose a unified and efficient nonparametric hypothesis testing procedure, and further demonstrate that the resulting test statistics have an asymptotic chi-squared distribution. In addition, the goodness-of-fit test is applied to test whether the model assumption is satisfied. The corresponding test is also useful for choosing basis functions and the number of knots for regression spline models in conjunction with the model selection criterion. We evaluate the finite sample performance of the proposed procedures with Monte Carlo simulation studies. The proposed methodology is illustrated by the analysis of an acquired immune deficiency syndrome (AIDS) data set.     

Adjusting O'Brien's test to control type I error for the generalized nonparametric Behrens-Fisher problem

O'Brien (1984, Biometrics 40, 1079-1087) introduced a simple nonparametric test procedure for testing whether multiple outcomes in one treatment group have consistently larger values than outcomes in the other treatment group. We first explore the theoretical properties of O'Brien's test. We then extend it to the general nonparametric Behrens-Fisher hypothesis problem when no assumption is made regarding the shape of the distributions. We provide conditions when O'Brien's test controls its error probability asymptotically and when it fails. We also provide adjusted tests when the conditions do not hold. Throughout this article, we do not assume that all outcomes are continuous. Simulations are performed to compare the adjusted tests to O'Brien's test. The difference is also illustrated using data from a Parkinson's disease clinical trial.     

Phenotyping of pig alpha 1B-glycoprotein (PO2) and haemopexin by 1D polyacrylamide gel electrophoresis and immunoblotting

Described is an alternative procedure for the phenotyping of pig alpha 1B-glycoprotein (PO2) and haemopexin. The procedure is based on the separation of serum samples by horizontal polyacrylamide gel electrophoresis, passive blotting onto a nitrocellulose (NC) sheet, and immunochemical detection using a mixture of a primary antibody (rabbit anti-pig alpha 1B or anti-pig haemopexin) and a peroxidase-labelled secondary antibody. Several NC copies can be obtained from a single gel and these can be developed with different monospecific antisera.     

Secondary abdominal contour surgery: a review of early and late reoperative surgery

A retrospective chart review of 400 abdominal contour operations produced a series of 24 patients who underwent both their primary and then their secondary abdominal contour surgeries with the senior author (Matarasso). The majority of patients were classified and treated according to the abdominoplasty classification system previously described; however, a subgroup could not be categorized according to this system. In this study, the authors identified the secondary abdominal contour surgical experience of one surgeon. A comparison was made between two groups of patients treated for both primary and secondary operations: group I, considered early, less than 18 months after the previous operation; and group II, considered late, 18 or more months after the previous operation. There was a significant difference between groups I and II (chi2 = 4.12, p = 0.05); most patients had their surgical procedures before 18 months. For patients who underwent either a miniabdominoplasty or a full primary abdominoplasty, there was a statistically significant difference between the number of patients treated in group I and the number in group II (Fisher's exact test, D = 0, p = 0.05). Next, the nature of the secondary procedure was determined to be either a revisional procedure or a completely new reoperative procedure. The majority of patients underwent revision or "touch-ups," accomplished with either liposuction alone or in combination with scar revision. There was no significant difference between types of primary and secondary procedures performed in group I or group II. Secondary abdominal contour surgery accounted for 6 percent (24 of 400) of all abdominal contour procedures performed by one surgeon. Complete secondary surgery, performing an additional open procedure, occurred in 21 percent of cases (five of 24). Revision surgery (scar revision or removal of dog-ears) was performed in 29 percent of all cases (seven of 24). There was a 4 percent (one of 24) complication rate requiring operative intervention. This rate is consistent with that reported in the literature for primary abdominal contour surgery. With the overall acceptance of aesthetic surgery increasing, the number of patients undergoing abdominoplasty increasing, an aging population, and the safety of secondary abdominal contour surgery suggested from this review, it is likely that plastic surgeons will see more patients requesting secondary abdominal contour surgery in the future.