Interim analysis and sample size reassessment

This article deals with sample size reassessment for adaptive two-stage designs based on conditional power arguments utilizing the variability observed at the first stage. Fisher's product test for the p-values from the disjoint samples at the two stages is considered in detail for the comparison of the means of two normal populations. We show that stopping rules allowing for the early acceptance of the null hypothesis that are optimal with respect to the average sample size may lead to a severe decrease of the overall power if the sample size is a priori underestimated. This problem can be overcome by choosing designs with low probabilities of early acceptance or by midtrial adaptations of the early acceptance boundary using the variability observed in the first stage. This modified procedure is negligibly anticonservative and preserves the power.     

An adaptive two-stage design with treatment selection using the conditional error function approach

As an approach to combining the phase II dose finding trial and phase III pivotal trials, we propose a two-stage adaptive design that selects the best among several treatments in the first stage and tests significance of the selected treatment in the second stage. The approach controls the type I error defined as the probability of selecting a treatment and claiming its significance when the selected treatment is indifferent from placebo, as considered in Bischoff and Miller (2005). Our approach uses the conditional error function and allows determining the conditional type I error function for the second stage based on information observed at the first stage in a similar way to that for an ordinary adaptive design without treatment selection. We examine properties such as expected sample size and stage-2 power of this design with a given type I error and a maximum stage-2 sample size under different hypothesis configurations. We also propose a method to find the optimal conditional error function of a simple parametric form to improve the performance of the design and have derived optimal designs under some hypothesis configurations. Application of this approach is illustrated by a hypothetical example.     

Sample Size Reassessment in Adaptive Clinical Trials Using a Bias Corrected Estimate

Point estimation in group sequential and adaptive trials is an important issue in analysing a clinical trial. Most literature in this area is only concerned with estimation after completion of a trial. Since adaptive designs allow reassessment of sample size during the trial, reliable point estimation of the true effect when continuing the trial is additionally needed. We present a bias adjusted estimator which allows a more exact sample size determination based on the conditional power principle than the naive sample mean does.     

Optimal promising zone designs

Clinical trials with adaptive sample size reassessment based on an unblinded analysis of interim results are perhaps the most popular class of adaptive designs (see Elsäßer et al., 2007). Such trials are typically designed by prespecifying a zone for the interim test statistic, termed the promising zone, along with a decision rule for increasing the sample size within that zone. Mehta and Pocock (2011) provided some examples of promising zone designs and discussed several procedures for controlling their type‐1 error. They did not, however, address how to choose the promising zone or the corresponding sample size reassessment rule, and proposed instead that the operating characteristics of alternative promising zone designs could be compared by simulation. Jennison and Turnbull (2015) developed an approach based on maximizing expected utility whereby one could evaluate alternative promising zone designs relative to a gold‐standard optimal design. In this paper, we show how, by eliciting a few preferences from the trial sponsor, one can construct promising zone designs that are both intuitive and achieve the Jennison and Turnbull (2015) gold‐standard for optimality.     

Interim analysis incorporating short‐ and long‐term binary endpoints

Designs incorporating more than one endpoint have become popular in drug development. One of such designs allows for incorporation of short‐term information in an interim analysis if the long‐term primary endpoint has not been yet observed for some of the patients. At first we consider a two‐stage design with binary endpoints allowing for futility stopping only based on conditional power under both fixed and observed effects. Design characteristics of three estimators: using primary long‐term endpoint only, short‐term endpoint only, and combining data from both are compared. For each approach, equivalent cut‐off point values for fixed and observed effect conditional power calculations can be derived resulting in the same overall power. While in trials stopping for futility the type I error rate cannot get inflated (it usually decreases), there is loss of power. In this study, we consider different scenarios, including different thresholds for conditional power, different amount of information available at the interim, different correlations and probabilities of success. We further extend the methods to adaptive designs with unblinded sample size reassessments based on conditional power with inverse normal method as the combination function. Two different futility stopping rules are considered: one based on the conditional power, and one from P‐values based on Z‐statistics of the estimators. Average sample size, probability to stop for futility and overall power of the trial are compared and the influence of the choice of weights is investigated.     

Application of adaptive designs--a review

A literature search has been performed to review applications of the adaptive design methodology based on the combination test or conditional error function approach. Some features of the 60 papers identified are summarized, e.g., the specific methodology used, calendar year, country, impact factor of the journal, number of planned and performed stages respectively, stopping for futility boundaries, type of adaptations and others. A selection of the ten recent publications in journals with the highest impact factors is discussed in more detail. Most applications up to now aim at sample size reassessment, the majority of papers is coming from Germany. Although we found that renowned journals allow for sufficient space to present the new statistical methodology in all its necessary details, the general impression is that the presentation of the adaptive designs methodology in applied papers has to be improved. Education and development of standards could help to achieve this.     

Adaptive designs with arbitrary dependence structure

Adaptive designs were originally developed for independent and uniformly distributed p‐values. There are trial settings where independence is not satisfied or where it may not be possible to check whether it is satisfied. In these cases, the test statistics and p‐values of each stage may be dependent. Since the probability of a type I error for a fixed adaptive design depends on the true dependence structure between the p‐values of the stages, control of the type I error rate might be endangered if the dependence structure is not taken into account adequately. In this paper, we address the problem of controlling the type I error rate in two‐stage adaptive designs if any dependence structure between the test statistics of the stages is admitted (worst case scenario). For this purpose, we pursue a copula approach to adaptive designs. For two‐stage adaptive designs without futility stop, we derive the probability of a type I error in the worst case, that is for the most adverse dependence structure between the p‐values of the stages. Explicit analytical considerations are performed for the class of inverse normal designs. A comparison with the significance level for independent and uniformly distributed p‐values is performed. For inverse normal designs without futility stop and equally weighted stages, it turns out that correcting for the worst case is too conservative as compared to a simple Bonferroni design.     

Efficiency Considerations for Group Sequential Designs with Adaptive Unblinded Sample Size Re-assessment

Clinical trials with adaptive sample size re-assessment, based on an analysis of the unblinded interim results (ubSSR), have gained in popularity due to uncertainty regarding the value of \(\delta \) at which to power the trial at the start of the study. While the statistical methodology for controlling the type-1 error of such designs is well established, there remain concerns that conventional group sequential designs with no ubSSR can accomplish the same goals with greater efficiency. The precise manner in which this efficiency comparison can be objectified has been difficult to quantify, however. In this paper, we present a methodology for making this comparison in a standard, well-accepted manner by plotting the unconditional power curves of the two approaches while holding constant their expected sample size, at each value of \(\delta \) in the range of interest. It is seen that under reasonable decision rules for increasing sample size (conservative promising zones, and no more than a 50% increase in sample size) there is little or no loss of efficiency for the adaptive designs in terms of unconditional power. The two approaches, however, have very different conditional power profiles. More generally, a methodology has been provided for comparing any design with ubSSR relative to a comparable group sequential design with no ubSSR, so one can determine whether the efficiency loss, if any, of the ubSSR design is offset by the advantages it confers for re-powering the study at the time of the interim analysis.     

Optimal conditional error functions for the control of conditional power

Ethical considerations and the competitive environment of clinical trials usually require that any given trial have sufficient power to detect a treatment advance. If at an interim analysis the available data are used to decide whether the trial is promising enough to be continued, investigators and sponsors often wish to have a high conditional power, which is the probability to reject the null hypothesis given the interim data and the alternative of interest. Under this requirement a design with interim sample size recalculation, which keeps the overall and conditional power at a prespecified value and preserves the overall type I error rate, is a reasonable alternative to a classical group sequential design, in which the conditional power is often too small. In this article two-stage designs with control of overall and conditional power are constructed that minimize the expected sample size, either for a simple point alternative or for a random mixture of alternatives given by a prior density for the efficacy parameter. The presented optimality result applies to trials with and without an interim hypothesis test; in addition, one can account for constraints such as a minimal sample size for the second stage. The optimal designs will be illustrated with an example, and will be compared to the frequently considered method of using the conditional type I error level of a group sequential design.     

Combining treatment selection and definitive testing

This is a discussion of the following two papers appearing in this special issue on adaptive designs: 'An adaptive hierarchical test procedure for selecting safe and efficient treatments' by Franz K?nig, Peter Bauer and Werner Brannath, and 'An adaptive two-stage design with treatment selection using the conditional error function approach' by Jixian Wang.     

Optimal adaptive two‐stage designs for phase II cancer clinical trials

In oncology, single‐arm two‐stage designs with binary endpoint are widely applied in phase II for the development of cytotoxic cancer therapies. Simon's optimal design with prefixed sample sizes in both stages minimizes the expected sample size under the null hypothesis and is one of the most popular designs. The search algorithms that are currently used to identify phase II designs showing prespecified characteristics are computationally intensive. For this reason, most authors impose restrictions on their search procedure. However, it remains unclear to what extent this approach influences the optimality of the resulting designs. This article describes an extension to fixed sample size phase II designs by allowing the sample size of stage two to depend on the number of responses observed in the first stage. Furthermore, we present a more efficient numerical algorithm that allows for an exhaustive search of designs. Comparisons between designs presented in the literature and the proposed optimal adaptive designs show that while the improvements are generally moderate, notable reductions in the average sample size can be achieved for specific parameter constellations when applying the new method and search strategy.     

Curtailment in single‐arm two‐stage phase II oncology trials

Two‐stage designs that allow for early stopping if the treatment is ineffective are commonly used in phase II oncology trials. A limitation of current designs is that early stopping is only allowed at the end of the first stage, even if it becomes evident during the trial that a significant result is unlikely. One way to overcome this limitation is to implement stochastic curtailment procedures that enable stopping the trial whenever the conditional power is below a pre‐specified threshold θ. In this paper, we present the results for implementing curtailment rules in either only the second stage or both stages of the designs. In total, 102 scenarios with different parameter settings were investigated using conditional power thresholds θ between 0 and 1 in steps of 0.01. An increase in θ results not only in a decrease of the actual Type I error rate and power but also of the expected sample size. Therefore, a reasonable balance has to be found when selecting a specific threshold value in the planning phase of a curtailed two‐stage design. Given that the effect of curtailment highly depends on the underlying design parameters, no general recommendation for θ can be made. However, up to , the loss in power was less than 5% for all investigated scenarios while savings of up to 50% in expected sample size occurred. In general, curtailment is most appropriate when the outcome can be observed fast or when accrual is slow so that adequate information for making early and frequent decisions is available.     

Conditional Point Estimation in Adaptive Group Sequential Test Designs

It is well known that point estimates in group sequential designs are biased. This also applies to adaptive designs that enable, e.g., data driven reassessments of group sample sizes. For triangular designs, Whitehead (1986) (Biometrika 73 , 573–581) proposed a bias adjusted estimate. But this estimate is not feasible in adaptive designs although it is in group sequential designs. Furthermore, there is a waste of information because it does not use the information at which stage the trial was stopped. We present a modification which does use this information and which is applicable to adaptive designs. The modified estimate achieves an improvement in group sequential designs and shows similar results in adaptive designs.     

A sample size adjustment procedure for clinical trials based on conditional power

When designing clinical trials, researchers often encounter the uncertainty in the treatment effect or variability assumptions. Hence the sample size calculation at the planning stage of a clinical trial may also be questionable. Adjustment of the sample size during the mid-course of a clinical trial has become a popular strategy lately. In this paper we propose a procedure for calculating additional sample size needed based on conditional power, and adjusting the final-stage critical value to protect the overall type-I error rate. Compared to other previous procedures, the proposed procedure uses the definition of the conditional type-I error directly without appealing to an extra special function for it. It has better flexibility in setting up interim decision rules and the final-stage test is a likelihood ratio test.     

A Comparison of Procedures for Adaptive Choice of Location Tests in Flexible Two‐Stage Designs

Although linear rank statistics for the two‐sample problem are distribution free tests, their power depends on the distribution of the data. In the planning phase of an experiment, researchers are often uncertain about the shape of this distribution and so the choice of test statistic for the analysis and the determination of the required sample size are based on vague information. Adaptive designs with interim analysis can potentially overcome both problems. And in particular, adaptive tests based on a selector statistic are a solution to the first. We investigate whether adaptive tests can be usefully implemented in flexible two‐stage designs to gain power. In a simulation study, we compare several methods for choosing a test statistic for the second stage of an adaptive design based on interim data with the procedure that applies adaptive tests in both stages. We find that the latter is a sensible approach that leads to the best results in most situations considered here. The different methods are illustrated using a clinical trial example.     

Adaptive patient enrichment designs in therapeutic trials

The utility of clinical trial designs with adaptive patient enrichment is investigated in an adequate and well‐controlled trial setting. The overall treatment effect is the weighted average of the treatment effects in the mutually exclusive subsets of the originally intended entire study population. The adaptive enrichment approaches permit assessment of treatment effect that may be applicable to specific nested patient (sub)sets due to heterogeneous patient characteristics and/or differential response to treatment, e.g. a responsive patient subset versus a lack of beneficial patient subset, in all patient (sub)sets studied. The adaptive enrichment approaches considered include three adaptive design scenarios: (i) total sample size fixed and with futility stopping, (ii) sample size adaptation and futility stopping, and (iii) sample size adaptation without futility stopping. We show that regardless of whether the treatment effect eventually assessed is applicable to the originally studied patient population or only to the nested patient subsets; it is possible to devise an adaptive enrichment approach that statistically outperforms one‐size‐fits‐all fixed design approach and the fixed design with a pre‐specified multiple test procedure. We emphasize the need of additional studies to replicate the finding of a treatment effect in an enriched patient subset. The replication studies are likely to need fewer number of patients because of an identified treatment effect size that is larger than the diluted overall effect size. The adaptive designs, when applicable, are along the line of efficiency consideration in a drug development program.     

Trendtests with adaptive scoring

The concept of adaptive two‐stage designs is applied to the problem of testing the equality of several normal means against an ordered (monotone) alternative. The likelihood‐ratio‐test proposed by Bartholomew is known to have favorable power properties when testing against a monotonic trend. Tests based on contrasts provide a flexible way to incorporate available information regarding the pattern of the unknown true means through appropriate specification of the scores. The basic idea of the presented concept is the combination of Bartholomew 's test (first stage) with an “adaptive score test” (second stage) which utilizes the information resulting from isotonic regression estimation at the first stage. In a Monte Carlo simulation study the adaptive scoring procedure is compared to the non‐adaptive two‐stage procedure using the Bartholomew test at both stages. We found that adaptive scoring may improve the power of the two stage design, in particular if the sample size at the first stage is considerably larger than at the second stage.     

Adaptive Cluster Sampling in the Context of Restoration

Abundance is an important population state variable for monitoring restoration progress. Efficient sampling often proves difficult, however, when populations are sparse and patchily distributed, such as early after restoration planting. Adaptive cluster sampling (ACS) can help by concentrating search effort in high density areas, improving the encounter rate and the ability to detect a population change over time. To illustrate the problem, I determined conventional design sample sizes for estimating abundance of 12 natural populations and 24 recently planted populations (divided among two preserves) of Lupinus perennis L. (wild blue lupine). I then determined the variance efficiency of ACS relative to simple random sampling at fixed effort and cost for 10 additional planted populations in two habitats (field vs. shrubland). Conventional design sample sizes to estimate lupine stem density with 10% or 20% margins of error were many times greater than initial sample size and would require sampling at least 90% of the study area. Differences in effort requirements were negligible for the two preserves and natural versus planted populations. At fixed sample size, ACS equaled or outperformed simple random sampling in 40% of populations; this shifted to 50% after correcting for travel time among sample units. ACS appeared to be a better strategy for inter‐seeded shrubland habitat than for planted field habitat. Restoration monitoring programs should consider adaptive sampling designs, especially when reliable abundance estimation under conventional designs proves elusive.     

Improving the Flexibility and Efficiency of Phase II Designs for Oncology Trials

Summary Phase II trials in oncology are usually conducted as single-arm two-stage designs with binary endpoints. Currently available adaptive design methods are tailored to comparative studies with continuous test statistics. Direct transfer of these methods to discrete test statistics results in conservative procedures and, therefore, in a loss in power. We propose a method based on the conditional error function principle that directly accounts for the discreteness of the outcome. It is shown how application of the method can be used to construct new phase II designs that are more efficient as compared to currently applied designs and that allow flexible mid-course design modifications. The proposed method is illustrated with a variety of frequently used phase II designs.     

Optimal spending functions for asymmetric group sequential designs

We present optimized group sequential designs where testing of a single parameter theta is of interest. We require specification of a loss function and of a prior distribution for theta. For the examples presented, we pre-specify Type I and II error rates and minimize the expected sample size over the prior distribution for theta. Minimizing the square of sample size rather than the sample size is found to produce designs with slightly less aggressive interim stopping rules and smaller maximum sample sizes with essentially identical expected sample size. We compare optimal designs using Hwang-Shih-DeCani and Kim-DeMets spending functions to fully optimized designs not restricted by a spending function family. In the examples selected, we also examine when there might be substantial benefit gained by adding an interim analysis. Finally, we provide specific optimal asymmetric spending function designs that should be generally useful and simply applied when a design with minimal expected sample size is desired.