A Comparison of Procedures for Adaptive Choice of Location Tests in Flexible Two‐Stage Designs

Although linear rank statistics for the two‐sample problem are distribution free tests, their power depends on the distribution of the data. In the planning phase of an experiment, researchers are often uncertain about the shape of this distribution and so the choice of test statistic for the analysis and the determination of the required sample size are based on vague information. Adaptive designs with interim analysis can potentially overcome both problems. And in particular, adaptive tests based on a selector statistic are a solution to the first. We investigate whether adaptive tests can be usefully implemented in flexible two‐stage designs to gain power. In a simulation study, we compare several methods for choosing a test statistic for the second stage of an adaptive design based on interim data with the procedure that applies adaptive tests in both stages. We find that the latter is a sensible approach that leads to the best results in most situations considered here. The different methods are illustrated using a clinical trial example.     

Adaptive Two Stage Designs and the Conditional Error Function

Proschan and Hunsberger (1995) suggest the use of a conditional error function to construct a two stage test that meets the α level and allows a very flexible reassessment of the sample size after the interim analysis. In this note we show that several adaptive designs can be formulated in terms of such an error function. The conditional power function defined similarly provides a simple method for sample size reassessment in adaptive two stage designs.     

Optimal promising zone designs

Clinical trials with adaptive sample size reassessment based on an unblinded analysis of interim results are perhaps the most popular class of adaptive designs (see Elsäßer et al., 2007). Such trials are typically designed by prespecifying a zone for the interim test statistic, termed the promising zone, along with a decision rule for increasing the sample size within that zone. Mehta and Pocock (2011) provided some examples of promising zone designs and discussed several procedures for controlling their type‐1 error. They did not, however, address how to choose the promising zone or the corresponding sample size reassessment rule, and proposed instead that the operating characteristics of alternative promising zone designs could be compared by simulation. Jennison and Turnbull (2015) developed an approach based on maximizing expected utility whereby one could evaluate alternative promising zone designs relative to a gold‐standard optimal design. In this paper, we show how, by eliciting a few preferences from the trial sponsor, one can construct promising zone designs that are both intuitive and achieve the Jennison and Turnbull (2015) gold‐standard for optimality.     

Interim analysis incorporating short‐ and long‐term binary endpoints

Designs incorporating more than one endpoint have become popular in drug development. One of such designs allows for incorporation of short‐term information in an interim analysis if the long‐term primary endpoint has not been yet observed for some of the patients. At first we consider a two‐stage design with binary endpoints allowing for futility stopping only based on conditional power under both fixed and observed effects. Design characteristics of three estimators: using primary long‐term endpoint only, short‐term endpoint only, and combining data from both are compared. For each approach, equivalent cut‐off point values for fixed and observed effect conditional power calculations can be derived resulting in the same overall power. While in trials stopping for futility the type I error rate cannot get inflated (it usually decreases), there is loss of power. In this study, we consider different scenarios, including different thresholds for conditional power, different amount of information available at the interim, different correlations and probabilities of success. We further extend the methods to adaptive designs with unblinded sample size reassessments based on conditional power with inverse normal method as the combination function. Two different futility stopping rules are considered: one based on the conditional power, and one from P‐values based on Z‐statistics of the estimators. Average sample size, probability to stop for futility and overall power of the trial are compared and the influence of the choice of weights is investigated.     

An efficient sample size adaptation strategy with adjustment of randomization ratio

In clinical trials, sample size reestimation is a useful strategy for mitigating the risk of uncertainty in design assumptions and ensuring sufficient power for the final analysis. In particular, sample size reestimation based on unblinded interim effect size can often lead to sample size increase, and statistical adjustment is usually needed for the final analysis to ensure that type I error rate is appropriately controlled. In current literature, sample size reestimation and corresponding type I error control are discussed in the context of maintaining the original randomization ratio across treatment groups, which we refer to as “proportional increase.” In practice, not all studies are designed based on an optimal randomization ratio due to practical reasons. In such cases, when sample size is to be increased, it is more efficient to allocate the additional subjects such that the randomization ratio is brought closer to an optimal ratio. In this research, we propose an adaptive randomization ratio change when sample size increase is warranted. We refer to this strategy as “nonproportional increase,” as the number of subjects increased in each treatment group is no longer proportional to the original randomization ratio. The proposed method boosts power not only through the increase of the sample size, but also via efficient allocation of the additional subjects. The control of type I error rate is shown analytically. Simulations are performed to illustrate the theoretical results.     

An iterative method to protect the type I error rate in bioequivalence studies under two‐stage adaptive 2×2 crossover designs

Bioequivalence studies are the pivotal clinical trials submitted to regulatory agencies to support the marketing applications of generic drug products. Average bioequivalence (ABE) is used to determine whether the mean values for the pharmacokinetic measures determined after administration of the test and reference products are comparable. Two‐stage 2×2 crossover adaptive designs (TSDs) are becoming increasingly popular because they allow making assumptions on the clinically meaningful treatment effect and a reliable guess for the unknown within‐subject variability. At an interim look, if ABE is not declared with an initial sample size, they allow to increase it depending on the estimated variability and to enroll additional subjects at a second stage, or to stop for futility in case of poor likelihood of bioequivalence. This is crucial because both parameters must clearly be prespecified in protocols, and the strategy agreed with regulatory agencies in advance with emphasis on controlling the overall type I error. We present an iterative method to adjust the significance levels at each stage which preserves the overall type I error for a wide set of scenarios which should include the true unknown variability value. Simulations showed adjusted significance levels higher than 0.0300 in most cases with type I error always below 5%, and with a power of at least 80%. TSDs work particularly well for coefficients of variation below 0.3 which are especially useful due to the balance between the power and the percentage of studies proceeding to stage 2. Our approach might support discussions with regulatory agencies.     

Sample size recalculation in multicenter randomized controlled clinical trials based on noncomparative data

Many late-phase clinical trials recruit subjects at multiple study sites. This introduces a hierarchical structure into the data that can result in a power-loss compared to a more homogeneous single-center trial. Building on a recently proposed approach to sample size determination, we suggest a sample size recalculation procedure for multicenter trials with continuous endpoints. The procedure estimates nuisance parameters at interim from noncomparative data and recalculates the sample size required based on these estimates. In contrast to other sample size calculation methods for multicenter trials, our approach assumes a mixed effects model and does not rely on balanced data within centers. It is therefore advantageous, especially for sample size recalculation at interim. We illustrate the proposed methodology by a study evaluating a diabetes management system. Monte Carlo simulations are carried out to evaluate operation characteristics of the sample size recalculation procedure using comparative as well as noncomparative data, assessing their dependence on parameters such as between-center heterogeneity, residual variance of observations, treatment effect size and number of centers. We compare two different estimators for between-center heterogeneity, an unadjusted and a bias-adjusted estimator, both based on quadratic forms. The type 1 error probability as well as statistical power are close to their nominal levels for all parameter combinations considered in our simulation study for the proposed unadjusted estimator, whereas the adjusted estimator exhibits some type 1 error rate inflation. Overall, the sample size recalculation procedure can be recommended to mitigate risks arising from misspecified nuisance parameters at the planning stage.     

An Adaptive Location‐Scale Test

Increasing locations are often accompanied by an increase in variability. In this case apparent heteroscedasticity can indicate that there are treatment effects and it is appropriate to consider an alternative involving differences in location as well as in scale. As a location‐scale test the sum of a location and a scale test statistic can be used. However, the power can be raised through weighting the sum. In order to select values for this weighting an adaptive design with an interim analysis is proposed: The data of the first stage are used to calculate the weights and with the second stage's data a weighted location‐scale test is carried out. The p‐values of the two stages are combined through Fisher's combination test. With a Lepage‐type location‐scale test it is illustrated that the resultant adaptive test can be more powerful than the ‘optimum’ test with no interim analysis. The principle to calculate weights, which cannot be reasonably chosen a priori, with the data of the first stage may be useful for other tests which utilize weighted statistics, too. Furthermore, the proposed test is illustrated with an example from experimental ecology.     

An adaptive two-stage design with treatment selection using the conditional error function approach

As an approach to combining the phase II dose finding trial and phase III pivotal trials, we propose a two-stage adaptive design that selects the best among several treatments in the first stage and tests significance of the selected treatment in the second stage. The approach controls the type I error defined as the probability of selecting a treatment and claiming its significance when the selected treatment is indifferent from placebo, as considered in Bischoff and Miller (2005). Our approach uses the conditional error function and allows determining the conditional type I error function for the second stage based on information observed at the first stage in a similar way to that for an ordinary adaptive design without treatment selection. We examine properties such as expected sample size and stage-2 power of this design with a given type I error and a maximum stage-2 sample size under different hypothesis configurations. We also propose a method to find the optimal conditional error function of a simple parametric form to improve the performance of the design and have derived optimal designs under some hypothesis configurations. Application of this approach is illustrated by a hypothetical example.     

Impacts on type I error rate with inappropriate use of learn and confirm in confirmatory adaptive design trials

A two-stage adaptive design trial is a single trial that combines the learning data from stage 1 (or phase II) and the confirming data in stage 2 (or phase III) for formal statistical testing. We call it a "Learn and Confirm" trial. The studywise type I error rate remains to be at issue in a "Learn and Confirm" trial. For studying multiple doses or multiple enpdoints, a "Learn and Confirm" adaptive design can be more attractive than a fixed design approach. This is because intuitively the learning data in stage 1 should not be subjected to type I error scrutiny if there is no formal interim analysis performed and only an adaptive selection of design parameters is made at stage 1. In this work, we conclude from extensive simulation studies that the intuition is most often misleading. That is, regardless of whether or not there is a formal interim analysis for making an adaptive selection, the type I error rates are always at risk of inflation. Inappropriate use of any "Learn and Confirm" strategy should not be overlooked.     

Data adaptive interim modification of sample sizes for candidate-gene association studies

OBJECTIVES: The use of conventional Transmission/Disequilibrium tests in the analysis of candidate-gene association studies requires the precise and complete pre-specification of the total number of trios to be sampled to obtain sufficient power at a certain significance level (type I error risk). In most of these studies, very little information about the genetic effect size will be available beforehand and thus it will be difficult to calculate a reasonable sample size. One would therefore wish to reassess the sample size during the course of a study. METHOD: We propose an adaptive group sequential procedure which allows for both early stopping of the study with rejection of the null hypothesis (H0) and for recalculation of the sample size based on interim effect size estimates when H0 cannot be rejected. The applicability of the method which was developed by Müller and Sch?fer [Biometrics 2001;57:886-891] in a clinical context is demonstrated by a numerical example. Monte Carlo simulations are performed comparing the adaptive procedure with a fixed sample and a conventional group sequential design. RESULTS: The main advantage of the adaptive procedure is its flexibility to allow for design changes in order to achieve a stabilized power characteristic while controlling the overall type I error and using the information already collected. CONCLUSIONS: Given these advantages, the procedure is a promising alternative to traditional designs.     

Optimal conditional error functions for the control of conditional power

Ethical considerations and the competitive environment of clinical trials usually require that any given trial have sufficient power to detect a treatment advance. If at an interim analysis the available data are used to decide whether the trial is promising enough to be continued, investigators and sponsors often wish to have a high conditional power, which is the probability to reject the null hypothesis given the interim data and the alternative of interest. Under this requirement a design with interim sample size recalculation, which keeps the overall and conditional power at a prespecified value and preserves the overall type I error rate, is a reasonable alternative to a classical group sequential design, in which the conditional power is often too small. In this article two-stage designs with control of overall and conditional power are constructed that minimize the expected sample size, either for a simple point alternative or for a random mixture of alternatives given by a prior density for the efficacy parameter. The presented optimality result applies to trials with and without an interim hypothesis test; in addition, one can account for constraints such as a minimal sample size for the second stage. The optimal designs will be illustrated with an example, and will be compared to the frequently considered method of using the conditional type I error level of a group sequential design.     

An improved double sampling procedure based on the variance

Sample size calculations for a continuous outcome require specification of the anticipated variance; inaccurate specification can result in an underpowered or overpowered study. For this reason, adaptive methods whereby sample size is recalculated using the variance of a subsample have become increasingly popular. The first proposal of this type (Stein, 1945, Annals of Mathematical Statistics 16, 243-258) used all of the data to estimate the mean difference but only the first stage data to estimate the variance. Stein's procedure is not commonly used because many people perceive it as ignoring relevant data. This is especially problematic when the first stage sample size is small, as would be the case if the anticipated total sample size were small. A more naive approach uses in the denominator of the final test statistic the variance estimate based on all of the data. Applying the Helmert transformation, we show why this naive approach underestimates the true variance and how to construct an unbiased estimate that uses all of the data. We prove that the type I error rate of our procedure cannot exceed alpha.     

Internal pilots for observational studies

Study planning often involves selecting an appropriate sample size. Power calculations require specifying an effect size and estimating “nuisance” parameters, e.g. the overall incidence of the outcome. For observational studies, an additional source of randomness must be estimated: the rate of the exposure. A poor estimate of any of these parameters will produce an erroneous sample size. Internal pilot (IP) designs reduce the risk of this error ‐ leading to better resource utilization ‐ by using revised estimates of the nuisance parameters at an interim stage to adjust the final sample size. In the clinical trials setting, where allocation to treatment groups is pre‐determined, IP designs have been shown to achieve the targeted power without introducing substantial inflation of the type I error rate. It has not been demonstrated whether the same general conclusions hold in observational studies, where exposure‐group membership cannot be controlled by the investigator. We extend the IP to observational settings. We demonstrate through simulations that implementing an IP, in which prevalence of the exposure can be re‐estimated at an interim stage, helps ensure optimal power for observational research with little inflation of the type I error associated with the final data analysis.     

Interim monitoring in sequential multiple assignment randomized trials

A sequential multiple assignment randomized trial (SMART) facilitates the comparison of multiple adaptive treatment strategies (ATSs) simultaneously. Previous studies have established a framework to test the homogeneity of multiple ATSs by a global Wald test through inverse probability weighting. SMARTs are generally lengthier than classical clinical trials due to the sequential nature of treatment randomization in multiple stages. Thus, it would be beneficial to add interim analyses allowing for an early stop if overwhelming efficacy is observed. We introduce group sequential methods to SMARTs to facilitate interim monitoring based on the multivariate chi-square distribution. Simulation studies demonstrate that the proposed interim monitoring in SMART (IM-SMART) maintains the desired type I error and power with reduced expected sample size compared to the classical SMART. Finally, we illustrate our method by reanalyzing a SMART assessing the effects of cognitive behavioral and physical therapies in patients with knee osteoarthritis and comorbid subsyndromal depressive symptoms.     

Population-Enrichment Adaptive Design Strategy for an Event-Driven Vaccine Efficacy Trial

A population-enrichment adaptive design allows a prospective use for study population selection. It has the flexibility allowing pre-specified modifications to an ongoing trial to mitigate the potential risk associated with the assumptions made at design stage. In this way, the trial can potentially encompass a broader target patient population, and move forward only with the subpopulations that appear to be benefiting from the treatment. Our work is motivated by a Phase III event-driven vaccine efficacy trial. Two target patient subpopulations were enrolled with the assumption that vaccine efficacy can be demonstrated based on the combined population. It is recognized due to the nature of patients’ underlying conditions, one subpopulation might respond to the treatment better than the other. To maximize the probability of demonstrating vaccine efficacy in at least one patient population while taking advantage of combining two subpopulations in one single trial, an adaptive design strategy with potential population enrichment is developed. Specifically, if the observed vaccine efficacy at interim for one subpopulation is not promising to warrant carrying forward, the population may be enriched with the other subpopulation with better performance. Simulations were conducted to evaluate the operational characteristics from a selection of interim analysis plans. This population-enrichment design provides a more efficient way as compared to the conventional approaches when targeting multiple subpopulations. If executed and planned with caution, this strategy can provide a greater chance of success of the trial and help maintain scientific and regulatory rigors.     

Efficiency Considerations for Group Sequential Designs with Adaptive Unblinded Sample Size Re-assessment

Clinical trials with adaptive sample size re-assessment, based on an analysis of the unblinded interim results (ubSSR), have gained in popularity due to uncertainty regarding the value of \(\delta \) at which to power the trial at the start of the study. While the statistical methodology for controlling the type-1 error of such designs is well established, there remain concerns that conventional group sequential designs with no ubSSR can accomplish the same goals with greater efficiency. The precise manner in which this efficiency comparison can be objectified has been difficult to quantify, however. In this paper, we present a methodology for making this comparison in a standard, well-accepted manner by plotting the unconditional power curves of the two approaches while holding constant their expected sample size, at each value of \(\delta \) in the range of interest. It is seen that under reasonable decision rules for increasing sample size (conservative promising zones, and no more than a 50% increase in sample size) there is little or no loss of efficiency for the adaptive designs in terms of unconditional power. The two approaches, however, have very different conditional power profiles. More generally, a methodology has been provided for comparing any design with ubSSR relative to a comparable group sequential design with no ubSSR, so one can determine whether the efficiency loss, if any, of the ubSSR design is offset by the advantages it confers for re-powering the study at the time of the interim analysis.     

Adaptive dose-finding: Proof of concept with type I error control

We consider an adaptive dose‐finding study with two stages. The doses for the second stage will be chosen based on the first stage results. Instead of considering pairwise comparisons with placebo, we apply one test to show an upward trend across doses. This is a possibility according to the ICH‐guideline for dose‐finding studies (ICH‐E4). In this article, we are interested in trend tests based on a single contrast or on the maximum of multiple contrasts. We are interested in flexibly choosing the Stage 2 doses including the possibility to add doses. If certain requirements for the interim decision rules are fulfilled, the final trend test that ignores the adaptive nature of the trial (naïve test) can control the type I error. However, for the more common case that these requirements are not fulfilled, we need to take the adaptivity into account and discuss a method for type I error control. We apply the general conditional error approach to adaptive dose‐finding and discuss special issues appearing in this application. We call the test based on this approach Adaptive Multiple Contrast Test. For an example, we illustrate the theory discussed before and compare the performance of several tests for the adaptive design in a simulation study.     

Modification of sample size in group sequential clinical trials

In group sequential clinical trials, sample size reestimation can be a complicated issue when it allows for change of sample size to be influenced by an observed sample path. Our simulation studies show that increasing sample size based on an interim estimate of the treatment difference can substantially inflate the probability of type I error in most practical situations. A new group sequential test procedure is developed by modifying the weights used in the traditional repeated significance two-sample mean test. The new test has the type I error probability preserved at the target level and can provide a substantial gain in power with the increase of sample size. Generalization of the new procedure is discussed.     

Statistical inference for self-designing clinical trials with a one-sided hypothesis

In the process of monitoring clinical trials, it seems appealing to use the interim findings to determine whether the sample size originally planned will provide adequate power when the alternative hypothesis is true, and to adjust the sample size if necessary. In the present paper, we propose a flexible sequential monitoring method following the work of Fisher (1998), in which the maximum sample size does not have to be specified in advance. The final test statistic is constructed based on a weighted average of the sequentially collected data, where the weight function at each stage is determined by the observed data prior to that stage. Such a weight function is used to maintain the integrity of the variance of the final test statistic so that the overall type I error rate is preserved. Moreover, the weight function plays an implicit role in termination of a trial when a treatment difference exists. Finally, the design allows the trial to be stopped early when the efficacy result is sufficiently negative. Simulation studies confirm the performance of the method.