Adaptive seamless clinical trials using early outcomes for treatment or subgroup selection: Methods,simulation model and their implementation in R

Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, for example, in terms of sample size and/or development time. It is well acknowledged that adaptive designs are more involved from a logistical perspective and require more upfront planning, often in the form of extensive simulation studies, than conventional approaches. Here, we present a framework for adaptive treatment and subgroup selection using the same notation, which links the somewhat disparate literature on treatment selection on one side and on subgroup selection on the other. Furthermore, we introduce a flexible and efficient simulation model that serves both designs. As primary endpoints often take a long time to observe, interim analyses are frequently informed by early outcomes. Therefore, all methods presented accommodate interim analyses informed by either the primary outcome or an early outcome. The R package asd , previously developed to simulate designs with treatment selection, was extended to include subgroup selection (so-called adaptive enrichment designs). Here, we describe the functionality of the R package asd and use it to present some worked-up examples motivated by clinical trials in chronic obstructive pulmonary disease and oncology. The examples both illustrate various features of the R package and provide insights into the operating characteristics of adaptive seamless studies.     

Blinded sample size reestimation in non-inferiority trials with binary endpoints

Sample size calculations in the planning of clinical trials depend on good estimates of the model parameters involved. When the estimates of these parameters have a high degree of uncertainty attached to them, it is advantageous to reestimate the sample size after an internal pilot study. For non-inferiority trials with binary outcome we compare the performance of Type I error rate and power between fixed-size designs and designs with sample size reestimation. The latter design shows itself to be effective in correcting sample size and power of the tests when misspecification of nuisance parameters occurs with the former design.     

Blinded and unblinded sample size reestimation in crossover trials balanced for period

The determination of the sample size required by a crossover trial typically depends on the specification of one or more variance components. Uncertainty about the value of these parameters at the design stage means that there is often a risk a trial may be under‐ or overpowered. For many study designs, this problem has been addressed by considering adaptive design methodology that allows for the re‐estimation of the required sample size during a trial. Here, we propose and compare several approaches for this in multitreatment crossover trials. Specifically, regulators favor reestimation procedures to maintain the blinding of the treatment allocations. We therefore develop blinded estimators for the within and between person variances, following simple or block randomization. We demonstrate that, provided an equal number of patients are allocated to sequences that are balanced for period, the proposed estimators following block randomization are unbiased. We further provide a formula for the bias of the estimators following simple randomization. The performance of these procedures, along with that of an unblinded approach, is then examined utilizing three motivating examples, including one based on a recently completed four‐treatment four‐period crossover trial. Simulation results show that the performance of the proposed blinded procedures is in many cases similar to that of the unblinded approach, and thus they are an attractive alternative.     

Adaptive patient enrichment designs in therapeutic trials

The utility of clinical trial designs with adaptive patient enrichment is investigated in an adequate and well‐controlled trial setting. The overall treatment effect is the weighted average of the treatment effects in the mutually exclusive subsets of the originally intended entire study population. The adaptive enrichment approaches permit assessment of treatment effect that may be applicable to specific nested patient (sub)sets due to heterogeneous patient characteristics and/or differential response to treatment, e.g. a responsive patient subset versus a lack of beneficial patient subset, in all patient (sub)sets studied. The adaptive enrichment approaches considered include three adaptive design scenarios: (i) total sample size fixed and with futility stopping, (ii) sample size adaptation and futility stopping, and (iii) sample size adaptation without futility stopping. We show that regardless of whether the treatment effect eventually assessed is applicable to the originally studied patient population or only to the nested patient subsets; it is possible to devise an adaptive enrichment approach that statistically outperforms one‐size‐fits‐all fixed design approach and the fixed design with a pre‐specified multiple test procedure. We emphasize the need of additional studies to replicate the finding of a treatment effect in an enriched patient subset. The replication studies are likely to need fewer number of patients because of an identified treatment effect size that is larger than the diluted overall effect size. The adaptive designs, when applicable, are along the line of efficiency consideration in a drug development program.     

Adaptive enrichment designs with a continuous biomarker

A popular design for clinical trials assessing targeted therapies is the two-stage adaptive enrichment design with recruitment in stage 2 limited to a biomarker-defined subgroup chosen based on data from stage 1. The data-dependent selection leads to statistical challenges if data from both stages are used to draw inference on treatment effects in the selected subgroup. If subgroups considered are nested, as when defined by a continuous biomarker, treatment effect estimates in different subgroups follow the same distribution as estimates in a group-sequential trial. This result is used to obtain tests controlling the familywise type I error rate (FWER) for six simple subgroup selection rules, one of which also controls the FWER for any selection rule. Two approaches are proposed: one based on multivariate normal distributions suitable if the number of possible subgroups, k, is small, and one based on Brownian motion approximations suitable for large k. The methods, applicable in the wide range of settings with asymptotically normal test statistics, are illustrated using survival data from a breast cancer trial.     

Blinded and unblinded internal pilot study designs for clinical trials with count data

Internal pilot studies are a popular design feature to address uncertainties in the sample size calculations caused by vague information on nuisance parameters. Despite their popularity, only very recently blinded sample size reestimation procedures for trials with count data were proposed and their properties systematically investigated. Although blinded procedures are favored by regulatory authorities, practical application is somewhat limited by fears that blinded procedures are prone to bias if the treatment effect was misspecified in the planning. Here, we compare unblinded and blinded procedures with respect to bias, error rates, and sample size distribution. We find that both procedures maintain the desired power and that the unblinded procedure is slightly liberal whereas the actual significance level of the blinded procedure is close to the nominal level. Furthermore, we show that in situations where uncertainty about the assumed treatment effect exists, the blinded estimator of the control event rate is biased in contrast to the unblinded estimator, which results in differences in mean sample sizes in favor of the unblinded procedure. However, these differences are rather small compared to the deviations of the mean sample sizes from the sample size required to detect the true, but unknown effect. We demonstrate that the variation of the sample size resulting from the blinded procedure is in many practically relevant situations considerably smaller than the one of the unblinded procedures. The methods are extended to overdispersed counts using a quasi‐likelihood approach and are illustrated by trials in relapsing multiple sclerosis.     

Sample size recalculation in internal pilot study designs: a review

The adequacy of sample size is important to clinical trials. In the planning phase of a trial, however, the investigators are often quite uncertain about the sizes of parameters which are needed for sample size calculations. A solution to this problem is mid-course recalculation of the sample size during the ongoing trial. In internal pilot study designs, nuisance parameters are estimated on the basis of interim data and the sample size is adjusted accordingly. This review attempts to give an overview on the available methods. It is written not only for biometricians who are already familar with the the topic and wish to update their knowledge but also for users new to the subject.     

Inverse Adaptive Cluster Sampling

Consider a population in which the variable of interest tends to be at or near zero for many of the population units but a subgroup exhibits values distinctly different from zero. Such a population can be described as rare in the sense that the proportion of elements having nonzero values is very small. Obtaining an estimate of a population parameter such as the mean or total that is nonzero is difficult under classical fixed sample-size designs since there is a reasonable probability that a fixed sample size will yield all zeroes. We consider inverse sampling designs that use stopping rules based on the number of rare units observed in the sample. We look at two stopping rules in detail and derive unbiased estimators of the population total. The estimators do not rely on knowing what proportion of the population exhibit the rare trait but instead use an estimated value. Hence, the estimators are similar to those developed for poststratification sampling designs. We also incorporate adaptive cluster sampling into the sampling design to allow for the case where the rare elements tend to cluster within the population in some manner. The formulas for the variances of the estimators do not allow direct analytic comparison of the efficiency of the various designs and stopping rules, so we provide the results of a small simulation study to obtain some insight into the differences among the stopping rules and sampling approaches. The results indicate that a modified stopping rule that incorporates an adaptive sampling component and utilizes an initial random sample of fixed size is the best in the sense of having the smallest variance.     

Biomarker-Defined Subgroup Selection Adaptive Design for Phase III Confirmatory Trial with Time-to-Event Data: Comparing Group Sequential and Various Adaptive Enrichment Designs

Predictive and prognostic biomarkers play an important role in personalized medicine to determine strategies for drug evaluation and treatment selection. In the context of continuous biomarkers, identification of an optimal cutoff for patient selection can be challenging due to limited information on biomarker predictive value, the biomarker’s distribution in the intended use population, and the complexity of the biomarker relationship to clinical outcomes. As a result, prespecified candidate cutoffs may be rationalized based on biological and practical considerations. In this context, adaptive enrichment designs have been proposed with interim decision rules to select a biomarker-defined subpopulation to optimize study performance. With a group sequential design as a reference, the performance of several proposed adaptive designs are evaluated and compared under various scenarios (e.g., sample size, study power, enrichment effects) where type I error rates are well controlled through closed testing procedures and where subpopulation selections are based upon the predictive probability of trial success. It is found that when the treatment is more effective in a subpopulation, these adaptive designs can improve study power substantially. Furthermore, we identified one adaptive design to have generally higher study power than the other designs under various scenarios.     

Designs of superiority and noninferiority trials for binary responses are noninterchangeable

To better understand the design of noninferiority trials for binary data, we identify analogies and contrasts between this and the more familiar superiority trial design. We restrict attention to the problem of detecting a difference between experimental and control response rates in the setting where there is no difference (piE - piC = 0) under the noninferiority alternative hypothesis and under the superiority null, and a matching difference between groups under the complementary hypotheses (/piE - piC/ = delta). Our derivation of the constrained maximum likelihood estimates (MLEs) reveals that superiority and noninferiority trials have different nuisance parameters--the marginal response rate and the control-group response rate, respectively. Our empirical results show that when individuals are allocated to treatment groups in the ratio that minimizes the overall sample size, balanced allocation is optimal only for superiority trials when the error rates are equal; otherwise imbalanced allocation is optimal. Different allocation ratios between trial types lead to different variances, and thus to different sample sizes. Finally, since the value of the marginal response rate--a design parameter in noninferiority trials--typically cannot be obtained from preliminary or published studies, we suggest a means of identifying a value that can be used. We conclude that full documentation of the design of a trial requires specification not only of the design parameters but also of the allocation ratio and the nuisance parameter, the value of which is not obvious under unequal allocation.     

Optimal promising zone designs

Clinical trials with adaptive sample size reassessment based on an unblinded analysis of interim results are perhaps the most popular class of adaptive designs (see Elsäßer et al., 2007). Such trials are typically designed by prespecifying a zone for the interim test statistic, termed the promising zone, along with a decision rule for increasing the sample size within that zone. Mehta and Pocock (2011) provided some examples of promising zone designs and discussed several procedures for controlling their type‐1 error. They did not, however, address how to choose the promising zone or the corresponding sample size reassessment rule, and proposed instead that the operating characteristics of alternative promising zone designs could be compared by simulation. Jennison and Turnbull (2015) developed an approach based on maximizing expected utility whereby one could evaluate alternative promising zone designs relative to a gold‐standard optimal design. In this paper, we show how, by eliciting a few preferences from the trial sponsor, one can construct promising zone designs that are both intuitive and achieve the Jennison and Turnbull (2015) gold‐standard for optimality.     

Bayesian Adaptive Randomization and Trial Monitoring with Predictive Probability for Time-to-Event Endpoint

There has been much development in Bayesian adaptive designs in clinical trials. In the Bayesian paradigm, the posterior predictive distribution characterizes the future possible outcomes given the currently observed data. Based on the interim time-to-event data, we develop a new phase II trial design by combining the strength of both Bayesian adaptive randomization and the predictive probability. By comparing the mean survival times between patients assigned to two treatment arms, more patients are assigned to the better treatment on the basis of adaptive randomization. We continuously monitor the trial using the predictive probability for early termination in the case of superiority or futility. We conduct extensive simulation studies to examine the operating characteristics of four designs: the proposed predictive probability adaptive randomization design, the predictive probability equal randomization design, the posterior probability adaptive randomization design, and the group sequential design. Adaptive randomization designs using predictive probability and posterior probability yield a longer overall median survival time than the group sequential design, but at the cost of a slightly larger sample size. The average sample size using the predictive probability method is generally smaller than that of the posterior probability design.     

Internal pilots for observational studies

Study planning often involves selecting an appropriate sample size. Power calculations require specifying an effect size and estimating “nuisance” parameters, e.g. the overall incidence of the outcome. For observational studies, an additional source of randomness must be estimated: the rate of the exposure. A poor estimate of any of these parameters will produce an erroneous sample size. Internal pilot (IP) designs reduce the risk of this error ‐ leading to better resource utilization ‐ by using revised estimates of the nuisance parameters at an interim stage to adjust the final sample size. In the clinical trials setting, where allocation to treatment groups is pre‐determined, IP designs have been shown to achieve the targeted power without introducing substantial inflation of the type I error rate. It has not been demonstrated whether the same general conclusions hold in observational studies, where exposure‐group membership cannot be controlled by the investigator. We extend the IP to observational settings. We demonstrate through simulations that implementing an IP, in which prevalence of the exposure can be re‐estimated at an interim stage, helps ensure optimal power for observational research with little inflation of the type I error associated with the final data analysis.     

Blinded sample size recalculation in multicentre trials with normally distributed outcome

The internal pilot study design enables to estimate nuisance parameters required for sample size calculation on the basis of data accumulated in an ongoing trial. By this, misspecifications made when determining the sample size in the planning phase can be corrected employing updated knowledge. According to regulatory guidelines, blindness of all personnel involved in the trial has to be preserved and the specified type I error rate has to be controlled when the internal pilot study design is applied. Especially in the late phase of drug development, most clinical studies are run in more than one centre. In these multicentre trials, one may have to deal with an unequal distribution of the patient numbers among the centres. Depending on the type of the analysis (weighted or unweighted), unequal centre sample sizes may lead to a substantial loss of power. Like the variance, the magnitude of imbalance is difficult to predict in the planning phase. We propose a blinded sample size recalculation procedure for the internal pilot study design in multicentre trials with normally distributed outcome and two balanced treatment groups that are analysed applying the weighted or the unweighted approach. The method addresses both uncertainty with respect to the variance of the endpoint and the extent of disparity of the centre sample sizes. The actual type I error rate as well as the expected power and sample size of the procedure is investigated in simulation studies. For the weighted analysis as well as for the unweighted analysis, the maximal type I error rate was not or only minimally exceeded. Furthermore, application of the proposed procedure led to an expected power that achieves the specified value in many cases and is throughout very close to it.     

Bayesian treatment screening and selection using subgroup-specific utilities of response and toxicity

A Bayesian design is proposed for randomized phase II clinical trials that screen multiple experimental treatments compared to an active control based on ordinal categorical toxicity and response. The underlying model and design account for patient heterogeneity characterized by ordered prognostic subgroups. All decision criteria are subgroup specific, including interim rules for dropping unsafe or ineffective treatments, and criteria for selecting optimal treatments at the end of the trial. The design requires an elicited utility function of the two outcomes that varies with the subgroups. Final treatment selections are based on posterior mean utilities. The methodology is illustrated by a trial of targeted agents for metastatic renal cancer, which motivated the design methodology. In the context of this application, the design is evaluated by computer simulation, including comparison to three designs that conduct separate trials within subgroups, or conduct one trial while ignoring subgroups, or base treatment selection on estimated response rates while ignoring toxicity.     

The effect of screening on some pretest-posttest test variances

The clinical trial design in which the endpoint is measured both at baseline and at the end of the study is used in a variety of situations. For two-group designs, test such as the t test or analysis of covariance are commonly used to evaluate treatment efficacy. Often such pretest-posttest trials restrict participation to subjects with a baseline measurement of the endpoint in a certain range. A range may define a disease, or it may be thought that subjects with extreme measurements are more responsive to treatment. This paper examines the effect of screening on the analysis of covariance and t-test variances relative to the population (i.e., unscreened) variances. Bivariate normal and bivariate gamma distributions are assumed for the (pretest, posttest) measurements. Because the sample size required to detect a specified difference between treatment and control is proportional to the variance, the results have direct application to setting sample size.     

Group sequential and adaptive designs - a review of basic concepts and points of discussion

In recent times, group sequential and adaptive designs for clinical trials have attracted great attention from industry, academia and regulatory authorities. These designs allow analyses on accumulating data - as opposed to classical, "fixed-sample" statistics. The rapid development of a great variety of statistical procedures is accompanied by a lively debate on their potential merits and shortcomings. The purpose of this review article is to ease orientation in both respects. First, we provide a concise overview of the essential technical concepts, with special emphasis on their interrelationships. Second, we give a structured review of the current controversial discussion on practical issues, opportunities and challenges of these new designs.     

Likelihood calculations to evaluate experimental designs to estimate genetic variances

Mixed model analyses via restricted maximum likelihood, fitting the so-called animal model, have become standard methodology for the estimation of genetic variances. Models involving multiple genetic variance components, due to different modes of gene action, are readily fitted. It is shown that likelihood-based calculations may provide insight into the quality of the resulting parameter estimates, and are directly applicable to the validation of experimental designs. This is illustrated for the example of a design suggested recently to estimate X-linked genetic variances. In particular, large sample variances and sampling correlations are demonstrated to provide an indication of 'problem' scenarios. Using simulation, it is shown that the profile likelihood function provides more appropriate estimates of confidence intervals than large sample variances. Examination of the likelihood function and its derivatives are recommended as part of the design stage of quantitative genetic experiments.     

Rearrangements in the 5′ Nontranslated Region and Phylogenetic Analyses of Cucumber Mosaic Virus RNA 3 Indicate Radial Evolution of Three Subgroups

Cucumber mosaic virus (CMV) has been divided into two subgroups based on serological data, peptide mapping of the coat protein, nucleic acid hybridization, and nucleotide sequence similarity. Analyses of a number of recently isolated strains suggest a further division of the subgroup I strains. Alignment of the 5' nontranslated regions of RNA 3 for 26 strains of CMV suggests the division of CMV into subgroups IA, IB, and II and suggests that rearrangements, deletions, and insertions in this region may have been the precursors of the subsequent radiation of each subgroup. Phylogeny analyses of CMV using the coat protein open reading frame of 53 strains strongly support the further division of subgroup I into IA and IB. In addition, strains within each subgroup radiate from a single point of origin, indicating that they have evolved from a single common ancestor for each subgroup.     

Adaptive trial designs: a review of barriers and opportunities

ABSTRACT: Adaptive designs allow planned modifications based on data accumulating within a study. The promise of greater flexibility and efficiency stimulates increasing interest in adaptive designs from clinical, academic, and regulatory parties. When adaptive designs are used properly, efficiencies can include a smaller sample size, a more efficient treatment development process, and an increased chance of correctly answering the clinical question of interest. However, improper adaptations can lead to biased studies. A broad definition of adaptive designs allows for countless variations, which creates confusion as to the statistical validity and practical feasibility of many designs. Determining properties of a particular adaptive design requires careful consideration of the scientific context and statistical assumptions. We first review several adaptive designs that garner the most current interest. We focus on the design principles and research issues that lead to particular designs being appealing or unappealing in particular applications. We separately discuss exploratory and confirmatory stage designs in order to account for the differences in regulatory concerns. We include adaptive seamless designs, which combine stages in a unified approach. We also highlight a number of applied areas, such as comparative effectiveness research, that would benefit from the use of adaptive designs. Finally, we describe a number of current barriers and provide initial suggestions for overcoming them in order to promote wider use of appropriate adaptive designs. Given the breadth of the coverage all mathematical and most implementation details are omitted for the sake of brevity. However, the interested reader will find that we provide current references to focused reviews and original theoretical sources which lead to details of the current state of the art in theory and practice.