A comparison of methods for constructing confidence intervals after phase II/III clinical trials

Recently, in order to accelerate drug development, trials that use adaptive seamless designs such as phase II/III clinical trials have been proposed. Phase II/III clinical trials combine traditional phases II and III into a single trial that is conducted in two stages. Using stage 1 data, an interim analysis is performed to answer phase II objectives and after collection of stage 2 data, a final confirmatory analysis is performed to answer phase III objectives. In this paper we consider phase II/III clinical trials in which, at stage 1, several experimental treatments are compared to a control and the apparently most effective experimental treatment is selected to continue to stage 2. Although these trials are attractive because the confirmatory analysis includes phase II data from stage 1, the inference methods used for trials that compare a single experimental treatment to a control and do not have an interim analysis are no longer appropriate. Several methods for analysing phase II/III clinical trials have been developed. These methods are recent and so there is little literature on extensive comparisons of their characteristics. In this paper we review and compare the various methods available for constructing confidence intervals after phase II/III clinical trials.     

Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: applications and practical considerations

Adaptive seamless phase II/III designs combine a phase II and a phase III study into one single confirmatory clinical trial. Several examples of such designs are presented, where the primary endpoint is binary, time-to-event or continuous. The interim adaptations considered include the selection of treatments and the selection of hypotheses related to a pre-specified subgroup of patients. Practical aspects concerning the planning and implementation of adaptive seamless confirmatory studies are also discussed.     

Adaptive seamless clinical trials using early outcomes for treatment or subgroup selection: Methods,simulation model and their implementation in R

Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, for example, in terms of sample size and/or development time. It is well acknowledged that adaptive designs are more involved from a logistical perspective and require more upfront planning, often in the form of extensive simulation studies, than conventional approaches. Here, we present a framework for adaptive treatment and subgroup selection using the same notation, which links the somewhat disparate literature on treatment selection on one side and on subgroup selection on the other. Furthermore, we introduce a flexible and efficient simulation model that serves both designs. As primary endpoints often take a long time to observe, interim analyses are frequently informed by early outcomes. Therefore, all methods presented accommodate interim analyses informed by either the primary outcome or an early outcome. The R package asd , previously developed to simulate designs with treatment selection, was extended to include subgroup selection (so-called adaptive enrichment designs). Here, we describe the functionality of the R package asd and use it to present some worked-up examples motivated by clinical trials in chronic obstructive pulmonary disease and oncology. The examples both illustrate various features of the R package and provide insights into the operating characteristics of adaptive seamless studies.     

Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: opportunities and limitations

This is a discussion of the following two papers in this special issue on adaptive designs: 'Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: General concepts' by Frank Bretz, Heinz Schmidli, Franz K?nig, Amy Racine and Willi Maurer, and 'Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: Applications and practical considerations' by Heinz Schmidli, Frank Bretz, Amy Racine and Willi Maurer.     

A SIMULATION OF WRIGHT'S SHIFTING-BALANCE PROCESS: MIGRATION AND THE THREE PHASES

Wright partitioned the shifting-balance process into three phases. Phase one is the shift of a deme within a population to the domain of a higher adaptive peak from that of the historical peak. Phase two is mass selection within a deme towards that higher peak. Phase three is the conversion of additional demes to the higher peak. The migration rate between demes is critical for the existence of phases one and three. Phase one requires small effective population sizes, hence low migration rates. Phase three is optimal under high migration rates that spread the most-fit genotype from deme to deme. Thus, a population-wide peak shift requires intermediate levels of migration. By altering the rates of phases one and three, migration affects the predominant direction of mass selection within a population. This study examines the degree to which migration, through its effects on phases one and three, determines the probability of a simulated population arriving at its genotypic optimum after 12,000 generations. These simulations reveal that there is a range of migration rates for which an entire population might be expected to shift to a higher peak. Below m = 0.001 peak shifts occur frequently (phases I and II) but are not successfully exported out of subpopulations (phase III), and above 0.01 peak shifts within demes (phase I and II), required to initiate phase III, become increasingly uncommon. Because it is unlikely that real populations will have uniform migration rates from generation to generation, the probable effects of varying migration rates on broadening the range of conditions producing peak shifts are discussed.     

PERSPECTIVE: A CRITIQUE OF SEWALL WRIGHT'S SHIFTING BALANCE THEORY OF EVOLUTION

We evaluate Sewall Wright's three-phase “shifting balance” theory of evolution, examining both the theoretical issues and the relevant data from nature and the laboratory. We conclude that while phases I and II of Wright's theory (the movement of populations from one “adaptive peak” to another via drift and selection) can occur under some conditions, genetic drift is often unnecessary for movement between peaks. Phase III of the shifting balance, in which adaptations spread from particular populations to the entire species, faces two major theoretical obstacles: (1) unlike adaptations favored by simple directional selection, adaptations whose fixation requires some genetic drift are often prevented from spreading by barriers to gene flow; and (2) it is difficult to assemble complex adaptations whose constituent parts arise via peak shifts in different demes. Our review of the data from nature shows that although there is some evidence for individual phases of the shifting balance process, there are few empirical observations explained better by Wright's three-phase mechanism than by simple mass selection. Similarly, artificial selection experiments fail to show that selection in subdivided populations produces greater response than does mass selection in large populations. The complexity of the shifting balance process and the difficulty of establishing that adaptive valleys have been crossed by genetic drift make it impossible to test Wright's claim that adaptations commonly originate by this process. In view of these problems, it seems unreasonable to consider the shifting balance process as an important explanation for the evolution of adaptations.     

How practical are adaptive designs likely to be for confirmatory trials?

This is a discussion of the following papers appearing in this special issue on adaptive designs: 'Confirmatory Seamless Phase II/III Clinical trials with Hypotheses Selection at Interim: General Concepts' by Frank Bretz, Heinz Schmidli, Franz K?nig, Amy Racine and Willi Maurer; and 'Confirmatory Seamless Phase II/III Clinical Trials with Hypotheses Selection at Interim: Applications and Practical Considerations' by Heinz Schmidli, Frank Bretz, Amy Racine and Willi Maurer.     

Improving the Flexibility and Efficiency of Phase II Designs for Oncology Trials

Summary Phase II trials in oncology are usually conducted as single-arm two-stage designs with binary endpoints. Currently available adaptive design methods are tailored to comparative studies with continuous test statistics. Direct transfer of these methods to discrete test statistics results in conservative procedures and, therefore, in a loss in power. We propose a method based on the conditional error function principle that directly accounts for the discreteness of the outcome. It is shown how application of the method can be used to construct new phase II designs that are more efficient as compared to currently applied designs and that allow flexible mid-course design modifications. The proposed method is illustrated with a variety of frequently used phase II designs.     

Test‐compatible confidence intervals for adaptive two‐stage single‐arm designs with binary endpoint

Inference after two‐stage single‐arm designs with binary endpoint is challenging due to the nonunique ordering of the sampling space in multistage designs. We illustrate the problem of specifying test‐compatible confidence intervals for designs with nonconstant second‐stage sample size and present two approaches that guarantee confidence intervals consistent with the test decision. Firstly, we extend the well‐known Clopper–Pearson approach of inverting a family of two‐sided hypothesis tests from the group‐sequential case to designs with fully adaptive sample size. Test compatibility is achieved by using a sample space ordering that is derived from a test‐compatible estimator. The resulting confidence intervals tend to be conservative but assure the nominal coverage probability. In order to assess the possibility of further improving these confidence intervals, we pursue a direct optimization approach minimizing the mean width of the confidence intervals. While the latter approach produces more stable coverage probabilities, it is also slightly anti‐conservative and yields only negligible improvements in mean width. We conclude that the Clopper–Pearson‐type confidence intervals based on a test‐compatible estimator are the best choice if the nominal coverage probability is not to be undershot and compatibility of test decision and confidence interval is to be preserved.     

Plan to be flexible: a commentary on adaptive designs

This is a discussion of the following three papers appearing in this special issue on adaptive designs: 'Nested repeated confidence intervals and switching between noninferiority and superiority' by Joachim Hartung and Guido Knapp; 'Confirmatory Seamless Phase II/III Clinical trials with Hypotheses Selection at Interim: General Concepts' by Frank Bretz, Heinz Schmidli, Franz K?nig, Amy Racine and Willi Maurer; and 'Confirmatory Seamless Phase Il/III Clinical Trials with Hypotheses Selection at Interim: Applications and Practical Considerations' by Heinz Schmidli, Frank Bretz, Amy Racine and Willi Maurer.     

Application of adaptive designs--a review

A literature search has been performed to review applications of the adaptive design methodology based on the combination test or conditional error function approach. Some features of the 60 papers identified are summarized, e.g., the specific methodology used, calendar year, country, impact factor of the journal, number of planned and performed stages respectively, stopping for futility boundaries, type of adaptations and others. A selection of the ten recent publications in journals with the highest impact factors is discussed in more detail. Most applications up to now aim at sample size reassessment, the majority of papers is coming from Germany. Although we found that renowned journals allow for sufficient space to present the new statistical methodology in all its necessary details, the general impression is that the presentation of the adaptive designs methodology in applied papers has to be improved. Education and development of standards could help to achieve this.     

An adaptive two-stage design with treatment selection using the conditional error function approach

As an approach to combining the phase II dose finding trial and phase III pivotal trials, we propose a two-stage adaptive design that selects the best among several treatments in the first stage and tests significance of the selected treatment in the second stage. The approach controls the type I error defined as the probability of selecting a treatment and claiming its significance when the selected treatment is indifferent from placebo, as considered in Bischoff and Miller (2005). Our approach uses the conditional error function and allows determining the conditional type I error function for the second stage based on information observed at the first stage in a similar way to that for an ordinary adaptive design without treatment selection. We examine properties such as expected sample size and stage-2 power of this design with a given type I error and a maximum stage-2 sample size under different hypothesis configurations. We also propose a method to find the optimal conditional error function of a simple parametric form to improve the performance of the design and have derived optimal designs under some hypothesis configurations. Application of this approach is illustrated by a hypothetical example.     

Cancer vaccines as a therapeutic modality: the long trek

The development of cancer vaccines has been one of the several false dawns in which initial promising Phase I and Phase II clinical data have not been followed up with conclusive Phase III trials. In this review, we describe some of the successes and failures, and review the most likely reasons for Phase III failure, such as protocol changes, which are common between Phase II and III, and poorly defined patient groups. Nevertheless, significant survival results have been reported with autologous vaccines for colorectal, renal and, more recently, prostate cancer. In addition, it is becoming evident that immunotherapy is potentially synergistic with other treatment modalities, such as chemotherapy, which can reduce T-regulatory activity that inhibits the immune response to cancer vaccines. This potential for synergy should allow cancer vaccines to become part of the standard treatment regimen for many common tumours.This article is a symposium paper from the “Robert Baldwin Symposium: 50 years of Cancer Immunotherapy”, held in Nottingham, Great Britain, on 30 June 2005.     

Phases of dormancy in Yam tubers (Dioscorea rotundata)

BACKGROUND AND AIMS: The control of dormancy in yam (Disocorea spp.) tubers is poorly understood and attempts to shorten the long dormant period (i.e. cause tubers to sprout or germinate much earlier) have been unsuccessful. The aim of this study was to identify and define the phases of dormancy in Dioscorea rotundata tubers, and to produce a framework within which dormancy can be more effectively studied. METHODS: Plants of 'TDr 131' derived from tissue culture were grown in a glasshouse simulating temperature and photoperiod at Ibadan (7 degrees N), Nigeria to produce tubers. Tubers were sampled on four occasions: 30 d before shoot senescence (149 days after planting, DAP), at shoot senescence (179 DAP), and twice during storage at a constant 25 degrees C (269 and 326 DAP). The development of the apical shoot bud was described from tissue sections. In addition, the responsiveness of shoot apical bud development to plant growth regulators (gibberellic acid, 2-chloroethanol and thiourea) applied to excised tuber sections was also examined 6 and 12 d after treatment. KEY RESULTS AND CONCLUSIONS: Three phases of tuber dormancy are proposed: Phase I, from tuber initiation to the appearance of the tuber germinating meristem; Phase II, from the tuber germinating meristem to initiation of foliar primordium; and Phase III, from foliar primordium to appearance of the shoot bud on the surface of the tuber. Phase I is the longest phase (approx. 220 d in 'TDr 131'), is not affected by PGRs and is proposed to be an endo-dormant phase. Phases II and III are shorter (<70 d in total), are influenced by PGRs and environmental conditions, and are therefore endo-/eco-dormant phases. To manipulate dormancy to allow off-season planting and more than one generation per year requires that the duration of Phase I is shortened.     

Seamlessly expanding a randomized phase II trial to phase III

A sequential Bayesian phase II/III design is proposed for comparative clinical trials. The design is based on both survival time and discrete early events that may be related to survival and assumes a parametric mixture model. Phase II involves a small number of centers. Patients are randomized between treatments throughout, and sequential decisions are based on predictive probabilities of concluding superiority of the experimental treatment. Whether to stop early, continue, or shift into phase III is assessed repeatedly in phase II. Phase III begins when additional institutions are incorporated into the ongoing phase II trial. Simulation studies in the context of a non-small-cell lung cancer trial indicate that the proposed method maintains overall size and power while usually requiring substantially smaller sample size and shorter trial duration when compared with conventional group-sequential phase III designs.     

Drop-the-losers design: normal case

Drop-the-losers designs are statistical designs which have two stages of a trial separated by a data based decision. In the first stage k experimental treatments and a control are administered. During a transition period, the empirically best experimental treatment is selected for continuation into the second phase, along with the control. At the study's end, inference focuses on the comparison of the selected treatment with the control using both stages' data. Traditional methods used to make inferences based on both stages' data can yield tests with higher than advertised levels of significance and confidence intervals with lower than advertised confidence. For normally distributed data, methods are provided to correct these deficiencies, providing confidence intervals with accurate levels of confidence. Drop-the-losers designs are particularly applicable to biopharmaceutical clinical trials where they can allow Phase II and Phase III clinical trials to be conducted under a single protocol with the use of all available data.     

Adaptive trial designs: a review of barriers and opportunities

ABSTRACT: Adaptive designs allow planned modifications based on data accumulating within a study. The promise of greater flexibility and efficiency stimulates increasing interest in adaptive designs from clinical, academic, and regulatory parties. When adaptive designs are used properly, efficiencies can include a smaller sample size, a more efficient treatment development process, and an increased chance of correctly answering the clinical question of interest. However, improper adaptations can lead to biased studies. A broad definition of adaptive designs allows for countless variations, which creates confusion as to the statistical validity and practical feasibility of many designs. Determining properties of a particular adaptive design requires careful consideration of the scientific context and statistical assumptions. We first review several adaptive designs that garner the most current interest. We focus on the design principles and research issues that lead to particular designs being appealing or unappealing in particular applications. We separately discuss exploratory and confirmatory stage designs in order to account for the differences in regulatory concerns. We include adaptive seamless designs, which combine stages in a unified approach. We also highlight a number of applied areas, such as comparative effectiveness research, that would benefit from the use of adaptive designs. Finally, we describe a number of current barriers and provide initial suggestions for overcoming them in order to promote wider use of appropriate adaptive designs. Given the breadth of the coverage all mathematical and most implementation details are omitted for the sake of brevity. However, the interested reader will find that we provide current references to focused reviews and original theoretical sources which lead to details of the current state of the art in theory and practice.     

A Bootstrap Procedure for Adaptive Selection of the Test Statistic in Flexible Two‐Stage Designs

Adaptive two‐stage designs allow a data‐driven change of design characteristics during the ongoing trial. One of the available options is an adaptive choice of the test statistic for the second stage of the trial based on the results of the interim analysis. Since there is often only a vague knowledge of the distribution shape of the primary endpoint in the planning phase of a study, a change of the test statistic may then be considered if the data indicate that the assumptions underlying the initial choice of the test are not correct. Collings and Hamilton proposed a bootstrap method for the estimation of the power of the two‐sample Wilcoxon test for shift alternatives. We use this approach for the selection of the test statistic. By means of a simulation study, we show that the gain in terms of power may be considerable when the initial assumption about the underlying distribution was wrong, whereas the loss is relatively small when in the first instance the optimal test statistic was chosen. The results also hold true for comparison with a one‐stage design. Application of the method is illustrated by a clinical trial example.     

Vasopressor challenges during chronic MAOI or TCA treatment in anesthetized dogs

Ten male mongrel dogs were treated in Phase I with tranylcypromine, 6 mg/kg IM b.i.d., for 21 days. Dogs were anesthetized at weekly intervals before, during, and up to four weeks after drug treatment with a combination of amylbarbital, 25 mg/kg, 1% enflurane in 70:30 N₂0:0₂) and fentanyl, 500 mcg. Bolus IV injections of norepinephrine, 0.1–0.6 mcg/kg, and ephedrine, 0.03–0.12 mg/kg were given while continuously recording arterial blood pressure, lead II of the ECG, heart rate, and rectal temperature. Awakening times were noted. In Phase II, the dogs were given imipramine, 25 mg/kg IM b.i.d., for 21 days. During Phase III, 14 days of tranylcypromine, 7 days of tranylcypromine plus imipramine, and 7 days of imipramine were administered. Anesthetic techniques were repeated in phases II and III. The fourth phase consisted of tranylcypromine injections, 6 mg IM b.i.d., and anesthesia with amylbarbital 25 mg/kg, 2.5% enflurane in 70:30 N₂0:0₂. Vasopressor challenges were repeated during each phase of the study. Following induction of anesthesia and prior to fentanyl challenge, baseline blood pressures and heart rates did not differ from control in Phase I, II, and III of this study. Responses to norepinephrine during all of the tranylcypromine phases were not significantly different from control but ephedrine responses were prolonged, peaking by the second week of treatment. During Phase II, dysrhythmias occurred following norepinephrine and ephedrine with one lethality following norepinephrine, 0.2 mcg/kg. Responses to norepinephrine and awakening times were significantly greater during Phase II compared to Phase I. In Phase III, during the first week of combined therapy the responses to norepinephrine were significantly greater than any other week of this phase. During Phase IV, resting blood pressure and the ephedrine responses were significantly increased during tranylcypromine when the anesthesia regimen did not include fentanyl. These results suggest that during initial treatment with tranylcypromine or imipramine, cardiovascular responses to vasopressor challenges were predicted by the pharmacology of the antidepressant. During tranylcypromine phases, we did not observe exaggerated cardiovascular effects during anesthesia and vasopressor challenges as had been previously reported.

The use of monoamine oxidase inhibitors (MAOIs), alone or in combination with tricyclic antidepressants (TCAs), has been increasing over the last few years because they appear to be more effective than TCAs alone in treating resistant affective disorders. (1,2) However, reluctance to use MAOIs has persisted because of reports of interactions with various foods and drugs. (3–6) In recent years, cell membrane receptor studies have shown that chronic treatment (greater than 14 to 21 days) with either MAOIs of TCAs results in physiological adaptations that partially reduce the interactions between drugs and their receptors. (7,8) Attenuation of arrhythmogenicity has been described in dogs after six weeks of TCA treatment. (9) Clinically, patients on chronic MAOI therapy have undergone elective surgery, including open-heart surgery, without adverse responses. (10,11) The cardiovascular response to analgesics, anesthetics, and vasopressors during the initial phase (less than 14 to 21 days) of treatment with MAOIs compared to TCAs has yet to be addressed. Therefore, the purpose of this study was to examine the cardiovascular responses of analgesic, anesthetic, and vasopressor agents before, during, and after the first three weeks of MAOI and TCA administration in dogs.