Monitoring futility and efficacy in phase II trials with Bayesian posterior distributions—A calibration approach

A multistage single arm phase II trial with binary endpoint is considered. Bayesian posterior probabilities are used to monitor futility in interim analyses and efficacy in the final analysis. For a beta‐binomial model, decision rules based on Bayesian posterior probabilities are converted to “traditional” decision rules in terms of number of responders among patients observed so far. Analytical derivations are given for the probability of stopping for futility and for the probability to declare efficacy. A workflow is presented on how to select the parameters specifying the Bayesian design, and the operating characteristics of the design are investigated. It is outlined how the presented approach can be transferred to statistical models other than the beta‐binomial model.     

Curtailment in single‐arm two‐stage phase II oncology trials

Two‐stage designs that allow for early stopping if the treatment is ineffective are commonly used in phase II oncology trials. A limitation of current designs is that early stopping is only allowed at the end of the first stage, even if it becomes evident during the trial that a significant result is unlikely. One way to overcome this limitation is to implement stochastic curtailment procedures that enable stopping the trial whenever the conditional power is below a pre‐specified threshold θ. In this paper, we present the results for implementing curtailment rules in either only the second stage or both stages of the designs. In total, 102 scenarios with different parameter settings were investigated using conditional power thresholds θ between 0 and 1 in steps of 0.01. An increase in θ results not only in a decrease of the actual Type I error rate and power but also of the expected sample size. Therefore, a reasonable balance has to be found when selecting a specific threshold value in the planning phase of a curtailed two‐stage design. Given that the effect of curtailment highly depends on the underlying design parameters, no general recommendation for θ can be made. However, up to , the loss in power was less than 5% for all investigated scenarios while savings of up to 50% in expected sample size occurred. In general, curtailment is most appropriate when the outcome can be observed fast or when accrual is slow so that adequate information for making early and frequent decisions is available.     

Adaptive patient enrichment designs in therapeutic trials

The utility of clinical trial designs with adaptive patient enrichment is investigated in an adequate and well‐controlled trial setting. The overall treatment effect is the weighted average of the treatment effects in the mutually exclusive subsets of the originally intended entire study population. The adaptive enrichment approaches permit assessment of treatment effect that may be applicable to specific nested patient (sub)sets due to heterogeneous patient characteristics and/or differential response to treatment, e.g. a responsive patient subset versus a lack of beneficial patient subset, in all patient (sub)sets studied. The adaptive enrichment approaches considered include three adaptive design scenarios: (i) total sample size fixed and with futility stopping, (ii) sample size adaptation and futility stopping, and (iii) sample size adaptation without futility stopping. We show that regardless of whether the treatment effect eventually assessed is applicable to the originally studied patient population or only to the nested patient subsets; it is possible to devise an adaptive enrichment approach that statistically outperforms one‐size‐fits‐all fixed design approach and the fixed design with a pre‐specified multiple test procedure. We emphasize the need of additional studies to replicate the finding of a treatment effect in an enriched patient subset. The replication studies are likely to need fewer number of patients because of an identified treatment effect size that is larger than the diluted overall effect size. The adaptive designs, when applicable, are along the line of efficiency consideration in a drug development program.     

Inverse Adaptive Cluster Sampling

Consider a population in which the variable of interest tends to be at or near zero for many of the population units but a subgroup exhibits values distinctly different from zero. Such a population can be described as rare in the sense that the proportion of elements having nonzero values is very small. Obtaining an estimate of a population parameter such as the mean or total that is nonzero is difficult under classical fixed sample-size designs since there is a reasonable probability that a fixed sample size will yield all zeroes. We consider inverse sampling designs that use stopping rules based on the number of rare units observed in the sample. We look at two stopping rules in detail and derive unbiased estimators of the population total. The estimators do not rely on knowing what proportion of the population exhibit the rare trait but instead use an estimated value. Hence, the estimators are similar to those developed for poststratification sampling designs. We also incorporate adaptive cluster sampling into the sampling design to allow for the case where the rare elements tend to cluster within the population in some manner. The formulas for the variances of the estimators do not allow direct analytic comparison of the efficiency of the various designs and stopping rules, so we provide the results of a small simulation study to obtain some insight into the differences among the stopping rules and sampling approaches. The results indicate that a modified stopping rule that incorporates an adaptive sampling component and utilizes an initial random sample of fixed size is the best in the sense of having the smallest variance.     

Conditional Rejection Probabilities of Student's t‐test and Design Adaptations

For clinical trials with interim analyses conditional rejection probabilities play an important role when stochastic curtailment or design adaptations are performed. The conditional rejection probability gives the conditional probability to finally reject the null hypothesis given the interim data. It is computed either under the null or the alternative hypothesis. We investigate the properties of the conditional rejection probability for the one sided, one sample t‐test and show that it can be non monotone in the interim mean of the data and non monotone in the non‐centrality parameter for the alternative. We give several proposals how to implement design adaptations (that are based on the conditional rejection probability) for the t‐test and give a numerical example. Additionally, the conditional rejection probability given the interim t‐statistic is investigated. It does not depend on the unknown σ and can be used in stochastic curtailment procedures. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

A Bootstrap Procedure for Adaptive Selection of the Test Statistic in Flexible Two‐Stage Designs

Adaptive two‐stage designs allow a data‐driven change of design characteristics during the ongoing trial. One of the available options is an adaptive choice of the test statistic for the second stage of the trial based on the results of the interim analysis. Since there is often only a vague knowledge of the distribution shape of the primary endpoint in the planning phase of a study, a change of the test statistic may then be considered if the data indicate that the assumptions underlying the initial choice of the test are not correct. Collings and Hamilton proposed a bootstrap method for the estimation of the power of the two‐sample Wilcoxon test for shift alternatives. We use this approach for the selection of the test statistic. By means of a simulation study, we show that the gain in terms of power may be considerable when the initial assumption about the underlying distribution was wrong, whereas the loss is relatively small when in the first instance the optimal test statistic was chosen. The results also hold true for comparison with a one‐stage design. Application of the method is illustrated by a clinical trial example.     

Exact and Asymptotic Power Determination For Dose‐Response Studies With Ordinal Response

In this article, we describe a conditional score test for detecting a monotone dose‐response relationship with ordinal response data. We consider three different versions of this test: asymptotic, conditional exact, and mid‐P conditional score test. Exact and asymptotic power formulae based on these tests will be studied. Asymptotic sample size formulae based on the asymptotic conditional score test will be derived. The proposed formulae are applied to a vaccination study and a developmental toxicity study for illustrative purposes. Actual significance level and exact power properties of these tests are compared in a small empirical study. The mid‐P conditional score test is observed to be the most powerful test with actual significance level close to the pre‐specified nominal level.     

Blinded Sample Size Recalculation for Normally Distributed Outcomes Using Long‐ and Short‐term Data

Clinical trials are often planned with high uncertainty about the variance of the primary outcome variable. A poor estimate of the variance, however, may lead to an over‐ or underpowered study. In the internal pilot study design, the sample variance is calculated at an interim step and the sample size can be adjusted if necessary. The available recalculation procedures use the data of those patients for sample size recalculation that have already completed the study. In this article, we consider a variance estimator that takes into account both the data at the endpoint and at an intermediate point of the treatment phase. We derive asymptotic properties of this estimator and the relating sample size recalculation procedure. In a simulation study, the performance of the proposed approach is evaluated and compared with the procedure that uses only long‐term data. Simulation results demonstrate that the sample size resulting from the proposed procedure shows in general a smaller variability. At the same time, the Type I error rate is not inflated and the achieved power is close to the desired value.     

The influence of sampling design on tree‐ring‐based quantification of forest growth

Tree‐rings offer one of the few possibilities to empirically quantify and reconstruct forest growth dynamics over years to millennia. Contemporaneously with the growing scientific community employing tree‐ring parameters, recent research has suggested that commonly applied sampling designs (i.e. how and which trees are selected for dendrochronological sampling) may introduce considerable biases in quantifications of forest responses to environmental change. To date, a systematic assessment of the consequences of sampling design on dendroecological and‐climatological conclusions has not yet been performed. Here, we investigate potential biases by sampling a large population of trees and replicating diverse sampling designs. This is achieved by retroactively subsetting the population and specifically testing for biases emerging for climate reconstruction, growth response to climate variability, long‐term growth trends, and quantification of forest productivity. We find that commonly applied sampling designs can impart systematic biases of varying magnitude to any type of tree‐ring‐based investigations, independent of the total number of samples considered. Quantifications of forest growth and productivity are particularly susceptible to biases, whereas growth responses to short‐term climate variability are less affected by the choice of sampling design. The world's most frequently applied sampling design, focusing on dominant trees only, can bias absolute growth rates by up to 459% and trends in excess of 200%. Our findings challenge paradigms, where a subset of samples is typically considered to be representative for the entire population. The only two sampling strategies meeting the requirements for all types of investigations are the (i) sampling of all individuals within a fixed area; and (ii) fully randomized selection of trees. This result advertises the consistent implementation of a widely applicable sampling design to simultaneously reduce uncertainties in tree‐ring‐based quantifications of forest growth and increase the comparability of datasets beyond individual studies, investigators, laboratories, and geographical boundaries.     

On Exact Unconditional Test for Linear Trend in Dose‐Response Studies

The one‐degree‐of‐freedom Cochran‐Armitage (C‐A) test statistic for linear trend has been widely applied in various dose‐response studies (e.g., anti‐ulcer medications and short‐term antibiotics, animal carcinogenicity bioassays and occupational toxicant studies). This approximate statistic relies, however, on asymptotic theory that is reliable only when the sample sizes are reasonably large and well balanced across dose levels. For small, sparse, or skewed data, the asymptotic theory is suspect and exact conditional method (based on the C‐A statistic) seems to provide a dependable alternative. Unfortunately, the exact conditional method is only practical for the linear logistic model from which the sufficient statistics for the regression coefficients can be obtained explicitly. In this article, a simple and efficient recursive polynomial multiplication algorithm for exact unconditional test (based on the C‐A statistic) for detecting a linear trend in proportions is derived. The method is applicable for all choices of the model with monotone trend including logistic, probit, arcsine, extreme value and one hit. We also show that this algorithm can be easily extended to exact unconditional power calculation for studies with up to a moderately large sample size. A real example is given to illustrate the applicability of the proposed method.     

Improved two‐stage group sequential procedures for testing a secondary endpoint after the primary endpoint achieves significance

In two‐stage group sequential trials with a primary and a secondary endpoint, the overall type I error rate for the primary endpoint is often controlled by an α‐level boundary, such as an O'Brien‐Fleming or Pocock boundary. Following a hierarchical testing sequence, the secondary endpoint is tested only if the primary endpoint achieves statistical significance either at an interim analysis or at the final analysis. To control the type I error rate for the secondary endpoint, this is tested using a Bonferroni procedure or any α‐level group sequential method. In comparison with marginal testing, there is an overall power loss for the test of the secondary endpoint since a claim of a positive result depends on the significance of the primary endpoint in the hierarchical testing sequence. We propose two group sequential testing procedures with improved secondary power: the improved Bonferroni procedure and the improved Pocock procedure. The proposed procedures use the correlation between the interim and final statistics for the secondary endpoint while applying graphical approaches to transfer the significance level from the primary endpoint to the secondary endpoint. The procedures control the familywise error rate (FWER) strongly by construction and this is confirmed via simulation. We also compare the proposed procedures with other commonly used group sequential procedures in terms of control of the FWER and the power of rejecting the secondary hypothesis. An example is provided to illustrate the procedures.     

A Shift Algorithm for Exact Computation in Sequential Tests with Binary Outcome

A simple shift algorithm is described enabling the exact determination of power functions and sample size distributions for a large variety of closed sequential two‐sample designs with a binary outcome variable. The test statistics are assumed to be based on relative frequencies of successes or failures, but the number of interim analyses, the monitoring times, and the continuation regions may be specified as desired. To give examples, exact properties of designs proposed by the program package EaSt (Cytel , 1992) are determined, and plans with interim analyses are considered where decisions are based on the conditional power given the observations obtained so far.     

Comparison of One‐Step and Two‐Step Meta‐Analysis Models Using Individual Patient Data

The problem of combining information from separate trials is a key consideration when performing a meta‐analysis or planning a multicentre trial. Although there is a considerable journal literature on meta‐analysis based on individual patient data (IPD), i.e. a one‐step IPD meta‐analysis, versus analysis based on summary data, i.e. a two‐step IPD meta‐analysis, recent articles in the medical literature indicate that there is still confusion and uncertainty as to the validity of an analysis based on aggregate data. In this study, we address one of the central statistical issues by considering the estimation of a linear function of the mean, based on linear models for summary data and for IPD. The summary data from a trial is assumed to comprise the best linear unbiased estimator, or maximum likelihood estimator of the parameter, along with its covariance matrix. The setup, which allows for the presence of random effects and covariates in the model, is quite general and includes many of the commonly employed models, for example, linear models with fixed treatment effects and fixed or random trial effects. For this general model, we derive a condition under which the one‐step and two‐step IPD meta‐analysis estimators coincide, extending earlier work considerably. The implications of this result for the specific models mentioned above are illustrated in detail, both theoretically and in terms of two real data sets, and the roles of balance and heterogeneity are highlighted. Our analysis also shows that when covariates are present, which is typically the case, the two estimators coincide only under extra simplifying assumptions, which are somewhat unrealistic in practice.     

A Comparison of Procedures for Adaptive Choice of Location Tests in Flexible Two‐Stage Designs

Although linear rank statistics for the two‐sample problem are distribution free tests, their power depends on the distribution of the data. In the planning phase of an experiment, researchers are often uncertain about the shape of this distribution and so the choice of test statistic for the analysis and the determination of the required sample size are based on vague information. Adaptive designs with interim analysis can potentially overcome both problems. And in particular, adaptive tests based on a selector statistic are a solution to the first. We investigate whether adaptive tests can be usefully implemented in flexible two‐stage designs to gain power. In a simulation study, we compare several methods for choosing a test statistic for the second stage of an adaptive design based on interim data with the procedure that applies adaptive tests in both stages. We find that the latter is a sensible approach that leads to the best results in most situations considered here. The different methods are illustrated using a clinical trial example.     

Optimal conditional error functions for the control of conditional power

Ethical considerations and the competitive environment of clinical trials usually require that any given trial have sufficient power to detect a treatment advance. If at an interim analysis the available data are used to decide whether the trial is promising enough to be continued, investigators and sponsors often wish to have a high conditional power, which is the probability to reject the null hypothesis given the interim data and the alternative of interest. Under this requirement a design with interim sample size recalculation, which keeps the overall and conditional power at a prespecified value and preserves the overall type I error rate, is a reasonable alternative to a classical group sequential design, in which the conditional power is often too small. In this article two-stage designs with control of overall and conditional power are constructed that minimize the expected sample size, either for a simple point alternative or for a random mixture of alternatives given by a prior density for the efficacy parameter. The presented optimality result applies to trials with and without an interim hypothesis test; in addition, one can account for constraints such as a minimal sample size for the second stage. The optimal designs will be illustrated with an example, and will be compared to the frequently considered method of using the conditional type I error level of a group sequential design.     

Optimal adaptive two‐stage designs for phase II cancer clinical trials

In oncology, single‐arm two‐stage designs with binary endpoint are widely applied in phase II for the development of cytotoxic cancer therapies. Simon's optimal design with prefixed sample sizes in both stages minimizes the expected sample size under the null hypothesis and is one of the most popular designs. The search algorithms that are currently used to identify phase II designs showing prespecified characteristics are computationally intensive. For this reason, most authors impose restrictions on their search procedure. However, it remains unclear to what extent this approach influences the optimality of the resulting designs. This article describes an extension to fixed sample size phase II designs by allowing the sample size of stage two to depend on the number of responses observed in the first stage. Furthermore, we present a more efficient numerical algorithm that allows for an exhaustive search of designs. Comparisons between designs presented in the literature and the proposed optimal adaptive designs show that while the improvements are generally moderate, notable reductions in the average sample size can be achieved for specific parameter constellations when applying the new method and search strategy.     

Prediction errors for state occupation and transition probabilities in multi‐state models

In this paper, we consider the estimation of prediction errors for state occupation probabilities and transition probabilities for multistate time‐to‐event data. We study prediction errors based on the Brier score and on the Kullback–Leibler score and prove their properness. In the presence of right‐censored data, two classes of estimators, based on inverse probability weighting and pseudo‐values, respectively, are proposed, and consistency properties of the proposed estimators are investigated. The second part of the paper is devoted to the estimation of dynamic prediction errors for state occupation probabilities for multistate models, conditional on being alive, and for transition probabilities. Cross‐validated versions are proposed. Our methods are illustrated on the CSL1 randomized clinical trial comparing prednisone versus placebo for liver cirrhosis patients.     

Continuous Covariates in Mark‐Recapture‐Recovery Analysis: A Comparison of Methods

Summary Time varying, individual covariates are problematic in experiments with marked animals because the covariate can typically only be observed when each animal is captured. We examine three methods to incorporate time varying, individual covariates of the survival probabilities into the analysis of data from mark‐recapture‐recovery experiments: deterministic imputation, a Bayesian imputation approach based on modeling the joint distribution of the covariate and the capture history, and a conditional approach considering only the events for which the associated covariate data are completely observed (the trinomial model). After describing the three methods, we compare results from their application to the analysis of the effect of body mass on the survival of Soay sheep (Ovis aries) on the Isle of Hirta, Scotland. Simulations based on these results are then used to make further comparisons. We conclude that both the trinomial model and Bayesian imputation method perform best in different situations. If the capture and recovery probabilities are all high, then the trinomial model produces precise, unbiased estimators that do not depend on any assumptions regarding the distribution of the covariate. In contrast, the Bayesian imputation method performs substantially better when capture and recovery probabilities are low, provided that the specified model of the covariate is a good approximation to the true data‐generating mechanism.     

Limited long‐term effectiveness of roller‐chopping for managing woody encroachment

The encroachment of woody plants into grasslands, woodland, and savanna has increased markedly over the past century, prompting the use of different physical methods to remove woody plants and restore grasses. Roller‐chopping is used extensively in the Americas, but little is known about its long‐term effectiveness for restoration, and whether its effectiveness varies with the intensity of encroachment. We compared the effects of roller‐chopping, under three treatment intensities (control, single treatment, double treatment), on woody plant density, ground cover, and groundstorey plants at sites of low, moderate, and high woody plant density in a semi‐arid eastern Australian woodland over 10 years. Both single and double treatment significantly altered the size distribution of Dodonaea viscosa, which comprised more than 85% of woody plants at all sites. Thus, roller‐chopping changed the size distribution of the community from an even‐size distribution to one dominated by shorter plants, irrespective of initial encroachment level. The effectiveness of roller‐chopping was strongly site‐specific, with significant reductions in density at low‐ and high‐density sites, but no clear trend in relation to the intensity of treatment (i.e. single cf. double treatment). The effectiveness of roller‐chopping was unsustained over the long term, with the suppressive effect on woody density diminishing over time. Grass cover increased with increasing intensity of woody removal, but only at the low‐density site and with some ill‐defined, variable, and short‐term effects on plant composition. Managers should consider that the short‐term effects may not adequately reflect the long‐term results of woody plant removal using the roller‐chopper.     

Dynamic inference in general nested case‐control designs

Nested case‐control designs are attractive in studies with a time‐to‐event endpoint if the outcome is rare or if interest lies in evaluating expensive covariates. The appeal is that these designs restrict to small subsets of all patients at risk just prior to the observed event times. Only these small subsets need to be evaluated. Typically, the controls are selected at random and methods for time‐simultaneous inference have been proposed in the literature. However, the martingale structure behind nested case‐control designs allows for more powerful and flexible non‐standard sampling designs. We exploit that structure to find simultaneous confidence bands based on wild bootstrap resampling procedures within this general class of designs. We show in a simulation study that the intended coverage probability is obtained for confidence bands for cumulative baseline hazard functions. We apply our methods to observational data about hospital‐acquired infections.