Inference in response-adaptive clinical trials when the enrolled population varies over time

A common assumption of data analysis in clinical trials is that the patient population, as well as treatment effects, do not vary during the course of the study. However, when trials enroll patients over several years, this hypothesis may be violated. Ignoring variations of the outcome distributions over time, under the control and experimental treatments, can lead to biased treatment effect estimates and poor control of false positive results. We propose and compare two procedures that account for possible variations of the outcome distributions over time, to correct treatment effect estimates, and to control type-I error rates. The first procedure models trends of patient outcomes with splines. The second leverages conditional inference principles, which have been introduced to analyze randomized trials when patient prognostic profiles are unbalanced across arms. These two procedures are applicable in response-adaptive clinical trials. We illustrate the consequences of trends in the outcome distributions in response-adaptive designs and in platform trials, and investigate the proposed methods in the analysis of a glioblastoma study.     

Nonparametric Methods in Crossover Trials

The crossover design is often used in biomedical trials since it eliminates between subject variability. This paper is concerned with the statistical analysis of data arising from such trials when assumptions like normality do not necessarily apply. Nonparametric analysis of the two-period, two-treatment design was first described by Koch in a paper 1972. The purpose of this paper is to study nonparametric methods in crossover designs with three or more treatments and an equal number of periods. The proposed test for direct treatment effects is based on within subject comparisons after removing a possible period effect. With only two treatments this test reduces to the twosided Wilcoxon signed rank test. By simulation experiments the validity of the significance level of the test when using the asymptotic distribution of the test statistic are manifested and the power against different alternatives illustrated. A test for first order carryover effects can be constructed by a straightforward generalization of the test proposed by Koch in 1972. However, since this test is based on between subject comparisons its power will be low. Our recommendation is to consider the crossover design rather than the parallel group design if the carryover effects are assumed to be neglible or positive and smaller then the direct treatment effects.     

A comparison of methods for constructing confidence intervals after phase II/III clinical trials

Recently, in order to accelerate drug development, trials that use adaptive seamless designs such as phase II/III clinical trials have been proposed. Phase II/III clinical trials combine traditional phases II and III into a single trial that is conducted in two stages. Using stage 1 data, an interim analysis is performed to answer phase II objectives and after collection of stage 2 data, a final confirmatory analysis is performed to answer phase III objectives. In this paper we consider phase II/III clinical trials in which, at stage 1, several experimental treatments are compared to a control and the apparently most effective experimental treatment is selected to continue to stage 2. Although these trials are attractive because the confirmatory analysis includes phase II data from stage 1, the inference methods used for trials that compare a single experimental treatment to a control and do not have an interim analysis are no longer appropriate. Several methods for analysing phase II/III clinical trials have been developed. These methods are recent and so there is little literature on extensive comparisons of their characteristics. In this paper we review and compare the various methods available for constructing confidence intervals after phase II/III clinical trials.     

The Use of Two‐Way Linear Mixed Models in Multitreatment Meta‐Analysis

Summary Meta‐analysis summarizes the results of a series of trials. When more than two treatments are included in the trials and when the set of treatments tested differs between trials, the combination of results across trials requires some care. Several methods have been proposed for this purpose, which feature under different labels, such as network meta‐analysis or mixed treatment comparisons. Two types of linear mixed model can be used for meta‐analysis. The one expresses the expected outcome of treatments as a contrast to a baseline treatment. The other uses a classical two‐way linear predictor with main effects for treatment and trial. In this article, we compare both types of model and explore under which conditions they give equivalent results. We illustrate practical advantages of the two‐way model using two published datasets. In particular, it is shown that between‐trial heterogeneity as well as inconsistency between different types of trial is straightforward to account for.     

Drop-the-losers design: normal case

Drop-the-losers designs are statistical designs which have two stages of a trial separated by a data based decision. In the first stage k experimental treatments and a control are administered. During a transition period, the empirically best experimental treatment is selected for continuation into the second phase, along with the control. At the study's end, inference focuses on the comparison of the selected treatment with the control using both stages' data. Traditional methods used to make inferences based on both stages' data can yield tests with higher than advertised levels of significance and confidence intervals with lower than advertised confidence. For normally distributed data, methods are provided to correct these deficiencies, providing confidence intervals with accurate levels of confidence. Drop-the-losers designs are particularly applicable to biopharmaceutical clinical trials where they can allow Phase II and Phase III clinical trials to be conducted under a single protocol with the use of all available data.     

Accurate mean comparisons for paired samples with missing data: An application to a smoking‐cessation trial

In this paper, we consider mean comparisons for paired samples in which a certain portion of the observations are missing. This type of data commonly arises in medical researches where the outcomes are assessed at two time points after the application of treatments. New methods for statistical inference are proposed by making finiteness correction based on asymptotic expansions of some intuitive statistics. The comparison methods naturally extend to the two‐group case after some suitable manipulations. Simulation study is carried out to demonstrate the numerical accuracy of the proposed methods. Data from a smoking‐cessation trial are used to illustrate the application of the methods.     

One-Sided Tests in Clinical Trials with Multiple Endpoints

Treatment comparisons in clinical trials often involve several endpoints. For example, one might wish to demonstrate that a new treatment is superior to the current standard for some components of the multivariate response vector and is not inferior, modulo biologically unimportant difference to the standard treatment for all other components. We introduce a new approach to multiple-endpoint testing that incorporates the essential univariate and multivariate features of the treatment effects. This approach is compared with existing methods in a simulation study and applied to data on rheumatoid arthritis patients receiving one of two treatments.     

Improving efficiency of inferences in randomized clinical trials using auxiliary covariates

Summary .   The primary goal of a randomized clinical trial is to make comparisons among two or more treatments. For example, in a two-arm trial with continuous response, the focus may be on the difference in treatment means; with more than two treatments, the comparison may be based on pairwise differences. With binary outcomes, pairwise odds ratios or log odds ratios may be used. In general, comparisons may be based on meaningful parameters in a relevant statistical model. Standard analyses for estimation and testing in this context typically are based on the data collected on response and treatment assignment only. In many trials, auxiliary baseline covariate information may also be available, and it is of interest to exploit these data to improve the efficiency of inferences. Taking a semiparametric theory perspective, we propose a broadly applicable approach to adjustment for auxiliary covariates to achieve more efficient estimators and tests for treatment parameters in the analysis of randomized clinical trials. Simulations and applications demonstrate the performance of the methods.     

The quality of clinical trials with Harpagophytum procumbens

OBJECTIVE: To examine systematically the quality of the clinical trials investigating the effectiveness of Harpagophytum products. METHODS: Literature searches and enquiries to experts identified 20 studies of treatment with various Harpagophytum products (powder, aqueous and ethanolic extracts) for exacerbations of chronic musculoskeletal pain. Eight were open uncontrolled observational studies, one comparing progress under treatment for pain in back, knee and hip pain. Two were open comparisons with conventional treatment, only one of which was randomised. Ten were double-blinded, randomised controlled comparisons, 8 with placebo and 2 with NSAID comparator treatments. Indices of the internal and external validities were examined by reference to a checklist to see how well the studies answered the questions: do Harpagophytum products work and do they work as well as more conventional comparator treatments? RESULTS: The uncontrolled trials, though providing useful preliminary estimates of the possible effect of treating various conditions, could not separate the effects of the Harpagophytum product from whatever placebo effect might have been exerted in the circumstances of the study. The 2 open comparisons were open to performance, detection and/or selection bias. Of the 8 randomised double blinded controlled comparisons with placebo, 6 were marred by lack of transparency, one could not provide definitive evidence from its pre-selected principal outcome measure, and one provided good quality evidence of a dose dependent superiority of effect over placebo, though this was with a product that is not generally available for clinical practice. One of the randomised controlled comparisons with comparator (Doloteffin versus rofecoxib) was intended only as a pilot and studied too few patients for definitive conclusions whereas the other did provide good evidence that the powder, Harpadol is not importantly less effective than the weak NSAID diacerhein. CONCLUSIONS: Evidence of effectiveness of Harpagophytum products is not transferrable from product to product. The results of some studies suggest some effectiveness for some products, but for none of the clinically available products is the quality of evidence totally satisfactory. It is better so far with products that contain at least 50 mg of harpagoside in the daily dosage than with products (which happen to be of ethanolic extraction) that contain less.     

Multiple Comparisons of Treatments with Stable Multivariate Tests in a Two‐Stage Adaptive Design,Including a Test for Non‐Inferiority

The application of stabilized multivariate tests is demonstrated in the analysis of a two‐stage adaptive clinical trial with three treatment arms. Due to the clinical problem, the multiple comparisons include tests of superiority as well as a test for non‐inferiority, where non‐inferiority is (because of missing absolute tolerance limits) expressed as linear contrast of the three treatments. Special emphasis is paid to the combination of the three sources of multiplicity – multiple endpoints, multiple treatments, and two stages of the adaptive design. Particularly, the adaptation after the first stage comprises a change of the a‐priori order of hypotheses.     

The compliance score as a regressor in randomized trials

The compliance score in randomized trials is a measure of the effect of randomization on treatment received. It is in principle a group-level pretreatment variable and so can be used where individual-level measures of treatment received can produce misleading inferences. The interpretation of models with the compliance score as a regressor of interest depends on the link function. Using the identity link can lead to valid inference about the effects of treatment received even in the presence of nonrandom noncompliance; such inference is more problematic for nonlinear links. We illustrate these points with data from two randomized trials.     

Bayesian Estimation of Small Effects in Exercise and Sports Science

The aim of this paper is to provide a Bayesian formulation of the so-called magnitude-based inference approach to quantifying and interpreting effects, and in a case study example provide accurate probabilistic statements that correspond to the intended magnitude-based inferences. The model is described in the context of a published small-scale athlete study which employed a magnitude-based inference approach to compare the effect of two altitude training regimens (live high-train low (LHTL), and intermittent hypoxic exposure (IHE)) on running performance and blood measurements of elite triathletes. The posterior distributions, and corresponding point and interval estimates, for the parameters and associated effects and comparisons of interest, were estimated using Markov chain Monte Carlo simulations. The Bayesian analysis was shown to provide more direct probabilistic comparisons of treatments and able to identify small effects of interest. The approach avoided asymptotic assumptions and overcame issues such as multiple testing. Bayesian analysis of unscaled effects showed a probability of 0.96 that LHTL yields a substantially greater increase in hemoglobin mass than IHE, a 0.93 probability of a substantially greater improvement in running economy and a greater than 0.96 probability that both IHE and LHTL yield a substantially greater improvement in maximum blood lactate concentration compared to a Placebo. The conclusions are consistent with those obtained using a ‘magnitude-based inference’ approach that has been promoted in the field. The paper demonstrates that a fully Bayesian analysis is a simple and effective way of analysing small effects, providing a rich set of results that are straightforward to interpret in terms of probabilistic statements.     

Principal stratification with predictors of compliance for randomized trials with 2 active treatments

In behavioral medicine trials, such as smoking cessation trials, 2 or more active treatments are often compared. Noncompliance by some subjects with their assigned treatment poses a challenge to the data analyst. The principal stratification framework permits inference about causal effects among subpopulations characterized by potential compliance. However, in the absence of prior information, there are 2 significant limitations: (1) the causal effects cannot be point identified for some strata and (2) individuals in the subpopulations (strata) cannot be identified. We propose to use additional information-compliance-predictive covariates-to help identify the causal effects and to help describe characteristics of the subpopulations. The probability of membership in each principal stratum is modeled as a function of these covariates. The model is constructed using marginal compliance models (which are identified) and a sensitivity parameter that captures the association between the 2 marginal distributions. We illustrate our methods in both a simulation study and an analysis of data from a smoking cessation trial.     

Characterisation of Certain Split-Block Designs with a Control

The paper deals with an incomplete split-block design in which one or two factors are split into two parts, the first one containing test treatments and the second one — a control treatment. The aim of the experiment carried out in such design is to compare the average of the test treatment effects with the control treatment effect and the test treatment effects with the control treatment effect individually. All those comparisons can be expressed by contrasts, elementary or basic ones. The aim of the paper is to characterise particular cases of the incomplete split-block designs with respect to general balance and to efficiency factors of the design with respect to the contrasts. In the paper we restrict our attention to the designs in which each block has two rows or/and two columns only. These designs are suitable for certain agricultural experiments.     

Reporting of noninferiority and equivalence randomized trials for major prostaglandins: A systematic survey of the ophthalmology literature

Aims

Implantable defibrillators are considered life-saving therapy in heart failure (CHF) patients. Surprisingly, the recent Sudden Cardiac Death in Heart Failure Trial (SCD-HeFT) reached an opposing conclusion from that of numerous other trials about their survival benefit in patients with advanced CHF. A critical analysis of common control trial design may explain this paradoxical finding, with important implications for future studies.

Methods and Results

Common control trials compare several intervention groups to a single rather than separate control groups. Though potentially requiring fewer patients than trials using separate controls, variation in the common control group will influence all comparisons and creates correlations between findings. During subgroup analyses, this dependency of outcomes may increase belief in the presence of a real subgroup effect when, in fact, it should increase skepticism. For example, a high (r = 0.92), statistically unlikely (p = 0.052) correlation between comparisons was observed across the subgroups reported in SCD-HeFT. Such concordance between amiodarone and a defibrillator across subgroups was unexpected, given how much the effects of these treatments significantly differed from one another in the main study. This suggests the study's subgroup findings (specifically the absence of benefit from defibrillators in advanced CHF) were not necessarily a consequence of treatment; more likely, they resulted from variation in what the treatments were compared against, the common control.

Conclusion

Common control trials can be more efficient than other designs, but induce dependence between treatment comparisons and require cautious interpretation.     

Robustness of Multiple Comparison Procedures: Treatment versus Control

Computer simulation techniques were used to investigate the Type I and Type II error rates of one parametric (Dunnett) and two nonparametric multiple comparison procedures for comparing treatments with a control under nonnormality and variance homogeneity. It was found that Dunnett's procedure is quite robust with respect to violations of the normality assumption. Power comparisons show that for small sample sizes Dunnett's procedure is superior to the nonparametric procedures also in non-normal cases, but for larger sample sizes the multiple analogue to Wilcoxon and Kruskal-Wallis rank statistics are superior to Dunnett's procedure in all considered nonnormal cases. Further investigations under nonnormality and variance heterogeneity show robustness properties with respect to the risks of first kind and power comparisons yield similar results as in the equal variance case.     

The likelihood as statistical evidence in multiple comparisons in clinical trials: no free lunch

The likelihood ratio summarizes the strength of statistical evidence for one simple pre-determined hypothesis versus another. However, it does not directly address the multiple comparisons problem. In this paper we discuss some concerns related to the application of likelihood ratio methods to several multiple comparisons issues in clinical trials, in particular, subgroup analysis, multiple variables, interim monitoring, and data driven choice of hypotheses.     

Rethinking historical controls

Inference from traditional historical controls, i.e. comparing a new treatment in a current series of patients with an old treatment in a previous series of patients, may be subject to a strong selection bias. To avoid this bias, Baker and Lindeman (1994) proposed the paired availability design. By applying this methodology to estimate the effect of epidural analgesia on the probability of Cesarean section, we made two important contributions with the current study. First, we generalized the methodology to include different types of availability and multiple time periods. Second, we investigated how well the paired availability design reduced selection bias by comparing results to those from a meta-analysis of randomized trials and a multivariate analysis of concurrent controls. The confidence interval from the paired availability approach differed considerably from that of the multivariate analysis of concurrent controls but was similar to that from the meta-analysis of randomized trials. Because we believe the multivariate analysis of concurrent controls omitted an important predictor and the meta-analysis of randomized trials was the gold standard for inference, we concluded that the paired availability design did, in fact, reduce selection bias.     

Estimating the Transition/Transversion Ratio from Independent Pairwise Comparisons with an Assumed Phylogeny

A method is presented for estimating the transition/transversion ratio (TI/TV), based on phylogenetically independent comparisons. TI/TV is a parameter of some models used in phylogeny estimation intended to reflect the fact that nucleotide substitutions are not all equally likely. Previous attempts to estimate TI/TV have commonly faced three problems: (1) few taxa; (2) nonindependence among pairwise comparisons; and (3) multiple hits make the apparent TI/TV between two sequences decrease over time since their divergence, giving a misleading impression of relative substitution probabilities. We have made use of the time dependency, modeling how the observed TI/TV changes over time and extrapolating to estimate the ``instantaneous' TI/TV—the relevant parameter for phylogenetic inference. To illustrate our method, TI/TV was estimated for two mammalian mitochondrial genes. For 26 pairs of cytochrome b sequences, the estimate of TI/TV was 5.5; 16 pairs of 12s rRNA yielded an estimate of 9.5. These estimates are higher than those given by the maximum likelihood method and than those obtained by averaging all possible pairwise comparisons (with or without a two-parameter correction for multiple substitutions). We discuss strengths, weaknesses, and further uses of our method. Received: 22 August 1995 / Accepted: 26 July 1996     

Multiple imputation methods for treatment noncompliance and nonresponse in randomized clinical trials

Summary .  Randomized clinical trials are a powerful tool for investigating causal treatment effects, but in human trials there are oftentimes problems of noncompliance which standard analyses, such as the intention-to-treat or as-treated analysis, either ignore or incorporate in such a way that the resulting estimand is no longer a causal effect. One alternative to these analyses is the complier average causal effect (CACE) which estimates the average causal treatment effect among a subpopulation that would comply under any treatment assigned. We focus on the setting of a randomized clinical trial with crossover treatment noncompliance (e.g., control subjects could receive the intervention and intervention subjects could receive the control) and outcome nonresponse. In this article, we develop estimators for the CACE using multiple imputation methods, which have been successfully applied to a wide variety of missing data problems, but have not yet been applied to the potential outcomes setting of causal inference. Using simulated data we investigate the finite sample properties of these estimators as well as of competing procedures in a simple setting. Finally we illustrate our methods using a real randomized encouragement design study on the effectiveness of the influenza vaccine.