Optimal adaptive randomized designs for clinical trials

Optimal decision-analytic designs are deterministic. Such designsare appropriately criticized in the context of clinical trialsbecause they are subject to assignment bias. On the other hand,balanced randomized designs may assign an excessive number ofpatients to a treatment arm that is performing relatively poorly.We propose a compromise between these two extremes, one thatachieves some of the good characteristics of both. We introducea constrained optimal adaptive design for a fully sequentialrandomized clinical trial with k arms and n patients. An r-designis one for which, at each allocation, each arm has probabilityat least r of being chosen, 0 r 1/k. An optimal design amongall r-designs is called r-optimal. An r₁-design is also an r₂-designif r₁ r₂. A design without constraint is the special case r = 0and a balanced randomized design is the special case r = 1/k.The optimization criterion is to maximize the expected overallutility in a Bayesian decision-analytic approach, where utilityis the sum over the utilities for individual patients over a‘patient horizon’ N. We prove analytically thatthere exists an r-optimal design such that each patient is assignedto a particular one of the arms with probability 1 – (k – 1)r,and to the remaining arms with probability r. We also show thatthe balanced design is asymptotically r-optimal for any givenr, 0 r < 1/k, as N/n  . This implies that everyr-optimal design is asymptotically optimal without constraint.Numerical computations using backward induction for k = 2arms show that, in general, this asymptotic optimality featurefor r-optimal designs can be accomplished with moderate trialsize n if the patient horizon N is large relative to n. We alsoshow that, in a trial with an r-optimal design, r < 1/2,fewer patients are assigned to an inferior arm than when followinga balanced design, even for r-optimal designs having the samestatistical power as a balanced design. We discuss extensionsto various clinical trial settings.     

Incorporating prior beliefs about selection bias into the analysis of randomized trials with missing outcomes

In randomized studies with missing outcomes, non-identifiable assumptions are required to hold for valid data analysis. As a result, statisticians have been advocating the use of sensitivity analysis to evaluate the effect of varying assumptions on study conclusions. While this approach may be useful in assessing the sensitivity of treatment comparisons to missing data assumptions, it may be dissatisfying to some researchers/decision makers because a single summary is not provided. In this paper, we present a fully Bayesian methodology that allows the investigator to draw a 'single' conclusion by formally incorporating prior beliefs about non-identifiable, yet interpretable, selection bias parameters. Our Bayesian model provides robustness to prior specification of the distributional form of the continuous outcomes.     

Adjustment for exploratory cut-off selection in randomized clinical trials with survival endpoint

Defining the target population based on predictive biomarkers plays an important role during clinical development. After establishing a relationship between a biomarker candidate and response to treatment in exploratory phases, a subsequent confirmatory trial ideally involves only subjects with high potential of benefiting from the new compound. In order to identify those subjects in case of a continuous biomarker, a cut-off is needed. Usually, a cut-off is chosen that resulted in a subgroup with a large observed treatment effect in an exploratory trial. However, such a data-driven selection may lead to overoptimistic expectations for the subsequent confirmatory trial. Treatment effect estimates, probability of success, and posterior probabilities are useful measures for deciding whether or not to conduct a confirmatory trial enrolling the biomarker-defined population. These measures need to be adjusted for selection bias. We extend previously introduced Approximate Bayesian Computation techniques for adjustment of subgroup selection bias to a time-to-event setting with cut-off selection. Challenges in this setting are that treatment effects become time-dependent and that subsets are defined by the biomarker distribution. Simulation studies show that the proposed method provides adjusted statistical measures which are superior to naïve Maximum Likelihood estimators as well as simple shrinkage estimators.     

Bounds on the average causal effects in randomized trials with noncompliance by covariate adjustment

In randomized trials with noncompliance, causal effects cannot be identified without strong assumptions. Therefore, several authors have considered bounds on the causal effects. Applying an idea of VanderWeele ( 2008 ), Chiba ( 2009 ) gave bounds on the average causal effects in randomized trials with noncompliance using the information on the randomized assignment, the treatment received and the outcome under monotonicity assumptions about covariates. But he did not consider any observed covariates. If there are some observed covariates such as age, gender, and race in a trial, we propose new bounds using the observed covariate information under some monotonicity assumptions similar to those of VanderWeele and Chiba. And we compare the three bounds in a real example.     

Validation of surrogate markers in multiple randomized clinical trials with repeated measurements: canonical correlation approach

Part of the recent literature on the evaluation of biomarkers as surrogate endpoints starts from a multitrial context, which leads to a definition of validity in terms of the quality of both trial-level and individual-level association between the surrogate and true endpoints (Buyse et al., 2000, Biostatistics1, 49-67). These authors concentrated on cross-sectional continuous responses. However, in many randomized clinical studies, repeated measurements are encountered on either or both endpoints. A challenge in this setting is the formulation of a simple and meaningful concept of "surrogacy."Alonso et al. (2003, Biometrical Journal45, 931-945) proposed the variance reduction factor (VRF) to evaluate surrogacy at the individual level. They also showed how and when this concept should be extended to study surrogacy at the trial level. Here, we approach the problem from the natural canonical correlation perspective. We define a class of canonical correlation functions that can be used to study surrogacy at the trial and individual level. We show that the VRF and the R2 measure defined by Buyse et al. (2000) follow as special cases. Simulations are conducted to evaluate the performance of different members of this family. The methodology is illustrated on data from a meta-analysis of five clinical trials comparing antipsychotic agents for the treatment of chronic schizophrenia.     

Simple sequential boundaries for treatment selection in multi-armed randomized clinical trials with a control

Summary .   In situations when many regimens are possible candidates for a large phase III study, but too few resources are available to evaluate each relative to the standard, conducting a multi-armed randomized selection trial is a useful strategy to remove inferior treatments from further consideration. When the study has a relatively quick endpoint such as an imaging-based lesion volume change in acute stroke patients, frequent interim monitoring of the trial is ethically and practically appealing to clinicians. In this article, I propose a class of sequential selection boundaries for multi-armed clinical trials, in which the objective is to select a treatment with a clinically significant improvement upon the control group, or to declare futility if no such treatment exists. The proposed boundaries are easy to implement in a blinded fashion, and can be applied on a flexible monitoring schedule in terms of calendar time. Design calibration with respect to prespecified levels of confidence is simple, and can be accomplished when the response rate of the control group is known only up to an interval. One of the proposed methods is applied to redesign a selection trial with an imaging endpoint in acute stroke patients, and is compared to an optimal two-stage design via simulations: The proposed method imposes smaller sample size on average than the two-stage design; this advantage is substantial when there is in fact a superior treatment to the control group.     

Case-only analysis of treatment-covariate interactions in clinical trials

Differential effectiveness of treatments across subgroups defined by pretreatment variables are of increasing interest, particularly in the expanding research field of pharmacogenomics. When the pretreatment variable is difficult to obtain or expensive to measure, but can be assessed at the end of the study using stored samples, nested case-control and case-cohort methods can be used to reduce costs in large efficacy trials with rare outcomes. Case-only methods are even more efficient, and reliable under a range of circumstances.     

Effects of covariate measurement error in the initial level and rate of change of an exposure variable

When repeated measures of an exposure variable are obtained on individuals, it can be of epidemiologic interest to relate the slope of this variable over time to a subsequent response. Subject-specific estimates of this slope are measured with error, as are corresponding estimates of the level of exposure, i.e., the intercept of a linear regression over time. Because the intercept is often correlated with the slope and may also be associated with the outcome, each error-prone covariate (intercept and slope) is a potential confounder, thereby tending to accentuate potential biases due to measurement error. Under a familiar mixed linear model for the exposure measurements, we present closed-form estimators for the true parameters of interest in the case of a continuous outcome with complete and equally timed follow-up for all subjects. Generalizations to handle incomplete follow-up, other types of outcome variables, and additional fixed covariates are illustrated via maximum likelihood. We provide examples using data from the Multicenter AIDS Cohort Study. In these examples, substantial adjustments are made to uncorrected parameter estimates corresponding to the health-related effects of exposure variable slopes over time. We illustrate the potential impact of such adjustments on the interpretation of an epidemiologic analysis.     

Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors

Background:

Clinical trials are commonly done without blinded outcome assessors despite the risk of bias. We wanted to evaluate the effect of nonblinded outcome assessment on estimated effects in randomized clinical trials with outcomes that involved subjective measurement scales.

Methods:

We conducted a systematic review of randomized clinical trials with both blinded and nonblinded assessment of the same measurement scale outcome. We searched PubMed, EMBASE, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, HighWire Press and Google Scholar for relevant studies. Two investigators agreed on the inclusion of trials and the outcome scale. For each trial, we calculated the difference in effect size (i.e., standardized mean difference between nonblinded and blinded assessments). A difference in effect size of less than 0 suggested that nonblinded assessors generated more optimistic estimates of effect. We pooled the differences in effect size using inverse variance random-effects meta-analysis and used metaregression to identify potential reasons for variation.

Results:

We included 24 trials in our review. The main meta-analysis included 16 trials (involving 2854 patients) with subjective outcomes. The estimated treatment effect was more beneficial when based on nonblinded assessors (pooled difference in effect size −0.23 [95% confidence interval (CI) −0.40 to −0.06]). In relative terms, nonblinded assessors exaggerated the pooled effect size by 68% (95% CI 14% to 230%). Heterogeneity was moderate (I² = 46%, p = 0.02) and unexplained by metaregression.

Interpretation:

We provide empirical evidence for observer bias in randomized clinical trials with subjective measurement scale outcomes. A failure to blind assessors of outcomes in such trials results in a high risk of substantial bias.A failure to blind assessors of outcomes in randomized clinical trials may result in bias. Observer bias, sometimes called “detection bias” or “ascertainment bias,” occurs when outcome assessments are systematically influenced by the assessors’ conscious or unconscious predispositions — for example, because of hope or expectations, often favouring the experimental intervention.¹Blinded outcome assessors are used in many trials to avoid such bias. However, the use of non-blinded assessors remains common,^2–4 especially in nonpharmacological trials; for example, non-blinded outcome assessment was used in 90% of trials involving orthopedic traumatology³ and 74% of trials involving strength training for muscles.⁴Unfortunately, the empirical evidence on observer bias in randomized clinical trials has been incomplete. Meta-epidemiological studies have compared double-blind trials with similar trials that were not double-blind.^5,6 However, such studies address blinding crudely because “double-blind” is an ambiguous term.^3,7 Furthermore, the risk of confounding is considerable in indirect between-trial analyses, as “double-blind” trials may have better overall methods and larger sample sizes than trials that are not reported as “double-blind.”A more reliable approach involves analyses of trials that use both blinded and nonblinded outcome assessors, because such a within-trial design provides a direct comparison between blinded and nonblinded assessments of the same outcome in the same patients. Our previous analysis of such trials with binary outcomes found substantial observer bias.⁸Although subjective measurement scales such as illness severity scores are popular, they may be susceptible to observer bias. They are frequently used as outcomes in clinical scenarios with no naturally distinct categories, and adjacent subcategories on a scale typically involve minor and vaguely defined differences.We decided to systematically review trials with both blinded and nonblinded assessment of outcomes using the same measurement scales. Our primary objective was to evaluate the impact of nonblinded outcome assessment on estimated treatment effects in randomized clinical trials. Our secondary objective was to examine reasons for variation in observer bias.     

Assessment of the efficacy of antimalarial drugs recommended by the National Malaria Control Programme in Madagascar: Up-dated baseline data from randomized and multi-site clinical trials

Background

In order to improve the monitoring of the antimalarial drug resistance in Madagascar, a new national network based on eight sentinel sites was set up. In 2006/2007, a multi-site randomized clinical trial was designed to assess the therapeutic efficacy of chloroquine (CQ), sulphadoxine-pyrimethamine (SP), amodiaquine (AQ) and artesunate plus amodiaquine combination (ASAQ), the antimalarial therapies recommended by the National Malaria Control Programme (NMCP).

Methods

Children between six months and 15 years of age, with uncomplicated falciparum malaria, were enrolled. Primary endpoints were the day-14 and day-28 risks of parasitological failure, either unadjusted or adjusted by genotyping. Risks of clinical and parasitological treatment failure after adjustment by genotyping were estimated using Kaplan-Meier survival analysis. Secondary outcomes included fever clearance, parasite clearance, change in haemoglobin levels between Day 0 and the last day of follow-up, and the incidence of adverse events.

Results

A total of 1,347 of 1,434 patients (93.9%) completed treatment and follow-up to day 28. All treatment regimens, except for the chloroquine (CQ) treatment group, resulted in clinical cure rates above 97.6% by day-14 and 96.7% by day-28 (adjusted by genotyping). Parasite and fever clearance was more rapid with artesunate plus amodiaquine, but the extent of haematological recovery on day-28 did not differ significantly between the four groups. No severe side-effects were observed during the follow-up period.

Conclusion

These findings (i) constitute an up-dated baseline data on the efficacy of antimalarial drugs recommended by the NMCP, (ii) show that antimalarial drug resistance remains low in Madagascar, except for CQ, compared to the bordering countries in the Indian Ocean region such as the Comoros Archipelago and (iii) support the current policy of ASAQ as the first-line treatment in uncomplicated falciparum malaria.

     

Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research

Background

The reporting of outcomes within published randomized trials has previously been shown to be incomplete, biased and inconsistent with study protocols. We sought to determine whether outcome reporting bias would be present in a cohort of government-funded trials subjected to rigorous peer review.

Methods

We compared protocols for randomized trials approved for funding by the Canadian Institutes of Health Research (formerly the Medical Research Council of Canada) from 1990 to 1998 with subsequent reports of the trials identified in journal publications. Characteristics of reported and unreported outcomes were recorded from the protocols and publications. Incompletely reported outcomes were defined as those with insufficient data provided in publications for inclusion in meta-analyses. An overall odds ratio measuring the association between completeness of reporting and statistical significance was calculated stratified by trial. Finally, primary outcomes specified in trial protocols were compared with those reported in publications.

Results

We identified 48 trials with 68 publications and 1402 outcomes. The median number of participants per trial was 299, and 44% of the trials were published in general medical journals. A median of 31% (10th–90th percentile range 5%–67%) of outcomes measured to assess the efficacy of an intervention (efficacy outcomes) and 59% (0%–100%) of those measured to assess the harm of an intervention (harm outcomes) per trial were incompletely reported. Statistically significant efficacy outcomes had a higher odds than nonsignificant efficacy outcomes of being fully reported (odds ratio 2.7; 95% confidence interval 1.5–5.0). Primary outcomes differed between protocols and publications for 40% of the trials.

Interpretation

Selective reporting of outcomes frequently occurs in publications of high-quality government-funded trials.Selective reporting of results from randomized trials can occur either at the level of end points within published studies (outcome reporting bias)¹ or at the level of entire trials that are selectively published (study publication bias).² Outcome reporting bias has previously been demonstrated in a broad cohort of published trials approved by a regional ethics committee.¹ The Canadian Institutes of Health Research (CIHR) — the primary federal funding agency, known before 2000 as the Medical Research Council of Canada (MRC) — recognized the need to address this issue and conducted an internal review process in 2002 to evaluate the reporting of results from its funded trials. The primary objectives were to determine (a) the prevalence of incomplete outcome reporting in journal publications of randomized trials; (b) the degree of association between adequate outcome reporting and statistical significance; and (c) the consistency between primary outcomes specified in trial protocols and those specified in subsequent journal publications.     

Efficient nonparametric estimation of causal effects in randomized trials with noncompliance

Causal approaches based on the potential outcome framework providea useful tool for addressing noncompliance problems in randomizedtrials. We propose a new estimator of causal treatment effectsin randomized clinical trials with noncompliance. We use theempirical likelihood approach to construct a profile randomsieve likelihood and take into account the mixture structurein outcome distributions, so that our estimator is robust toparametric distribution assumptions and provides substantialfinite-sample efficiency gains over the standard instrumentalvariable estimator. Our estimator is asymptotically equivalentto the standard instrumental variable estimator, and it canbe applied to outcome variables with a continuous, ordinal orbinary scale. We apply our method to data from a randomizedtrial of an intervention to improve the treatment of depressionamong depressed elderly patients in primary care practices.     

Longitudinal analysis of pre- and post-treatment measurements with equal baseline assumptions in randomized trials

For continuous variables of randomized controlled trials, recently, longitudinal analysis of pre- and posttreatment measurements as bivariate responses is one of analytical methods to compare two treatment groups. Under random allocation, means and variances of pretreatment measurements are expected to be equal between groups, but covariances and posttreatment variances are not. Under random allocation with unequal covariances and posttreatment variances, we compared asymptotic variances of the treatment effect estimators in three longitudinal models. The data-generating model has equal baseline means and variances, and unequal covariances and posttreatment variances. The model with equal baseline means and unequal variance–covariance matrices has a redundant parameter. In large sample sizes, these two models keep a nominal type I error rate and have high efficiency. The model with equal baseline means and equal variance–covariance matrices wrongly assumes equal covariances and posttreatment variances. Only under equal sample sizes, this model keeps a nominal type I error rate. This model has the same high efficiency with the data-generating model under equal sample sizes. In conclusion, longitudinal analysis with equal baseline means performed well in large sample sizes. We also compared asymptotic properties of longitudinal models with those of the analysis of covariance (ANCOVA) and t-test.     

Quantifying the impact of fixed effects modeling of clusters in multiple imputation for cluster randomized trials

In cluster randomized trials (CRTs), identifiable clusters rather than individuals are randomized to study groups. Resulting data often consist of a small number of clusters with correlated observations within a treatment group. Missing data often present a problem in the analysis of such trials, and multiple imputation (MI) has been used to create complete data sets, enabling subsequent analysis with well-established analysis methods for CRTs. We discuss strategies for accounting for clustering when multiply imputing a missing continuous outcome, focusing on estimation of the variance of group means as used in an adjusted t-test or ANOVA. These analysis procedures are congenial to (can be derived from) a mixed effects imputation model; however, this imputation procedure is not yet available in commercial statistical software. An alternative approach that is readily available and has been used in recent studies is to include fixed effects for cluster, but the impact of using this convenient method has not been studied. We show that under this imputation model the MI variance estimator is positively biased and that smaller intraclass correlations (ICCs) lead to larger overestimation of the MI variance. Analytical expressions for the bias of the variance estimator are derived in the case of data missing completely at random, and cases in which data are missing at random are illustrated through simulation. Finally, various imputation methods are applied to data from the Detroit Middle School Asthma Project, a recent school-based CRT, and differences in inference are compared.     

Quantification of length‐bias in screening trials with covariate‐dependent test sensitivity

Length‐biased sampling exists in screening programs where longer duration disease is detected during the preclinical stage because a longer sojourn time (preclinical duration) has a higher probability of being screen detected. By modeling the course of disease, we quantify the effect of length‐biased sampling on clinical duration when cases are subject to periodic screening with variable test sensitivity. We use the highly flexible bivariate lognormal density to jointly model preclinical and clinical durations, and we model screening test sensitivity as a function of the sojourn time and number of previous false negative screens. We show that the mean clinical duration among screen‐detected cases can be up to 40% higher, with shrinking standard deviation, than those among nonscreen‐detected cases, due to biased sampling alone, irrespective of any possible benefit (increased survival time arising from earlier detection or reduction in mortality). These findings will aid in the design and interpretation of screening trials.