A simple meta-analytic approach for using a binary surrogate endpoint to predict the effect of intervention on true endpoint

A surrogate endpoint is an endpoint that is obtained sooner, at lower cost, or less invasively than the true endpoint for a health outcome and is used to make conclusions about the effect of intervention on the true endpoint. In this approach, each previous trial with surrogate and true endpoints contributes an estimated predicted effect of intervention on true endpoint in the trial of interest based on the surrogate endpoint in the trial of interest. These predicted quantities are combined in a simple random-effects meta-analysis to estimate the predicted effect of intervention on true endpoint in the trial of interest. Validation involves comparing the average prediction error of the aforementioned approach with (i) the average prediction error of a standard meta-analysis using only true endpoints in the other trials and (ii) the average clinically meaningful difference in true endpoints implicit in the trials. Validation is illustrated using data from multiple randomized trials of patients with advanced colorectal cancer in which the surrogate endpoint was tumor response and the true endpoint was median survival time.     

Bayesian adaptive trial design for a newly validated surrogate endpoint

The evaluation of surrogate endpoints for primary use in future clinical trials is an increasingly important research area, due to demands for more efficient trials coupled with recent regulatory acceptance of some surrogates as 'valid.' However, little consideration has been given to how a trial that utilizes a newly validated surrogate endpoint as its primary endpoint might be appropriately designed. We propose a novel Bayesian adaptive trial design that allows the new surrogate endpoint to play a dominant role in assessing the effect of an intervention, while remaining realistically cautious about its use. By incorporating multitrial historical information on the validated relationship between the surrogate and clinical endpoints, then subsequently evaluating accumulating data against this relationship as the new trial progresses, we adaptively guard against an erroneous assessment of treatment based upon a truly invalid surrogate. When the joint outcomes in the new trial seem plausible given similar historical trials, we proceed with the surrogate endpoint as the primary endpoint, and do so adaptively-perhaps stopping the trial for early success or inferiority of the experimental treatment, or for futility. Otherwise, we discard the surrogate and switch adaptive determinations to the original primary endpoint. We use simulation to test the operating characteristics of this new design compared to a standard O'Brien-Fleming approach, as well as the ability of our design to discriminate trustworthy from untrustworthy surrogates in hypothetical future trials. Furthermore, we investigate possible benefits using patient-level data from 18 adjuvant therapy trials in colon cancer, where disease-free survival is considered a newly validated surrogate endpoint for overall survival.     

Predicting treatment effect from surrogate endpoints and historical trials: an extrapolation involving probabilities of a binary outcome or survival to a specific time

Using multiple historical trials with surrogate and true endpoints, we consider various models to predict the effect of treatment on a true endpoint in a target trial in which only a surrogate endpoint is observed. This predicted result is computed using (1) a prediction model (mixture, linear, or principal stratification) estimated from historical trials and the surrogate endpoint of the target trial and (2) a random extrapolation error estimated from successively leaving out each trial among the historical trials. The method applies to either binary outcomes or survival to a particular time that is computed from censored survival data. We compute a 95% confidence interval for the predicted result and validate its coverage using simulation. To summarize the additional uncertainty from using a predicted instead of true result for the estimated treatment effect, we compute its multiplier of standard error. Software is available for download.     

The validation of surrogate endpoints in meta-analyses of randomized experiments

The validation of surrogate endpoints has been studied by Prentice (1989). He presented a definition as well as a set of criteria, which are equivalent only if the surrogate and true endpoints are binary. Freedman et al. (1992) supplemented these criteria with the so-called 'proportion explained'. Buyse and Molenberghs (1998) proposed replacing the proportion explained by two quantities: (1) the relative effect linking the effect of treatment on both endpoints and (2) an individual-level measure of agreement between both endpoints. The latter quantity carries over when data are available on several randomized trials, while the former can be extended to be a trial-level measure of agreement between the effects of treatment of both endpoints. This approach suggests a new method for the validation of surrogate endpoints, and naturally leads to the prediction of the effect of treatment upon the true endpoint, given its observed effect upon the surrogate endpoint. These ideas are illustrated using data from two sets of multicenter trials: one comparing chemotherapy regimens for patients with advanced ovarian cancer, the other comparing interferon-alpha with placebo for patients with age-related macular degeneration.     

Clinical and biomarker endpoint analysis in neoadjuvant endocrine therapy trials

Neoadjuvant endocrine therapy trials for breast cancer are now a widely accepted investigational approach for oncology cooperative group and pharmaceutical company research programs. However, there remains considerable uncertainty regarding the most suitable endpoints for these studies, in part, because short-term clinical, radiological or biomarker responses have not been fully validated as surrogate endpoints that closely relate to long-term breast cancer outcome. This shortcoming must be addressed before neoadjuvant endocrine treatment can be used as a triage strategy designed to identify patients with endocrine therapy “curable” disease. In this summary, information from published studies is used as a basis to critique clinical trial designs and to suggest experimental endpoints for future validation studies. Three aspects of neoadjuvant endocrine therapy designs are considered: the determination of response; the assessment of surgical outcomes; and biomarker endpoint analysis. Data from the letrozole 024 (LET 024) trial that compared letrozole and tamoxifen is used to illustrate a combined endpoint analysis that integrates both clinical and biomarker information. In addition, the concept of a “cell cycle response” is explored as a simple post-treatment endpoint based on Ki67 analysis that might have properties similar to the pathological complete response endpoint used in neoadjuvant chemotherapy trials.     

Carotid intimal-media thickness as a surrogate for cardiovascular disease events in trials of HMG-CoA reductase inhibitors

BACKGROUND: Surrogate measures for cardiovascular disease events have the potential to increase greatly the efficiency of clinical trials. A leading candidate for such a surrogate is the progression of intima-media thickness (IMT) of the carotid artery; much experience has been gained with this endpoint in trials of HMG-CoA reductase inhibitors (statins). METHODS AND RESULTS: We examine two separate systems of criteria that have been proposed to define surrogate endpoints, based on clinical and statistical arguments. We use published results and a formal meta-analysis to evaluate whether progression of carotid IMT meets these criteria for HMG-CoA reductase inhibitors (statins).IMT meets clinical-based criteria to serve as a surrogate endpoint for cardiovascular events in statin trials, based on relative efficiency, linkage to endpoints, and congruency of effects. Results from a meta-analysis and post-trial follow-up from a single published study suggest that IMT meets established statistical criteria by accounting for intervention effects in regression models. CONCLUSION: Carotid IMT progression meets accepted definitions of a surrogate for cardiovascular disease endpoints in statin trials. This does not, however, establish that it may serve universally as a surrogate marker in trials of other agents.     

Sample Size Reassessment and Hypothesis Testing in Adaptive Survival Trials

Mid-study design modifications are becoming increasingly accepted in confirmatory clinical trials, so long as appropriate methods are applied such that error rates are controlled. It is therefore unfortunate that the important case of time-to-event endpoints is not easily handled by the standard theory. We analyze current methods that allow design modifications to be based on the full interim data, i.e., not only the observed event times but also secondary endpoint and safety data from patients who are yet to have an event. We show that the final test statistic may ignore a substantial subset of the observed event times. An alternative test incorporating all event times is found, where a conservative assumption must be made in order to guarantee type I error control. We examine the power of this approach using the example of a clinical trial comparing two cancer therapies.     

Challenges in the development of glaucoma neuroprotection therapy

Glaucoma, a disease of the optic nerve and retina, causes blindness in millions of people worldwide. Currently available therapies for this disease only attempt to reduce intraocular pressure, the major risk factor, without addressing the associated optic neuropathy and retinopathy. Development of glaucoma neuroprotective treatment is therefore a pressing unmet medical need. Unfortunately, many challenges hinder this effort, including an incomplete understanding of the mechanism of pathogenesis, leading to uncertain therapeutic targets and confounded by not yet validated preclinical models. Most importantly, with slow disease progression and a less than ideal endpoint measurement method, clinical trials are necessarily large, lengthy, expensive and, to many, prohibitive. No easy solution is available to overcome these challenges. Increased commitment to basic mechanistic research is an essential foundation for dealing with this problem. Innovations in clinical trials with novel surrogate endpoints, nontraditional study designs and the use of surrogate diseases might shorten the study time, reduce the patient sample size and consequently lower the budgetary hurdle for the development of new therapies.     

On assessing surrogacy in a single trial setting using a semicompeting risks paradigm

Summary .  There has been a recent emphasis on the identification of biomarkers and other biologic measures that may be potentially used as surrogate endpoints in clinical trials. We focus on the setting of data from a single clinical trial. In this article, we consider a framework in which the surrogate must occur before the true endpoint. This suggests viewing the surrogate and true endpoints as semicompeting risks data; this approach is new to the literature on surrogate endpoints and leads to an asymmetrical treatment of the surrogate and true endpoints. However, such a data structure also conceptually complicates many of the previously considered measures of surrogacy in the literature. We propose novel estimation and inferential procedures for the relative effect and adjusted association quantities proposed by Buyse and Molenberghs (1998, Biometrics 54, 1014–1029). The proposed methodology is illustrated with application to simulated data, as well as to data from a leukemia study.     

General independent censoring in event-driven trials with staggered entry

Randomized clinical trials with time-to-event endpoints are frequently stopped after a prespecified number of events has been observed. This practice leads to dependent data and nonrandom censoring, which can in general not be solved by conditioning on the underlying baseline information. In case of staggered study entry, matters are complicated substantially. The present paper demonstrates that the study design at hand entails general independent censoring in the counting process sense, provided that the analysis is based on study time information only. To illustrate that the filtrations must not use abundant information, we simulated data of event-driven trials and evaluated them by means of Cox regression models with covariates for the calendar times. The Breslow curves of the cumulative baseline hazard showed considerable deviations, which implies that the analysis is disturbed by conditioning on the calendar time variables. A second simulation study further revealed that Efron's classical bootstrap, unlike the (martingale-based) wild bootstrap, may lead to biased results in the given setting, as the assumption of random censoring is violated. This is exemplified by an analysis of data on immunotherapy in patients with advanced, previously treated nonsmall cell lung cancer.     

Interim analysis incorporating short‐ and long‐term binary endpoints

Designs incorporating more than one endpoint have become popular in drug development. One of such designs allows for incorporation of short‐term information in an interim analysis if the long‐term primary endpoint has not been yet observed for some of the patients. At first we consider a two‐stage design with binary endpoints allowing for futility stopping only based on conditional power under both fixed and observed effects. Design characteristics of three estimators: using primary long‐term endpoint only, short‐term endpoint only, and combining data from both are compared. For each approach, equivalent cut‐off point values for fixed and observed effect conditional power calculations can be derived resulting in the same overall power. While in trials stopping for futility the type I error rate cannot get inflated (it usually decreases), there is loss of power. In this study, we consider different scenarios, including different thresholds for conditional power, different amount of information available at the interim, different correlations and probabilities of success. We further extend the methods to adaptive designs with unblinded sample size reassessments based on conditional power with inverse normal method as the combination function. Two different futility stopping rules are considered: one based on the conditional power, and one from P‐values based on Z‐statistics of the estimators. Average sample size, probability to stop for futility and overall power of the trial are compared and the influence of the choice of weights is investigated.     

Which and How Many Patients Should Be Included in Randomised Controlled Trials to Demonstrate the Efficacy of Biologics in Primary Sj?gren’s Syndrome?

Objective

The goal of this study was to determine how the choice of the primary endpoint influenced sample size estimates in randomised controlled trials (RCTs) of treatments for primary Sjögren’s syndrome (pSS).

Methods

We reviewed all studies evaluating biotechnological therapies in pSS to identify their inclusion criteria and primary endpoints. Then, in a large cohort (ASSESS), we determined the proportion of patients who would be included in RCTs using various inclusion criteria sets. Finally, we used the population of a large randomised therapeutic trial in pSS (TEARS) to assess the impact of various primary objectives and endpoints on estimated sample sizes. These analyses were performed only for the endpoints indicating greater efficacy of rituximab compared to the placebo.

Results

We identified 18 studies. The most common inclusion criteria were short disease duration; systemic involvement; high mean visual analogue scale (VAS) scores for dryness, pain, and fatigue; and biological evidence of activity. In the ASSESS cohort, 35 percent of patients had recent-onset disease (lower than 4 years), 68 percent systemic manifestations, 68 percent high scores on two of three VASs, and 52 percent biological evidence of activity. The primary endpoints associated with the smallest sample sizes (nlower than 200) were a VAS dryness score improvement higher to 20 mm by week 24 or variable improvements (10, 20, or 30 mm) in fatigue VAS by week 6 or 16. For patients with systemic manifestations, the ESSDAI change may be the most logical endpoint, as it reflects all domains of disease activity. However, the ESSDAI did not improve significantly with rituximab therapy in the TEARS study. Ultrasound score improvement produced the smallest sample size estimate in the TEARS study.

Conclusion

This study provides valuable information for designing future RCTs on the basis of previously published studies. Previous RCTs used inclusion criteria that selected a small part of the entire pSS population. The endpoint was usually based on VASs assessing patient complaints. In contrast to VAS dryness cut-offs, VAS fatigue cut-offs did not affect estimated sample sizes. SGUS improvement produced the smallest estimated sample size. Further studies are required to validate standardised SGUS modalities and assessment criteria. Thus, researchers should strive to develop a composite primary endpoint and to determine its best cut-off and assessment time point.     

Proportion of treatment effect mediated by surrogate endpoints

One of the central aims in randomized clinical trials is to find well‐validated surrogate endpoints to reduce the sample size and/or duration of trials. Clinical researchers and practitioners have proposed various surrogacy measures for assessing candidate surrogate endpoints. However, most existing surrogacy measures have the following shortcomings: (i) they often fall outside the range [0,1], (ii) they are imprecisely estimated, and (iii) they ignore the interaction associations between a treatment and candidate surrogate endpoints in the evaluation of the surrogacy level. To overcome these difficulties, we propose a new surrogacy measure, the proportion of treatment effect mediated by candidate surrogate endpoints (PMS), based on the decomposition of the treatment effect into direct, indirect, and interaction associations mediated by candidate surrogate endpoints. In addition, we validate the advantages of PMS through Monte Carlo simulations and the application to empirical data from ORIENT (the Olmesartan Reducing Incidence of Endstage Renal Disease in Diabetic Nephropathy Trial).     

Using a dose-finding benchmark to quantify the loss incurred by dichotomization in Phase II dose-ranging studies

While there is recognition that more informative clinical endpoints can support better decision-making in clinical trials, it remains a common practice to categorize endpoints originally measured on a continuous scale. The primary motivation for this categorization (and most commonly dichotomization) is the simplicity of the analysis. There is, however, a long argument that this simplicity can come at a high cost. Specifically, larger sample sizes are needed to achieve the same level of accuracy when using a dichotomized outcome instead of the original continuous endpoint. The degree of “loss of information” has been studied in the contexts of parallel-group designs and two-stage Phase II trials. Limited attention, however, has been given to the quantification of the associated losses in dose-ranging trials. In this work, we propose an approach to estimate the associated losses in Phase II dose-ranging trials that is free of the actual dose-ranging design used and depends on the clinical setting only. The approach uses the notion of a nonparametric optimal benchmark for dose-finding trials, an evaluation tool that facilitates the assessment of a dose-finding design by providing an upper bound on its performance under a given scenario in terms of the probability of the target dose selection. After demonstrating how the benchmark can be applied to Phase II dose-ranging trials, we use it to quantify the dichotomization losses. Using parameters from real clinical trials in various therapeutic areas, it is found that the ratio of sample sizes needed to obtain the same precision using continuous and binary (dichotomized) endpoints varies between 70% and 75% under the majority of scenarios but can drop to 50% in some cases.     

Informative noncompliance in endpoint trials

Noncompliance with study medications is an important issue in the design of endpoint clinical trials. Including noncompliant patient data in an intention-to-treat analysis could seriously decrease study power. Standard methods for calculating sample size account for noncompliance, but all assume that noncompliance is noninformative, i.e., that the risk of discontinuation is independent of the risk of experiencing a study endpoint. Using data from several published clinical trials (OPTIMAAL, LIFE, RENAAL, SOLVD-Prevention and SOLVD-Treatment), we demonstrate that this assumption is often untrue, and we discuss the effect of informative noncompliance on power and sample size.     

A flexible design for advanced Phase I/II clinical trials with continuous efficacy endpoints

There is growing interest in integrated Phase I/II oncology clinical trials involving molecularly targeted agents (MTA). One of the main challenges of these trials are nontrivial dose–efficacy relationships and administration of MTAs in combination with other agents. While some designs were recently proposed for such Phase I/II trials, the majority of them consider the case of binary toxicity and efficacy endpoints only. At the same time, a continuous efficacy endpoint can carry more information about the agent's mechanism of action, but corresponding designs have received very limited attention in the literature. In this work, an extension of a recently developed information‐theoretic design for the case of a continuous efficacy endpoint is proposed. The design transforms the continuous outcome using the logistic transformation and uses an information–theoretic argument to govern selection during the trial. The performance of the design is investigated in settings of single‐agent and dual‐agent trials. It is found that the novel design leads to substantial improvements in operating characteristics compared to a model‐based alternative under scenarios with nonmonotonic dose/combination–efficacy relationships. The robustness of the design to missing/delayed efficacy responses and to the correlation in toxicity and efficacy endpoints is also investigated.     

Assessment of DNA flow cytometry as a surrogate end point biomarker in a bladder cancer chemoprevention trial

Although conventional cytology represents the most widely performed cytometric analysis of bladder cancer cells, DNA flow cytometry has, over the past decade, been increasingly used to evaluate cell proliferation and DNA ploidy in cells from bladder washings. We have investigated whether DNA flow cytometry and conventional cytology of epithelial cells obtained from bladder washings provide reliable surrogate endpoint biomarkers in clinical chemoprevention trials. We used cytometric and clinical data from a chemoprevention trial of the synthetic retinoid Fenretinide on 99 patients with superficial bladder cancer. A total of 642 bladder washing specimens obtained from the patients at 4 month intervals was analyzed. Intra-individual agreement and correlation of flow cytometric DNA ploidy (diploid vs. aneuploid), DNA Index, Hyper-Diploid-Fraction (proportion of cells with DNA content higher than 2C), and conventional cytologic examination, as assessed by kappa statistics and Spearman's correlation test, were poor from baseline through 24 months. Moreover, no correlation was found between DNA ploidy and cytology at each time point. The same results were obtained when the analyses were stratified by treatment group. In addition, the association between the results of bladder washing (by either DNA flow cytometry or cytology) and concomitant tumor recurrence was significant only for abnormal cytology, while neither biomarker was predictive of tumor recurrence at the subsequent visit. During the time of this study only four patients progressed to muscle-invasive bladder cancer, indicating the "low-risk" features of the patient population. We conclude that DNA flow cytometry and conventional cytology on epithelial cells obtained from bladder washings do not appear to provide suitable surrogate endpoint biomarkers during the early stages of bladder carcinogenesis.     

Prentice's approach and the meta-analytic paradigm: a reflection on the role of statistics in the evaluation of surrogate endpoints

We put a perspective on the strengths and limitations of statistical methods for the evaluation of surrogate endpoints. Whereas using several trials overcomes some of the limitations of a single-trial framework (Prentice, 1989, Statistics in Medicine 8, 431-440), arguably the evaluation of surrogate endpoints can never be done using only statistical evidence but such evidence should be seen as but one component in a decision-making process that involves, among others, a number of clinical and biological considerations. We briefly present a hierarchical framework that incorporates ideas from Prentice's work and is uniformly applicable to different types of surrogate and true clinical outcomes.     

Sample Size Requirements for Evaluation of Bridging Evidence

This paper addresses issues concerning methodologies on the sample size required for statistical evaluation of bridging evidence for a registration of pharmaceutical products in a new region. The bridging data can be either in the Complete Clinical Data Package (CCDP) generated during clinical drug development for submission to the original region or from a bridging study conducted in the new region after the pharmaceutical product was approved in the original region. When the data are in the CCDP, the randomized parallel dose‐response design stratified to the ethnic factors and region will generate internally valid data for evaluating similarity concurrently between the regions for assessment of the ability of extrapolation to the new region. Formula for sample size under this design is derived. The required sample size for evaluation of similarity between the regions can be at least four times as large as that needed for evaluation of treatment effects only. For a bridging study conducted in the new region in which the data of the foreign and new regions are not generated concurrently, a hierarchical model approach to incorporating the foreign bridging information into the data generated by the bridging study is suggested. The sample size required is evaluated. In general, the required sample size for the bridging trials in the new region is inversely proportional to equivalence limits, variability of primary endpoints, and the number of patients of the trials conducted in the original region.     

The analysis of multiple endpoints in clinical trials

Treatment comparisons in randomized clinical trials usually involve several endpoints such that conventional significance testing can seriously inflate the overall Type I error rate. One option is to select a single primary endpoint for formal statistical inference, but this is not always feasible. Another approach is to apply Bonferroni correction (i.e., multiply each P-value by the total number of endpoints). Its conservatism for correlated endpoints is examined for multivariate normal data. A third approach is to derive an appropriate global test statistic and this paper explores one such test applicable to any set of asymptotically normal test statistics. Quantitative, binary, and survival endpoints are all considered within this general framework. Two examples are presented and the relative merits of the proposed strategies are discussed.