Validation of surrogate markers in multiple randomized clinical trials with repeated measurements: canonical correlation approach

Part of the recent literature on the evaluation of biomarkers as surrogate endpoints starts from a multitrial context, which leads to a definition of validity in terms of the quality of both trial-level and individual-level association between the surrogate and true endpoints (Buyse et al., 2000, Biostatistics1, 49-67). These authors concentrated on cross-sectional continuous responses. However, in many randomized clinical studies, repeated measurements are encountered on either or both endpoints. A challenge in this setting is the formulation of a simple and meaningful concept of "surrogacy."Alonso et al. (2003, Biometrical Journal45, 931-945) proposed the variance reduction factor (VRF) to evaluate surrogacy at the individual level. They also showed how and when this concept should be extended to study surrogacy at the trial level. Here, we approach the problem from the natural canonical correlation perspective. We define a class of canonical correlation functions that can be used to study surrogacy at the trial and individual level. We show that the VRF and the R2 measure defined by Buyse et al. (2000) follow as special cases. Simulations are conducted to evaluate the performance of different members of this family. The methodology is illustrated on data from a meta-analysis of five clinical trials comparing antipsychotic agents for the treatment of chronic schizophrenia.     

Meta-analysis for surrogacy: accelerated failure time models and semicompeting risks modeling

There has been great recent interest in the medical and statistical literature in the assessment and validation of surrogate endpoints as proxies for clinical endpoints in medical studies. More recently, authors have focused on using metaanalytical methods for quantification of surrogacy. In this article, we extend existing procedures for analysis based on the accelerated failure time model to this setting. An advantage of this approach relative to proportional hazards model is that it allows for analysis in the semicompeting risks setting, where we model the region where the surrogate endpoint occurs before the true endpoint. Several estimation methods and attendant inferential procedures are presented. In addition, between- and within-trial methods for evaluating surrogacy are developed; a novel principal components procedure is developed for quantifying trial-level surrogacy. The methods are illustrated by application to data from several studies in colorectal cancer.     

Carotid intimal-media thickness as a surrogate for cardiovascular disease events in trials of HMG-CoA reductase inhibitors

BACKGROUND: Surrogate measures for cardiovascular disease events have the potential to increase greatly the efficiency of clinical trials. A leading candidate for such a surrogate is the progression of intima-media thickness (IMT) of the carotid artery; much experience has been gained with this endpoint in trials of HMG-CoA reductase inhibitors (statins). METHODS AND RESULTS: We examine two separate systems of criteria that have been proposed to define surrogate endpoints, based on clinical and statistical arguments. We use published results and a formal meta-analysis to evaluate whether progression of carotid IMT meets these criteria for HMG-CoA reductase inhibitors (statins).IMT meets clinical-based criteria to serve as a surrogate endpoint for cardiovascular events in statin trials, based on relative efficiency, linkage to endpoints, and congruency of effects. Results from a meta-analysis and post-trial follow-up from a single published study suggest that IMT meets established statistical criteria by accounting for intervention effects in regression models. CONCLUSION: Carotid IMT progression meets accepted definitions of a surrogate for cardiovascular disease endpoints in statin trials. This does not, however, establish that it may serve universally as a surrogate marker in trials of other agents.     

The evaluation of multiple surrogate endpoints

Surrogate endpoints are desirable because they typically result in smaller, faster efficacy studies compared with the ones using the clinical endpoints. Research on surrogate endpoints has received substantial attention lately, but most investigations have focused on the validity of using a single biomarker as a surrogate. Our paper studies whether the use of multiple markers can improve inferences about a treatment's effects on a clinical endpoint. We propose a joint model for a time to clinical event and for repeated measures over time on multiple biomarkers that are potential surrogates. This model extends the formulation of Xu and Zeger (2001, in press) and Fawcett and Thomas (1996, Statistics in Medicine 15, 1663-1685). We propose two complementary measures of the relative benefit of multiple surrogates as opposed to a single one. Markov chain Monte Carlo is implemented to estimate model parameters. The methodology is illustrated with an analysis of data from a schizophrenia clinical trial.     

Semiparametric inference for surrogate endpoints with bivariate censored data

Summary .   Considerable attention has been recently paid to the use of surrogate endpoints in clinical research. We deal with the situation where the two endpoints are both right censored. While proportional hazards analyses are typically used for this setting, their use leads to several complications. In this article, we propose the use of the accelerated failure time model for analysis of surrogate endpoints. Based on the model, we then describe estimation and inference procedures for several measures of surrogacy. A complication is that potentially both the independent and dependent variable are subject to censoring. We adapt the Theil–Sen estimator to this problem, develop the associated asymptotic results, and propose a novel resampling-based technique for calculating the variances of the proposed estimators. The finite-sample properties of the estimation methodology are assessed using simulation studies, and the proposed procedures are applied to data from an acute myelogenous leukemia clinical trial.     

Prentice's approach and the meta-analytic paradigm: a reflection on the role of statistics in the evaluation of surrogate endpoints

We put a perspective on the strengths and limitations of statistical methods for the evaluation of surrogate endpoints. Whereas using several trials overcomes some of the limitations of a single-trial framework (Prentice, 1989, Statistics in Medicine 8, 431-440), arguably the evaluation of surrogate endpoints can never be done using only statistical evidence but such evidence should be seen as but one component in a decision-making process that involves, among others, a number of clinical and biological considerations. We briefly present a hierarchical framework that incorporates ideas from Prentice's work and is uniformly applicable to different types of surrogate and true clinical outcomes.     

A simple meta-analytic approach for using a binary surrogate endpoint to predict the effect of intervention on true endpoint

A surrogate endpoint is an endpoint that is obtained sooner, at lower cost, or less invasively than the true endpoint for a health outcome and is used to make conclusions about the effect of intervention on the true endpoint. In this approach, each previous trial with surrogate and true endpoints contributes an estimated predicted effect of intervention on true endpoint in the trial of interest based on the surrogate endpoint in the trial of interest. These predicted quantities are combined in a simple random-effects meta-analysis to estimate the predicted effect of intervention on true endpoint in the trial of interest. Validation involves comparing the average prediction error of the aforementioned approach with (i) the average prediction error of a standard meta-analysis using only true endpoints in the other trials and (ii) the average clinically meaningful difference in true endpoints implicit in the trials. Validation is illustrated using data from multiple randomized trials of patients with advanced colorectal cancer in which the surrogate endpoint was tumor response and the true endpoint was median survival time.     

Proportion of treatment effect mediated by surrogate endpoints

One of the central aims in randomized clinical trials is to find well‐validated surrogate endpoints to reduce the sample size and/or duration of trials. Clinical researchers and practitioners have proposed various surrogacy measures for assessing candidate surrogate endpoints. However, most existing surrogacy measures have the following shortcomings: (i) they often fall outside the range [0,1], (ii) they are imprecisely estimated, and (iii) they ignore the interaction associations between a treatment and candidate surrogate endpoints in the evaluation of the surrogacy level. To overcome these difficulties, we propose a new surrogacy measure, the proportion of treatment effect mediated by candidate surrogate endpoints (PMS), based on the decomposition of the treatment effect into direct, indirect, and interaction associations mediated by candidate surrogate endpoints. In addition, we validate the advantages of PMS through Monte Carlo simulations and the application to empirical data from ORIENT (the Olmesartan Reducing Incidence of Endstage Renal Disease in Diabetic Nephropathy Trial).     

Counterfactual links to the proportion of treatment effect explained by a surrogate marker

In a randomized clinical trial, a statistic that measures the proportion of treatment effect on the primary clinical outcome that is explained by the treatment effect on a surrogate outcome is a useful concept. We investigate whether a statistic proposed to estimate this proportion can be given a causal interpretation as defined by models of counterfactual variables. For the situation of binary surrogate and outcome variables, two counterfactual models are considered, both of which include the concept of the proportion of the treatment effect, which acts through the surrogate. In general, the statistic does not equal either of the two proportions from the counterfactual models, and can be substantially different. Conditions are given for which the statistic does equal the counterfactual model proportions. A randomized clinical trial with potential surrogate endpoints is undertaken in a scientific context; this context will naturally place constraints on the parameters of the counterfactual model. We conducted a simulation experiment to investigate what impact these constraints had on the relationship between the proportion explained (PE) statistic and the counterfactual model proportions. We found that observable constraints had very little impact on the agreement between the statistic and the counterfactual model proportions, whereas unobservable constraints could lead to more agreement.     

The validation of surrogate endpoints in meta-analyses of randomized experiments

The validation of surrogate endpoints has been studied by Prentice (1989). He presented a definition as well as a set of criteria, which are equivalent only if the surrogate and true endpoints are binary. Freedman et al. (1992) supplemented these criteria with the so-called 'proportion explained'. Buyse and Molenberghs (1998) proposed replacing the proportion explained by two quantities: (1) the relative effect linking the effect of treatment on both endpoints and (2) an individual-level measure of agreement between both endpoints. The latter quantity carries over when data are available on several randomized trials, while the former can be extended to be a trial-level measure of agreement between the effects of treatment of both endpoints. This approach suggests a new method for the validation of surrogate endpoints, and naturally leads to the prediction of the effect of treatment upon the true endpoint, given its observed effect upon the surrogate endpoint. These ideas are illustrated using data from two sets of multicenter trials: one comparing chemotherapy regimens for patients with advanced ovarian cancer, the other comparing interferon-alpha with placebo for patients with age-related macular degeneration.     

A measure of the proportion of treatment effect explained by a surrogate marker

Randomized clinical trials with rare primary endpoints or long duration times are costly. Because of this, there has been increasing interest in replacing the true endpoint with an earlier measured marker. However, surrogate markers must be appropriately validated. A quantitative measure for the proportion of treatment effect explained by the marker in a specific trial is a useful concept. Freedman, Graubard, and Schatzkin (1992, Statistics in Medicine 11, 167-178) suggested such a measure of surrogacy by the ratio of regression coefficients for the treatment indicator from two separate models with or without adjusting for the surrogate marker. However, it has been shown that this measure is very variable and there is no guarantee that the two models both fit. In this article, we propose alternative measures of the proportion explained that adapts an idea in Tsiatis, DeGruttola, and Wulfsohn (1995, Journal of the American Statistical Association 90, 27-37). The new measures require fewer assumptions in estimation and allow more flexibility in modeling. The estimates of these different measures are compared using data from an ophthalmology clinical trial and a series of simulation studies. The results suggest that the new measures are less variable.     

Solutions for surrogacy validation with longitudinal outcomes for a gene therapy

Valid surrogate endpoints S can be used as a substitute for a true outcome of interest T to measure treatment efficacy in a clinical trial. We propose a causal inference approach to validate a surrogate by incorporating longitudinal measurements of the true outcomes using a mixed modeling approach, and we define models and quantities for validation that may vary across the study period using principal surrogacy criteria. We consider a surrogate-dependent treatment efficacy curve that allows us to validate the surrogate at different time points. We extend these methods to accommodate a delayed-start treatment design where all patients eventually receive the treatment. Not all parameters are identified in the general setting. We apply a Bayesian approach for estimation and inference, utilizing more informative prior distributions for selected parameters. We consider the sensitivity of these prior assumptions as well as assumptions of independence among certain counterfactual quantities conditional on pretreatment covariates to improve identifiability. We examine the frequentist properties (bias of point and variance estimates, credible interval coverage) of a Bayesian imputation method. Our work is motivated by a clinical trial of a gene therapy where the functional outcomes are measured repeatedly throughout the trial.     

Predicting treatment effect from surrogate endpoints and historical trials: an extrapolation involving probabilities of a binary outcome or survival to a specific time

Using multiple historical trials with surrogate and true endpoints, we consider various models to predict the effect of treatment on a true endpoint in a target trial in which only a surrogate endpoint is observed. This predicted result is computed using (1) a prediction model (mixture, linear, or principal stratification) estimated from historical trials and the surrogate endpoint of the target trial and (2) a random extrapolation error estimated from successively leaving out each trial among the historical trials. The method applies to either binary outcomes or survival to a particular time that is computed from censored survival data. We compute a 95% confidence interval for the predicted result and validate its coverage using simulation. To summarize the additional uncertainty from using a predicted instead of true result for the estimated treatment effect, we compute its multiplier of standard error. Software is available for download.     

Bayesian adaptive trial design for a newly validated surrogate endpoint

The evaluation of surrogate endpoints for primary use in future clinical trials is an increasingly important research area, due to demands for more efficient trials coupled with recent regulatory acceptance of some surrogates as 'valid.' However, little consideration has been given to how a trial that utilizes a newly validated surrogate endpoint as its primary endpoint might be appropriately designed. We propose a novel Bayesian adaptive trial design that allows the new surrogate endpoint to play a dominant role in assessing the effect of an intervention, while remaining realistically cautious about its use. By incorporating multitrial historical information on the validated relationship between the surrogate and clinical endpoints, then subsequently evaluating accumulating data against this relationship as the new trial progresses, we adaptively guard against an erroneous assessment of treatment based upon a truly invalid surrogate. When the joint outcomes in the new trial seem plausible given similar historical trials, we proceed with the surrogate endpoint as the primary endpoint, and do so adaptively-perhaps stopping the trial for early success or inferiority of the experimental treatment, or for futility. Otherwise, we discard the surrogate and switch adaptive determinations to the original primary endpoint. We use simulation to test the operating characteristics of this new design compared to a standard O'Brien-Fleming approach, as well as the ability of our design to discriminate trustworthy from untrustworthy surrogates in hypothetical future trials. Furthermore, we investigate possible benefits using patient-level data from 18 adjuvant therapy trials in colon cancer, where disease-free survival is considered a newly validated surrogate endpoint for overall survival.     

Endpoints in the design of clinical trials for primary sclerosing cholangitis

Primary sclerosing cholangitis is an enigmatic disease affecting the bile ducts, eventually leading to liver failure necessitating liver transplantation in many cases. There is currently no therapy that has proven to halt disease progression. One of the reasons for this is the lack of proper endpoints to measure the effect of medical intervention on the course of the disease. Relevant clinical endpoints such as death or liver transplantation occur too infrequently in this orphan disease to be used as endpoints in phase 2 or 3 trials. It is therefore of utmost importance to identify appropriate surrogate endpoints that are reasonably likely to measure true clinical benefit. This article will discuss a number of surrogate endpoints that are likely candidates to serve this role. This article is part of a Special Issue entitled: Cholangiocytes in Health and Diseaseedited by Jesus Banales, Marco Marzioni, Nicholas LaRusso and Peter Jansen.     

The Use of Surrogate Endpoints in Regulating Medicines for Cardio-Renal Disease: Opinions of Stakeholders

Aim

There is discussion whether medicines can be authorized on the market based on evidence from surrogate endpoints. We assessed opinions of different stakeholders on this topic.

Methods

We conducted an online questionnaire that targeted various stakeholder groups (regulatory agencies, pharmaceutical industry, academia, relevant public sector organisations) and medical specialties (cardiology or nephrology vs. other). Participants were enrolled through purposeful sampling. We inquired for conditions under which surrogate endpoints can be used, the validity of various cardio-renal biomarkers and new approaches for biomarker use.

Results

Participants agreed that surrogate endpoints can be used when the surrogate is scientifically valid (5-point Likert response format, mean score: 4.3, SD: 0.9) or when there is an unmet clinical need (mean score: 3.8, SD: 1.2). Industry participants agreed to a greater extent than regulators and academics. However, out of four proposed surrogates (blood pressure (BP), HbA1c, albuminuria, CRP) for cardiovascular outcomes or end-stage renal disease, only use of BP for cardiovascular outcomes was deemed moderately accurate (mean: 3.6, SD: 1.1). Specialists in cardiology or nephrology tended to be more positive about the use of surrogate endpoints.

Conclusion

Stakeholders in drug development do not oppose to the use of surrogate endpoints in drug marketing authorization, but most surrogates are not considered valid. To solve this impasse, increased efforts are required to validate surrogate endpoints and to explore alternative ways to use them.     

Properties of a nonparametric test for early comparison of treatments in clinical trials in the presence of surrogate endpoints

A nonparametric test is derived for comparing treatments with respect to the final endpoint in clinical trials in which the final endpoint has been observed for a random subset of patients, but results are available for a surrogate endpoint for a larger sample of patients. The test is an adaptation of the Wilcoxon-Mann-Whitney two-sample test, with an adjustment that involves a comparison of the ranks of the surrogate endpoints between patients with and without final endpoints. The validity of the test depends on the assumption that the patients with final endpoints represent a random sample of the patients registered in the study. This assumption is viable in trials in which the final endpoint is evaluated at a "landmark" timepoint in the patients' natural history. A small sample simulation study demonstrates that the test has a size that is close to the nominal value for all configurations evaluated. When compared with the conventional test based only on the final endpoints, the new test delivers substantial increases in power only when the surrogate endpoint is highly correlated with the true endpoint. Our research indicates that, in the absence of modeling assumptions, auxiliary information derived from surrogate endpoints can provide significant additional information only under special circumstances.     

On the Use of Published Demographic Data for Population-Level Risk Assessment in Birds

As the practice of using population models for wildlife risk assessment has become more common, so has the practice of using surrogate data, typically taken from the published scientific literature, as inputs for demographic models. This practice clearly exposes the user to inferential errors. However, it is likely to continue because demographic data are expensive to gather. We review potential errors associated with the use of previously published demographic data and how those errors propagate into the endpoints of demographic projection models. We suggest methods for inferring bias in model endpoints when multiple and opposing biases are present in the demographic input data. We provide an example using Eastern Meadowlarks (Sturnella magna), a common songbird in Midwestern grasslands and agro-ecosystems. We conclude with a brief review of methods that could improve inference made using published demographic data, including methods from life-history theory, meta-analysis, and Bayesian statistics.     

A bayesian approach to improved estimation of causal effect predictiveness for a principal surrogate endpoint

Summary The literature on potential outcomes has shown that traditional methods for characterizing surrogate endpoints in clinical trials based only on observed quantities can fail to capture causal relationships between treatments, surrogates, and outcomes. Building on the potential-outcomes formulation of a principal surrogate, we introduce a Bayesian method to estimate the causal effect predictiveness (CEP) surface and quantify a candidate surrogate's utility for reliably predicting clinical outcomes. In considering the full joint distribution of all potentially observable quantities, our Bayesian approach has the following features. First, our approach illuminates implicit assumptions embedded in previously-used estimation strategies that have been shown to result in poor performance. Second, our approach provides tools for making explicit and scientifically-interpretable assumptions regarding associations about which observed data are not informative. Through simulations based on an HIV vaccine trial, we found that the Bayesian approach can produce estimates of the CEP surface with improved performance compared to previous methods. Third, our approach can extend principal-surrogate estimation beyond the previously considered setting of a vaccine trial where the candidate surrogate is constant in one arm of the study. We illustrate this extension through an application to an AIDS therapy trial where the candidate surrogate varies in both treatment arms.     

A shrinkage approach for estimating a treatment effect using intermediate biomarker data in clinical trials

In clinical trials, a biomarker (S ) that is measured after randomization and is strongly associated with the true endpoint (T) can often provide information about T and hence the effect of a treatment (Z ) on T. A useful biomarker can be measured earlier than T and cost less than T. In this article, we consider the use of S as an auxiliary variable and examine the information recovery from using S for estimating the treatment effect on T, when S is completely observed and T is partially observed. In an ideal but often unrealistic setting, when S satisfies Prentice's definition for perfect surrogacy, there is the potential for substantial gain in precision by using data from S to estimate the treatment effect on T. When S is not close to a perfect surrogate, it can provide substantial information only under particular circumstances. We propose to use a targeted shrinkage regression approach that data-adaptively takes advantage of the potential efficiency gain yet avoids the need to make a strong surrogacy assumption. Simulations show that this approach strikes a balance between bias and efficiency gain. Compared with competing methods, it has better mean squared error properties and can achieve substantial efficiency gain, particularly in a common practical setting when S captures much but not all of the treatment effect and the sample size is relatively small. We apply the proposed method to a glaucoma data example.