Testing for heterogeneity in the utility of a surrogate marker

In studies that require long-term and/or costly follow-up of participants to evaluate a treatment, there is often interest in identifying and using a surrogate marker to evaluate the treatment effect. While several statistical methods have been proposed to evaluate potential surrogate markers, available methods generally do not account for or address the potential for a surrogate to vary in utility or strength by patient characteristics. Previous work examining surrogate markers has indicated that there may be such heterogeneity, that is, that a surrogate marker may be useful (with respect to capturing the treatment effect on the primary outcome) for some subgroups, but not for others. This heterogeneity is important to understand, particularly if the surrogate is to be used in a future trial to replace the primary outcome. In this paper, we propose an approach and estimation procedures to measure the surrogate strength as a function of a baseline covariate W and thus examine potential heterogeneity in the utility of the surrogate marker with respect to W. Within a potential outcome framework, we quantify the surrogate strength/utility using the proportion of treatment effect on the primary outcome that is explained by the treatment effect on the surrogate. We propose testing procedures to test for evidence of heterogeneity, examine finite sample performance of these methods via simulation, and illustrate the methods using AIDS clinical trial data.     

Robust approach to combining multiple markers to improve surrogacy

Identifying effective and valid surrogate markers to make inference about a treatment effect on long-term outcomes is an important step in improving the efficiency of clinical trials. Replacing a long-term outcome with short-term and/or cheaper surrogate markers can potentially shorten study duration and reduce trial costs. There is sizable statistical literature on methods to quantify the effectiveness of a single surrogate marker. Both parametric and nonparametric approaches have been well developed for different outcome types. However, when there are multiple markers available, methods for combining markers to construct a composite marker with improved surrogacy remain limited. In this paper, building on top of the optimal transformation framework of Wang et al. (2020), we propose a novel calibrated model fusion approach to optimally combine multiple markers to improve surrogacy. Specifically, we obtain two initial estimates of optimal composite scores of the markers based on two sets of models with one set approximating the underlying data distribution and the other directly approximating the optimal transformation function. We then estimate an optimal calibrated combination of the two estimated scores which ensures both validity of the final combined score and optimality with respect to the proportion of treatment effect explained by the final combined score. This approach is unique in that it identifies an optimal combination of the multiple surrogates without strictly relying on parametric assumptions while borrowing modeling strategies to avoid fully nonparametric estimation which is subject to the curse of dimensionality. Our identified optimal transformation can also be used to directly quantify the surrogacy of this identified combined score. Theoretical properties of the proposed estimators are derived, and the finite sample performance of the proposed method is evaluated through simulation studies. We further illustrate the proposed method using data from the Diabetes Prevention Program study.     

Solutions for surrogacy validation with longitudinal outcomes for a gene therapy

Valid surrogate endpoints S can be used as a substitute for a true outcome of interest T to measure treatment efficacy in a clinical trial. We propose a causal inference approach to validate a surrogate by incorporating longitudinal measurements of the true outcomes using a mixed modeling approach, and we define models and quantities for validation that may vary across the study period using principal surrogacy criteria. We consider a surrogate-dependent treatment efficacy curve that allows us to validate the surrogate at different time points. We extend these methods to accommodate a delayed-start treatment design where all patients eventually receive the treatment. Not all parameters are identified in the general setting. We apply a Bayesian approach for estimation and inference, utilizing more informative prior distributions for selected parameters. We consider the sensitivity of these prior assumptions as well as assumptions of independence among certain counterfactual quantities conditional on pretreatment covariates to improve identifiability. We examine the frequentist properties (bias of point and variance estimates, credible interval coverage) of a Bayesian imputation method. Our work is motivated by a clinical trial of a gene therapy where the functional outcomes are measured repeatedly throughout the trial.     

A measure of the proportion of treatment effect explained by a surrogate marker

Randomized clinical trials with rare primary endpoints or long duration times are costly. Because of this, there has been increasing interest in replacing the true endpoint with an earlier measured marker. However, surrogate markers must be appropriately validated. A quantitative measure for the proportion of treatment effect explained by the marker in a specific trial is a useful concept. Freedman, Graubard, and Schatzkin (1992, Statistics in Medicine 11, 167-178) suggested such a measure of surrogacy by the ratio of regression coefficients for the treatment indicator from two separate models with or without adjusting for the surrogate marker. However, it has been shown that this measure is very variable and there is no guarantee that the two models both fit. In this article, we propose alternative measures of the proportion explained that adapts an idea in Tsiatis, DeGruttola, and Wulfsohn (1995, Journal of the American Statistical Association 90, 27-37). The new measures require fewer assumptions in estimation and allow more flexibility in modeling. The estimates of these different measures are compared using data from an ophthalmology clinical trial and a series of simulation studies. The results suggest that the new measures are less variable.     

The validation of surrogate endpoints in meta-analyses of randomized experiments

The validation of surrogate endpoints has been studied by Prentice (1989). He presented a definition as well as a set of criteria, which are equivalent only if the surrogate and true endpoints are binary. Freedman et al. (1992) supplemented these criteria with the so-called 'proportion explained'. Buyse and Molenberghs (1998) proposed replacing the proportion explained by two quantities: (1) the relative effect linking the effect of treatment on both endpoints and (2) an individual-level measure of agreement between both endpoints. The latter quantity carries over when data are available on several randomized trials, while the former can be extended to be a trial-level measure of agreement between the effects of treatment of both endpoints. This approach suggests a new method for the validation of surrogate endpoints, and naturally leads to the prediction of the effect of treatment upon the true endpoint, given its observed effect upon the surrogate endpoint. These ideas are illustrated using data from two sets of multicenter trials: one comparing chemotherapy regimens for patients with advanced ovarian cancer, the other comparing interferon-alpha with placebo for patients with age-related macular degeneration.     

Estimation and inference for the causal effect of receiving treatment on a multinomial outcome

Summary .  This article considers the analysis of two-arm randomized trials with noncompliance, which have a multinomial outcome. We first define the causal effect in these trials as some function of outcome distributions of compliers with and without treatment (e.g., the complier average causal effect, the measure of stochastic superiority of treatment over control for compliers), then estimate the causal effect with the likelihood method. Next, based on the likelihood-ratio (LR) statistic, we test those functions of or the equality of the outcome distributions of compliers with and without treatment. Although the corresponding LR statistic follows a chi-squared  (χ²)  distribution asymptotically when the true values of parameters are in the interior of the parameter space under the null, its asymptotic distribution is not  χ²  when the true values of parameters are on the boundary of the parameter space under the null. Therefore, we propose a bootstrap/double bootstrap version of a LR test for the causal effect in these trials. The methods are illustrated by an analysis of data from a randomized trial of an encouragement intervention to improve adherence to prescribed depression treatments among depressed elderly patients in primary care practices.     

Analysis of covariance in randomized trials: More precision and valid confidence intervals,without model assumptions

“Covariate adjustment” in the randomized trial context refers to an estimator of the average treatment effect that adjusts for chance imbalances between study arms in baseline variables (called “covariates”). The baseline variables could include, for example, age, sex, disease severity, and biomarkers. According to two surveys of clinical trial reports, there is confusion about the statistical properties of covariate adjustment. We focus on the analysis of covariance (ANCOVA) estimator, which involves fitting a linear model for the outcome given the treatment arm and baseline variables, and trials that use simple randomization with equal probability of assignment to treatment and control. We prove the following new (to the best of our knowledge) robustness property of ANCOVA to arbitrary model misspecification: Not only is the ANCOVA point estimate consistent (as proved by Yang and Tsiatis, 2001) but so is its standard error. This implies that confidence intervals and hypothesis tests conducted as if the linear model were correct are still asymptotically valid even when the linear model is arbitrarily misspecified, for example, when the baseline variables are nonlinearly related to the outcome or there is treatment effect heterogeneity. We also give a simple, robust formula for the variance reduction (equivalently, sample size reduction) from using ANCOVA. By reanalyzing completed randomized trials for mild cognitive impairment, schizophrenia, and depression, we demonstrate how ANCOVA can achieve variance reductions of 4 to 32%.     

Analyzing a randomized trial on breast self-examination with noncompliance and missing outcomes

Recently, instrumental variables methods have been used to address non-compliance in randomized experiments. Complicating such analyses is often the presence of missing data. The standard model for missing data, missing at random (MAR), has some unattractive features in this context. In this paper we compare MAR-based estimates of the complier average causal effect (CACE) with an estimator based on an alternative, nonignorable model for the missing data process, developed by Frangakis and Rubin (1999, Biometrika, 86, 365-379). We also introduce a new missing data model that, like the Frangakis-Rubin model, is specially suited for models with instrumental variables, but makes different substantive assumptions. We analyze these issues in the context of a randomized trial of breast self-examination (BSE). In the study two methods of teaching BSE, consisting of either mailed information about BSE (the standard treatment) or the attendance of a course involving theoretical and practical sessions (the new treatment), were compared with the aim of assessing whether teaching programs could increase BSE practice and improve examination skills. The study was affected by the two sources of bias mentioned above: only 55% of women assigned to receive the new treatment complied with their assignment and 35% of the women did not respond to the post-test questionnaire. Comparing the causal estimand of the new treatment using the MAR, Frangakis-Rubin, and our new approach, the results suggest that for these data the MAR assumption appears least plausible, and that the new model appears most plausible among the three choices.     

Proportion of treatment effect mediated by surrogate endpoints

One of the central aims in randomized clinical trials is to find well‐validated surrogate endpoints to reduce the sample size and/or duration of trials. Clinical researchers and practitioners have proposed various surrogacy measures for assessing candidate surrogate endpoints. However, most existing surrogacy measures have the following shortcomings: (i) they often fall outside the range [0,1], (ii) they are imprecisely estimated, and (iii) they ignore the interaction associations between a treatment and candidate surrogate endpoints in the evaluation of the surrogacy level. To overcome these difficulties, we propose a new surrogacy measure, the proportion of treatment effect mediated by candidate surrogate endpoints (PMS), based on the decomposition of the treatment effect into direct, indirect, and interaction associations mediated by candidate surrogate endpoints. In addition, we validate the advantages of PMS through Monte Carlo simulations and the application to empirical data from ORIENT (the Olmesartan Reducing Incidence of Endstage Renal Disease in Diabetic Nephropathy Trial).     

Generalizing causal inferences from individuals in randomized trials to all trial‐eligible individuals

We consider methods for causal inference in randomized trials nested within cohorts of trial‐eligible individuals, including those who are not randomized. We show how baseline covariate data from the entire cohort, and treatment and outcome data only from randomized individuals, can be used to identify potential (counterfactual) outcome means and average treatment effects in the target population of all eligible individuals. We review identifiability conditions, propose estimators, and assess the estimators' finite‐sample performance in simulation studies. As an illustration, we apply the estimators in a trial nested within a cohort of trial‐eligible individuals to compare coronary artery bypass grafting surgery plus medical therapy vs. medical therapy alone for chronic coronary artery disease.     

Estimating treatment effect heterogeneity for binary outcomes via Dirichlet multinomial constraints

In a randomized two-group parallel trial the mean causal effect is typically estimated as the difference in means or proportions for patients receiving, say, either treatment (T) or control (C). Treatment effect heterogeneity (TEH), or unit-treatment interaction, the variability of the causal effect (defined in terms of potential outcomes) across individuals, is often ignored. Since only one of the outcomes, either Y(T) or Y(C), is observed for each unit in such studies, the TEH is not directly estimable. For convenience, it is often assumed to be minimal or zero. We are particularly interested in the 'treatment risk' for binary outcomes, that is, the proportion of individuals who would succeed on C but fail on T. Previous work has shown that the treatment risk can be bounded (Albert, Gadbury and Mascha, 2005), and that the confidence interval width around it can be narrowed using clustered or correlated data (Mascha and Albert, 2006). Without further parameter constraints, treatment risk is unidentifiable. We show, however, that the treatment risk can be directly estimated when the four underlying population counts comprising the joint distribution of the potential outcomes, Y(T) and Y(C), follow constraints consistent with the Dirichlet multinomial. We propose a test of zero treatment risk and show it to have good size and power. Methods are applied to both a randomized as well as a non-randomized study. Implications for medical decision-making at the policy and individual levels are discussed.     

Related Causal Frameworks for Surrogate Outcomes

Summary .  Four major frameworks have been developed for evaluating surrogate markers in randomized trials: one based on conditional independence of observable variables, another based on direct and indirect effects, a third based on a meta-analysis, and a fourth based on principal stratification. The first two of these fit into a paradigm we call the causal-effects (CE) paradigm, in which, for a good surrogate, the effect of treatment on the surrogate, combined with the effect of the surrogate on the clinical outcome, allow prediction of the effect of the treatment on the clinical outcome. The last two approaches fall into the causal-association (CA) paradigm, in which the effect of the treatment on the surrogate is associated with its effect on the clinical outcome. We consider the CE paradigm first, and consider identifying assumptions and some simple estimation procedures; we then consider the CA paradigm. We examine the relationships among these approaches and associated estimators. We perform a small simulation study to illustrate properties of the various estimators under different scenarios, and conclude with a discussion of the applicability of both paradigms.     

Generalized causal mediation analysis

The goal of mediation analysis is to assess direct and indirect effects of a treatment or exposure on an outcome. More generally, we may be interested in the context of a causal model as characterized by a directed acyclic graph (DAG), where mediation via a specific path from exposure to outcome may involve an arbitrary number of links (or "stages"). Methods for estimating mediation (or pathway) effects are available for a continuous outcome and a continuous mediator related via a linear model, while for a categorical outcome or categorical mediator, methods are usually limited to two-stage mediation. We present a method applicable to multiple stages of mediation and mixed variable types using generalized linear models. We define pathway effects using a potential outcomes framework and present a general formula that provides the effect of exposure through any specified pathway. Some pathway effects are nonidentifiable and their estimation requires an assumption regarding the correlation between counterfactuals. We provide a sensitivity analysis to assess the impact of this assumption. Confidence intervals for pathway effect estimates are obtained via a bootstrap method. The method is applied to a cohort study of dental caries in very low birth weight adolescents. A simulation study demonstrates low bias of pathway effect estimators and close-to-nominal coverage rates of confidence intervals. We also find low sensitivity to the counterfactual correlation in most scenarios.     

Constructing a confidence interval for the fraction who benefit from treatment,using randomized trial data

The fraction who benefit from treatment is the proportion of patients whose potential outcome under treatment is better than that under control. Inference on this parameter is challenging since it is only partially identifiable, even in our context of a randomized trial. We propose a new method for constructing a confidence interval for the fraction, when the outcome is ordinal or binary. Our confidence interval procedure is pointwise consistent. It does not require any assumptions about the joint distribution of the potential outcomes, although it has the flexibility to incorporate various user‐defined assumptions. Our method is based on a stochastic optimization technique involving a second‐order, asymptotic approximation that, to the best of our knowledge, has not been applied to biomedical studies. This approximation leads to statistics that are solutions to quadratic programs, which can be computed efficiently using optimization tools. In simulation, our method attains the nominal coverage probability or higher, and can have narrower average width than competitor methods. We apply it to a trial of a new intervention for stroke.     

A Bayesian Approach to Surrogacy Assessment Using Principal Stratification in Clinical Trials

Summary A surrogate marker (S) is a variable that can be measured earlier and often more easily than the true endpoint (T) in a clinical trial. Most previous research has been devoted to developing surrogacy measures to quantify how well S can replace T or examining the use of S in predicting the effect of a treatment (Z). However, the research often requires one to fit models for the distribution of T given S and Z. It is well known that such models do not have causal interpretations because the models condition on a postrandomization variable S. In this article, we directly model the relationship among T, S, and Z using a potential outcomes framework introduced by Frangakis and Rubin (2002, Biometrics 58 , 21–29). We propose a Bayesian estimation method to evaluate the causal probabilities associated with the cross‐classification of the potential outcomes of S and T when S and T are both binary. We use a log‐linear model to directly model the association between the potential outcomes of S and T through the odds ratios. The quantities derived from this approach always have causal interpretations. However, this causal model is not identifiable from the data without additional assumptions. To reduce the nonidentifiability problem and increase the precision of statistical inferences, we assume monotonicity and incorporate prior belief that is plausible in the surrogate context by using prior distributions. We also explore the relationship among the surrogacy measures based on traditional models and this counterfactual model. The method is applied to the data from a glaucoma treatment study.     

Evaluating candidate principal surrogate endpoints

SUMMARY: Frangakis and Rubin (2002, Biometrics 58, 21-29) proposed a new definition of a surrogate endpoint (a "principal" surrogate) based on causal effects. We introduce an estimand for evaluating a principal surrogate, the causal effect predictiveness (CEP) surface, which quantifies how well causal treatment effects on the biomarker predict causal treatment effects on the clinical endpoint. Although the CEP surface is not identifiable due to missing potential outcomes, it can be identified by incorporating a baseline covariate(s) that predicts the biomarker. Given case-cohort sampling of such a baseline predictor and the biomarker in a large blinded randomized clinical trial, we develop an estimated likelihood method for estimating the CEP surface. This estimation assesses the "surrogate value" of the biomarker for reliably predicting clinical treatment effects for the same or similar setting as the trial. A CEP surface plot provides a way to compare the surrogate value of multiple biomarkers. The approach is illustrated by the problem of assessing an immune response to a vaccine as a surrogate endpoint for infection.     

G-estimation of structural nested restricted mean time lost models to estimate effects of time-varying treatments on a failure time outcome

G-estimation of structural nested models (SNMs) plays an important role in estimating the effects of time-varying treatments with appropriate adjustment for time-dependent confounding. As SNMs for a failure time outcome, structural nested accelerated failure time models (SNAFTMs) and structural nested cumulative failure time models have been developed. The latter models are included in the class of structural nested mean models (SNMMs) and are not involved in artificial censoring, which induces several difficulties in g-estimation of SNAFTMs. Recently, restricted mean time lost (RMTL), which corresponds to the area under a distribution function up to a restriction time, is attracting attention in clinical trial communities as an appropriate summary measure of a failure time outcome. In this study, we propose another SNMM for a failure time outcome, which is called structural nested RMTL model (SNRMTLM) and describe randomized and observational g-estimation procedures that use different assumptions for the treatment mechanism in a randomized trial setting. We also provide methods to estimate marginal RMTLs under static treatment regimes using estimated SNRMTLMs. A simulation study evaluates finite-sample performances of the proposed methods compared with the conventional intention-to-treat and per-protocol analyses. We illustrate the proposed methods using data from a randomized controlled trial for cardiovascular disease with treatment changes. G-estimation of SNRMTLMs is a useful tool to estimate the effects of time-varying treatments on a failure time outcome.     

Assessing treatment effect heterogeneity in clinical trials with blocked binary outcomes

This paper addresses treatment effect heterogeneity (also referred to, more compactly, as 'treatment heterogeneity') in the context of a controlled clinical trial with binary endpoints. Treatment heterogeneity, variation in the true (causal) individual treatment effects, is explored using the concept of the potential outcome. This framework supposes the existance of latent responses for each subject corresponding to each possible treatment. In the context of a binary endpoint, treatment heterogeniety may be represented by the parameter, pi2, the probability that an individual would have a failure on the experimental treatment, if received, and would have a success on control, if received. Previous research derived bounds for pi2 based on matched pairs data. The present research extends this method to the blocked data context. Estimates (and their variances) and confidence intervals for the bounds are derived. We apply the new method to data from a renal disease clinical trial. In this example, bounds based on the blocked data are narrower than the corresponding bounds based only on the marginal success proportions. Some remaining challenges (including the possibility of further reducing bound widths) are discussed.     

Applications of Extensions of Bivariate Rank Sum Statistics to the Crossover Design to Compare Two Treatments Through Four Sequence Groups

Summary This article describes applications of extensions of bivariate rank sum statistics to the crossover design with four sequence groups for two treatments. A randomized clinical trial in ophthalmology provides motivating background for the discussion. The bilateral design for this study has four sequence groups T:T, T:P, P:T, and P:P, respectively, for T as test treatment or P as placebo in the corresponding order for the left and right eyes. This article describes how to use the average of the separate Wilcoxon rank sum statistics for the left and right eyes for the overall comparison between T and P with the correlation between the two eyes taken into account. An extension of this criterion with better sensitivity to potential differences between T and P through reduction of the applicable variance has discussion in terms of a conceptual model with constraints for within‐side homogeneity of groups with the same treatment and between‐side homogeneity of the differences between T and P. Goodness of fit for this model can have assessment with test statistics for its corresponding constraints. Simulation studies for the conceptual model confirm better power for the extended test statistic with its full invocation than other criteria without this property. The methods summarized here are illustrated for the motivating clinical trial in ophthalmology, but they are applicable to other situations with the crossover design with four sequence groups for either two locations for two treatments at the same time for a patient or two successive periods for the assigned treatments for a recurrent disorder. This article also notes that the methods based on its conceptual model can have unsatisfactory power for departures from that model where the difference between T and P via the T:T and P:P groups is not similar to that via the T:P and P:T groups, as might occur when T has a systemic effect in a bilateral trial. For this situation, more robust test statistics have identification, but there is recognition that the parallel groups design with only the T:T and P:P groups may be more useful than the bilateral design with four sequence groups.     

A simple meta-analytic approach for using a binary surrogate endpoint to predict the effect of intervention on true endpoint

A surrogate endpoint is an endpoint that is obtained sooner, at lower cost, or less invasively than the true endpoint for a health outcome and is used to make conclusions about the effect of intervention on the true endpoint. In this approach, each previous trial with surrogate and true endpoints contributes an estimated predicted effect of intervention on true endpoint in the trial of interest based on the surrogate endpoint in the trial of interest. These predicted quantities are combined in a simple random-effects meta-analysis to estimate the predicted effect of intervention on true endpoint in the trial of interest. Validation involves comparing the average prediction error of the aforementioned approach with (i) the average prediction error of a standard meta-analysis using only true endpoints in the other trials and (ii) the average clinically meaningful difference in true endpoints implicit in the trials. Validation is illustrated using data from multiple randomized trials of patients with advanced colorectal cancer in which the surrogate endpoint was tumor response and the true endpoint was median survival time.