Sensitivity analyses comparing outcomes only existing in a subset selected post-randomization, conditional on covariates, with application to HIV vaccine trials

In many experiments, researchers would like to compare between treatments and outcome that only exists in a subset of participants selected after randomization. For example, in preventive HIV vaccine efficacy trials it is of interest to determine whether randomization to vaccine causes lower HIV viral load, a quantity that only exists in participants who acquire HIV. To make a causal comparison and account for potential selection bias we propose a sensitivity analysis following the principal stratification framework set forth by Frangakis and Rubin (2002, Biometrics58, 21-29). Our goal is to assess the average causal effect of treatment assignment on viral load at a given baseline covariate level in the always infected principal stratum (those who would have been infected whether they had been assigned to vaccine or placebo). We assume stable unit treatment values (SUTVA), randomization, and that subjects randomized to the vaccine arm who became infected would also have become infected if randomized to the placebo arm (monotonicity). It is not known which of those subjects infected in the placebo arm are in the always infected principal stratum, but this can be modeled conditional on covariates, the observed viral load, and a specified sensitivity parameter. Under parametric regression models for viral load, we obtain maximum likelihood estimates of the average causal effect conditional on covariates and the sensitivity parameter. We apply our methods to the world's first phase III HIV vaccine trial.     

Sensitivity analysis for the assessment of causal vaccine effects on viral load in HIV vaccine trials

Vaccines with limited ability to prevent HIV infection may positively impact the HIV/AIDS pandemic by preventing secondary transmission and disease in vaccine recipients who become infected. To evaluate the impact of vaccination on secondary transmission and disease, efficacy trials assess vaccine effects on HIV viral load and other surrogate endpoints measured after infection. A standard test that compares the distribution of viral load between the infected subgroups of vaccine and placebo recipients does not assess a causal effect of vaccine, because the comparison groups are selected after randomization. To address this problem, we formulate clinically relevant causal estimands using the principal stratification framework developed by Frangakis and Rubin (2002, Biometrics 58, 21-29), and propose a class of logistic selection bias models whose members identify the estimands. Given a selection model in the class, procedures are developed for testing and estimation of the causal effect of vaccination on viral load in the principal stratum of subjects who would be infected regardless of randomization assignment. We show how the procedures can be used for a sensitivity analysis that quantifies how the causal effect of vaccination varies with the presumed magnitude of selection bias.     

Chop-Lump Tests for Vaccine Trials

Summary This article proposes new tests to compare the vaccine and placebo groups in randomized vaccine trials when a small fraction of volunteers become infected. A simple approach that is consistent with the intent‐to‐treat principle is to assign a score, say W, equal to 0 for the uninfecteds and some postinfection outcome X > 0 for the infecteds. One can then test the equality of this skewed distribution of W between the two groups. This burden of illness (BOI) test was introduced by Chang, Guess, and Heyse (1994, Statistics in Medicine 13 , 1807–1814). If infections are rare, the massive number of 0s in each group tends to dilute the vaccine effect and this test can have poor power, particularly if the X's are not close to zero. Comparing X in just the infecteds is no longer a comparison of randomized groups and can produce misleading conclusions. Gilbert, Bosch, and Hudgens (2003, Biometrics 59 , 531–541) and Hudgens, Hoering, and Self (2003, Statistics in Medicine 22 , 2281–2298) introduced tests of the equality of X in a subgroup—the principal stratum of those “doomed” to be infected under either randomization assignment. This can be more powerful than the BOI approach, but requires unexaminable assumptions. We suggest new “chop‐lump” Wilcoxon and t‐tests (CLW and CLT) that can be more powerful than the BOI tests in certain situations. When the number of volunteers in each group are equal, the chop‐lump tests remove an equal number of zeros from both groups and then perform a test on the remaining W's, which are mostly >0. A permutation approach provides a null distribution. We show that under local alternatives, the CLW test is always more powerful than the usual Wilcoxon test provided the true vaccine and placebo infection rates are the same. We also identify the crucial role of the “gap” between 0 and the X's on power for the t‐tests. The chop‐lump tests are compared to established tests via simulation for planned HIV and malaria vaccine trials. A reanalysis of the first phase III HIV vaccine trial is used to illustrate the method.     

Clustered encouragement designs with individual noncompliance: bayesian inference with randomization,and application to advance directive forms

In many studies comparing a new 'target treatment' with a control target treatment, the received treatment does not always agree with assigned treatment-that is, the compliance is imperfect. An obvious example arises when ethical or practical constraints prevent even the randomized assignment of receipt of the new target treatment but allow the randomized assignment of the encouragement to receive this treatment. In fact, many randomized experiments where compliance is not enforced by the experimenter (e.g. with non-blinded assignment) may be more accurately thought of as randomized encouragement designs. Moreover, often the assignment of encouragement is at the level of clusters (e.g. doctors) where the compliance with the assignment varies across the units (e.g. patients) within clusters. We refer to such studies as 'clustered encouragement designs' (CEDs) and they arise relatively frequently (e.g. Sommer and Zeger, 1991; McDonald et al., 1992; Dexter et al., 1998) Here, we propose Bayesian methodology for causal inference for the effect of the new target treatment versus the control target treatment in the randomized CED with all-or-none compliance at the unit level, which generalizes the approach of Hirano et al. (2000) in important and surprisingly subtle ways, to account for the clustering, which is necessary for statistical validity. We illustrate our methods using data from a recent study exploring the role of physician consulting in increasing patients' completion of Advance Directive forms.     

Bounds on the average causal effects in randomized trials with noncompliance by covariate adjustment

In randomized trials with noncompliance, causal effects cannot be identified without strong assumptions. Therefore, several authors have considered bounds on the causal effects. Applying an idea of VanderWeele ( 2008 ), Chiba ( 2009 ) gave bounds on the average causal effects in randomized trials with noncompliance using the information on the randomized assignment, the treatment received and the outcome under monotonicity assumptions about covariates. But he did not consider any observed covariates. If there are some observed covariates such as age, gender, and race in a trial, we propose new bounds using the observed covariate information under some monotonicity assumptions similar to those of VanderWeele and Chiba. And we compare the three bounds in a real example.     

The large sample bounds on the principal strata effect with application to a prostate cancer prevention trial

Issues of post-randomization selection bias and truncation-by-death can arise in randomized clinical trials; for example, in a cancer prevention trial, an outcome such as cancer severity is undefined for individuals who do not develop cancer. Restricting analysis to a subpopulation selected after randomization can give rise to biased outcome comparisons. One approach to deal with such issues is to consider the principal strata effect (PSE, or equally, the survivor average causal effect). PSE is defined as the effect of treatment on the outcome among the subpopulation that would have been selected under either treatment arm. Unfortunately, the PSE cannot generally be estimated without the identifying assumptions; however, the bounds can be derived using a deterministic causal model. In this paper, we propose a number of assumptions for deriving the bounds with narrow width. The assumptions and bounds, which differ from those introduced by Zhang and Rubin (2003), are illustrated using data from a randomized prostate cancer prevention trial.     

Methods for clustered encouragement design studies with noncompliance and missing data

Encouragement design studies are particularly useful for estimating the effect of an intervention that cannot itself be randomly administered to some and not to others. They require a randomly selected group receive extra encouragement to undertake the treatment of interest, where the encouragement typically takes the form of additional information or incentives. We consider a "clustered encouragement design" (CED), where the randomization is at the level of the clusters (e.g. physicians), but the compliance with assignment is at the level of the units (e.g. patients) within clusters. Noncompliance and missing data are particular problems in encouragement design studies, where encouragement to take the treatment, rather than the treatment itself, is randomized. The motivating study looks at whether computer-based care suggestions can improve patient outcomes in veterans with chronic heart failure. Since physician adherence has been inadequate, the original study focused on methods to improve physician adherence, although an equally important question is whether physician adherence improves patient outcomes. Here, we reanalyze the data to determine the effect of physician adherence on patient outcomes. We propose causal inference methodology for the effect of a treatment versus a control in a randomized CED study with all-or-none compliance at the unit level. These methods extend the current approaches to account for nonignorable missing data and use an alternative approach to inference using multiple imputation methods, which have been successfully applied to a wide variety of missing data problems and have recently been applied to the potential outcomes framework of causal inference (Taylor and Zhou, 2009b).     

The role of mucosal immunity in prevention of HIV transmission

Vaccines designed to prevent mucosal transmission of HIV should establish multiple immune effectors in vaccine recipients, including antibodies which are capable of blocking HIV entry at mucosal epithelial barriers and of preventing initial infection of target cells in the mucosa. Immunological analyses of HIV-resistant humans and data obtained in nonhuman primate vaccine studies indicate that both secretory and serum antibodies may play an important role in protection against mucosal transmission of HIV or SIV, whereas cytotoxic T cells are required for clearance of mucosal infection and prevention of systemic spread. This review summarizes the roles of IgA and IgG antibodies in preventing mucosal infection by other viral and bacterial pathogens, and then discusses the various mechanisms by which antibodies might contribute to protection against HIV at mucosal surfaces. These include prevention of mucosal contact, blocking attachment of virus or infected cells to epithelial cells, interception of virus during transepithelial transport, neutralization of virus in the mucosa, and elimination of locally infected cells through antibody-dependent cell-mediated cytotoxic reactions. The regional nature of mucosal immune responses is reviewed in light of its relevance to HIV vaccine development. We conclude that mucosal immunization should be considered a component of vaccine strategies against HIV.     

Adaptive Randomization for Multiarm Comparative Clinical Trials Based on Joint Efficacy/Toxicity Outcomes

Summary .  We present an outcome-adaptive randomization (AR) scheme for comparative clinical trials in which the primary endpoint is a joint efficacy/toxicity outcome. Under the proposed scheme, the randomization probabilities are unbalanced adaptively in favor of treatments with superior joint outcomes characterized by higher efficacy and lower toxicity. This type of scheme is advantageous from the patients' perspective because on average, more patients are randomized to superior treatments. We extend the approximate Bayesian time-to-event model in Cheung and Thall (2002,  Biometrics   58, 89–97) to model the joint efficacy/toxicity outcomes and perform posterior computation based on a latent variable approach. Consequently, this allows us to incorporate essential information about patients with incomplete follow-up. Based on the computed posterior probabilities, we propose an AR scheme that favors the treatments with larger joint probabilities of efficacy and no toxicity. We illustrate our methodology with a leukemia trial that compares three treatments in terms of their 52-week molecular remission rates and 52-week toxicity rates.     

Augmented designs to assess immune response in vaccine trials

This article introduces methods for use in vaccine clinical trials to help determine whether the immune response to a vaccine is actually causing a reduction in the infection rate. This is not easy because immune response to the (say HIV) vaccine is only observed in the HIV vaccine arm. If we knew what the HIV-specific immune response in placebo recipients would have been, had they been vaccinated, this immune response could be treated essentially like a baseline covariate and an interaction with treatment could be evaluated. Relatedly, the rate of infection by this baseline covariate could be compared between the two groups and a causative role of immune response would be supported if infection risk decreased with increasing HIV immune response only in the vaccine group. We introduce two methods for inferring this HIV-specific immune response. The first involves vaccinating everyone before baseline with an irrelevant vaccine, for example, rabies. Randomization ensures that the relationship between the immune responses to the rabies and HIV vaccines observed in the vaccine group is the same as what would have been seen in the placebo group. We infer a placebo volunteer's response to the HIV vaccine using their rabies response and a prediction model from the vaccine group. The second method entails vaccinating all uninfected placebo patients at the closeout of the trial with the HIV vaccine and recording immune response. We pretend this immune response at closeout is what they would have had at baseline. We can then infer what the distribution of immune response among placebo infecteds would have been. Such designs may help elucidate the role of immune response in preventing infections. More pointedly, they could be helpful in the decision to improve or abandon an HIV vaccine with mediocre performance in a phase III trial.     

Mycobacterium tuberculosis Infection Interferes with HIV Vaccination in Mice

Tuberculosis (TB) has emerged as the most prominent bacterial disease found in human immunodeficiency virus (HIV)-positive individuals worldwide. Due to high prevalence of asymptomatic Mycobacterium tuberculosis (Mtb) infections, the future HIV vaccine in areas highly endemic for TB will often be administrated to individuals with an ongoing Mtb infection. The impact of concurrent Mtb infection on the immunogenicity of a HIV vaccine candidate, MultiHIV DNA/protein, was investigated in mice. We found that, depending on the vaccination route, mice infected with Mtb before the administration of the HIV vaccine showed impairment in both the magnitude and the quality of antibody and T cell responses to the vaccine components p24Gag and gp160Env. Mice infected with Mtb prior to intranasal HIV vaccination exhibited reduced p24Gag-specific serum IgG and IgA, and suppressed gp160Env-specific serum IgG as compared to respective titers in uninfected HIV-vaccinated controls. Importantly, in Mtb-infected mice that were HIV-vaccinated by the intramuscular route the virus neutralizing activity in serum was significantly decreased, relative to uninfected counterparts. In addition mice concurrently infected with Mtb had fewer p24Gag-specific IFN-γ-expressing T cells and multifunctional T cells in their spleens. These results suggest that Mtb infection might interfere with the outcome of prospective HIV vaccination in humans.     

A comparison of eight methods for the dual-endpoint evaluation of efficacy in a proof-of-concept HIV vaccine trial

To support the design of the world's first proof-of-concept (POC) efficacy trial of a cell-mediated immunity-based HIV vaccine, we evaluate eight methods for testing the composite null hypothesis of no-vaccine effect on either the incidence of HIV infection or the viral load set point among those infected, relative to placebo. The first two methods use a single test applied to the actual values or ranks of a burden-of-illness (BOI) outcome that combines the infection and viral load endpoints. The other six methods combine separate tests for the two endpoints using unweighted or weighted versions of the two-part z, Simes', and Fisher's methods. Based on extensive simulations that were used to design the landmark POC trial, the BOI methods are shown to have generally low power for rejecting the composite null hypothesis (and hence advancing the vaccine to a subsequent large-scale efficacy trial). The unweighted Simes' and Fisher's combination methods perform best overall. Importantly, this conclusion holds even after the test for the viral load component is adjusted for bias that can be introduced by conditioning on a postrandomization event (HIV infection). The adjustment is derived using a selection bias model based on the principal stratification framework of causal inference.     

Predicting the impact of a nonsterilizing vaccine against human immunodeficiency virus

Studies of human immunodeficiency virus (HIV) vaccines in animal models suggest that it is difficult to induce complete protection from infection (sterilizing immunity) but that it is possible to reduce the viral load and to slow or prevent disease progression following infection. We have developed an age-structured epidemiological model of the effects of a disease-modifying HIV vaccine that incorporates the intrahost dynamics of infection, a transmission rate and host mortality that depend on the viral load, the possible evolution and transmission of vaccine escape mutant viruses, a finite duration of vaccine protection, and possible changes in sexual behavior. Using this model, we investigated the long-term outcome of a disease-modifying vaccine and utilized uncertainty analysis to quantify the effects of our lack of precise knowledge of various parameters. Our results suggest that the extent of viral load reduction in vaccinated infected individuals (compared to unvaccinated individuals) is the key predictor of vaccine efficacy. Reductions in viral load of about 1 log(10) copies ml(-1) would be sufficient to significantly reduce HIV-associated mortality in the first 20 years after the introduction of vaccination. Changes in sexual risk behavior also had a strong impact on the epidemic outcome. The impact of vaccination is dependent on the population in which it is used, with disease-modifying vaccines predicted to have the most impact in areas of low prevalence and rapid epidemic growth. Surprisingly, the extent to which vaccination alters disease progression, the rate of generation of escape mutants, and the transmission of escape mutants are predicted to have only a weak impact on the epidemic outcome over the first 25 years after the introduction of a vaccine.     

Microdevices for examining immunological responses of single cells to HIV

More than 60 million people in the world have been diagnosed with HIV infections since the virus was recognized as the causative agent of AIDS in the 1980s. Even though more than half of the infected patients have died, effective disease treatment and prevention measures have not been established. ART (antiretroviral therapy) is the only proven HIV treatment that sustains the suppression of patient viraemia. Current routine approaches to treat HIV infections are targeted at developing vaccines that will induce humoral or cell memory immune responses. However, developing an effective vaccine has been challenging because the HIV mutates rapidly, which allows the virus to evade immune surveillances established against the previous strain. In addition, the virus is able to quickly establish a reservoir and treatment is difficult because of the general lack of knowledge about HIV immune response mechanisms. This review introduces common disease symptoms and the progression of HIV infection with a brief summary of the current treatment approaches. Different cellular immune responses against HIV are also discussed, with emphasis on a nanotechnology research that has focused on probing T-cell response to HIV infection. Furthermore, we discuss recent noteworthy nanotechnology updates on T-cell response screening that is focused on HIV infection. Finally, we review potential future treatment strategies based on the correlations between T-cell response and HIV infection.     

Assessing Biases in the Evaluation of Classification Assays for HIV Infection Recency

Identifying recent HIV infection cases has important public health and clinical implications. It is essential for estimating incidence rates to monitor epidemic trends and evaluate the effectiveness of interventions. Detecting recent cases is also important for HIV prevention given the crucial role that recently infected individuals play in disease transmission, and because early treatment onset can improve the clinical outlook of patients while reducing transmission risk. Critical to this enterprise is the development and proper assessment of accurate classification assays that, based on cross-sectional samples of viral sequences, help determine infection recency status. In this work we assess some of the biases present in the evaluation of HIV recency classification algorithms that rely on measures of within-host viral diversity. Particularly, we examine how the time since infection (TSI) distribution of the infected subjects from which viral samples are drawn affect performance metrics (e.g., area under the ROC curve, sensitivity, specificity, accuracy and precision), potentially leading to misguided conclusions about the efficacy of classification assays. By comparing the performance of a given HIV recency assay using six different TSI distributions (four simulated TSI distributions representing different epidemic scenarios, and two empirical TSI distributions), we show that conclusions about the overall efficacy of the assay depend critically on properties of the TSI distribution. Moreover, we demonstrate that an assay with high overall classification accuracy, mainly due to properly sorting members of the well-represented groups in the validation dataset, can still perform notoriously poorly when sorting members of the less represented groups. This is an inherent issue of classification and diagnostics procedures that is often underappreciated. Thus, this work underscores the importance of acknowledging and properly addressing evaluation biases when proposing new HIV recency assays.     

Allocation of patients to treatment in clinical trials.

This article is intended as a practical guide to the various methods of patient assignment in clinical trials. Topics discussed include a critical appraisal of non-randomized studies, methods of restricted randomization such as random permuted blocks and the biased coin technique, the extent to which stratification is necessary and the methods available, the possible benefits of randomization with a greater proportion of patients on a new treatment, factorial designs, crossover designs, randomized consent designs and adaptive assignment procedures. With all this diversity of approach it needs to be remembered that the effective implementation and reliability of a relatively straightforward randomization scheme may be more important than attempting theoretical optimality with more complex designs.     

Drugs of abuse and HIV infection/replication: implications for mother-fetus transmission

Human immunodeficiency virus (HIV) infection and progression of acquired immunodeficiency syndrome (AIDS) can be modulated by a number of cofactors, including drugs of abuse. Opioids, cocaine, cannabinoids, methamphetamine (METH), alcohol, and other substances of abuse have been implicated as risk factors for HIV infection, as they all have the potential to compromise host immunity and facilitate viral replication. Although epidemiologic evidence regarding the impact of drugs of abuse on HIV disease progression is mixed, in vitro studies as well as studies using in vivo animal models have indicated that drugs of abuse have the ability to enhance HIV infection/replication. Drugs of abuse may also be a risk factor for perinatal transmission of HIV. Because high levels of viral load in maternal blood are associated with increased risk of HIV vertical transmission, it is likely that drugs of abuse play an important role in promoting mother-fetus transmission. Furthermore, because the neonatal immune system differs qualitatively from the adult system, it is possible that maternal exposure to drugs of abuse would exacerbate neonatal immunity defects, facilitating HIV infection of neonate immune cells and promoting HIV vertical transmission. The availability and use of antiretroviral therapy for women infected with HIV increase, there is an increasing interest in determining the impact of drug abuse on efficacy of AIDS Clinical Trials Group (ACTG)-standardized treatment regimens for woman infected with HIV in the context of HIV vertical transmission.     

Estimation of survival distributions of treatment policies in two-stage randomization designs in clinical trials

Some clinical trials follow a design where patients are randomized to a primary therapy at entry followed by another randomization to maintenance therapy contingent upon disease remission. Ideally, analysis would allow different treatment policies, i.e., combinations of primary and maintenance therapy if specified up-front, to be compared. Standard practice is to conduct separate analyses for the primary and follow-up treatments, which does not address this issue directly. We propose consistent estimators for the survival distribution and mean restricted survival time for each treatment policy in such two-stage studies and derive large-sample properties. The methods are demonstrated on a leukemia clinical trial data set and through simulation.     

Commentary on "Principal stratification - a goal or a tool?" by Judea Pearl

This commentary takes up Pearl's welcome challenge to clearly articulate the scientific value of principal stratification estimands that we and colleagues have investigated, in the area of randomized placebo-controlled preventive vaccine efficacy trials, especially trials of HIV vaccines. After briefly arguing that certain principal stratification estimands for studying vaccine effects on post-infection outcomes are of genuine scientific interest, the bulk of our commentary argues that the "causal effect predictiveness" (CEP) principal stratification estimand for evaluating immune biomarkers as surrogate endpoints is not of ultimate scientific interest, because it evaluates surrogacy restricted to the setting of a particular vaccine efficacy trial, but is nevertheless useful for guiding the selection of primary immune biomarker endpoints in Phase I/II vaccine trials and for facilitating assessment of transportability/bridging surrogacy.