Sequential parallel comparison design for “gold standard” noninferiority trials with a prespecified margin

Three‐arm noninferiority trials (involving an experimental treatment, a reference treatment, and a placebo)—called the “gold standard” noninferiority trials—are conducted in patients with mental disorders whenever feasible, but often fail to show superiority of the experimental treatment and/or the reference treatment over the placebo. One possible reason is that some of the patients receiving the placebo show apparent improvement in the clinical condition. An approach to addressing this problem is the use of the sequential parallel comparison design (SPCD). Nonetheless, the SPCD has not yet been discussed in relation to gold standard noninferiority trials. In this article, our aim was to develop a hypothesis‐testing method and its corresponding sample size calculation method for gold standard noninferiority trials with the SPCD. In a simulation, we show that the proposed hypothesis‐testing method achieves the nominal type I error rate and power and that the proposed sample size calculation method has adequate power accuracy.     

Assessment of Noninferiority (and Equivalence) for Simple Crossover Trials Using Bayesian Approach

To assess the noninferiority or equivalence of a general drug to a standard one, researchers generally use the odds ratio of patient response rates to evaluate the relative treatment efficacy. In this paper, we use a logistic random effects model to test noninferiority and equivalence based on the odds ratio of patient response rates for crossover trials with binary data. We use Bayesian sampling algorithm, data augmentation, and scaled mixture of normal representation to implement the approach and improve efficiency. The performance of the proposed approach is assessed via simulation and real data examples.     

Statistical methods in recent HIV noninferiority trials: reanalysis of 11 trials

Background

In recent years the “noninferiority” trial has emerged as the new standard design for HIV drug development among antiretroviral patients often with a primary endpoint based on the difference in success rates between the two treatment groups. Different statistical methods have been introduced to provide confidence intervals for that difference. The main objective is to investigate whether the choice of the statistical method changes the conclusion of the trials.

Methods

We presented 11 trials published in 2010 using a difference in proportions as the primary endpoint. In these trials, 5 different statistical methods have been used to estimate such confidence intervals. The five methods are described and applied to data from the 11 trials. The noninferiority of the new treatment is not demonstrated if the prespecified noninferiority margin it includes in the confidence interval of the treatment difference.

Results

Results indicated that confidence intervals can be quite different according to the method used. In many situations, however, conclusions of the trials are not altered because point estimates of the treatment difference were too far from the prespecified noninferiority margins. Nevertheless, in few trials the use of different statistical methods led to different conclusions. In particular the use of “exact” methods can be very confusing.

Conclusion

Statistical methods used to estimate confidence intervals in noninferiority trials have a strong impact on the conclusion of such trials.     

A Novel Approach to Delayed-Start Analyses for Demonstrating Disease-Modifying Effects in Alzheimer’s Disease

One method for demonstrating disease modification is a delayed-start design, consisting of a placebo-controlled period followed by a delayed-start period wherein all patients receive active treatment. To address methodological issues in previous delayed-start approaches, we propose a new method that is robust across conditions of drug effect, discontinuation rates, and missing data mechanisms. We propose a modeling approach and test procedure to test the hypothesis of noninferiority, comparing the treatment difference at the end of the delayed-start period with that at the end of the placebo-controlled period. We conducted simulations to identify the optimal noninferiority testing procedure to ensure the method was robust across scenarios and assumptions, and to evaluate the appropriate modeling approach for analyzing the delayed-start period. We then applied this methodology to Phase 3 solanezumab clinical trial data for mild Alzheimer’s disease patients. Simulation results showed a testing procedure using a proportional noninferiority margin was robust for detecting disease-modifying effects; conditions of high and moderate discontinuations; and with various missing data mechanisms. Using all data from all randomized patients in a single model over both the placebo-controlled and delayed-start study periods demonstrated good statistical performance. In analysis of solanezumab data using this methodology, the noninferiority criterion was met, indicating the treatment difference at the end of the placebo-controlled studies was preserved at the end of the delayed-start period within a pre-defined margin. The proposed noninferiority method for delayed-start analysis controls Type I error rate well and addresses many challenges posed by previous approaches. Delayed-start studies employing the proposed analysis approach could be used to provide evidence of a disease-modifying effect. This method has been communicated with FDA and has been successfully applied to actual clinical trial data accrued from the Phase 3 clinical trials of solanezumab.     

On Analysis of the Difference of Two Exposure-Adjusted Poisson Rates with Stratification: From Asymptotic to Exact Approaches

Long-term studies are frequently conducted to assess the treatment effect on rare diseases or the safety of a new treatment. To account for differential follow-up often encountered in long-term studies, exposure-adjusted incidence rates are used in evaluating the treatment effect. The difference of rates is sometimes used to quantify the treatment’s public health impact because the reciprocal of this difference can be interpreted as “the number needed to treat (or number needed to vaccinate) in order to cure (or prevent) 1 case of disease.” In this paper we focus on the stratified analysis of the difference of two exposure-adjusted rates in the setting of superiority, noninferiority and super-superiority hypothesis testing. After a brief review of asymptotic methods, we derive an exact method that guarantees control of the type I error. But it is conservative for noninferiority and super-superiority testing because of the search of the maximum tail probability over a multidimensional nuisance parameter space. Then, we present an approximate exact method where the p-value is estimated at the maximum likelihood estimates of the nuisance parameter. This method is identical to exact method for superiority testing and reduces the conservatism for noninferiority and super-superiority testing. In addition, a quasi-exact method is discussed. A real-life vaccine clinical trial example is used to illustrate these methods. Finally, we compare the performance of these methods via empirical studies and make some general practical recommendations.     

Sequential tests for noninferiority and superiority

The problem of simultaneous sequential tests for noninferiority and superiority of a treatment, as compared to an active control, is considered in terms of continuous hierarchical families of one-sided null hypotheses, in the framework of group sequential and adaptive two-stage designs. The crucial point is that the decision boundaries for the individual null hypotheses may vary over the parameter space. This allows one to construct designs where, e.g., a rigid stopping criterion is chosen, rejecting or accepting all individual null hypotheses simultaneously. Another possibility is to use monitoring type stopping boundaries, which leave some flexibility to the experimenter: he can decide, at the interim analysis, whether he is satisfied with the noninferiority margin achieved at this stage, or wants to go for more at the second stage. In the case where he proceeds to the second stage, he may perform midtrial design modifications (e.g., reassess the sample size). The proposed approach allows one to "spend," e.g., less of alpha for an early proof of noninferiority than for an early proof of superiority, and is illustrated by typical examples.     

Exact-corrected confidence interval for risk difference in noninferiority binomial trials

A novel confidence interval estimator is proposed for the risk difference in noninferiority binomial trials. The proposed confidence interval, which is dependent on the prespecified noninferiority margin, is consistent with an exact unconditional test that preserves the type-I error and has improved power, particularly for smaller sample sizes, compared to the confidence interval by Chan and Zhang. The improved performance of the proposed confidence interval is theoretically justified and demonstrated with simulations and examples. An R package is also distributed that implements the proposed methods along with other confidence interval estimators.     

Power of the Wilcoxon–Mann–Whitney test for non‐inferiority in the presence of death‐censored observations

In clinical trials with patients in a critical state, death may preclude measurement of a quantitative endpoint of interest, and even early measurements, for example for intention‐to‐treat analysis, may not be available. For example, a non‐negligible proportion of patients with acute pulmonary embolism will die before 30 day measurements on the efficacy of thrombolysis can be obtained. As excluding such patients may introduce bias, alternative analyses, and corresponding means for sample size calculation are needed. We specifically consider power analysis in a randomized clinical trial setting in which the goal is to demonstrate noninferiority of a new treatment as compared to a reference treatment. Also, a nonparametric approach may be needed due to the distribution of the quantitative endpoint of interest. While some approaches have been developed in a composite endpoint setting, our focus is on the continuous endpoint affected by death‐related censoring, for which no approach for noninferiority is available. We propose a solution based on ranking the quantitative outcome and assigning worst rank scores to the patients without quantitative outcome because of death. Based on this, we derive power formulae for a noninferiority test in the presence of death‐censored observations, considering settings with and without ties. The approach is illustrated for an exemplary clinical trial in pulmonary embolism. The results there show a substantial effect of death on power, also depending on differential effects in the two trial arms. Therefore, use of the proposed formulae is advisable whenever there is death to be expected before measurement of a quantitative primary outcome of interest.     

Repeated confidence intervals in self-designing clinical trials and switching between noninferiority and superiority

In self-designing clinical trials, repeated confidence intervals are derived for the parameter of interest where the results of the independent study stages are combined using the generalized inverse chi-square-method. The confidence intervals can be calculated at each interim analysis and always hold the predefined overall nominal confidence level. Moreover, the confidence intervals calculated during the course of the trial are nested in the sense that a calculated interval is completely contained in all the previously calculated intervals. During the course of the self-designing trial the sample sizes as well as the number of study stages can be determined simultaneously in a completely adaptive way. The adaptive procedure allows an early stop for significance. The clinical trial may be originally designed either to show noninferiority or superiority. However, in each interim analysis, it is possible to change the planning from showing superiority to showing noninferiority or vice versa. Since the repeated confidence intervals are nested, there is no risk to loose the noninferiority once showed when, after an interim analysis, the trial is continued in an attempt to reach superiority. A simulation study investigates the behavior of the considered confidence intervals. The performance of the derived nested repeated confidence intervals is also demonstrated in examples showing both kinds of switching during an ongoing trial.     

Designs of superiority and noninferiority trials for binary responses are noninterchangeable

To better understand the design of noninferiority trials for binary data, we identify analogies and contrasts between this and the more familiar superiority trial design. We restrict attention to the problem of detecting a difference between experimental and control response rates in the setting where there is no difference (piE - piC = 0) under the noninferiority alternative hypothesis and under the superiority null, and a matching difference between groups under the complementary hypotheses (/piE - piC/ = delta). Our derivation of the constrained maximum likelihood estimates (MLEs) reveals that superiority and noninferiority trials have different nuisance parameters--the marginal response rate and the control-group response rate, respectively. Our empirical results show that when individuals are allocated to treatment groups in the ratio that minimizes the overall sample size, balanced allocation is optimal only for superiority trials when the error rates are equal; otherwise imbalanced allocation is optimal. Different allocation ratios between trial types lead to different variances, and thus to different sample sizes. Finally, since the value of the marginal response rate--a design parameter in noninferiority trials--typically cannot be obtained from preliminary or published studies, we suggest a means of identifying a value that can be used. We conclude that full documentation of the design of a trial requires specification not only of the design parameters but also of the allocation ratio and the nuisance parameter, the value of which is not obvious under unequal allocation.     

Unconditional Exact Tests for Equivalence or Noninferiority for Paired Binary Endpoints

Problems of establishing equivalence or noninferiority between two medical diagnostic procedures involve comparisons of the response rates between correlated proportions. When the sample size is small, the asymptotic tests may not be reliable. This article proposes an unconditional exact test procedure to assess equivalence or noninferiority. Two statistics, a sample-based test statistic and a restricted maximum likelihood estimation (RMLE)-based test statistic, to define the rejection region of the exact test are considered. We show the p-value of the proposed unconditional exact tests can be attained at the boundary point of the null hypothesis. Assessment of equivalence is often based on a comparison of the confidence limits with the equivalence limits. We also derive the unconditional exact confidence intervals on the difference of the two proportion means for the two test statistics. A typical data set of comparing two diagnostic procedures is analyzed using the proposed unconditional exact and asymptotic methods. The p-value from the unconditional exact tests is generally larger than the p-value from the asymptotic tests. In other words, an exact confidence interval is generally wider than the confidence interval obtained from an asymptotic test.     

Bayesian design of noninferiority trials for medical devices using historical data

Summary We develop a new Bayesian approach of sample size determination (SSD) for the design of noninferiority clinical trials. We extend the fitting and sampling priors of Wang and Gelfand (2002, Statistical Science 17 , 193–208) to Bayesian SSD with a focus on controlling the type I error and power. Historical data are incorporated via a hierarchical modeling approach as well as the power prior approach of Ibrahim and Chen (2000, Statistical Science 15 , 46–60). Various properties of the proposed Bayesian SSD methodology are examined and a simulation‐based computational algorithm is developed. The proposed methodology is applied to the design of a noninferiority medical device clinical trial with historical data from previous trials.     

Sample size calculations for noninferiority trials with Poisson distributed count data

Clinical trials with Poisson distributed count data as the primary outcome are common in various medical areas such as relapse counts in multiple sclerosis trials or the number of attacks in trials for the treatment of migraine. In this article, we present approximate sample size formulae for testing noninferiority using asymptotic tests which are based on restricted or unrestricted maximum likelihood estimators of the Poisson rates. The Poisson outcomes are allowed to be observed for unequal follow‐up schemes, and both the situations that the noninferiority margin is expressed in terms of the difference and the ratio are considered. The exact type I error rates and powers of these tests are evaluated and the accuracy of the approximate sample size formulae is examined. The test statistic using the restricted maximum likelihood estimators (for the difference test problem) and the test statistic that is based on the logarithmic transformation and employs the maximum likelihood estimators (for the ratio test problem) show favorable type I error control and can be recommended for practical application. The approximate sample size formulae show high accuracy even for small sample sizes and provide power values identical or close to the aspired ones. The methods are illustrated by a clinical trial example from anesthesia.     

Statistical analysis of noninferiority trials with a rate ratio in small-sample matched-pair designs

Testing of noninferiority has become increasingly important in modern medicine as a means of comparing a new test procedure to a currently available test procedure. Asymptotic methods have recently been developed for analyzing noninferiority trials using rate ratios under the matched-pair design. In small samples, however, the performance of these asymptotic methods may not be reliable, and they are not recommended. In this article, we investigate alternative methods that are desirable for assessing noninferiority trials, using the rate ratio measure under small-sample matched-pair designs. In particular, we propose an exact and an approximate exact unconditional test, along with the corresponding confidence intervals based on the score statistic. The exact unconditional method guarantees the type I error rate will not exceed the nominal level. It is recommended for when strict control of type I error (protection against any inflated risk of accepting inferior treatments) is required. However, the exact method tends to be overly conservative (thus, less powerful) and computationally demanding. Via empirical studies, we demonstrate that the approximate exact score method, which is computationally simple to implement, controls the type I error rate reasonably well and has high power for hypothesis testing. On balance, the approximate exact method offers a very good alternative for analyzing correlated binary data from matched-pair designs with small sample sizes. We illustrate these methods using two real examples taken from a crossover study of soft lenses and a Pneumocystis carinii pneumonia study. We contrast the methods with a hypothetical example.     

Sample size determination for establishing equivalence/noninferiority via ratio of two proportions in matched-pair design

In this article, we propose approximate sample size formulas for establishing equivalence or noninferiority of two treatments in match-pairs design. Using the ratio of two proportions as the equivalence measure, we derive sample size formulas based on a score statistic for two types of analyses: hypothesis testing and confidence interval estimation. Depending on the purpose of a study, these formulas can be used to provide a sample size estimate that guarantees a prespecified power of a hypothesis test at a certain significance level or controls the width of a confidence interval with a certain confidence level. Our empirical results confirm that these score methods are reliable in terms of true size, coverage probability, and skewness. A liver scan detection study is used to illustrate the proposed methods.     

Comparison of Transversus Abdominis Plane Infiltration with Liposomal Bupivacaine versus Continuous Epidural Analgesia versus Intravenous Opioid Analgesia

Epidural analgesia is considered the standard of care but cannot be provided to all patients Liposomal bupivacaine has been approved for field blocks such as transversus abdominis plane (TAP) blocks but has not been clinically compared against other modalities. In this retrospective propensity matched cohort study we thus tested the primary hypothesis that TAP infiltration are noninferior (not worse) to continuous epidural analgesia and superior (better) to intravenous opioid analgesia in patients recovering from major lower abdominal surgery. 318 patients were propensity matched on 18 potential factors among three groups (106 per group): 1) TAP infiltration with bupivacaine liposome; 2) continuous Epidural analgesia with plain bupivacaine; and; 3) intravenous patient-controlled analgesia (IV PCA). We claimed TAP noninferior (not worse) over Epidural if TAP was noninferior (not worse) on total morphine-equivalent opioid and time-weighted average pain score (10-point scale) within first 72 hours after surgery with noninferiority deltas of 1 (10-point scale) for pain and an increase less of 20% in the mean morphine equivalent opioid consumption. We claimed TAP or Epidural groups superior (better) over IV PCA if TAP or Epidural was superior on opioid consumption and at least noninferior on pain outcome. Multivariable linear regressions within the propensity-matched cohorts were used to model total morphine-equivalent opioid dose and time-weighted average pain score within first 72 hours after surgery; joint hypothesis framework was used for formal testing. TAP infiltration were noninferior to Epidural on both primary outcomes (p<0.001). TAP infiltration were noninferior to IV PCA on pain scores (p = 0.001) but we did not find superiority on opioid consumption (p = 0.37). We did not find noninferiority of Epidural over IV PCA on pain scores (P = 0.13) and nor did we find superiority on opioid consumption (P = 0.98). TAP infiltration with liposomal bupivacaine and continuous epidural analgesia were similar in terms of pain and opioid consumption, and not worse in pain compared with IV PCA. TAP infiltrations might be a reasonable alternative to epidural analgesia in abdominal surgical patients. A large randomized trial comparing these techniques is justified.     

Plan to be flexible: a commentary on adaptive designs

This is a discussion of the following three papers appearing in this special issue on adaptive designs: 'Nested repeated confidence intervals and switching between noninferiority and superiority' by Joachim Hartung and Guido Knapp; 'Confirmatory Seamless Phase II/III Clinical trials with Hypotheses Selection at Interim: General Concepts' by Frank Bretz, Heinz Schmidli, Franz K?nig, Amy Racine and Willi Maurer; and 'Confirmatory Seamless Phase Il/III Clinical Trials with Hypotheses Selection at Interim: Applications and Practical Considerations' by Heinz Schmidli, Frank Bretz, Amy Racine and Willi Maurer.     

Bayesian meta-experimental design: evaluating cardiovascular risk in new antidiabetic therapies to treat type 2 diabetes

Recent guidance from the Food and Drug Administration for the evaluation of new therapies in the treatment of type 2 diabetes (T2DM) calls for a program-wide meta-analysis of cardiovascular (CV) outcomes. In this context, we develop a new Bayesian meta-analysis approach using survival regression models to assess whether the size of a clinical development program is adequate to evaluate a particular safety endpoint. We propose a Bayesian sample size determination methodology for meta-analysis clinical trial design with a focus on controlling the type I error and power. We also propose the partial borrowing power prior to incorporate the historical survival meta data into the statistical design. Various properties of the proposed methodology are examined and an efficient Markov chain Monte Carlo sampling algorithm is developed to sample from the posterior distributions. In addition, we develop a simulation-based algorithm for computing various quantities, such as the power and the type I error in the Bayesian meta-analysis trial design. The proposed methodology is applied to the design of a phase 2/3 development program including a noninferiority clinical trial for CV risk assessment in T2DM studies.     

Noninferiority trials

Noninferiority trials are intended to show that the effect of a new treatment is not worse than that of an active control by more than a specified margin. These trials have a number of inherent weaknesses that superiority trials do not: no internal demonstration of assay sensitivity, no single conservative analysis approach, lack of protection from bias by blinding, and difficulty in specifying the noninferiority margin. Noninferiority trials may sometimes be necessary when a placebo group can not be ethically included, but it should be recognized that the results of such trials are not as credible as those from a superiority trial.     

Immunogenicity noninferiority study of 2 doses and 3 doses of an Escherichia coli-produced HPV bivalent vaccine in girls vs. 3 doses in young women

A new HPV-16/18 bivalent vaccine expressed by the Escherichia coli has been proven to be efficacious in adult women. A randomized, immunogenicity noninferiority study of this candidate vaccine was conducted in December 2015 in China. Girls aged 9–14 years were randomized to receive 2 doses at months 0 and 6(n=301) or 3 doses at months 0, 1 and 6(n=304). Girls aged 15–17 years(n=149) and women aged 18–26 years(n=225) received 3 doses. The objectives included noninferiority analysis of the IgG geometric mean concentration(GMC) ratio(95% CI, lower bound0.5) to HPV-16 and HPV-18 at month 7 in girls compared with women. In the per-protocol set, the GMC ratio of IgG was noninferior for girls aged 9–17 years receiving 3 doses compared with women(1.76(95% CI, 1.56, 1.99) for HPV-16 and 1.93(95% CI, 1.69, 2.21) for HPV-18) and noninferior for girls aged 9–14 years receiving 2 doses compared with women(1.45(95% CI, 1.25, 1.62) for HPV-16 and 1.17(95% CI,1.02, 1.33) for HPV-18). Noninferiority was also demonstrated for neutralizing antibodies. The immunogenicity of the HPV vaccine in girls receiving 3 or 2 doses was noninferior compared with that in young adult women.