Unconditional Exact Tests for Equivalence or Noninferiority for Paired Binary Endpoints

Problems of establishing equivalence or noninferiority between two medical diagnostic procedures involve comparisons of the response rates between correlated proportions. When the sample size is small, the asymptotic tests may not be reliable. This article proposes an unconditional exact test procedure to assess equivalence or noninferiority. Two statistics, a sample-based test statistic and a restricted maximum likelihood estimation (RMLE)-based test statistic, to define the rejection region of the exact test are considered. We show the p-value of the proposed unconditional exact tests can be attained at the boundary point of the null hypothesis. Assessment of equivalence is often based on a comparison of the confidence limits with the equivalence limits. We also derive the unconditional exact confidence intervals on the difference of the two proportion means for the two test statistics. A typical data set of comparing two diagnostic procedures is analyzed using the proposed unconditional exact and asymptotic methods. The p-value from the unconditional exact tests is generally larger than the p-value from the asymptotic tests. In other words, an exact confidence interval is generally wider than the confidence interval obtained from an asymptotic test.     

Designs of superiority and noninferiority trials for binary responses are noninterchangeable

To better understand the design of noninferiority trials for binary data, we identify analogies and contrasts between this and the more familiar superiority trial design. We restrict attention to the problem of detecting a difference between experimental and control response rates in the setting where there is no difference (piE - piC = 0) under the noninferiority alternative hypothesis and under the superiority null, and a matching difference between groups under the complementary hypotheses (/piE - piC/ = delta). Our derivation of the constrained maximum likelihood estimates (MLEs) reveals that superiority and noninferiority trials have different nuisance parameters--the marginal response rate and the control-group response rate, respectively. Our empirical results show that when individuals are allocated to treatment groups in the ratio that minimizes the overall sample size, balanced allocation is optimal only for superiority trials when the error rates are equal; otherwise imbalanced allocation is optimal. Different allocation ratios between trial types lead to different variances, and thus to different sample sizes. Finally, since the value of the marginal response rate--a design parameter in noninferiority trials--typically cannot be obtained from preliminary or published studies, we suggest a means of identifying a value that can be used. We conclude that full documentation of the design of a trial requires specification not only of the design parameters but also of the allocation ratio and the nuisance parameter, the value of which is not obvious under unequal allocation.     

A Novel Approach to Delayed-Start Analyses for Demonstrating Disease-Modifying Effects in Alzheimer’s Disease

One method for demonstrating disease modification is a delayed-start design, consisting of a placebo-controlled period followed by a delayed-start period wherein all patients receive active treatment. To address methodological issues in previous delayed-start approaches, we propose a new method that is robust across conditions of drug effect, discontinuation rates, and missing data mechanisms. We propose a modeling approach and test procedure to test the hypothesis of noninferiority, comparing the treatment difference at the end of the delayed-start period with that at the end of the placebo-controlled period. We conducted simulations to identify the optimal noninferiority testing procedure to ensure the method was robust across scenarios and assumptions, and to evaluate the appropriate modeling approach for analyzing the delayed-start period. We then applied this methodology to Phase 3 solanezumab clinical trial data for mild Alzheimer’s disease patients. Simulation results showed a testing procedure using a proportional noninferiority margin was robust for detecting disease-modifying effects; conditions of high and moderate discontinuations; and with various missing data mechanisms. Using all data from all randomized patients in a single model over both the placebo-controlled and delayed-start study periods demonstrated good statistical performance. In analysis of solanezumab data using this methodology, the noninferiority criterion was met, indicating the treatment difference at the end of the placebo-controlled studies was preserved at the end of the delayed-start period within a pre-defined margin. The proposed noninferiority method for delayed-start analysis controls Type I error rate well and addresses many challenges posed by previous approaches. Delayed-start studies employing the proposed analysis approach could be used to provide evidence of a disease-modifying effect. This method has been communicated with FDA and has been successfully applied to actual clinical trial data accrued from the Phase 3 clinical trials of solanezumab.     

Sample size determination for establishing equivalence/noninferiority via ratio of two proportions in matched-pair design

In this article, we propose approximate sample size formulas for establishing equivalence or noninferiority of two treatments in match-pairs design. Using the ratio of two proportions as the equivalence measure, we derive sample size formulas based on a score statistic for two types of analyses: hypothesis testing and confidence interval estimation. Depending on the purpose of a study, these formulas can be used to provide a sample size estimate that guarantees a prespecified power of a hypothesis test at a certain significance level or controls the width of a confidence interval with a certain confidence level. Our empirical results confirm that these score methods are reliable in terms of true size, coverage probability, and skewness. A liver scan detection study is used to illustrate the proposed methods.     

Testing a molecular clock without an outgroup: derivations of induced priors on branch-length restrictions in a Bayesian framework

We propose a Bayesian method for testing molecular clock hypotheses for use with aligned sequence data from multiple taxa. Our method utilizes a nonreversible nucleotide substitution model to avoid the necessity of specifying either a known tree relating the taxa or an outgroup for rooting the tree. We employ reversible jump Markov chain Monte Carlo to sample from the posterior distribution of the phylogenetic model parameters and conduct hypothesis testing using Bayes factors, the ratio of the posterior to prior odds of competing models. Here, the Bayes factors reflect the relative support of the sequence data for equal rates of evolutionary change between taxa versus unequal rates, averaged over all possible phylogenetic parameters, including the tree and root position. As the molecular clock model is a restriction of the more general unequal rates model, we use the Savage-Dickey ratio to estimate the Bayes factors. The Savage-Dickey ratio provides a convenient approach to calculating Bayes factors in favor of sharp hypotheses. Critical to calculating the Savage-Dickey ratio is a determination of the prior induced on the modeling restrictions. We demonstrate our method on a well-studied mtDNA sequence data set consisting of nine primates. We find strong support against a global molecular clock, but do find support for a local clock among the anthropoids. We provide mathematical derivations of the induced priors on branch length restrictions assuming equally likely trees. These derivations also have more general applicability to the examination of prior assumptions in Bayesian phylogenetics.     

Statistical methods for the analysis of two-arm non-inferiority trials with binary outcomes

The aim of this contribution is to give an overview of approaches to testing for non-inferiority of one out of two binomial distributions as compared to the other in settings involving independent samples (the paired samples case is not considered here but the major conclusions and recommendations can be shown to hold for both sampling schemes). In principle, there is an infinite number of different ways of defining (one-sided) equivalence in any multiparameter setting. In the binomial two-sample problem, the following three choices of a measure of dissimilarity between the underlying distributions are of major importance for real applications: the odds ratio (OR), the relative risk (RR), and the difference (DEL) of both binomial parameters. It is shown that for all three possibilities of formulating the hypotheses of a non-inferiority problem concerning two binomial proportions, reasonable testing procedures providing exact control over the type-I error risk are available. As a particularly useful and versatile way of handling mathematically nonnatural parametrizations like RR and DELTA, the approach through Bayesian posterior probabilities of hypotheses with respect to some non-informative reference prior has much to recommend it. In order to ensure that the corresponding testing procedure be valid in the classical, i.e. frequentist sense, it suffices to use straightforward computational techniques yielding suitably corrected nominal significance levels. In view of the availability of testing procedures with satisfactory properties for all parametrizations of main practical interest, the discussion of the pros and cons of these methods has to focus on the question of which of the underlying measures of dissimilarity should be preferred on grounds of logic and intuition. It is argued that the OR clearly merits to be given preference also with regard to this latter kind of criteria since the non-inferiority hypotheses defined in terms of the other parametric functions are bounded by lines which cross the boundaries of the parameter space. From this fact, we conclude that the exact Fisher type test for one-sided equivalence provides the most reasonable approach to the confirmatory analysis of non-inferiority trials involving two independent samples of binary data. The marked conservatism of the nonrandomized version of this test can largely be removed by using a suitably increased nominal significance level (depending, in addition to the target level, on the sample sizes and the equivalence margin), or by replacing it with a Bayesian test for non-inferiority with respect to the odds ratio.     

Testing hypotheses in case-control studies--equivalence of Mantel-Haenszel statistics and logit score tests.

The two approaches in common use for the analysis of case-control studies are cross-classification by confounding variables, and modeling the logarithm of the odds ratio as a function of exposure and confounding variables. We show here that score statistics derived from the likelihood function in the latter approach are identical to the Mantel-Haenszel test statistics appropriate for the former approach. This identity holds in the most general situation considered, testing for marginal homogeneity in mK tables. This equivalence is demonstrated by a permutational argument which leads to a general likelihood expression in which the exposure variable may be a vector of discrete and/or continuous variables and in which more than two comparison groups may be considered. This likelihood can be used in analyzing studies in which there are multiple controls for each case or in which several disease categories are being compared. The possibility of including continuous variables makes this likelihood useful in situations that cannot be treated using the Mantel-Haenszel cross-classification approach.     

Sample size calculations for noninferiority trials with Poisson distributed count data

Clinical trials with Poisson distributed count data as the primary outcome are common in various medical areas such as relapse counts in multiple sclerosis trials or the number of attacks in trials for the treatment of migraine. In this article, we present approximate sample size formulae for testing noninferiority using asymptotic tests which are based on restricted or unrestricted maximum likelihood estimators of the Poisson rates. The Poisson outcomes are allowed to be observed for unequal follow‐up schemes, and both the situations that the noninferiority margin is expressed in terms of the difference and the ratio are considered. The exact type I error rates and powers of these tests are evaluated and the accuracy of the approximate sample size formulae is examined. The test statistic using the restricted maximum likelihood estimators (for the difference test problem) and the test statistic that is based on the logarithmic transformation and employs the maximum likelihood estimators (for the ratio test problem) show favorable type I error control and can be recommended for practical application. The approximate sample size formulae show high accuracy even for small sample sizes and provide power values identical or close to the aspired ones. The methods are illustrated by a clinical trial example from anesthesia.     

Methods for assessing noninferiority with censored data

In this paper we present the existing approaches to the problem of showing noninferiority with randomly right censored data. The main focus is on the choice of the discrepancy measure which is used to define the deviation from the classical null hypothesis, i.e. the noninferiority margin. Most methods are based on certain parametric or semiparametric assumptions. In contrast, a new, completely nonparametric approach is suggested and discussed.     

Sequential parallel comparison design for “gold standard” noninferiority trials with a prespecified margin

Three‐arm noninferiority trials (involving an experimental treatment, a reference treatment, and a placebo)—called the “gold standard” noninferiority trials—are conducted in patients with mental disorders whenever feasible, but often fail to show superiority of the experimental treatment and/or the reference treatment over the placebo. One possible reason is that some of the patients receiving the placebo show apparent improvement in the clinical condition. An approach to addressing this problem is the use of the sequential parallel comparison design (SPCD). Nonetheless, the SPCD has not yet been discussed in relation to gold standard noninferiority trials. In this article, our aim was to develop a hypothesis‐testing method and its corresponding sample size calculation method for gold standard noninferiority trials with the SPCD. In a simulation, we show that the proposed hypothesis‐testing method achieves the nominal type I error rate and power and that the proposed sample size calculation method has adequate power accuracy.     

Bayesian design of noninferiority trials for medical devices using historical data

Summary We develop a new Bayesian approach of sample size determination (SSD) for the design of noninferiority clinical trials. We extend the fitting and sampling priors of Wang and Gelfand (2002, Statistical Science 17 , 193–208) to Bayesian SSD with a focus on controlling the type I error and power. Historical data are incorporated via a hierarchical modeling approach as well as the power prior approach of Ibrahim and Chen (2000, Statistical Science 15 , 46–60). Various properties of the proposed Bayesian SSD methodology are examined and a simulation‐based computational algorithm is developed. The proposed methodology is applied to the design of a noninferiority medical device clinical trial with historical data from previous trials.     

Modeling and E-M estimation of haplotype-specific relative risks from genotype data for a case-control study of unrelated individuals

The US National Cancer Institute has recently sponsored the formation of a Cohort Consortium (http://2002.cancer.gov/scpgenes.htm) to facilitate the pooling of data on very large numbers of people, concerning the effects of genes and environment on cancer incidence. One likely goal of these efforts will be generate a large population-based case-control series for which a number of candidate genes will be investigated using SNP haplotype as well as genotype analysis. The goal of this paper is to outline the issues involved in choosing a method of estimating haplotype-specific risk estimates for such data that is technically appropriate and yet attractive to epidemiologists who are already comfortable with odds ratios and logistic regression. Our interest is to develop and evaluate extensions of methods, based on haplotype imputation, that have been recently described (Schaid et al., Am J Hum Genet, 2002, and Zaykin et al., Hum Hered, 2002) as providing score tests of the null hypothesis of no effect of SNP haplotypes upon risk, which may be used for more complex tasks, such as providing confidence intervals, and tests of equivalence of haplotype-specific risks in two or more separate populations. In order to do so we (1) develop a cohort approach towards odds ratio analysis by expanding the E-M algorithm to provide maximum likelihood estimates of haplotype-specific odds ratios as well as genotype frequencies; (2) show how to correct the cohort approach, to give essentially unbiased estimates for population-based or nested case-control studies by incorporating the probability of selection as a case or control into the likelihood, based on a simplified model of case and control selection, and (3) finally, in an example data set (CYP17 and breast cancer, from the Multiethnic Cohort Study) we compare likelihood-based confidence interval estimates from the two methods with each other, and with the use of the single-imputation approach of Zaykin et al. applied under both null and alternative hypotheses. We conclude that so long as haplotypes are well predicted by SNP genotypes (we use the Rh2 criteria of Stram et al. [1]) the differences between the three methods are very small and in particular that the single imputation method may be expected to work extremely well.     

In situ gamma-spectrometry several years after deposition of radiocesium

A new method is introduced for deriving radiocesium soil contaminations and kerma rates in air from in situ gamma-ray spectrometric measurements. The approach makes use of additional information about gamma-ray attenuation given by the peak-to-valley ratio, which is the ratio of the count rates for primary and forward scattered photons. In situ measurements are evaluated by comparing the experimental data with the results of Monte Carlo simulations of photon transport and detector response. The influence of photons emitted by natural radionuclides on the calculation of the peak-to-valley ratio is carefully analysed. The new method has been applied to several post-Chernobyl measurements and the results agreed well with those of soil sampling.     

Saline-filled breast implant safety and efficacy: a multicenter retrospective review

Our center undertook an industry-funded, outcomes-based, multicenter, retrospective review to evaluate the safety and efficacy of saline-filled breast implants. Our review was part of a pre-market approval review process mandated by the U.S. Food and Drug Administration. The design of our review was modeled on a Plastic Surgery Educational Foundation outcomes study previously conducted by our center. For this study, several significant changes were made to our previous protocol, including improved patient tracking, stronger biostatistical support, and a mandatory 10-year minimum patient follow-up period. Physician-reported and patient-reported data on 450 patients with 882 saline-filled breast implants placed between January 1, 1980, and June 30, 1986, were obtained. Mean patient follow-up period was 13.0 years. Most implants (93.9 percent) were placed for breast augmentation. Seventy-four percent were placed in a submammary position; 25.6 percent, subpectorally; and 0.2 percent, subcutaneously. The overall complication rate was 20.2 percent. Reoperation for capsular contracture or implant deflation was necessary in 104 of 450 patients (23.1 percent). Deflation occurred in 73 implants (8.3 percent) and was underreported according to the physicians' record review alone. This deflation rate is higher than the 5.5 percent previously reported by our center. However, 26 of these 73 deflations (35.6 percent) occurred in a single cohort of patients at one center using Surgitek saline implants. If this cohort is excluded, the deflation rate drops to 5.8 percent, a figure more consistent with data published in the literature and found in our previous study. Of the 73 deflations, spontaneous deflation was reported for 50 (74.6 percent), and the remainder were iatrogenic. Actuarial survival of the non-Surgitek implants was 98.4 percent to 99.8 percent at 5 years and 96.9 percent to 98.9 percent at 10 years (95 percent confidence interval). Risk factors for implant deflation included the use of Surgitek saline-filled implants (odds ratio = 17.5, p < 0.01), use of Heyer-Schulte and Mentor model 1800 implants (odds ratio = 3.0, p < 0.01), and implant size greater than 450 cc (odds ratio = 1.01, p < 0.02). Risk factors for capsular contracture included submammary implant position (odds ratio = 2.05, p = 0.03) and implant size greater than 450 cc (odds ratio = 1.01, p < 0.01). Overall, satisfaction was high: 93 percent of patients were "satisfied" or "very satisfied" with their implants. As in our earlier study, risk factors for patient dissatisfaction were reconstruction after mastectomy (odds ratio = 7.6, p = 0.011), significant breast firmness (odds ratio = 6.2, p < 0.001), and patient desire for smaller implants (odds ratio = 3.0, p = 0.020). In conclusion, our review provides additional outcomes-based evidence that saline-filled breast implants remain a safe, effective alternative to silicone gel-filled models.     

Statistical methods in recent HIV noninferiority trials: reanalysis of 11 trials

Background

In recent years the “noninferiority” trial has emerged as the new standard design for HIV drug development among antiretroviral patients often with a primary endpoint based on the difference in success rates between the two treatment groups. Different statistical methods have been introduced to provide confidence intervals for that difference. The main objective is to investigate whether the choice of the statistical method changes the conclusion of the trials.

Methods

We presented 11 trials published in 2010 using a difference in proportions as the primary endpoint. In these trials, 5 different statistical methods have been used to estimate such confidence intervals. The five methods are described and applied to data from the 11 trials. The noninferiority of the new treatment is not demonstrated if the prespecified noninferiority margin it includes in the confidence interval of the treatment difference.

Results

Results indicated that confidence intervals can be quite different according to the method used. In many situations, however, conclusions of the trials are not altered because point estimates of the treatment difference were too far from the prespecified noninferiority margins. Nevertheless, in few trials the use of different statistical methods led to different conclusions. In particular the use of “exact” methods can be very confusing.

Conclusion

Statistical methods used to estimate confidence intervals in noninferiority trials have a strong impact on the conclusion of such trials.     

Occupational Injury Risk Assessment Using Injury Severity Odds Ratios: Male Falls from Heights in the Danish Construction Industry, 1993-1999

To allow for an improved targeted approach to occupational injury research and prevention, detailed risk analyses of fatal, serious, and minor occupational injuries were completed with a focus on male falls from heights in construction. Reported lost-time injuries in Denmark (1993 to 1999) were analyzed for proportions, relative rates, and an injury severity odds ratio to assess relative hazards and reporting. Different types of injury risks, such as elevation falls through surfaces, were identified by contrasting fatal, serious and minor injuries. Trade-specific analyses provided evidence that the carpentry-trade group merits increased attention, as this group has excessively high proportions, rates and hazards for falls from heights, compared to the entire construction industry. Age-specific analyses of workers aged 20 to 59 revealed that the rates of serious-injury falls from heights increase with increasing age. This relationship was inverted for elevation fall injuries from roof surfaces. A focus on construction and falls from heights is necessary not only in terms of fatal injuries, but also for serious injuries. The use of an injury severity odds ratio to assess relative hazards and reporting is a useful supplement to proportions and injury rates in contributing to more precise targeting of subgroups for primary injury prevention.     

Presenting the uncertainties of odds ratios using empirical-Bayes prediction intervals

Quantifying exposure-disease associations is a central issue in epidemiology. Researchers of a study often present an odds ratio (or a logarithm of odds ratio, logOR) estimate together with its confidence interval (CI), for each exposure they examined. Here the authors advocate using the empirical-Bayes-based 'prediction intervals' (PIs) to bound the uncertainty of logORs. The PI approach is applicable to a panel of factors believed to be exchangeable (no extra information, other than the data itself, is available to distinguish some logORs from the others). The authors demonstrate its use in a genetic epidemiological study on age-related macular degeneration (AMD). The proposed PIs can enjoy straightforward probabilistic interpretations--a 95% PI has a probability of 0.95 to encompass the true value, and the expected number of true values that are being encompassed is 0.95m for a total of m 95% PIs. The PI approach is theoretically more efficient (producing shorter intervals) than the traditional CI approach. In the AMD data, the average efficiency gain is 51.2%. The PI approach is advocated to present the uncertainties of many logORs in a study, for its straightforward probabilistic interpretations and higher efficiency while maintaining the nominal coverage probability.     

Fatal peptic ulcer complications and the use of non-steroidal anti-inflammatory drugs,aspirin, and corticosteroids.

Although non-steroidal anti-inflammatory drugs are known to cause peptic ulcer and its complications, controversy exists about the number of deaths from ulcer which are attributable to their use. A case-control study was therefore performed to determine whether prior use of non-steroidal and other anti-inflammatory compounds was associated with an increased case fatality rate from complications of peptic ulcer. Non-steroidal anti-inflammatory drugs were used by 39% of a series of 80 patients who had died from peptic ulcer complications and by 37% of 160 controls who were survivors matched for sex, age, ulcer site, and nature of complication (odds ratio 1.1; 95% confidence interval 0.6 to 2.1). Similarly, the rates of prior use of aspirin by cases and controls were almost identical (odds ratio 1.2; 95% confidence interval 0.5 to 1.9). Thus neither nonsteroidal anti-inflammatory drugs nor aspirin were associated with increased case fatality rates from peptic ulcer complications. In contrast, corticosteroids were associated with an increased mortality (odds ratio 4.2; 95% confidence interval 0.9 to 25.6). Although this increase in the estimated relative risk was not statistically significant, a review of the case records indicated that most deaths in steroid users were due to serious sepsis, indicating that there might be a causal association between use of the drugs and the mode of death.