Non-inferiority of new procedure to standard procedure in stratified matched-pair design

We consider the statistical testing for non-inferiority of a new treatment compared with the standard one under matched-pair setting in a stratified study or in several trials. A non-inferiority test based on the efficient scores and a Mantel-Haenszel (M-H) like procedure with restricted maximum likelihood estimators (RMLEs) of nuisance parameters and their corresponding sample size formulae are presented. We evaluate the above tests and the M-H type Wald test in level and power. The stratified score test is conservative and provides the best power. The M-H like procedure with RMLEs gives an accurate level. However, the Wald test is anti-conservative and we suggest caution when it is used. The unstratified score test is not biased but it is less powerful than the stratified score test when base-line probabilities related to strata are not the same. This investigation shows that the stratified score test possesses optimum statistical properties in testing non-inferiority. A common difference between two proportions across strata is the basic assumption of the stratified tests, we present appropriate tests to validate the assumption and related remarks.     

On testing simultaneously non-inferiority in two multiple primary endpoints and superiority in at least one of them

In a clinical trial with an active treatment and a placebo the situation may occur that two (or even more) primary endpoints may be necessary to describe the active treatment's benefit. The focus of our interest is a more specific situation with two primary endpoints in which superiority in one of them would suffice given that non-inferiority is observed in the other. Several proposals exist in the literature for dealing with this or similar problems, but prove insufficient or inadequate at a closer look (e.g. Bloch et al. (2001, 2006) or Tamhane and Logan (2002, 2004)). For example, we were unable to find a good reason why a bootstrap p-value for superiority should depend on the initially selected non-inferiority margins or on the initially selected type I error alpha. We propose a hierarchical three step procedure, where non-inferiority in both variables must be proven in the first step, superiority has to be shown by a bivariate test (e.g. Holm (1979), O'Brien (1984), Hochberg (1988), a bootstrap (Wang (1998)), or L?uter (1996)) in the second step, and then superiority in at least one variable has to be verified in the third step by a corresponding univariate test. All statistical tests are performed at the same one-sided significance level alpha. From the above mentioned bivariate superiority tests we preferred L?uter's SS test and the Holm procedure for the reason that these have been proven to control the type I error strictly, irrespective of the correlation structure among the primary variables and the sample size applied. A simulation study reveals that the performance regarding power of the bivariate test depends to a considerable degree on the correlation and on the magnitude of the expected effects of the two primary endpoints. Therefore, the recommendation of which test to choose depends on knowledge of the possible correlation between the two primary endpoints. In general, L?uter's SS procedure in step 2 shows the best overall properties, whereas Holm's procedure shows an advantage if both a positive correlation between the two variables and a considerable difference between their standardized effect sizes can be expected.     

Superiority inferences on individual endpoints following noninferiority testing in clinical trials

We consider the problem of drawing superiority inferences on individual endpoints following non-inferiority testing. R?hmel et al. (2006) pointed out this as an important problem which had not been addressed by the previous procedures that only tested for global superiority. R?hmel et al. objected to incorporating the non-inferiority tests in the assessment of the global superiority test by exploiting the relationship between the two, since the results of the latter test then depend on the non-inferiority margins specified for the former test. We argue that this is justified, besides the fact that it enhances the power of the global superiority test. We provide a closed testing formulation which generalizes the three-step procedure proposed by R?hmel et al. for two endpoints. For the global superiority test, R?hmel et al. suggest using the L?uter (1996) test which is modified to make it monotone. The resulting test not only is complicated to use, but the modification does not readily extend to more than two endpoints, and it is less powerful in general than several of its competitors. This is verified in a simulation study. Instead, we suggest applying the one-sided likelihood ratio test used by Perlman and Wu (2004) or the union-intersection t(max) test used by Tamhane and Logan (2004).     

Choice of delta: requirements and reality--results of a systematic review

An essential problem in planning clinical non-inferiority or equivalence studies is the specification of the 'irrelevant difference' (irrelevance margin; delta). This quantifies the amount of non-inferiority or difference, respectively, between a new test therapy and an established standard treatment which is to be considered as tolerable. In the past, most recommendations and guidelines for clinical non-inferiority and equivalence studies contained only general statements and formulations concerning the specification of delta. The current unsatisfactory situation was the reason for performing a systematic review of published clinical non-inferiority and equivalence studies. It was the aim to gain an overview on the irrelevance margins used in such studies, and on reasons for choosing the particular margins. For the sake of comparability, the irrelevance margins were converted into standardized differences and odds ratios. Overall, there were 332 non-inferiority or equivalence trials obtained by means of an extensive literature search. The results of the systematic review show that current requirements on the choice of delta and the reality of recent clinical non-inferiority and equivalence trials differ substantially. In about one half of the trials a difference of 0.5 standard deviations or more was regarded as 'irrelevant' explicitly or implicitly. Estimates of standard-placebo differences formed the basis of the irrelevance margin in less than every tenth trial. Reasons for this very low proportion might be (1) the possibly resulting very small irrelevance margins, and (2) unsolved problems of the requirements themselves. Overall, it seems that a more global definition of 'irrelevance' might be warranted.     

Blinded sample size reestimation in non-inferiority trials with binary endpoints

Sample size calculations in the planning of clinical trials depend on good estimates of the model parameters involved. When the estimates of these parameters have a high degree of uncertainty attached to them, it is advantageous to reestimate the sample size after an internal pilot study. For non-inferiority trials with binary outcome we compare the performance of Type I error rate and power between fixed-size designs and designs with sample size reestimation. The latter design shows itself to be effective in correcting sample size and power of the tests when misspecification of nuisance parameters occurs with the former design.     

Non-inferiority testing with a variable margin

There has been growing interest, when comparing an experimental treatment with an active control with respect to a binary outcome, in allowing the non-inferiority margin to depend on the unknown success rate in the control group. It does not seem universally recognized, however, that the statistical test should appropriately adjust for the uncertainty surrounding the non-inferiority margin. In this paper, we inspect a naive procedure that treats an "observed margin" as if it were fixed a priori, and explain why it might not be valid. We then derive a class of tests based on the delta method, including the Wald test and the score test, for a smooth margin. An alternative derivation is given for the asymptotic distribution of the likelihood ratio statistic, again for a smooth margin. We discuss the asymptotic behavior of these tests when applied to a piecewise smooth margin. A simple condition on the margin function is given which allows the likelihood ratio test to carry over to a piecewise smooth margin using the same critical value as for a smooth margin. Simulation experiments are conducted, under a smooth margin and a piecewise linear margin, to evaluate the finite-sample performance of the asymptotic tests studied.     

Re‐Formulating Non‐inferiority Trials as Superiority Trials: The Case of Binary Outcomes

Non‐inferiority trials are conducted for a variety of reasons including to show that a new treatment has a negligible reduction in efficacy or safety when compared to the current standard treatment, or a more complex setting of showing that a new treatment has a negligible reduction in efficacy when compared to the current standard yet is superior in terms of other treatment characteristics. The latter reason for conducting a non‐inferiority trial presents the challenge of deciding on a balance between a suitable reduction in efficacy, known as the non‐inferiority margin, in return for a gain in other important treatment characteristics/findings. It would be ideal to alleviate the dilemma on the choice of margin in this setting by reverting to a traditional superiority trial design where a single p ‐value for superiority of both the most important endpoint (efficacy) and the most important finding (treatment characteristic) is provided. We discuss how this can be done using the information‐preserving composite endpoint (IPCE) approach and consider binary outcome cases in which the combination of efficacy and treatment characteristics, but not one itself, paints a clear picture that the novel treatment is superior to the active control (© 2009 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Statistical methods for the analysis of two-arm non-inferiority trials with binary outcomes

The aim of this contribution is to give an overview of approaches to testing for non-inferiority of one out of two binomial distributions as compared to the other in settings involving independent samples (the paired samples case is not considered here but the major conclusions and recommendations can be shown to hold for both sampling schemes). In principle, there is an infinite number of different ways of defining (one-sided) equivalence in any multiparameter setting. In the binomial two-sample problem, the following three choices of a measure of dissimilarity between the underlying distributions are of major importance for real applications: the odds ratio (OR), the relative risk (RR), and the difference (DEL) of both binomial parameters. It is shown that for all three possibilities of formulating the hypotheses of a non-inferiority problem concerning two binomial proportions, reasonable testing procedures providing exact control over the type-I error risk are available. As a particularly useful and versatile way of handling mathematically nonnatural parametrizations like RR and DELTA, the approach through Bayesian posterior probabilities of hypotheses with respect to some non-informative reference prior has much to recommend it. In order to ensure that the corresponding testing procedure be valid in the classical, i.e. frequentist sense, it suffices to use straightforward computational techniques yielding suitably corrected nominal significance levels. In view of the availability of testing procedures with satisfactory properties for all parametrizations of main practical interest, the discussion of the pros and cons of these methods has to focus on the question of which of the underlying measures of dissimilarity should be preferred on grounds of logic and intuition. It is argued that the OR clearly merits to be given preference also with regard to this latter kind of criteria since the non-inferiority hypotheses defined in terms of the other parametric functions are bounded by lines which cross the boundaries of the parameter space. From this fact, we conclude that the exact Fisher type test for one-sided equivalence provides the most reasonable approach to the confirmatory analysis of non-inferiority trials involving two independent samples of binary data. The marked conservatism of the nonrandomized version of this test can largely be removed by using a suitably increased nominal significance level (depending, in addition to the target level, on the sample sizes and the equivalence margin), or by replacing it with a Bayesian test for non-inferiority with respect to the odds ratio.     

The Use of Two‐Way Linear Mixed Models in Multitreatment Meta‐Analysis

Summary Meta‐analysis summarizes the results of a series of trials. When more than two treatments are included in the trials and when the set of treatments tested differs between trials, the combination of results across trials requires some care. Several methods have been proposed for this purpose, which feature under different labels, such as network meta‐analysis or mixed treatment comparisons. Two types of linear mixed model can be used for meta‐analysis. The one expresses the expected outcome of treatments as a contrast to a baseline treatment. The other uses a classical two‐way linear predictor with main effects for treatment and trial. In this article, we compare both types of model and explore under which conditions they give equivalent results. We illustrate practical advantages of the two‐way model using two published datasets. In particular, it is shown that between‐trial heterogeneity as well as inconsistency between different types of trial is straightforward to account for.     

Sample Size Estimation for Non-Inferiority Trials: Frequentist Approach versus Decision Theory Approach

Background

Non-inferiority trials are performed when the main therapeutic effect of the new therapy is expected to be not unacceptably worse than that of the standard therapy, and the new therapy is expected to have advantages over the standard therapy in costs or other (health) consequences. These advantages however are not included in the classic frequentist approach of sample size calculation for non-inferiority trials. In contrast, the decision theory approach of sample size calculation does include these factors. The objective of this study is to compare the conceptual and practical aspects of the frequentist approach and decision theory approach of sample size calculation for non-inferiority trials, thereby demonstrating that the decision theory approach is more appropriate for sample size calculation of non-inferiority trials.

Methods

The frequentist approach and decision theory approach of sample size calculation for non-inferiority trials are compared and applied to a case of a non-inferiority trial on individually tailored duration of elastic compression stocking therapy compared to two years elastic compression stocking therapy for the prevention of post thrombotic syndrome after deep vein thrombosis.

Results

The two approaches differ substantially in conceptual background, analytical approach, and input requirements. The sample size calculated according to the frequentist approach yielded 788 patients, using a power of 80% and a one-sided significance level of 5%. The decision theory approach indicated that the optimal sample size was 500 patients, with a net value of €92 million.

Conclusions

This study demonstrates and explains the differences between the classic frequentist approach and the decision theory approach of sample size calculation for non-inferiority trials. We argue that the decision theory approach of sample size estimation is most suitable for sample size calculation of non-inferiority trials.     

Testing superiority and non-inferiority hypotheses in active controlled clinical trials

Switching between testing for superiority and non-inferiority has been an important statistical issue in the design and analysis of active controlled clinical trial. In practice, it is often conducted with a two-stage testing procedure. It has been assumed that there is no type I error rate adjustment required when either switching to test for non-inferiority once the data fail to support the superiority claim or switching to test for superiority once the null hypothesis of non-inferiority is rejected with a pre-specified non-inferiority margin in a generalized historical control approach. However, when using a cross-trial comparison approach for non-inferiority testing, controlling the type I error rate sometimes becomes an issue with the conventional two-stage procedure. We propose to adopt a single-stage simultaneous testing concept as proposed by Ng (2003) to test both non-inferiority and superiority hypotheses simultaneously. The proposed procedure is based on Fieller's confidence interval procedure as proposed by Hauschke et al. (1999).     

A regulatory perspective on choice of margin and statistical inference issue in non-inferiority trials

Without a placebo arm, any non-inferiority inference involving assessment of the placebo effect under the active control trial setting is difficult. The statistical risk for falsely concluding non-inferiority cannot be evaluated unless the constancy assumption approximately holds that the effect of the active control under the historical trial setting where the control effect can be assessed carries to the noninferiority trial setting. The constancy assumption cannot be checked because of missing the placebo arm in the non-inferiority trial. Depending on how serious the violation of the assumption is thought to be, one may need to seek an alternative design strategy that includes a cushion for a very conservative non-inferiority analysis or shows superiority of the experimental treatment over the control. Determination of the non-inferiority margin depends on what objective the non-inferiority analysis is intended to achieve. The margin can be a fixed margin or a margin functionally defined. Between-trial differences always exist and need to be properly considered.     

Foraging ecology of insectivorous birds in a mixed forest of Hong Kong

In a mixed forest in Hong Kong, the foraging ecology of nine species of insectivorous birds was studied. Leaves and branches of diameters smaller than 2 cm were the most frequently searched microhabitats. Gleaning was the most frequently used foraging method. Apart from Blue-winged Minla and Japanese White-eye, no two species used similar proportions of vertical strata and microhabitats at the same time. Bird species using similar proportion of microhabitats were foraging in different proportion of vertical strata. This niche segregation enabled the bird species to coexist in the same habitat. Velvet-fronted Nuthatch differed from other species by its more frequent use of branches of diameters larger than 2 cm and tree trunks. This might be one of the reasons why this exotic species successfully established a breeding population in the study area.     

Variations of the germinable soil seed bank along the altitudinal gradient in the northwestern Red Sea region

In a mixed forest in Hong Kong, the foraging ecology of nine species of insectivorous birds was studied. Leaves and branches of diameters smaller than 2 cm were the most frequently searched microhabitats. Gleaning was the most frequently used foraging method. Apart from Blue-winged Minla and Japanese White-eye, no two species used similar proportions of vertical strata and microhabitats at the same time. Bird species using similar proportion of microhabitats were foraging in different proportion of vertical strata. This niche segregation enabled the bird species to coexist in the same habitat. Velvet-fronted Nuthatch differed from other species by its more frequent use of branches of diameters larger than 2 cm and tree trunks. This might be one of the reasons why this exotic species successfully established a breeding population in the study area.     

Foraging ecology of insectivorous birds in a mixed forest of Hong Kong

In a mixed forest in Hong Kong, the foraging ecology of nine species of insectivorous birds was studied. Leaves and branches of diameters smaller than 2 cm were the most frequently searched microhabitats. Gleaning was the most frequently used foraging method. Apart from Blue-winged Minla and Japanese White-eye, no two species used similar proportions of vertical strata and microhabitats at the same time. Bird species using similar proportion of microhabitats were foraging in different proportion of vertical strata. This niche segregation enabled the bird species to coexist in the same habitat. Velvet-fronted Nuthatch differed from other species by its more frequent use of branches of diameters larger than 2 cm and tree trunks. This might be one of the reasons why this exotic species successfully established a breeding population in the study area.     

A note on the covariance of the Mantel-Haenszel log-odds-ratio estimator and the sample marginal rates

A simple technique, developed in Phillips (unpublished Ph.D. dissertation, University of Windsor, Windsor, Ontario, 1987), is used to approximate cov(theta MH, pi), i = 1, 2, where theta MH is the Mantel-Haenszel log-odds-ratio estimator for a 2 x 2 x K table and the pi are the sample marginal proportions. These results are then applied to obtain an approximate variance estimate of an adjusted risk difference based on the Mantel-Haenszel odds-ratio estimator.     

A group sequential type design for three‐arm non‐inferiority trials with binary endpoints

The three‐arm design with a test treatment, an active control and a placebo group is the gold standard design for non‐inferiority trials if it is ethically justifiable to expose patients to placebo. In this paper, we first use the closed testing principle to establish the hierarchical testing procedure for the multiple comparisons involved in the three‐arm design. For the effect preservation test we derive the explicit formula for the optimal allocation ratios. We propose a group sequential type design, which naturally accommodates the hierarchical testing procedure. Under this proposed design, Monte Carlo simulations are conducted to evaluate the performance of the sequential effect preservation test when the variance of the test statistic is estimated based on the restricted maximum likelihood estimators of the response rates under the null hypothesis. When there are uncertainties for the placebo response rate, the proposed design demonstrates better operating characteristics than the fixed sample design.     

Quantifying sequence proportions in a DNA‐based diet study using Ion Torrent amplicon sequencing: which counts count?

A goal of many environmental DNA barcoding studies is to infer quantitative information about relative abundances of different taxa based on sequence read proportions generated by high‐throughput sequencing. However, potential biases associated with this approach are only beginning to be examined. We sequenced DNA amplified from faeces (scats) of captive harbour seals (Phoca vitulina) to investigate whether sequence counts could be used to quantify the seals’ diet. Seals were fed fish in fixed proportions, a chordate‐specific mitochondrial 16S marker was amplified from scat DNA and amplicons sequenced using an Ion Torrent PGM?. For a given set of bioinformatic parameters, there was generally low variability between scat samples in proportions of prey species sequences recovered. However, proportions varied substantially depending on sequencing direction, level of quality filtering (due to differences in sequence quality between species) and minimum read length considered. Short primer tags used to identify individual samples also influenced species proportions. In addition, there were complex interactions between factors; for example, the effect of quality filtering was influenced by the primer tag and sequencing direction. Resequencing of a subset of samples revealed some, but not all, biases were consistent between runs. Less stringent data filtering (based on quality scores or read length) generally produced more consistent proportional data, but overall proportions of sequences were very different than dietary mass proportions, indicating additional technical or biological biases are present. Our findings highlight that quantitative interpretations of sequence proportions generated via high‐throughput sequencing will require careful experimental design and thoughtful data analysis.     

Exact Tests using Two Correlated Binomial Variables in Contemporary Cancer Clinical Trials

New therapy strategies for the treatment of cancer are rapidly emerging because of recent technology advances in genetics and molecular biology. Although newer targeted therapies can improve survival without measurable changes in tumor size, clinical trial conduct has remained nearly unchanged. When potentially efficacious therapies are tested, current clinical trial design and analysis methods may not be suitable for detecting therapeutic effects. We propose an exact method with respect to testing cytostatic cancer treatment using correlated bivariate binomial random variables to simultaneously assess two primary outcomes. The method is easy to implement. It does not increase the sample size over that of the univariate exact test and in most cases reduces the sample size required. Sample size calculations are provided for selected designs.     

Aggression and nest spacing in single and mixed species groups of seabirds

When heterospecific seabirds are part of a nesting colony, there may be less opportunity for conspecifics to come in direct contact with each other, resulting in lower intraspecific aggressiveness. To determine if individuals spend less time in aggressive behavior when nesting in conspecific rather than heterospecific groups, we compared the behavior of black skimmers (Rhynchops niger) nesting with gull-billed terns (Sterna nilotica) in three mixed species subcolonies to those of black skimmers in three single species subcolonies. In contrast to our predictions, black skimmers spent significantly less time in aggressive behaviors when nesting in single species subcolonies than when nesting with heterospecifics. Although skimmers in mixed species subcolonies tended to have more aggressive interactions with skimmers than terns, this may be a function of subcolony composition; the proportions of aggressive interactions with conspecifics were similar to the proportions of conspecifics in each subcolony. However, within the mixed species subcolonies, skimmers that nested nearer to terns were involved in aggressive interactions significantly less than skimmers that nested closer to conspecifics. Also, skimmers nested closer to their nearest neighbor when it was a gull-billed tern than when it was another skimmer. Regardless of which species they nested closest to, skimmers were more aggressive towards other skimmers than to terns within the mixed species subcolonies. Distance to nearest neighbor's nest did not differ significantly between the colony types, and did not seem to influence the duration of aggressive activity in the single species subcolonies. In the mixed species subcolonies, however, the time spent in aggressive behavior increased as the distance to nearest neighbor increased. It appears that of the several benefits that have been proposed of mixed species colonies, reduced time spent in conspecific aggression is not among them. However, within a mixed species colony, an individual can reduce time spent in aggressive interactions by nesting near heterospecifics. Received: 16 September 1996 / Accepted: 24 February 1997