Frequentist performance of Bayesian confidence intervals for comparing proportions in 2 x 2 contingency tables

This article investigates the performance, in a frequentist sense, of Bayesian confidence intervals (CIs) for the difference of proportions, relative risk, and odds ratio in 2 x 2 contingency tables. We consider beta priors, logit-normal priors, and related correlated priors for the two binomial parameters. The goal was to analyze whether certain settings for prior parameters tend to provide good coverage performance regardless of the true association parameter values. For the relative risk and odds ratio, we recommend tail intervals over highest posterior density (HPD) intervals, for invariance reasons. To protect against potentially very poor coverage probabilities when the effect is large, it is best to use a diffuse prior, and we recommend the Jeffreys prior. Otherwise, with relatively small samples, Bayesian CIs using more informative (even uniform) priors tend to have poorer performance than the frequentist CIs based on inverting score tests, which perform uniformly quite well for these parameters.     

On small-sample confidence intervals for parameters in discrete distributions

The traditional definition of a confidence interval requires the coverage probability at any value of the parameter to be at least the nominal confidence level. In constructing such intervals for parameters in discrete distributions, less conservative behavior results from inverting a single two-sided test than inverting two separate one-sided tests of half the nominal level each. We illustrate for a variety of discrete problems, including interval estimation of a binomial parameter, the difference and the ratio of two binomial parameters for independent samples, and the odds ratio.     

Improved exact confidence intervals for the odds ratio in two independent binomial samples

For two independent binomial samples, the usual exact confidence interval for the odds ratio based on the conditional approach can be very conservative. Recently, Agresti and Min (2002) showed that the unconditional intervals are preferable to conditional intervals with small sample sizes. We use the unconditional approach to obtain a modified interval, which has shorter length, and its coverage probability is closer to and at least the nominal confidence coefficient.     

Test-based exact confidence intervals for the difference of two binomial proportions

Confidence intervals are often provided to estimate a treatment difference. When the sample size is small, as is typical in early phases of clinical trials, confidence intervals based on large sample approximations may not be reliable. In this report, we propose test-based methods of constructing exact confidence intervals for the difference in two binomial proportions. These exact confidence intervals are obtained from the unconditional distribution of two binomial responses, and they guarantee the level of coverage. We compare the performance of these confidence intervals to ones based on the observed difference alone. We show that a large improvement can be achieved by using the standardized Z test with a constrained maximum likelihood estimate of the variance.     

Simultaneous confidence intervals for comparing biodiversity indices estimated from overdispersed count data

Diversity indices might be used to assess the impact of treatments on the relative abundance patterns in species communities. When several treatments are to be compared, simultaneous confidence intervals for the differences of diversity indices between treatments may be used. The simultaneous confidence interval methods described until now are either constructed or validated under the assumption of the multinomial distribution for the abundance counts. Motivated by four example data sets with background in agricultural and marine ecology, we focus on the situation when available replications show that the count data exhibit extra‐multinomial variability. Based on simulated overdispersed count data, we compare previously proposed methods assuming multinomial distribution, a method assuming normal distribution for the replicated observations of the diversity indices and three different bootstrap methods to construct simultaneous confidence intervals for multiple differences of Simpson and Shannon diversity indices. The focus of the simulation study is on comparisons to a control group. The severe failure of asymptotic multinomial methods in overdispersed settings is illustrated. Among the bootstrap methods, the widely known Westfall–Young method performs best for the Simpson index, while for the Shannon index, two methods based on stratified bootstrap and summed count data are preferable. The methods application is illustrated for an example.     

Simultaneous confidence intervals from randomization tests with application in testing bioequivalence with multiple endpoints

We propose a method to construct simultaneous confidence intervals for a parameter vector from inverting a series of randomization tests (RT). The randomization tests are facilitated by an efficient multivariate Robbins–Monro procedure that takes the correlation information of all components into account. The estimation method does not require any distributional assumption of the population other than the existence of the second moments. The resulting simultaneous confidence intervals are not necessarily symmetric about the point estimate of the parameter vector but possess the property of equal tails in all dimensions. In particular, we present the constructing the mean vector of one population and the difference between two mean vectors of two populations. Extensive simulation is conducted to show numerical comparison with four methods. We illustrate the application of the proposed method to test bioequivalence with multiple endpoints on some real data.     

Symmetric tests and confidence intervals for survival probabilities and quantiles of censored survival data

We describe existing tests and introduce two new tests concerning the value of a survival function. These tests may be used to construct a confidence interval for the survival probability at a given time or for a quantile of the survival distribution. Simulation studies show that error rates can differ substantially from their nominal values, particularly at survival probabilities close to zero or one. We recommend our new constrained bootstrap test for its good overall performance.     

Local multiplicity adjustment for the spatial scan statistic using the Gumbel distribution

The spatial scan statistic is an important and widely used tool for cluster detection. It is based on the simultaneous evaluation of the statistical significance of the maximum likelihood ratio test statistic over a large collection of potential clusters. In most cluster detection problems, there is variation in the extent of local multiplicity across the study region. For example, using a fixed maximum geographic radius for clusters, urban areas typically have many overlapping potential clusters, whereas rural areas have relatively few. The spatial scan statistic does not account for local multiplicity variation. We describe a previously proposed local multiplicity adjustment based on a nested Bonferroni correction and propose a novel adjustment based on a Gumbel distribution approximation to the distribution of a local scan statistic. We compare the performance of all three statistics in terms of power and a novel unbiased cluster detection criterion. These methods are then applied to the well-known New York leukemia dataset and a Wisconsin breast cancer incidence dataset.     

Mid‐P confidence intervals for group testing based on the total number of positive groups

In the estimation of proportions by group testing, unequal sized groups results in an ambiguous ordering of the sample space, which complicates the construction of exact confidence intervals. The total number of positive groups is shown to be a suitable statistic for ordering outcomes, provided its ties are broken by the MLE. We propose an interval estimation method based on this quantity, with a mid‐P correction. Coverage is evaluated using group testing problems in plant disease assessment and virus transmission by insect vectors. The proposed method provides good coverage in a range of situations, and compares favorably with existing exact methods.     

A mixed approach for proving non-inferiority in clinical trials with binary endpoints

When a new treatment is compared to an established one in a randomized clinical trial, it is standard practice to statistically test for non-inferiority rather than for superiority. When the endpoint is binary, one usually compares two treatments using either an odds-ratio or a difference of proportions. In this paper, we propose a mixed approach which uses both concepts. One first defines the non-inferiority margin using an odds-ratio and one ultimately proves non-inferiority statistically using a difference of proportions. The mixed approach is shown to be more powerful than the conventional odds-ratio approach when the efficacy of the established treatment is known (with good precision) and high (e.g. with more than 56% of success). The gain of power achieved may lead in turn to a substantial reduction in the sample size needed to prove non-inferiority. The mixed approach can be generalized to ordinal endpoints.     

On logit confidence intervals for the odds ratio with small samples

Unless the true association is very strong, simple large-sample confidence intervals for the odds ratio based on the delta method perform well even for small samples. Such intervals include the Woolf logit interval and the related Gart interval based on adding .5 before computing the log odds ratio estimate and its standard error. The Gart interval smooths the observed counts toward the model of equiprobability, but one obtains better coverage probabilities by smoothing toward the independence model and by extending the interval in the appropriate direction when a cell count is zero.     

Correction: Simultaneous confidence regions for closed tests,including Holm‐, Hochberg‐, and Hommel‐related procedures

Algorithm 1 in Guilbaud (2012, p. 327) in Biometrical Journal (DOI: 10.1002/bimj.201100123 ) reproduced a recently detected index error in a theorem concerning a shortcut for rejection decisions for certain multiple‐testing procedures as it was stated in Bernhard et al. (2004, p. 8) in Statistical Papers (DOI: 10.1007/BF02778266 ). This short article provides: (i) the correction to be made to Algorithm 1 and (ii) a brief discussion of the consequences. Although the theoretical developments in Guilbaud (2012) are not affected, the numerical illustrations in Section 7 are affected. A corrected version of that section is given in the Supporting Information.     

A note on permutation tests for variance components in multilevel generalized linear mixed models

In many applications of generalized linear mixed models to multilevel data, it is of interest to test whether a random effects variance component is zero. It is well known that the usual asymptotic chi-square distribution of the likelihood ratio and score statistics under the null does not necessarily hold. In this note we propose a permutation test, based on randomly permuting the indices associated with a given level of the model, that has the correct Type I error rate under the null. Results from a simulation study suggest that it is more powerful than tests based on mixtures of chi-square distributions. The proposed test is illustrated using data on the familial aggregation of sleep disturbance.