Approximate interval estimation of the ratio of binomial parameters: a review and corrections for skewness

Various methods for finding confidence intervals for the ratio of binomial parameters are reviewed and evaluated numerically. It is found that the method based on likelihood scores (Koopman, 1984, Biometrics 40, 513-517; Miettinen and Nurminen, 1985, Statistics in Medicine 4, 213-226) performs best in achieving the nominal confidence coefficient, but it may distribute the tail probabilities quite disparately. Using general theory of Bartlett (1953, Biometrika 40, 306-317; 1955, Biometrika 42, 201-203), we correct this method for asymptotic skewness. Following Gart (1985, Biometrika 72, 673-677), we extend this correction to the case of estimating the common ratio in a series of two-by-two tables. Computing algorithms are given and applied to numerical examples. Parallel methods for the odds ratio and the ratio of Poisson parameters are noted.     

Some Remarks on the Estimation of the Ratio of the Expectation Values of a Two-dimensional Normal Random Variable (Correction of the Theorem of Milliken)

It is possible to show that the theorem of Fieller yields an exact confidence region derivable from the likelihood ratio test. The statement of Milliken (1982) that this confidence region is conservative is refuted. Bounds for the conditional confidence coefficient under the condition that the confidence region is a “meaningful” confidence interval are given.     

Evaluation of exact and asymptotic interval estimators in logistic analysis of matched case-control studies.

We compare six methods for constructing confidence intervals for a single parameter in stratified logistic regression. Three of these are based on inversion of standard asymptotic tests--namely, the Wald, the score, and the likelihood ratio tests. The other three are based on the exact distribution of the sufficient statistic for the parameter of interest. These include the traditional exact method of constructing confidence intervals, and two others, the mid-P and mean-P methods, which are modifications of this procedure that aim at reducing the conservative bias of the exact method. Using efficient algorithms, the six methods are compared by determination of their exact coverage levels in a series of conditional sample spaces. An incident case-control study of lung cancer in women is used to further illustrate the differences among the various methods. Computation of coverage functions is seen as a useful graphical diagnostic tool for assessing the appropriateness of different methods. The mid-P and the score methods are seen to have better coverage properties than the other four.     

Confidence Limits on the Ratio of the Mean Failure Times Based on Paired Exponential Observations

This paper discusses interval estimation for the ratio of the mean failure times on the basis of paired exponential observations. This paper considers five interval estimators: the confidence interval using an idea similar to Fieller's theorem (CIFT), the confidence interval using an exact parametric test (CIEP), the confidence interval using the marginal likelihood ratio test (CILR), the confidence interval assuming no matching effect (CINM), and the confidence interval using a locally most powerful test (CIMP). To evaluate and compare the performance of these five interval estimators, this paper applies Monte Carlo simulation. This paper notes that with respect to the coverage probability, use of the CIFT, CILR, or CIMP, although which are all derived based on large sample theory, can perform well even when the number of pairs n is as small as 10. As compared with use of the CILR, this paper finds that use of the CIEP with equal tail probabilities is likely to lose efficiency. However, this loss can be reduced by using the optimal tail probabilities to minimize the average length when n is small (<20). This paper further notes that use of the CIMP is preferable to the CIEP in a variety of situations considered here. In fact, the average length of the CIMP with use of the optimal tail probabilities can even be shorter than that of the CILR. When the intraclass correlation between failure times within pairs is 0 (i.e., the failure times within the same pair are independent), the CINM, which is derived for two independent samples, is certainly the best one among the five interval estimators considered here. When there is an intraclass correlation but which is small (<0.10), the CIFT is recommended for obtaining a relatively short interval estimate without sacrificing the loss of the coverage probability. When the intraclass correlation is moderate or large, either the CILR or the CIMP with the optimal tail probabilities is preferable to the others. This paper also notes that if the intraclass correlation between failure times within pairs is large, use of the CINM can be misleading, especially when the number of pairs is large.     

Problems with existing procedures to calculate exact unconditional P-values for non-inferiority/superiority and confidence intervals for two binomials and how to resolve them

Recently several papers have been published that deal with the construction of exact unconditional tests for non-inferiority and confidence intervals based on the approximative unconditional restricted maximum likelihood test for two binomial random variables. Soon after the papers have been published the commercially available software for exact tests StatXact has incorporated the new methods. There are however gaps in the proofs which since have not been resolved adequately. Further it turned out that the methods for testing non-inferiority are not coherent and test for non-inferiority can easily come to different conclusions compared to the confidence interval inclusion rule. In this paper, a proposal is made how to resolve the open problems. Berger and Boos (1994) developed the confidence interval method for testing equality of two proportions. StatXact (Version 5) has extended this method for shifted hypotheses. It is shown that at least for unbalanced designs (i.e. largely different sample sizes) the Berger and Boos method can lead to controversial results.     

Likelihood-Weighted Confidence Intervals for the Difference of Two Binomial Proportions

Two new methods for computing confidence intervals for the difference δ = p₁ — p₂ between two binomial proportions (p₁, p₂) are proposed. Both the Mid-P and Max-P likelihood weighted intervals are constructed by mapping the tail probabilities from the two-dimensional (p₁, p₂)-space into a one-dimensional function of δ based on the likelihood weights. This procedure may be regarded as a natural extension of the CLOPPER-PEARSON (1934) interval to the two-sample case where the weighted tail probability is α/2 at each end on the δ scale. The probability computation is based on the exact distribution rather than a large sample approximation. Extensive computation was carried out to evaluate the coverage probability and expected width of the likelihood-weighted intervals, and of several other methods. The likelihood-weighted intervals compare very favorably with the standard asymptotic interval and with intervals proposed by HAUCK and ANDERSON (1986), COX and SNELL (1989), SANTNER and SNELL (1980), SANTNER and YAMAGAMI (1993), and PESKUN (1993). In particular, the Mid-P likelihood-weighted interval provides a good balance between accurate coverage probability and short interval width in both small and large samples. The Mid-P interval is also comparable to COE and TAMHANE'S (1993) interval, which has the best performance in small samples.     

Simultaneous confidence intervals for comparing binomial parameters

SUMMARY: To compare proportions with several independent binomial samples, we recommend a method of constructing simultaneous confidence intervals that uses the studentized range distribution with a score statistic. It applies to a variety of measures, including the difference of proportions, odds ratio, and relative risk. For the odds ratio, a simulation study suggests that the method has coverage probability closer to the nominal value than ad hoc approaches such as the Bonferroni implementation of Wald or "exact" small-sample pairwise intervals. It performs well even for the problematic but practically common case in which the binomial parameters are relatively small. For the difference of proportions, the proposed method has performance comparable to a method proposed by Piegorsch (1991, Biometrics 47, 45-52).     

Testing Equality of Means and Simultaneous Confidence Intervals in Repeated Measures with Missing Data

In this paper, repeated measures with intraclass correlation model is considered when the observations are missing at random. An exact test for the equality of the mean components and simultaneous confidence intervals (Scheffé and Bonferroni inequality types) are given for linear contrasts of the mean components when the missing observations are of a monotone type. When the missing observations are not of the monotone type, the maximum likelihood estimates are obtained numerically by iterative methods given in Srivastava and Carter (1986). These estimators are then used to obtain asymptotic tests and confidence intervals for the equality of mean components and linear contrasts, respectively. An example is given to illustrate the method.     

Unconditional Exact Tests for Equivalence or Noninferiority for Paired Binary Endpoints

Problems of establishing equivalence or noninferiority between two medical diagnostic procedures involve comparisons of the response rates between correlated proportions. When the sample size is small, the asymptotic tests may not be reliable. This article proposes an unconditional exact test procedure to assess equivalence or noninferiority. Two statistics, a sample-based test statistic and a restricted maximum likelihood estimation (RMLE)-based test statistic, to define the rejection region of the exact test are considered. We show the p-value of the proposed unconditional exact tests can be attained at the boundary point of the null hypothesis. Assessment of equivalence is often based on a comparison of the confidence limits with the equivalence limits. We also derive the unconditional exact confidence intervals on the difference of the two proportion means for the two test statistics. A typical data set of comparing two diagnostic procedures is analyzed using the proposed unconditional exact and asymptotic methods. The p-value from the unconditional exact tests is generally larger than the p-value from the asymptotic tests. In other words, an exact confidence interval is generally wider than the confidence interval obtained from an asymptotic test.     

Improved exact confidence intervals for the odds ratio in two independent binomial samples

For two independent binomial samples, the usual exact confidence interval for the odds ratio based on the conditional approach can be very conservative. Recently, Agresti and Min (2002) showed that the unconditional intervals are preferable to conditional intervals with small sample sizes. We use the unconditional approach to obtain a modified interval, which has shorter length, and its coverage probability is closer to and at least the nominal confidence coefficient.     

Confidence intervals of effect size for paired studies

To construct a confidence interval of effect size in paired studies, we propose four approximate methods--Wald method, variance-stabilizing transformation method, and signed and modified signed log-likelihood ratio methods. We compare these methods using simulation to determine those that have good performance in terms of coverage probability. In particular, simulations show that the modified signed log-likelihood ratio method produces a confidence interval with a nearly exact coverage probability and highly accurate and symmetric error probabilities even for very small samples. We apply the methods to data from an iron deficiency anemia study.     

A Monte Carlo maximum likelihood method for estimating uncertainty arising from shared errors in exposures in epidemiological studies of nuclear workers

Errors in the estimation of exposures or doses are a major source of uncertainty in epidemiological studies of cancer among nuclear workers. This paper presents a Monte Carlo maximum likelihood method that can be used for estimating a confidence interval that reflects both statistical sampling error and uncertainty in the measurement of exposures. The method is illustrated by application to an analysis of all cancer (excluding leukemia) mortality in a study of nuclear workers at the Oak Ridge National Laboratory (ORNL). Monte Carlo methods were used to generate 10,000 data sets with a simulated corrected dose estimate for each member of the cohort based on the estimated distribution of errors in doses. A Cox proportional hazards model was applied to each of these simulated data sets. A partial likelihood, averaged over all of the simulations, was generated; the central risk estimate and confidence interval were estimated from this partial likelihood. The conventional unsimulated analysis of the ORNL study yielded an excess relative risk (ERR) of 5.38 per Sv (90% confidence interval 0.54-12.58). The Monte Carlo maximum likelihood method yielded a slightly lower ERR (4.82 per Sv) and wider confidence interval (0.41-13.31).     

Interval estimation of the kappa coefficient with binary classification and an equal marginal probability model

We derive a likelihood score method for interval estimation of the intraclass version of the kappa coefficient of agreement with binary classification using a general theory of Bartlett (1953, Biometrika 40, 306-317). By exact evaluation, we investigate statistical properties of the score method, the chi-square goodness-of-fit procedure (Donner and Eliasziw, 1992, Statistics in Medicine 11, 1511-1519; Hale and Fleiss, 1993, Biometrics 49, 523-534), and a crude confidence interval for small and medium sample sizes. Actual coverage percentages of the score and chi-square methods are satisfactorily close to the nominal confidence coefficient, while that of the crude method is quite unsatisfactory. The expected length of the score method is shorter than that of the chi-square procedure when the response rate is very small or very large.     

Development and validation of a computational method for assessment of missense variants in hypertrophic cardiomyopathy

Assessing the significance of novel genetic variants revealed by DNA sequencing is a major challenge to the integration of genomic techniques with medical practice. Many variants remain difficult to classify by traditional genetic methods. Computational methods have been developed that could contribute to classifying these variants, but they have not been properly validated and are generally not considered mature enough to be used effectively in a clinical setting. We developed a computational method for predicting the effects of missense variants detected in patients with hypertrophic cardiomyopathy (HCM). We used a curated clinical data set of 74 missense variants in six genes associated with HCM to train and validate an automated predictor. The predictor is based on support vector regression and uses phylogenetic and structural features specific to genes involved in HCM. Ten-fold cross validation estimated our predictor's sensitivity at 94% (95% confidence interval: 83%-98%) and specificity at 89% (95% confidence interval: 72%-100%). This corresponds to an odds ratio of 10 for a prediction of pathogenic (95% confidence interval: 4.0-infinity), or an odds ratio of 9.9 for a prediction of benign (95% confidence interval: 4.6-21). Coverage (proportion of variants for which a prediction was made) was 57% (95% confidence interval: 49%-64%). This performance exceeds that of existing methods that are not specifically designed for HCM. The accuracy of this predictor provides support for the clinical use of automated predictions alongside family segregation and population frequency data in the interpretation of new missense variants and suggests future development of similar tools for other diseases.     

An Exact Confidence Interval for the Ratio of Two Related Proportions

I propose an exact confidence interval for the ratio of two proportions when the proportions are not independent. One application is to estimate the population prevalence using a screening test with perfect specificity but imperfect sensitivity. The population prevalence is the ratio of the observed prevalence divided by the test's sensitivity. I describe a method to calculate exact confidence intervals for this problem and compare these results with approximate confidence intervals given previously.     

On the use of concordant pairs in matched case-control studies

A new estimator of the common odds ratio in one-to-one matched case-control studies is proposed. The connection between this estimator and the James-Stein estimating procedure is highlighted through the argument of estimating functions. Comparisons are made between this estimator, the conditional maximum likelihood estimator, and the estimator ignoring the matching in terms of finite sample bias, mean squared error, coverage probability, and length of confidence interval. In many situations, the new estimator is found to be more efficient than the conditional maximum likelihood estimator without being as biased as the estimator that ignores matching. The extension to multiple risk factors is also outlined.     

Interval estimation of the median lethal dose

Assuming a logistic dose-response curve, one can construct a confidence interval for the LD50 from the asymptotic likelihood ratio test. Reasons are given for preferring this likelihood ratio interval to the established interval calculated by applying Fieller's theorem to the maximum likelihood estimates.     

Exact unconditional inference for risk ratio in a correlated 2 x 2 table with structural zero

In this article, we consider small-sample statistical inference for rate ratio (RR) in a correlated 2 x 2 table with a structural zero in one of the off-diagonal cells. Existing Wald's test statistic and logarithmic transformation test statistic will be adopted for this purpose. Hypothesis testing and confidence interval construction based on large-sample theory will be reviewed first. We then propose reliable small-sample exact unconditional procedures for hypothesis testing and confidence interval construction. We present empirical results to evince the better confidence interval performance of our proposed exact unconditional procedures over the traditional large-sample procedures in small-sample designs. Unlike the findings given in Lui (1998, Biometrics 54, 706-711), our empirical studies show that the existing asymptotic procedures may not attain a prespecified confidence level even in moderate sample-size designs (e.g., n = 50). Our exact unconditional procedures on the other hand do not suffer from this problem. Hence, the asymptotic procedures should be applied with caution. We propose two approximate unconditional confidence interval construction methods that outperform the existing asymptotic ones in terms of coverage probability and expected interval width. Also, we empirically demonstrate that the approximate unconditional tests are more powerful than their associated exact unconditional tests. A real data set from a two-step tuberculosis testing study is used to illustrate the methodologies.     

On logit confidence intervals for the odds ratio with small samples

Unless the true association is very strong, simple large-sample confidence intervals for the odds ratio based on the delta method perform well even for small samples. Such intervals include the Woolf logit interval and the related Gart interval based on adding .5 before computing the log odds ratio estimate and its standard error. The Gart interval smooths the observed counts toward the model of equiprobability, but one obtains better coverage probabilities by smoothing toward the independence model and by extending the interval in the appropriate direction when a cell count is zero.     

Methods for inferring phylogenies from nucleic acid sequence data by using maximum likelihood and linear invariants

Likelihood methods and methods using invariants are procedures for inferring the evolutionary relationships among species through statistical analysis of nucleic acid sequences. A likelihood-ratio test may be used to determine the feasibility of any tree for which the maximum likelihood can be computed. The method of linear invariants described by Cavender, which includes Lake's method of evolutionary parsimony as a special case, is essentially a form of the likelihood-ratio method. In the case of a small number of species (four or five), these methods may be used to find a confidence set for the correct tree. An exact version of Lake's asymptotic chi 2 test has been mentioned by Holmquist et al. Under very general assumptions, a one-sided exact test is appropriate, which greatly increases power.