Precision Intervals for Estimates of the Difference in Success Rates for Binary Random Variables Based on the Permutation Principle

Fisher's exact test is a very commonly applied test in clinical trials with a binary outcome variable (e.g. success/failure). However confidence statements about the difference of success rates are usually based on the normal approximation. This may sometimes lead to the confusing statement that the test is statistically significant at a prespecified level while the corresponding confidence interval includes the zero difference and vice versa. Here, we construct precision intervals for the difference of success rates from two independent samples based on the permutation principle which are in perfect agreement with the discrete (permutation) test and compare it to examples from the literature. APL programs are provided.