Approximating the distribution of maximally selected McNemar's statistics

It is common in epidemiologic analyses to summarize continuous outcomes as falling above or below a threshold. With paired data and with a threshold chosen without reference to the outcomes, McNemar's test of marginal homogeneity may be applied to the resulting dichotomous pairs when testing for equality of the marginal distributions of the underlying continuous outcomes. If the threshold is chosen to maximize the test statistic, however, referring the resulting test statistic to the nominal chi 2 distribution is incorrect; instead, the p-value must be adjusted for the multiple comparisons. Here the distribution of a maximally selected McNemar's statistic is derived, and it is shown that an approximation due to Durbin (1985, Journal of Applied Probability 22, 99-122) may be used to estimate approximate p-values. The methodology is illustrated by an application to measurements of insulin-like growth factor-I (IGF-I) in matched prostate cancer cases and controls from the Physicians' Health Study. The results of simulation experiments that assess the accuracy of the approximation in moderate sample sizes are reported.