On ROC analysis with nonbinary reference standard

Statistical methods for the evaluation of the accuracy of diagnostic tests usually assume a binary true disease status. However, this assumption may not be realistic in practical settings in which “disease” is defined by dichotomizing continuous or ordinal categorical measures using a pre‐specified threshold value. In this paper, we focus on the analysis of studies in which both the diagnostic test and the reference standard are reported as continuous measures. We propose a semiparametric model for estimating the sensitivity, specificity, and the ROC curve as functions of reference standard thresholds. Under suitable order restrictions on the mean of the test result variable, fitting is done via two alternative approaches: isotonic regression and monotone smoothing splines. The model provides the basis to assess the effect of varying reference standard threshold on the performance of a diagnostic test. An example to evaluate the ability of the maximal SUV‐lean (standardized uptake value normalized to lean body mass) in predicting axillary node involvement in women diagnosed with breast cancer is presented.