A permutation test to compare receiver operating characteristic curves

We developed a permutation test in our earlier paper (Venkatraman and Begg, 1996, Biometrika 83, 835-848) to test the equality of receiver operating characteristic curves based on continuous paired data. Here we extend the underlying concepts to develop a permutation test for continuous unpaired data, and we study its properties through simulations.     

A semiparametric approach for the two-sample comparison of survival times with long-term survivors

In the two-sample comparison of survival times with long-term survivors, the overall difference between the two distributions reflects differences occurring in early follow-up for susceptible subjects and in long-term follow-up for nonsusceptible subjects. In this setting, we propose statistics for testing (i) no overall, (ii) no short-term, and (iii) no long-term difference between the two distributions to be compared. The statistics are derived as follows. A semiparametric model is defined that characterizes a short-term effect and a long-term effect. By approximating this model about no difference in early survival, a time-dependent proportional hazards model is obtained. The statistics are obtained from this working model. The asymptotic distributions of the statistics for testing no overall or no short-term effects are ascertained, while that of the statistic for testing no long-term effect is valid only when the short-term effect is small. Simulation studies investigate the power properties of the proposed tests for different configurations. The results show the interesting behavior of the proposed tests for situations where a short-term effect is expected. An example investigating the impact of progesterone receptors status on local tumor relapse for patients with early breast cancer illustrates the use of the proposed tests.     

ROC-based utility function maximization for feature selection and classification with applications to high-dimensional protease data

SUMMARY: In medical diagnosis, the diseased and nondiseased classes are usually unbalanced and one class may be more important than the other depending on the diagnosis purpose. Most standard classification methods, however, are designed to maximize the overall accuracy and cannot incorporate different costs to different classes explicitly. In this article, we propose a novel nonparametric method to directly maximize the weighted specificity and sensitivity of the receiver operating characteristic curve. Combining advances in machine learning, optimization theory, and statistics, the proposed method has excellent generalization property and assigns different error costs to different classes explicitly. We present experiments that compare the proposed algorithms with support vector machines and regularized logistic regression using data from a study on HIV-1 protease as well as six public available datasets. Our main conclusion is that the performance of proposed algorithm is significantly better in most cases than the other classifiers tested. Software package in MATLAB is available upon request.     

Latent Class Modeling Approaches for Assessing Diagnostic Error without a Gold Standard: With Applications to p53 Immunohistochemical Assays in Bladder Tumors

Improved characterization of tumors for purposes of guiding treatment decisions for cancer patients will require that accurate and reproducible assays be developed for a variety of tumor markers. No gold standards exist for most tumor marker assays. Therefore, estimates of assay sensitivity and specificity cannot be obtained unless a latent class model-based approach is used. Our goal in this article is to estimate sensitivity and specificity for p53 immunohistochemical assays of bladder tumors using data from a reproducibility study conducted by the National Cancer Institute Bladder Tumor Marker Network. We review latent class modeling approaches proposed by previous authors, and we find that many of these approaches impose assumptions about specimen heterogeneity that are not consistent with the biology of bladder tumors. We present flexible mixture model alternatives that are biologically plausible for our example, and we use them to estimate sensitivity and specificity for our p53 assay example. These mixture models are shown to offer an improvement over other methods in a variety of settings, but we caution that, in general, care must be taken in applying latent class models.