Estimation of the Youden Index and its associated cutoff point

The Youden Index is a frequently used summary measure of the ROC (Receiver Operating Characteristic) curve. It both, measures the effectiveness of a diagnostic marker and enables the selection of an optimal threshold value (cutoff point) for the marker. In this paper we compare several estimation procedures for the Youden Index and its associated cutoff point. These are based on (1) normal assumptions; (2) transformations to normality; (3) the empirical distribution function; (4) kernel smoothing. These are compared in terms of bias and root mean square error in a large variety of scenarios by means of an extensive simulation study. We find that the empirical method which is the most commonly used has the overall worst performance. In the estimation of the Youden Index the kernel is generally the best unless the data can be well transformed to achieve normality whereas in estimation of the optimal threshold value results are more variable.     

Construction of confidence intervals for the maximum of the Youden index and the corresponding cutoff point of a continuous biomarker

Evaluation of the overall accuracy of biomarkers might be based on average measures of the sensitivity for all possible specificities ‐and vice versa‐ or equivalently the area under the receiver operating characteristic (ROC) curve that is typically used in such settings. In practice clinicians are in need of a cutoff point to determine whether intervention is required after establishing the utility of a continuous biomarker. The Youden index can serve both purposes as an overall index of a biomarker's accuracy, that also corresponds to an optimal, in terms of maximizing the Youden index, cutoff point that in turn can be utilized for decision making. In this paper, we provide new methods for constructing confidence intervals for both the Youden index and its corresponding cutoff point. We explore approaches based on the delta approximation under the normality assumption, as well as power transformations to normality and nonparametric kernel‐ and spline‐based approaches. We compare our methods to existing techniques through simulations in terms of coverage and width. We then apply the proposed methods to serum‐based markers of a prospective observational study involving diagnosis of late‐onset sepsis in neonates.     

Evaluating the Predictive Value of Biomarkers with Stratified Case‐Cohort Design

Summary Identification of novel biomarkers for risk assessment is important for both effective disease prevention and optimal treatment recommendation. Discovery relies on the precious yet limited resource of stored biological samples from large prospective cohort studies. Case‐cohort sampling design provides a cost‐effective tool in the context of biomarker evaluation, especially when the clinical condition of interest is rare. Existing statistical methods focus on making efficient inference on relative hazard parameters from the Cox regression model. Drawing on recent theoretical development on the weighted likelihood for semiparametric models under two‐phase studies ( Breslow and Wellner, 2007 ), we propose statistical methods to evaluate accuracy and predictiveness of a risk prediction biomarker, with censored time‐to‐event outcome under stratified case‐cohort sampling. We consider nonparametric methods and a semiparametric method. We derive large sample properties of proposed estimators and evaluate their finite sample performance using numerical studies. We illustrate new procedures using data from Framingham Offspring Study to evaluate the accuracy of a recently developed risk score incorporating biomarker information for predicting cardiovascular disease.