ROC graphs for assessing the ability of a diagnostic marker to detect three disease classes with an umbrella ordering

Receiver operating characteristic (ROC) curves and the area under these curves are commonly used to assess the ability of a continuous diagnostic marker (e.g., DNA methylation markers) to correctly classify subjects as having a particular disease or not (e.g., cancer). These approaches, however, are not applicable to settings where the gold standard yields more than two disease states or classes. ROC surfaces and the volume under the surfaces have been proposed for settings with more than two disease classes. These approaches, however, do not allow one to assess the ability of a marker to differentiate two disease classes from a third disease class without requiring a monotone order for the three disease classes under study. That is, existing approaches do not accommodate an umbrella ordering of disease classes. This article proposes the construction of an ROC graph that is applicable for an umbrella ordering. Furthermore, this article proposes that a summary measure for this umbrella ROC graph can be used to summarize the classification accuracy, and corresponding variance estimates can be obtained using U-statistics theory or bootstrap methods. The proposed methods are illustrated using data from a study assessing the ability of a DNA methylation marker to correctly classify lung specimens into three histologic classes: squamous cell carcinoma, large cell carcinoma, and nontumor lung.     

ROC Volumes – Should They Be Used?

Invited comment on Alonzo, T.A. and Nakas, C.T. (2007). Comparison of ROC umbrella volumes with an application to the assessment of lung cancer diagnostic markers. Biometrical Journal 49, 654-664. Issues remain as to the appropriateness of volumes as summary measures of accuracy when distinguishing between multiple disease classes and when comparing accuracy of tests.     

The Umbrella Volume: Is it Useful in Applications?

Invited comment on Alonzo, T. A. and Nakas, C. T. (2007). Comparison of ROC umbrella volumes with an application to the assessment of lung cancer diagnostic markers. Biometrical Journal 49, 654-664. The umbrella volume, proposed as a parameter for diagnostic validity in a specific three group situation might be difficult to interpret in clinical applications. It should be used very cautiously.     

Combining diagnostic test results to increase accuracy

When multiple diagnostic tests are performed on an individual or multiple disease markers are available it may be possible to combine the information to diagnose disease. We consider how to choose linear combinations of markers in order to optimize diagnostic accuracy. The accuracy index to be maximized is the area or partial area under the receiver operating characteristic (ROC) curve. We propose a distribution-free rank-based approach for optimizing the area under the ROC curve and compare it with logistic regression and with classic linear discriminant analysis (LDA). It has been shown that the latter method optimizes the area under the ROC curve when test results have a multivariate normal distribution for diseased and non-diseased populations. Simulation studies suggest that the proposed non-parametric method is efficient when data are multivariate normal.The distribution-free method is generalized to a smooth distribution-free approach to: (i) accommodate some reasonable smoothness assumptions; (ii) incorporate covariate effects; and (iii) yield optimized partial areas under the ROC curve. This latter feature is particularly important since it allows one to focus on a region of the ROC curve which is of most relevance to clinical practice. Neither logistic regression nor LDA necessarily maximize partial areas. The approaches are illustrated on two cancer datasets, one involving serum antigen markers for pancreatic cancer and the other involving longitudinal prostate specific antigen data.     

Elevated Levels of the Complement Activation Product C4d in Bronchial Fluids for the Diagnosis of Lung Cancer

Molecular markers in bronchial fluids may contribute to the diagnosis of lung cancer. We previously observed a significant increase of C4d-containing complement degradation fragments in bronchoalveolar lavage (BAL) supernatants from lung cancer patients in a cohort of 50 cases and 22 controls (CUN cohort). The present study was designed to determine the diagnostic performance of these complement fragments (hereinafter jointly referred as C4d) in bronchial fluids. C4d levels were determined in BAL supernatants from two independent cohorts: the CU cohort (25 cases and 26 controls) and the HUVR cohort (60 cases and 98 controls). A series of spontaneous sputum samples from 68 patients with lung cancer and 10 controls was also used (LCCCIO cohort). Total protein content, complement C4, complement C5a, and CYFRA 21-1 were also measured in all cohorts. C4d levels were significantly increased in BAL samples from lung cancer patients. The area under the ROC curve was 0.82 (95%CI = 0.71–0.94) and 0.67 (95%CI = 0.58–0.76) for the CU and HUVR cohorts, respectively. In addition, unlike the other markers, C4d levels in BAL samples were highly consistent across the CUN, CU and HUVR cohorts. Interestingly, C4d test markedly increased the sensitivity of bronchoscopy in the two cohorts in which cytological data were available (CUN and HUVR cohorts). Finally, in the LCCCIO cohort, C4d levels were higher in sputum supernatants from patients with lung cancer (area under the ROC curve: 0.7; 95%CI = 0.56–0.83). In conclusion, C4d is consistently elevated in bronchial fluids from lung cancer patients and may be used to improve the diagnosis of the disease.     

Combining several screening tests: optimality of the risk score

The development of biomarkers for cancer screening is an active area of research. While several biomarkers exist, none is sufficiently sensitive and specific on its own for population screening. It is likely that successful screening programs will require combinations of multiple markers. We consider how to combine multiple disease markers for optimal performance of a screening program. We show that the risk score, defined as the probability of disease given data on multiple markers, is the optimal function in the sense that the receiver operating characteristic (ROC) curve is maximized at every point. Arguments draw on the Neyman-Pearson lemma. This contrasts with the corresponding optimality result of classic decision theory, which is set in a Bayesian framework and is based on minimizing an expected loss function associated with decision errors. Ours is an optimality result defined from a strictly frequentist point of view and does not rely on the notion of associating costs with misclassifications. The implication for data analysis is that binary regression methods can be used to yield appropriate relative weightings of different biomarkers, at least in large samples. We propose some modifications to standard binary regression methods for application to the disease screening problem. A flexible biologically motivated simulation model for cancer biomarkers is presented and we evaluate our methods by application to it. An application to real data concerning two ovarian cancer biomarkers is also presented. Our results are equally relevant to the more general medical diagnostic testing problem, where results of multiple tests or predictors are combined to yield a composite diagnostic test. Moreover, our methods justify the development of clinical prediction scores based on binary regression.     

A Bayesian hierarchical non-linear regression model in receiver operating characteristic analysis of clustered continuous diagnostic data

Receiver operating characteristic (ROC) analysis is a useful evaluative method of diagnostic accuracy. A Bayesian hierarchical nonlinear regression model for ROC analysis was developed. A validation analysis of diagnostic accuracy was conducted using prospective multi-center clinical trial prostate cancer biopsy data collected from three participating centers. The gold standard was based on radical prostatectomy to determine local and advanced disease. To evaluate the diagnostic performance of PSA level at fixed levels of Gleason score, a normality transformation was applied to the outcome data. A hierarchical regression analysis incorporating the effects of cluster (clinical center) and cancer risk (low, intermediate, and high) was performed, and the area under the ROC curve (AUC) was estimated.     

Combining biomarkers to detect disease with application to prostate cancer

In early detection of disease, combinations of biomarkers promise improved discrimination over diagnostic tests based on single markers. An example of this is in prostate cancer screening, where additional markers have been sought to improve the specificity of the conventional Prostate-Specific Antigen (PSA) test. A marker of particular interest is the percent free PSA. Studies evaluating the benefits of percent free PSA reflect the need for a methodological approach that is statistically valid and useful in the clinical setting. This article presents methods that address this need. We focus on and-or combinations of biomarker results that we call logic rules and present novel definitions for the ROC curve and the area under the curve (AUC) that are applicable to this class of combination tests. Our estimates of the ROC and AUC are amenable to statistical inference including comparisons of tests and regression analysis. The methods are applied to data on free and total PSA levels among prostate cancer cases and matched controls enrolled in the Physicians' Health Study.     

Increased levels of albumin in bronchial washing fluid of patients with bronchial carcinoma. Could albumin be considered as a tumor marker?

BACKGROUND: The aim of this study was to evaluate the significance of albumin in bronchial washing fluid (BWF) and its relationship to three tumor markers (CEA, CA 19-9 and NSE). METHODS: Serum and BWF samples were collected in a group of 60 patients. Albumin and tumor markers in the BWF and serum of three groups: a control group (CG), a chronic bronchitis group (CBG) and a lung cancer group (CaG), were analyzed in a prospective cross-sectional study. The diagnostic yields of the tests in each environment (serum and BWF) were evaluated by using as cutoff points the values of the corresponding 90th percentile of CG and CBG taken together. RESULTS: A significant difference in albumin level (p < 0.001) was noted in the BWF of patients with cancer compared with the other two groups. In addition, a significant difference in CEA level (p < 0.001) was observed in the serum of cancer patients compared with the other two groups. The cutoff values for CEA in serum and albumin in BWF were 2.20 ng/mL and 2.00 g/dL, respectively. The areas under the corresponding ROC curves were 93% and 97%. Combination of CEA-serum and albumin-BWF by logistic regression analysis increased their diagnostic value. CONCLUSION: Measurement of albumin levels in BWF could be a useful additional diagnostic tool to differentiate malignant from non-malignant lung diseases. Moreover, the combined measurement of CEA in serum and albumin in BWF could be of aid in the follow-up of lung cancer patients.     

A parametric comparison of diagnostic accuracy with three ordinal diagnostic groups

Many medical diagnostic studies involve three ordinal diagnostic groups in which the diagnostic accuracy can be summarized by the volume or partial volume under a Receiver Operating Characteristic (ROC) surface. We study in this paper the statistical comparison of diagnostic accuracy from multiple diagnostic tests when three ordinal diagnostic groups are involved. Under the assumption that the multiple diagnostic tests follow a multivariate normal distribution within each diagnostic group, we provide the asymptotic variance and covariance for the maximum likelihood estimates of the volumes under the ROC surfaces from multiple diagnostic tests and propose statistical tests to test whether the diagnostic accuracy as measured by the volume under the ROC surface is the same for multiple diagnostic tests. We also propose a confidence interval estimate to the difference of two volumes under two ROC surfaces. Our approach depends crucially on the assumptions of normal distributions on diagnostic tests, which might not be robust when such assumptions are violated. Finally, we apply our proposed methodology to a real data set of 118 subjects to compare the diagnostic accuracy of early stage Alzheimer's disease (AD) from multiple neuropsychological tests.     

血清多肿瘤标志物蛋白芯片检测在乳腺癌诊断中的价值

目的:探讨血清多肿瘤标志物蛋白芯片检测系统在乳腺癌诊断中的临床价值。方法:临床确诊的乳腺癌患者307例为乳腺癌组,非乳腺癌的其他恶性肿瘤患者495例为对照组。应用多肿瘤标志物蛋白芯片检测系统检测12种肿瘤标志物水平,评价血清肿瘤标志物的在乳腺癌组与对照组之间的差异。结果:CA153,CEA,Free-PSA这三项指标为诊断乳腺癌的独立相关因素(P<0.05),比较三项指标ROC曲线下面积可见,CA153对于鉴别乳腺癌准确性更高,其敏感性、特异性分别为78.92和56.14,女性乳腺癌患者Free-PSA可见明显升高,对乳腺癌有特殊提示意义,手术前后标志物CA199、CA242、Ferrin、CA125水平差异有统计学意义。结论:在临床常用的肿瘤标记物中,CA153,CEA,Free_PSA水平的升高与乳腺癌发生独立相关,其中CA153具有更高的诊断准确性,Free_PSA水平升高对乳腺癌的诊断有特别提示意义。     

CEA和CYFRA21-1在原发性肺癌诊断中的应用价值分析

目的:评价血清癌胚抗原(carcinoembryonic antigen,CEA)和细胞角蛋白19的片段(CYFRA21-1)水平对原发性肺癌的诊断价值。方法:回顾性分析2012年5月~2013年5月我科收治的329例肺癌患者和192例肺部良性病变患者的临床资料。结果:肺癌患者血清CEA或CYFRA21-1水平随着肺癌的分期呈现逐渐上升的趋势(P0.001,P0.01)。以血清CEA≥3.4μg/L作为诊断条件诊断肺癌的灵敏度、特异度和阳性预计值预计值分别为67%、62%和75.2%;以血清CYFRA-21≥5.0 ug/L作为诊断条件诊断原发性肺癌的灵敏度、特异度和阳性预计值预计值分别为48%、87%和86.3%;以血清CEA≥3.4 ug/L和CYFRA-21≥5.0 ug/L共同作为诊断条件推断原发性肺癌的灵敏度、特异度和阳性预计值预计值分别为48%、87%和86.3%;良性肺部疾病患者中血清CEA和CYFRA-21水平同时升高者只有2%。结论:血清CEA≥3.4 ug/L和CYFRA-21≥5.0 ug/L同时升高对肺癌有具有重要的诊断价值,可有效的鉴别原发性肺癌与其它良性病变。     

Characterization of serum biomarkers for detection of early stage ovarian cancer

We have previously reported the identification of three ovarian cancer biomarker panels comprised of SELDI-TOF-MS peaks representing 14 differentially expressed serum proteins for the diagnosis of ovarian cancer. Using micro-LC-MS/MS, we identified five m/z peaks as transthyretin (TTR 13.9 kDa, TTR fragment 12.9 kDa), beta-hemoglobin (Hb, 15.9 kDa), apolipoprotein AI (ApoAI, 28 kDa) and transferrin (TF, 79 kDa). Western and/or ELISA methods confirmed the differential expression of TTR, Hb, and TF, and multivariate analyses resulted in improving the detection of early stage ovarian tumors (low malignant potential and malignant; receiver operating characteristic, ROC 0.933) as compared to cancer antigen CA125 alone (ROC 0.833). Interestingly, when CA125 was included with our markers in the multivariate analysis, the ROC increased to 0.959. Furthermore, multivariate analysis with only the mucinous subtype of early stage ovarian tumors, showed our markers to greatly improve the detection of disease (ROC 0.959) as compared to CA125 alone (ROC 0.613). Interestingly, the combination of CA125 with our markers did not seem to further improve the detection of mucinous tumors (ROC 0.955). We conclude that TTR, Hb, ApoAI and TF, when combined with CA125 should significantly improve the detection of early stage ovarian cancer.     

CAXII Is a sero-diagnostic marker for lung cancer

To develop sero-diagnostic markers for lung cancer, we generated monoclonal antibodies using pulmonary adenocarcinoma (AD)-derived A549 cells as antigens by employing the random immunization method. Hybridoma supernatants were immunohistochemically screened for antibodies with AMeX-fixed and paraffin-embedded A549 cell preparations. Positive clones were monocloned twice through limiting dilutions. From the obtained monoclonal antibodies, we selected an antibody designated as KU-Lu-5 which showed intense membrane staining of A549 cells. Based on immunoprecipitation and MADLI TOF/TOF-MS analysis, this antibody was recognized as carbonic anhydrase XII (CAXII). To evaluate the utility of this antibody as a sero-diagnostic marker for lung cancer, we performed dot blot analysis with a training set consisting of sera from 70 lung cancer patients and 30 healthy controls. The CAXII expression levels were significantly higher in lung cancer patients than in healthy controls in the training set (P<0.0001), and the area under the curve of ROC was 0.794, with 70.0% specificity and 82.9% sensitivity. In lung cancers, expression levels of CAXII were significantly higher in patients with squamous cell carcinoma (SCC) than with AD (P = 0.035). Furthermore, CAXII was significantly higher in well- and moderately differentiated SCCs than in poorly differentiated ones (P = 0.027). To further confirm the utility of serum CAXII levels as a sero-diagnostic marker, an additional set consisting of sera from 26 lung cancer patients and 30 healthy controls was also investigated by dot blot analysis as a validation study. Serum CAXII levels were also significantly higher in lung cancer patients than in healthy controls in the validation set (P = 0.030). Thus, the serum CAXII levels should be applicable markers discriminating lung cancer patients from healthy controls. To our knowledge, this is the first report providing evidence that CAXII may be a novel sero-diagnostic marker for lung cancer.     

Molecular-genetic markers in lung cancer diagnostics

The major approaches to different lung cancer marker development are outlined in the review, including genetic, epigenetic, protein, transcryptomic, proteomic, metabolic, and miRNA markers. As far as epigenetic changes are among the earliest events in malignant transformation, methylated markers are thoroughly discussed. Special attention is given to minimally invasive tumor markers, which could be detected in easily accessible biological fluids, because they can be useful for screening and early diagnostics of cancer (before its clinical manifestation) as well as for verification of standard methods of diagnostics. Extracellular nucleic acids, circulating in blood (cirNA), are highlighted as the potential source of material for the early lung cancer diagnostics, prediction of antitumor treatment efficiency, post-treatment monitoring and disease prognosis.     

Markers of small cell lung cancer

Lung cancer is the number one cause of cancer death; however, no specific serum biomarker is available till date for detection of early lung cancer. Despite good initial response to chemotherapy, small-cell lung cancer (SCLC) has a poor prognosis. Therefore, it is important to identify molecular markers that might influence survival and may serve as potential therapeutic targets. The review aims to summarize the current knowledge of serum biomarkers in SCLC to improve diagnostic efficiency in the detection of tumor progression in lung cancer. The current knowledge on the known serum cytokines and tumor biomarkers of SCLC is emphasized. Recent findings in the search for novel diagnostic and therapeutic molecular markers using the emerging genomic technology for detecting lung cancer are also described. It is believed that implementing these new research techniques will facilitate and improve early detection, prognostication and better treatment of SCLC.     

血清细胞角蛋白19、神经元特异性烯醇化酶及鳞状上皮细胞癌抗原在小细胞肺癌诊断及病情评估中的作用研究

摘要 目的：分析血清细胞角蛋白19（CK19）、神经元特异性烯醇化酶（NSE）及鳞状上皮细胞癌抗原（SCCA）在小细胞肺癌诊断及病情评估中的作用。方法：选择我院自2020年1月至2022年7月收治的85例小细胞肺癌患者作为观察组,另选同期的85例肺部良性疾病患者作为对照组。检测两组血清CK19、NSE、SCCA表达水平,比较两组血清CK19、NSE、SCCA表达水平及其阳性率,使用ROC曲线下面积（AUC）分析上述指标对小细胞肺癌的诊断效能,观察小细胞肺癌不同分期患者血清CK19、NSE、SCCA表达水平的差异性,分析观察组治疗前后血清CK19、NSE、SCCA表达水平的变化情况。结果：观察组血清CK19、NSE、SCCA表达水平均较对照组高（P＜0.05）;观察组血清CK19、NSE和SCCA的阳性率均较对照组高（P＜0.05）;经ROC曲线分析,血清CK19、NSE联合SCCA诊断小细胞肺癌的敏感度为91.23 %,特异度为84.67 %,AUC为0.910;在85例小细胞肺癌患者中,临床分期为局限期33例、广泛期52例;广泛期患者血清CK19、NSE、SCCA表达水平均高于局限期患者（P＜0.05）。结论：血清CK19、NSE联合SCCA诊断小细胞肺癌的效能较好,三者均与患者病情严重程度有关,有利于此病的早期诊断和病情评估,值得进一步研究应用。     

小细胞肺癌分子标记物研究进展

肺癌仍然是现在最常见的恶性肿瘤之一。小细胞肺癌(Smallcelllungcancer,SCLC)是肺癌中恶性程度最高的一种类型,与吸烟密切相关,其较早发生远处转移和播散导致预后差,目前的主要治疗手段有手术、化学治疗、放射治疗。但其具有初始化放疗敏感,却很快耐受的特点,导致了它总体预后不良,生存期短。如何寻求一种有效的疾病预后、疗效判断标记物,显得尤为重要。本文主要介绍近年来在小细胞肺癌中分子标记物的研究进展,包括神经内分泌的相关蛋白、凋亡蛋白抑制剂(Survivin)、相关酶类及膜蛋白,这些分子标记物与小细胞肺癌疾病的进展、预后密切相关,能够为临床的疾病治疗评估提供潜在可行的方法。但是,这些标记物仍存在特异性不高的问题,最终应用于临床实践,仍需要更多的临床研究。     

血清NSE、SCCA及CEA在肺癌早期诊断和预后预测中的应用价值研究

目的:研究血清神经元特异性烯醇化酶(NSE)、鳞状细胞癌抗原(SCCA)及癌胚抗原(CEA)在肺癌早期诊断和预后预测中的应用价值。方法:选择我院2013年1月~2017年1月收治的110例肺癌患者(肺癌组)及同期96例肺部良性疾病患者(肺良性病组)和85例门诊健康体检者(对照组)。比较各组血清NSE、SCCA及CEA水平,采用受试者工作特征(ROC)曲线分析以上指标对肺癌的诊断价值。结果:肺癌组血清NSE、SCCA、CEA水平高于肺良性病组及对照组,肺良性病组血清NSE、SCCA、CEA水平高于对照组(P<0.05)。肺癌Ⅲ+Ⅳ组血清NSE、SCCA及CEA水平高于Ⅰ+Ⅱ组(P<0.05)。小细胞肺癌组血清NSE水平高于鳞癌组、腺癌组,鳞癌组血清SCCA水平高于腺癌组及小细胞肺癌组,腺癌组血清CEA水平高于鳞癌组及小细胞肺癌组(P<0.05)。NSE<16.0μg/L者平均无疾病进展生存期(PFS)长于NSE≥16.0μg/L,SCCA<1.5μg/L者平均PFS长于SCCA≥1.5μg/L,CEA<5.0μg/L平均PFS长于CEA≥5.0μg/L(P<0.05)。NSE、SCCA和CEA及三者联合诊断肺癌的ROC曲线下面积分别为0.880、0.651、0.830及0.937,NSE+SCCA+CEA联合诊断的曲线下面积高于单个指标单独诊断(P<0.05)。结论:血清NSE、SCCA及CEA对肺癌的诊断有重要的参考价值,且有利于肺癌的分期、分型及预后评价。