Combining diagnostic test results to increase accuracy

When multiple diagnostic tests are performed on an individual or multiple disease markers are available it may be possible to combine the information to diagnose disease. We consider how to choose linear combinations of markers in order to optimize diagnostic accuracy. The accuracy index to be maximized is the area or partial area under the receiver operating characteristic (ROC) curve. We propose a distribution-free rank-based approach for optimizing the area under the ROC curve and compare it with logistic regression and with classic linear discriminant analysis (LDA). It has been shown that the latter method optimizes the area under the ROC curve when test results have a multivariate normal distribution for diseased and non-diseased populations. Simulation studies suggest that the proposed non-parametric method is efficient when data are multivariate normal.The distribution-free method is generalized to a smooth distribution-free approach to: (i) accommodate some reasonable smoothness assumptions; (ii) incorporate covariate effects; and (iii) yield optimized partial areas under the ROC curve. This latter feature is particularly important since it allows one to focus on a region of the ROC curve which is of most relevance to clinical practice. Neither logistic regression nor LDA necessarily maximize partial areas. The approaches are illustrated on two cancer datasets, one involving serum antigen markers for pancreatic cancer and the other involving longitudinal prostate specific antigen data.     

ROC analysis of diagnostic performance utilizing serum antiproteases in cancer patients

Assessment of the accuracy of diagnostic procedures is made independent of diagnostic criteria by means of a receiver-operating-characteristics (ROC) curve. We performed ROC analysis for the major serum antiproteases: alpha-1-antitrypsin (A1AT) and alpha-2-macroglobulin (A2M), in 99 cancer patients compared with 71 normal individuals. A1AT and A2M were significantly higher in cancer patients (p less than 0.0005). By comparing true positive and false positive rates for different serum levels, ROC analysis showed that serum A1AT quantification seems more useful in clinical practice than serum A2M.     

Adjusting confounders in ranking biomarkers: a model-based ROC approach

High-throughput studies have been extensively conducted in the research of complex human diseases. As a representative example, consider gene-expression studies where thousands of genes are profiled at the same time. An important objective of such studies is to rank the diagnostic accuracy of biomarkers (e.g. gene expressions) for predicting outcome variables while properly adjusting for confounding effects from low-dimensional clinical risk factors and environmental exposures. Existing approaches are often fully based on parametric or semi-parametric models and target evaluating estimation significance as opposed to diagnostic accuracy. Receiver operating characteristic (ROC) approaches can be employed to tackle this problem. However, existing ROC ranking methods focus on biomarkers only and ignore effects of confounders. In this article, we propose a model-based approach which ranks the diagnostic accuracy of biomarkers using ROC measures with a proper adjustment of confounding effects. To this end, three different methods for constructing the underlying regression models are investigated. Simulation study shows that the proposed methods can accurately identify biomarkers with additional diagnostic power beyond confounders. Analysis of two cancer gene-expression studies demonstrates that adjusting for confounders can lead to substantially different rankings of genes.     

Partial AUC estimation and regression

Accurate diagnosis of disease is a critical part of health care. New diagnostic and screening tests must be evaluated based on their abilities to discriminate diseased from nondiseased states. The partial area under the receiver operating characteristic (ROC) curve is a measure of diagnostic test accuracy. We present an interpretation of the partial area under the curve (AUC), which gives rise to a nonparametric estimator. This estimator is more robust than existing estimators, which make parametric assumptions. We show that the robustness is gained with only a moderate loss in efficiency. We describe a regression modeling framework for making inference about covariate effects on the partial AUC. Such models can refine knowledge about test accuracy. Model parameters can be estimated using binary regression methods. We use the regression framework to compare two prostate-specific antigen biomarkers and to evaluate the dependence of biomarker accuracy on the time prior to clinical diagnosis of prostate cancer.     

Model checking for ROC regression analysis

Summary .   The receiver operating characteristic (ROC) curve is a prominent tool for characterizing the accuracy of a continuous diagnostic test. To account for factors that might influence the test accuracy, various ROC regression methods have been proposed. However, as in any regression analysis, when the assumed models do not fit the data well, these methods may render invalid and misleading results. To date, practical model-checking techniques suitable for validating existing ROC regression models are not yet available. In this article, we develop cumulative residual-based procedures to graphically and numerically assess the goodness of fit for some commonly used ROC regression models, and show how specific components of these models can be examined within this framework. We derive asymptotic null distributions for the residual processes and discuss resampling procedures to approximate these distributions in practice. We illustrate our methods with a dataset from the cystic fibrosis registry.     

Profile-likelihood inference for highly accurate diagnostic tests

We consider profile-likelihood inference based on the multinomial distribution for assessing the accuracy of a diagnostic test. The methods apply to ordinal rating data when accuracy is assessed using the area under the receiver operating characteristic (ROC) curve. Simulation results suggest that the derived confidence intervals have acceptable coverage probabilities, even when sample sizes are small and the diagnostic tests have high accuracies. The methods extend to stratified settings and situations in which the ratings are correlated. We illustrate the methods using data from a clinical trial on the detection of ovarian cancer.     

An outcome model for human bladder cancer: A comprehensive study based on weighted gene co‐expression network analysis

The precision evaluation of prognosis is crucial for clinical treatment decision of bladder cancer (BCa). Therefore, establishing an effective prognostic model for BCa has significant clinical implications. We performed WGCNA and DEG screening to initially identify the candidate genes. The candidate genes were applied to construct a LASSO Cox regression analysis model. The effectiveness and accuracy of the prognostic model were tested by internal/external validation and pan‐cancer validation and time‐dependent ROC. Additionally, a nomogram based on the parameter selected from univariate and multivariate cox regression analysis was constructed. Eight genes were eventually screened out as progression‐related differentially expressed candidates in BCa. LASSO Cox regression analysis identified 3 genes to build up the outcome model in E‐MTAB‐4321 and the outcome model had good performance in predicting patient progress free survival of BCa patients in discovery and test set. Subsequently, another three datasets also have a good predictive value for BCa patients' OS and DFS. Time‐dependent ROC indicated an ideal predictive accuracy of the outcome model. Meanwhile, the nomogram showed a good performance and clinical utility. In addition, the prognostic model also exhibits good performance in pan‐cancer patients. Our outcome model was the first prognosis model for human bladder cancer progression prediction via integrative bioinformatics analysis, which may aid in clinical decision‐making.     

Semi-parametric area under the curve regression method for diagnostic studies with ordinal data

The classification accuracy of new diagnostic tests is based on receiver operating characteristic (ROC) curves. The area under the ROC curve (AUC) is one of the well-accepted summary measures for describing the accuracy of diagnostic tests. The AUC summary measure can vary by patient and testing characteristics. Thus, the performance of the test may be different in certain subpopulation of patients and readers. For this purpose, we propose a direct semi-parametric regression model for the non-parametric AUC measure for ordinal data while accounting for discrete and continuous covariates. The proposed method can be used to estimate the AUC value under degenerate data where certain rating categories are not observed. We will discuss the non-standard asymptotic theory, since the estimating functions were based on cross-correlated random variables. Simulation studies based on different classification models showed that the proposed model worked reasonably well with small percent bias and percent mean-squared error. The proposed method was applied to the prostate cancer study to estimate the AUC for four readers, and the carotid vessel study with age, gender, history of previous stroke, and total number of risk factors as covariates, to estimate the accuracy of the diagnostic test in the presence of subject-level covariates.     

Combining biomarkers to detect disease with application to prostate cancer

In early detection of disease, combinations of biomarkers promise improved discrimination over diagnostic tests based on single markers. An example of this is in prostate cancer screening, where additional markers have been sought to improve the specificity of the conventional Prostate-Specific Antigen (PSA) test. A marker of particular interest is the percent free PSA. Studies evaluating the benefits of percent free PSA reflect the need for a methodological approach that is statistically valid and useful in the clinical setting. This article presents methods that address this need. We focus on and-or combinations of biomarker results that we call logic rules and present novel definitions for the ROC curve and the area under the curve (AUC) that are applicable to this class of combination tests. Our estimates of the ROC and AUC are amenable to statistical inference including comparisons of tests and regression analysis. The methods are applied to data on free and total PSA levels among prostate cancer cases and matched controls enrolled in the Physicians' Health Study.     

Serum markers change for intraocular metastasis in renal cell carcinoma

Objective: Renal cell carcinoma is prone to early metastasis. In general, intraocular metastasis (IOM) is not common. In the present study, we studied the relationship between different biochemical indicators and the occurrence of IOM in renal cancer patients, and identified the potential risk factors.Methods: A retrospective analysis of the clinical data of 214 patients with renal cell carcinoma from October 2001 to August 2016 was carried out. The difference and correlation of various indicators between the two groups with or without IOM was analyzed, and binary logistic regression analysis was used to explore the risk factors of IOM in renal cancer patients. The diagnostic value of each independent related factor was calculated according to the receiver operating curve (ROC).Results: The level of neuron-specific enolase (NSE) in renal cell carcinoma patients with IOM was significantly higher than that in patients without IOM (P<0.05). There was no significant difference in alkaline phosphatase (ALP), hemoglobin (Hb), serum calcium concentration, α fetoprotein (AFP), carcinoembryonic antigen (CEA), CA-125 etc. between IOM group and non-IOM (NIOM) group (P>0.05). Binary logistic regression analysis showed that NSE was an independent risk factor for IOM in renal cell carcinoma patients (P<0.05). ROC curve shows that the factor has high accuracy in predicting IOM, and the area under the curve (AUC) is 0.774. The cut-off value of NSE was 49.5 U/l, the sensitivity was 72.2% and the specificity was 80.1%.Conclusion: NSE concentration is a risk factor for IOM in patients with renal cell cancer. If the concentration of NSE in the patient’s body is ≥49.5 U/l, disease monitoring and eye scans should be strengthened.     

An interpretation for the ROC curve and inference using GLM procedures

The accuracy of a medical diagnostic test is often summarized in a receiver operating characteristic (ROC) curve. This paper puts forth an interpretation for each point on the ROC curve as being a conditional probability of a test result from a random diseased subject exceeding that from a random nondiseased subject. This interpretation gives rise to new methods for making inference about ROC curves. It is shown that inference can be achieved with binary regression techniques applied to indicator variables constructed from pairs of test results, one component of the pair being from a diseased subject and the other from a nondiseased subject. Within the generalized linear model (GLM) binary regression framework, ROC curves can be estimated, and we highlight a new semiparametric estimator. Covariate effects can also be evaluated with the GLM models. The methodology is applied to a pancreatic cancer dataset where we use the regression framework to compare two different serum biomarkers. Asymptotic distribution theory is developed to facilitate inference and to provide insight into factors influencing variability of estimated model parameters.     

Semi-parametric ROC regression analysis with placement values

Advances in technology provide new diagnostic tests for early detection of disease. Frequently, these tests have continuous outcomes. One popular method to summarize the accuracy of such a test is the Receiver Operating Characteristic (ROC) curve. Methods for estimating ROC curves have long been available. To examine covariate effects, Pepe (1997, 2000) and Alonzo and Pepe (2002) proposed distribution-free approaches based on a parametric regression model for the ROC curve. Cai and Pepe (2002) extended the parametric ROC regression model by allowing an arbitrary non-parametric baseline function. In this paper, while we follow the same semi-parametric setting as in that paper, we highlight a new estimator that offers several improvements over the earlier work: superior efficiency, the ability to estimate the covariate effects without estimating the non-parametric baseline function and easy implementation with standard software. The methodology is applied to a case control dataset where we evaluate the accuracy of the prostate-specific antigen as a biomarker for early detection of prostate cancer. Simulation studies suggest that the new estimator under the semi-parametric model, while always being more robust, has efficiency that is comparable to or better than the Alonzo and Pepe (2002) estimator from the parametric model.     

Combining several screening tests: optimality of the risk score

The development of biomarkers for cancer screening is an active area of research. While several biomarkers exist, none is sufficiently sensitive and specific on its own for population screening. It is likely that successful screening programs will require combinations of multiple markers. We consider how to combine multiple disease markers for optimal performance of a screening program. We show that the risk score, defined as the probability of disease given data on multiple markers, is the optimal function in the sense that the receiver operating characteristic (ROC) curve is maximized at every point. Arguments draw on the Neyman-Pearson lemma. This contrasts with the corresponding optimality result of classic decision theory, which is set in a Bayesian framework and is based on minimizing an expected loss function associated with decision errors. Ours is an optimality result defined from a strictly frequentist point of view and does not rely on the notion of associating costs with misclassifications. The implication for data analysis is that binary regression methods can be used to yield appropriate relative weightings of different biomarkers, at least in large samples. We propose some modifications to standard binary regression methods for application to the disease screening problem. A flexible biologically motivated simulation model for cancer biomarkers is presented and we evaluate our methods by application to it. An application to real data concerning two ovarian cancer biomarkers is also presented. Our results are equally relevant to the more general medical diagnostic testing problem, where results of multiple tests or predictors are combined to yield a composite diagnostic test. Moreover, our methods justify the development of clinical prediction scores based on binary regression.     

Study on gastric cancer blood plasma based on surface-enhanced Raman spectroscopy combined with multivariate analysis

A surface-enhanced Raman spectroscopy (SERS) method combined with multivariate analysis was developed for non-invasive gastric cancer detection. SERS measurements were performed on two groups of blood plasma samples: one group from 32 gastric patients and the other group from 33 healthy volunteers. Tentative assignments of the Raman bands in the measured SERS spectra suggest interesting cancer-specific biomolecular changes, including an increase in the relative amounts of nucleic acid, collagen, phospholipids and phenylalanine and a decrease in the percentage of amino acids and saccharide in the blood plasma of gastric cancer patients as compared with those of healthy subjects. Principal components analysis (PCA) and linear discriminant analysis (LDA) were employed to develop effective diagnostic algorithms for classification of SERS spectra between normal and cancer plasma with high sensitivity (79.5%) and specificity (91%). A receiver operating characteristic (ROC) curve was employed to assess the accuracy of diagnostic algorithms based on PCA-LDA. The results from this exploratory study demonstrate that SERS plasma analysis combined with PCA-LDA has tremendous potential for the non-invasive detection of gastric cancers.     

Prediction-based structured variable selection through the receiver operating characteristic curves

In many clinical settings, a commonly encountered problem is to assess accuracy of a screening test for early detection of a disease. In these applications, predictive performance of the test is of interest. Variable selection may be useful in designing a medical test. An example is a research study conducted to design a new screening test by selecting variables from an existing screener with a hierarchical structure among variables: there are several root questions followed by their stem questions. The stem questions will only be asked after a subject has answered the root question. It is therefore unreasonable to select a model that only contains stem variables but not its root variable. In this work, we propose methods to perform variable selection with structured variables when predictive accuracy of a diagnostic test is the main concern of the analysis. We take a linear combination of individual variables to form a combined test. We then maximize a direct summary measure of the predictive performance of the test, the area under a receiver operating characteristic curve (AUC of an ROC), subject to a penalty function to control for overfitting. Since maximizing empirical AUC of the ROC of a combined test is a complicated nonconvex problem (Pepe, Cai, and Longton, 2006, Biometrics62, 221-229), we explore the connection between the empirical AUC and a support vector machine (SVM). We cast the problem of maximizing predictive performance of a combined test as a penalized SVM problem and apply a reparametrization to impose the hierarchical structure among variables. We also describe a penalized logistic regression variable selection procedure for structured variables and compare it with the ROC-based approaches. We use simulation studies based on real data to examine performance of the proposed methods. Finally we apply developed methods to design a structured screener to be used in primary care clinics to refer potentially psychotic patients for further specialty diagnostics and treatment.     

Comparison of ROC umbrella volumes with an application to the assessment of lung cancer diagnostic markers

Receiver operating characteristic (ROC) analysis is widely used to assess the ability of diagnostic markers to correctly classify into one of two disease classes. ROC surfaces and umbrella surfaces generalize the utility of ROC analysis when there are three disease classes. Identification of lung cancer diagnostic markers is an active area of research since prognosis for those diagnosed with lung cancer is so poor and there is not an accurate method for early detection of lung cancer. A study conducted for the assessment of DNA methylation markers motivated the comparison of ROC umbrella surfaces which is developed in this article using U-statistics and bootstrap methodology.     

动态增强磁共振扫描与超声弹性成像对乳腺癌的诊断价值探讨

目的:探讨动态增强磁共振成像扫描与超声弹性成像对乳腺癌良恶性肿瘤的诊断价值,为临床诊断提供影像学依据。方法:回顾性分析2009年10月至2013年5月在我院经穿刺或手术病理证实为乳腺癌的59例患者的临床资料,患者术前均行超声与动态增强MR检查。依据病理组织活检和临床随访分别评价动态增强MR和UE对乳腺癌诊断的准确性。结果:DCE-MRI检测共发现病灶59个,55个初步诊断乳腺恶性肿瘤(BI-RADS 4-5),4个诊断为良性(BI-RADS 3),诊断准确率为93.22%(55/59)。UE对59个病灶进行评分,54个评分为乳腺恶性肿瘤,5个评分为良性,诊断率为91.53%(54/59)。UE检测乳腺癌的敏感性明显低于DCE-MRI及DCE-MRI+UE,DCE-MRI检测乳腺癌的特异性明显低于UE及DCE-MRI+UE,差异具有统计学意义(P0.05)。DCE-MRI+UE诊断乳腺癌的准确率为96.61%(57/59),明显高于DCE-MRI或UE单独检测的准确率(P0.05)。结论:动态增强MR诊断乳腺癌的敏感性较高,而超声弹性成像的特异性较好,两者联合可提高诊断准确率,对乳腺癌的早期诊断具有重要的临床应用价值。     

A six-microRNA risk score model predicts prognosis in esophageal squamous cell carcinoma

Esophageal cancer ranks the eighth most common cancer and the sixth most common cause of cancer death worldwide. MicroRNAs (miRNAs) are small noncoding RNAs that regulate a wide variety of cancer-related cellular processes. In the current study, a series of previously published gene expression microarray data from Gene Expression Ominus and The Cancer Genome Atlas were downloaded and further divided into training, internal, and external validation sets. Least absolute shrinkage and selectionator operator Cox regression model along with 10-fold cross-validation was performed to select the miRNAs associated with the prognosis of esophageal squamous cell carcinoma (ESCC) and constructed a six-miRNA signature. Then the prediction accuracy of this signature was assessed in validation and test set using Kaplan–Meier analysis, time-dependent receiver operating characteristic (ROC) curves and dynamic area under the ROC curve. According to the result, the prediction accuracy of miRNA signature was much better than that of tumor–node–metastasis (TNM) stage in all the three sets. Stratified analysis also demonstrated that the predict ability of this signature was independent of TNM stage. Finally, function experiments including apoptosis and colony formation assay were performed to further reveal the regulatory role of miRNAs in ESCC. Our study demonstrated the promising potential application of this novel six-miRNA signature as an independent biomarker for survival prediction of ESCC patients.     

ROC curve estimation when covariates affect the verification process

A receiver operating characteristic (ROC) curve is commonly used to measure the accuracy of a medical test. It is a plot of the true positive fraction (sensitivity) against the false positive fraction (1-specificity) for increasingly stringent positivity criterion. Bias can occur in estimation of an ROC curve if only some of the tested patients are selected for disease verification and if analysis is restricted only to the verified cases. This bias is known as verification bias. In this paper, we address the problem of correcting for verification bias in estimation of an ROC curve when the verification process and efficacy of the diagnostic test depend on covariates. Our method applies the EM algorithm to ordinal regression models to derive ML estimates for ROC curves as a function of covariates, adjusted for covariates affecting the likelihood of being verified. Asymptotic variance estimates are obtained using the observed information matrix of the observed data. These estimates are derived under the missing-at-random assumption, which means that selection for disease verification depends only on the observed data, i.e., the test result and the observed covariates. We also address the issues of model selection and model checking. Finally, we illustrate the proposed method on data from a two-phase study of dementia disorders, where selection for verification depends on the screening test result and age.