Semiparametric transformation models for multiple continuous biomarkers in ROC analysis

Recent technological advances continue to provide noninvasive and more accurate biomarkers for evaluating disease status. One standard tool for assessing the accuracy of diagnostic tests is the receiver operating characteristic (ROC) curve. Few statistical methods exist to accommodate multiple continuous‐scale biomarkers in the framework of ROC analysis. In this paper, we propose a method to integrate continuous‐scale biomarkers to optimize classification accuracy. Specifically, we develop semiparametric transformation models for multiple biomarkers. We assume that unknown and marker‐specific transformations of biomarkers follow a multivariate normal distribution. Our models accommodate biomarkers subject to limits of detection and account for the dependence among biomarkers by including a subject‐specific random effect. We also propose a diagnostic measure using an optimal linear combination of the transformed biomarkers. Our diagnostic rule does not depend on any monotone transformation of biomarkers and is not sensitive to extreme biomarker values. Nonparametric maximum likelihood estimation (NPMLE) is used for inference. We show that the parameter estimators are asymptotically normal and efficient. We illustrate our semiparametric approach using data from the Endometriosis, Natural History, Diagnosis, and Outcomes (ENDO) study.     

Adjusting confounders in ranking biomarkers: a model-based ROC approach

High-throughput studies have been extensively conducted in the research of complex human diseases. As a representative example, consider gene-expression studies where thousands of genes are profiled at the same time. An important objective of such studies is to rank the diagnostic accuracy of biomarkers (e.g. gene expressions) for predicting outcome variables while properly adjusting for confounding effects from low-dimensional clinical risk factors and environmental exposures. Existing approaches are often fully based on parametric or semi-parametric models and target evaluating estimation significance as opposed to diagnostic accuracy. Receiver operating characteristic (ROC) approaches can be employed to tackle this problem. However, existing ROC ranking methods focus on biomarkers only and ignore effects of confounders. In this article, we propose a model-based approach which ranks the diagnostic accuracy of biomarkers using ROC measures with a proper adjustment of confounding effects. To this end, three different methods for constructing the underlying regression models are investigated. Simulation study shows that the proposed methods can accurately identify biomarkers with additional diagnostic power beyond confounders. Analysis of two cancer gene-expression studies demonstrates that adjusting for confounders can lead to substantially different rankings of genes.     

Partial AUC estimation and regression

Accurate diagnosis of disease is a critical part of health care. New diagnostic and screening tests must be evaluated based on their abilities to discriminate diseased from nondiseased states. The partial area under the receiver operating characteristic (ROC) curve is a measure of diagnostic test accuracy. We present an interpretation of the partial area under the curve (AUC), which gives rise to a nonparametric estimator. This estimator is more robust than existing estimators, which make parametric assumptions. We show that the robustness is gained with only a moderate loss in efficiency. We describe a regression modeling framework for making inference about covariate effects on the partial AUC. Such models can refine knowledge about test accuracy. Model parameters can be estimated using binary regression methods. We use the regression framework to compare two prostate-specific antigen biomarkers and to evaluate the dependence of biomarker accuracy on the time prior to clinical diagnosis of prostate cancer.     

Estimating prevalence and test accuracy in disease ecology: How Bayesian latent class analysis can boost or bias imperfect test results

1. Obtaining accurate estimates of disease prevalence is crucial for the monitoring and management of wildlife populations but can be difficult if different diagnostic tests yield conflicting results and if the accuracy of each diagnostic test is unknown. Bayesian latent class analysis (BLCA) modeling offers a potential solution, providing estimates of prevalence levels and diagnostic test accuracy under the realistic assumption that no diagnostic test is perfect.
2. In typical applications of this approach, the specificity of one test is fixed at or close to 100%, allowing the model to simultaneously estimate the sensitivity and specificity of all other tests, in addition to infection prevalence. In wildlife systems, a test with near‐perfect specificity is not always available, so we simulated data to investigate how decreasing this fixed specificity value affects the accuracy of model estimates.
3. We used simulations to explore how the trade‐off between diagnostic test specificity and sensitivity impacts prevalence estimates and found that directional biases depend on pathogen prevalence. Both the precision and accuracy of results depend on the sample size, the diagnostic tests used, and the true infection prevalence, so these factors should be considered when applying BLCA to estimate disease prevalence and diagnostic test accuracy in wildlife systems. A wildlife disease case study, focusing on leptospirosis in California sea lions, demonstrated the potential for Bayesian latent class methods to provide reliable estimates under real‐world conditions.
4. We delineate conditions under which BLCA improves upon the results from a single diagnostic across a range of prevalence levels and sample sizes, demonstrating when this method is preferable for disease ecologists working in a wide variety of pathogen systems.

     

Insights into latent class analysis of diagnostic test performance

Latent class analysis is used to assess diagnostic test accuracy when a gold standard assessment of disease is not available but results of multiple imperfect tests are. We consider the simplest setting, where 3 tests are observed and conditional independence (CI) is assumed. Closed-form expressions for maximum likelihood parameter estimates are derived. They show explicitly how observed 2- and 3-way associations between test results are used to infer disease prevalence and test true- and false-positive rates. Although interesting and reasonable under CI, the estimators clearly have no basis when it fails. Intuition for bias induced by conditional dependence follows from the analytic expressions. Further intuition derives from an Expectation Maximization (EM) approach to calculating the estimates. We discuss implications of our results and related work for settings where more than 3 tests are available. We conclude that careful justification of assumptions about the dependence between tests in diseased and nondiseased subjects is necessary in order to ensure unbiased estimates of prevalence and test operating characteristics and to provide these estimates clinical interpretations. Such justification must be based in part on a clear clinical definition of disease and biological knowledge about mechanisms giving rise to test results.     

Bayesian sample size determination for prevalence and diagnostic test studies in the absence of a gold standard test

Planning studies involving diagnostic tests is complicated by the fact that virtually no test provides perfectly accurate results. The misclassification induced by imperfect sensitivities and specificities of diagnostic tests must be taken into account, whether the primary goal of the study is to estimate the prevalence of a disease in a population or to investigate the properties of a new diagnostic test. Previous work on sample size requirements for estimating the prevalence of disease in the case of a single imperfect test showed very large discrepancies in size when compared to methods that assume a perfect test. In this article we extend these methods to include two conditionally independent imperfect tests, and apply several different criteria for Bayesian sample size determination to the design of such studies. We consider both disease prevalence studies and studies designed to estimate the sensitivity and specificity of diagnostic tests. As the problem is typically nonidentifiable, we investigate the limits on the accuracy of parameter estimation as the sample size approaches infinity. Through two examples from infectious diseases, we illustrate the changes in sample sizes that arise when two tests are applied to individuals in a study rather than a single test. Although smaller sample sizes are often found in the two-test situation, they can still be prohibitively large unless accurate information is available about the sensitivities and specificities of the tests being used.     

The diagnostic performance of current tumour markers in surveillance for recurrent testicular cancer: A diagnostic test accuracy systematic review

In this diagnostic test accuracy systematic review we summarise the evidence on the diagnostic accuracy of blood α-fetoprotein (AFP), human chorionic gonadotropin (HCG) and lactate dehydrogenase (LDH) in surveillance for testicular cancer recurrence in adults. We searched four electronic databases for studies that reported the diagnostic accuracy of HCG, AFP, and/or LDH in sufficient detail for sensitivity and specificity to be calculated by extracting a 2 × 2 table comparing biomarker positivity with testicular cancer recurrence. Screening, data extraction and QUADAS-2 quality assessment were completed by two independent reviewers. From 2406 studies, nine met our inclusion criteria. Eight reported data at the per-patient level. Sample sizes were small (range 5 to 449 patients) and clinical heterogeneity precluded meta-analysis. In most studies the specificity for recurrence with AFP and HCG was high (90–100%) but sensitivity was often relatively low, suggesting that many recurrences would not be detected by tumour markers alone. The diagnostic performance of LDH appears poorer. Studies were methodologically weak, with probable selection, incorporation and partial verification bias, and many studies were excluded for not reporting on recurrence-free patients. Limitations including small sample sizes, high heterogeneity, and inconsistent and incomplete reporting mean these results must be interpreted with caution. Despite inclusion of biomarkers in international surveillance guidance, there remains a lack of high quality evidence about their accuracy, optimal thresholds, and the most effective surveillance strategy in relation to contemporary investigative modalities. Higher quality research using data from modern-day follow-up cohorts is necessary to identify opportunities to reduce unnecessary testing.     

A cautionary note on the robustness of latent class models for estimating diagnostic error without a gold standard

Modeling diagnostic error without a gold standard has been an active area of biostatistical research. In a majority of the approaches, model-based estimates of sensitivity, specificity, and prevalence are derived from a latent class model in which the latent variable represents an individual's true unobserved disease status. For simplicity, initial approaches assumed that the diagnostic test results on the same subject were independent given the true disease status (i.e., the conditional independence assumption). More recently, various authors have proposed approaches for modeling the dependence structure between test results given true disease status. This note discusses a potential problem with these approaches. Namely, we show that when the conditional dependence between tests is misspecified, estimators of sensitivity, specificity, and prevalence can be biased. Importantly, we demonstrate that with small numbers of tests, likelihood comparisons and other model diagnostics may not be able to distinguish between models with different dependence structures. We present asymptotic results that show the generality of the problem. Further, data analysis and simulations demonstrate the practical implications of model misspecification. Finally, we present some guidelines about the use of these models for practitioners.     

A general approach to sample size determination for prevalence surveys that use dual test protocols

We develop a Bayesian simulation based approach for determining the sample size required for estimating a binomial probability and the difference between two binomial probabilities where we allow for dependence between two fallible diagnostic procedures. Examples include estimating the prevalence of disease in a single population based on results from two imperfect diagnostic tests applied to sampled individuals, or surveys designed to compare the prevalences of two populations using diagnostic outcomes that are subject to misclassification. We propose a two stage procedure in which the tests are initially assumed to be independent conditional on true disease status (i.e. conditionally independent). An interval based sample size determination scheme is performed under this assumption and data are collected and used to test the conditional independence assumption. If the data reveal the diagnostic tests to be conditionally dependent, structure is added to the model to account for dependence and the sample size routine is repeated in order to properly satisfy the criterion under the correct model. We also examine the impact on required sample size when adding an extra heterogeneous population to a study.     

Leveraging nonlinear dynamic models to predict progression of neuroimaging biomarkers

Using biomarkers to model disease course effectively and make early prediction is a challenging but critical path to improving diagnostic accuracy and designing preventive trials for neurological disorders. Leveraging the domain knowledge that certain neuroimaging biomarkers may reflect the disease pathology, we propose a model inspired by the neural mass model from cognitive neuroscience to jointly model nonlinear dynamic trajectories of the biomarkers. Under a nonlinear mixed‐effects model framework, we introduce subject‐ and biomarker‐specific random inflection points to characterize the critical time of underlying disease progression as reflected in the biomarkers. A latent liability score is shared across biomarkers to pool information. Our model allows assessing how the underlying disease progression will affect the trajectories of the biomarkers, and, thus, is potentially useful for individual disease management or preventive therapeutics. We propose an EM algorithm for maximum likelihood estimation, where in the E step, a normal approximation is used to facilitate numerical integration. We perform extensive simulation studies and apply the method to analyze data from a large multisite natural history study of Huntington's Disease (HD). The results show that some neuroimaging biomarker inflection points are early signs of the HD onset. Finally, we develop an online tool to provide the individual prediction of the biomarker trajectories given the medical history and baseline measurements.     

Improved classification of Alzheimer's disease data via removal of nuisance variability

Diagnosis of Alzheimer's disease is based on the results of neuropsychological tests and available supporting biomarkers such as the results of imaging studies. The results of the tests and the values of biomarkers are dependent on the nuisance features, such as age and gender. In order to improve diagnostic power, the effects of the nuisance features have to be removed from the data. In this paper, four types of interactions between classification features and nuisance features were identified. Three methods were tested to remove these interactions from the classification data. In stratified analysis, a homogeneous subgroup was generated from a training set. Data correction method utilized linear regression model to remove the effects of nuisance features from data. The third method was a combination of these two methods. The methods were tested using all the baseline data from the Alzheimer's Disease Neuroimaging Initiative database in two classification studies: classifying control subjects from Alzheimer's disease patients and discriminating stable and progressive mild cognitive impairment subjects. The results show that both stratified analysis and data correction are able to statistically significantly improve the classification accuracy of several neuropsychological tests and imaging biomarkers. The improvements were especially large for the classification of stable and progressive mild cognitive impairment subjects, where the best improvements observed were 6% units. The data correction method gave better results for imaging biomarkers, whereas stratified analysis worked well with the neuropsychological tests. In conclusion, the study shows that the excess variability caused by nuisance features should be removed from the data to improve the classification accuracy, and therefore, the reliability of diagnosis making.     

Bayesian approaches to modeling the conditional dependence between multiple diagnostic tests

Many analyses of results from multiple diagnostic tests assume the tests are statistically independent conditional on the true disease status of the subject. This assumption may be violated in practice, especially in situations where none of the tests is a perfectly accurate gold standard. Classical inference for models accounting for the conditional dependence between tests requires that results from at least four different tests be used in order to obtain an identifiable solution, but it is not always feasible to have results from this many tests. We use a Bayesian approach to draw inferences about the disease prevalence and test properties while adjusting for the possibility of conditional dependence between tests, particularly when we have only two tests. We propose both fixed and random effects models. Since with fewer than four tests the problem is nonidentifiable, the posterior distributions are strongly dependent on the prior information about the test properties and the disease prevalence, even with large sample sizes. If the degree of correlation between the tests is known a priori with high precision, then our methods adjust for the dependence between the tests. Otherwise, our methods provide adjusted inferences that incorporate all of the uncertainty inherent in the problem, typically resulting in wider interval estimates. We illustrate our methods using data from a study on the prevalence of Strongyloides infection among Cambodian refugees to Canada.     

Serial determination of CEA and CA 15.3 in breast cancer follow-up: An assessment of their diagnostic accuracy for the detection of tumour recurrences

We studied the diagnostic accuracy of carcinoembryonic antigen (CEA) and cancer antigen 15.3 (CA 15.3) in detecting breast cancer recurrence. Biomarker follow-up determinations, made over 900 patients, were related to local–regional or distant recurrence using statistical models for longitudinal data. The diagnostic accuracy was quantified in terms of sensitivity, specificity and Youden index. The biomarkers were poorly predictive of local–regional recurrence. As for distant recurrence, the best diagnostic accuracy was obtained considering the two biomarkers jointly and combining two positivity criteria: a value above the normal limit or a difference between two consecutive measurements greater than the critical difference for at least one biomarker. A third criterion, based on within-patient comparison between follow-up determinations and a baseline, failed to improve the above result. CEA and CA 15.3 might play a role in patient monitoring during follow-up for the search of distant recurrence.     

Prospects for developing an accurate diagnostic biomarker panel for low prevalence cancers

Background

Early detection screening of asymptomatic populations for low prevalence cancers requires a highly specific test in order to limit the cost and anxiety produced by falsely positive identifications. Most solid cancers are a heterogeneous collection of diseases as they develop from various combinations of genetic lesions and epigenetic modifications. Therefore, it is unlikely that a single test will discriminate all cases of any particular cancer type. We propose a novel, intuitive biomarker panel design that accommodates disease heterogeneity by allowing for diverse biomarker selection that increases diagnostic accuracy.

Methods

Using characteristics of nine pancreatic ductal adenocarcinoma (PDAC) biomarkers measured in human sera, we modeled the behavior of biomarker panels consisting of a sum of indicator variables representing a subset of biomarkers within a larger biomarker data set. We then chose a cutoff for the sum to force specificity to be high and delineated the number of biomarkers required for adequate sensitivity of PDAC in our panel design.

Results

The model shows that a panel consisting of 40 non-correlated biomarkers characterized individually by 32% sensitivity at 95% specificity would require any 7 biomarkers to be above their respective thresholds and would result in a panel specificity and sensitivity of 99% each.

Conclusions

A highly accurate blood-based diagnostic panel can be developed from a reasonable number of individual serum biomarkers that are relatively weak classifiers when used singly. A panel constructed as described is advantageous in that a high level of specificity can be forced, accomplishing a prerequisite for screening asymptomatic populations for low-prevalence cancers.

     

Mathematical modelling and computer simulation of Brownian motion and hybridisation of nanoparticle–bioprobe–polymer complexes in the low concentration limit

A wide variety of nano-biotechnological applications are being developed for nanoparticles based on in vitro diagnostic and imaging systems. Some of these systems allow highly sensitive detection of molecular biomarkers. Frequently, the very low concentration of the biomarkers makes very difficult the mathematical simulation of the motion of nanoparticles based on classical, partial differential equations. We address the issue of incubation times for low concentration systems using Monte Carlo simulations. We describe a mathematical model and computer simulation of Brownian motion of nanoparticle–bioprobe–polymer contrast agent complexes and their hybridisation to immobilised targets. We present results for the dependence of incubation times on the number of particles available for detection, and the geometric layout of the DNA-detection assay on the chip.     

The impact of cerebrospinal fluid biomarkers on the diagnosis of Alzheimer's disease

The cerebrospinal fluid (CSF) biomarkers β-amyloid(1-42) (Aβ(1-42)), total tau protein (T-tau), and tau phosphorylated at threonine 181 (P-tau(181P)) are gradually finding their way into routine clinical practice as an affirmative diagnostic tool for Alzheimer's disease (AD). These biomarkers have also been implemented in the revised diagnostic criteria for AD. The combination of the CSF biomarkers Aβ(1-42), T-tau, and P-tau(181P) leads to high (around 80%) levels of sensitivity, specificity, and diagnostic accuracy for discrimination between AD and controls (including psychiatric disorders like depression) and can be applied for diagnosing AD in the predementia phases of the disease (mild cognitive impairment). The added value of CSF biomarkers could lie within those cases in which the clinical diagnostic work-up is not able to discriminate between AD and non-AD dementias. However, their discriminatory power for the differential diagnosis of dementia is suboptimal. Other CSF biomarkers, especially those that are reflective of the pathology of non-AD dementia etiologies, could improve the accuracy of differential dementia diagnosis. CSF biomarkers will be of help to establish a correct and early AD diagnosis, even in the preclinical stages of the disease, which will be of importance once disease-modifying drugs for AD become available. Variation in biomarker measurements still jeopardize the introduction of CSF biomarkers into routine clinical practice and clinical trials, but several national and international standardization initiatives are ongoing.     

Differentially Expressed RNA from Public Microarray Data Identifies Serum Protein Biomarkers for Cross-Organ Transplant Rejection and Other Conditions

Serum proteins are routinely used to diagnose diseases, but are hard to find due to low sensitivity in screening the serum proteome. Public repositories of microarray data, such as the Gene Expression Omnibus (GEO), contain RNA expression profiles for more than 16,000 biological conditions, covering more than 30% of United States mortality. We hypothesized that genes coding for serum- and urine-detectable proteins, and showing differential expression of RNA in disease-damaged tissues would make ideal diagnostic protein biomarkers for those diseases. We showed that predicted protein biomarkers are significantly enriched for known diagnostic protein biomarkers in 22 diseases, with enrichment significantly higher in diseases for which at least three datasets are available. We then used this strategy to search for new biomarkers indicating acute rejection (AR) across different types of transplanted solid organs. We integrated three biopsy-based microarray studies of AR from pediatric renal, adult renal and adult cardiac transplantation and identified 45 genes upregulated in all three. From this set, we chose 10 proteins for serum ELISA assays in 39 renal transplant patients, and discovered three that were significantly higher in AR. Interestingly, all three proteins were also significantly higher during AR in the 63 cardiac transplant recipients studied. Our best marker, serum PECAM1, identified renal AR with 89% sensitivity and 75% specificity, and also showed increased expression in AR by immunohistochemistry in renal, hepatic and cardiac transplant biopsies. Our results demonstrate that integrating gene expression microarray measurements from disease samples and even publicly-available data sets can be a powerful, fast, and cost-effective strategy for the discovery of new diagnostic serum protein biomarkers.     

Identification of biomarkers for colorectal cancer through proteomics-based approaches

The early detection of colorectal cancer is one of the great challenges in the battle against this disease. However, owing to its heterogeneous character, single markers are not likely to provide sufficient diagnostic power to be used in colorectal cancer population screens. This review provides an overview of recent studies aimed at the discovery of new diagnostic protein markers through proteomics-based approaches. It indicates that studies that start with the proteomic analysis of tumor tissue or tumor cell lines (near the source) have a high potential to yield novel and colorectal cancer-specific biomarkers. In the next step, the diagnostic accuracy of these candidate markers can be assessed by a targeted ELISA assay using serum from colorectal cancer patients and healthy controls. Instead, direct proteomic analysis of serum yields predominantly secondary markers composed of fragments of abundant serum proteins that may be associated with tumor-associated protease activity, and alternatively, immunoproteomic analysis of the serum antibody repertoire provides a valuable tool to identify the molecular imprint of colorectal cancer-associated antigens directly from patient serum samples. The latter approach also allows a relatively easy translation into targeted assays. Eventually, multimarker assays should be developed to reach a diagnostic accuracy that meets the stringent criteria for colorectal cancer screening at the population level.     

Prospective studies of diagnostic test accuracy when disease prevalence is low

Prospective studies of diagnostic test accuracy have important advantages over retrospective designs. Yet, when the disease being detected by the diagnostic test(s) has a low prevalence rate, a prospective design can require an enormous sample of patients. We consider two strategies to reduce the costs of prospective studies of binary diagnostic tests: stratification and two-phase sampling. Utilizing neither, one, or both of these strategies provides us with four study design options: (1) the conventional design involving a simple random sample (SRS) of patients from the clinical population; (2) a stratified design where patients from higher-prevalence subpopulations are more heavily sampled; (3) a simple two-phase design using a SRS in the first phase and selection for the second phase based on the test results from the first; and (4) a two-phase design with stratification in the first phase. We describe estimators for sensitivity and specificity and their variances for each design, along with sample size estimation. We offer some recommendations for choosing among the various designs. We illustrate the study designs with two examples.