A novel design for estimating relative accuracy of screening tests when complete disease verification is not feasible

The accuracy (sensitivity and specificity) of a new screening test can be compared with that of a standard test by applying both tests to a group of subjects in which disease status can be determined by a gold standard (GS) test. However, it is not always feasible to administer a GS test to all study subjects. For example, a study is planned to determine whether a new screening test for cervical cancer ("ThinPrep") is better than the standard test ("Pap"), and in this setting it is not feasible (or ethical) to determine disease status by biopsy in order to identify women with and without disease for participation in a study. When determination of disease status is not possible for all study subjects, the relative accuracy of two screening tests can still be estimated by using a paired screen-positive (PSP) design in which all subjects receive both screening tests, but only have the GS test if one of the screening tests is positive. Unfortunately in the cervical cancer example, the PSP design is also infeasible because it is not technically possible to administer both the ThinPrep and Pap at the same time. In this article, we describe a randomized paired screen-positive (RPSP) design in which subjects are randomized to receive one of the two screening tests initially, and only receive the other screening test and GS if the first screening test is positive. We derive maximum likelihood estimators and confidence intervals for the relative accuracy of the two screening tests, and assess the small sample behavior of these estimators using simulation studies. Sample size formulae are derived and applied to the cervical cancer screening trial example, and the efficiency of the RPSP design is compared with other designs.     

New Techniques in the Evaluation of Cerebrovascular Disease

Numerous reports in the literature indicate that various noninvasive vascular techniques can now be used to evaluate atherosclerosis at the carotid bifurcation. This article reviews noninvasive screening techniques currently available and being developed. Particular emphasis has been placed on the practicality of these techniques as well as their limitations. Our conclusions are that noninvasive techniques cannot be used as definitive screening tests for cerebrovascular disease. Although these tests are frequently useful when positive, the false negative rate of these tests would appear to be significant and variable in different hands. Nonstenotic ulcers are usually not detected and total occlusion often not differentiated from stenosis. These tests should be viewed as the beginning rather than the end result of a developing field. At present, contrast arteriography remains the definitive test to evaluate the presence and significance of extracranial cerebrovascular disease.     

Estimation of disease prevalence, true positive rate, and false positive rate of two screening tests when disease verification is applied on only screen-positives: a hierarchical model using multi-center data

Objectives: A model is proposed to estimate and compare cervical cancer screening test properties for third world populations when only subjects with a positive screen receive the gold standard test. Two fallible screening tests are compared, VIA and VILI. Methods: We extend the model of Berry et al. [1] to the multi-site case in order to pool information across sites and form better estimates for prevalences of cervical cancer, the true positive rates (TPRs), and false positive rates (FPRs). For 10 centers in five African countries and India involving more than 52,000 women, Bayesian methods were applied when gold standard results for subjects who screened negative on both tests were treated as missing. The Bayesian methods employed suitably correct for the missing screen negative subjects. The study included gold standard verification for all cases, making it possible to validate model-based estimation of accuracy using only outcomes of women with positive VIA or VILI result (ignoring verification of double negative screening test results) with the observed full data outcomes. Results: Across the sites, estimates for the sensitivity of VIA ranged from 0.792 to 0.917 while for VILI sensitivities ranged from 0.929 to 0.977. False positive estimates ranged from 0.056 to 0.256 for VIA and 0.085 to 0.269 for VILI. The pooled estimates for the TPR of VIA and VILI are 0.871 and 0.968, respectively, compared to the full data values of 0.816 and 0.918. Similarly, the pooled estimates for the FPR of VIA and VILI are 0.134 and 0.146, respectively, compared to the full data values of 0.144 and 0.146. Globally, we found VILI had a statistically significant higher sensitivity but no statistical difference for the false positive rates could be determined. Conclusion: Hierarchical Bayesian methods provide a straight forward approach to estimate screening test properties, prevalences, and to perform comparisons for screening studies where screen negative subjects do not receive the gold standard test. The hierarchical model with random effects used to analyze the sites simultaneously resulted in improved estimates compared to the single-site analyses with improved TPR estimates and nearly identical FPR estimates to the full data outcomes. Furthermore, higher TPRs but similar FPRs were observed for VILI compared to VIA.     

Testing the Independence of Two Diagnostic Tests

Consider two diagnostic procedures having binary outcomes. If one of the tests results in a positive finding, a more definitive diagnostic procedure will be administered to establish the presence or absence of a disease. The use of both tests will improve the overall screening sensitivity when the two tests are independent, compared with employing two tests that are positively correlated. We estimate the correlation coefficient of the two tests and derive statistical methods for testing the independence of the two diagnostic procedures conditional on disease status. The statistical tests are used to investigate the independence of mammography and clinical breast exams aimed at establishing the benefit of early detection of breast cancer. The data used in the analysis are obtained from periodic screening examinations of three randomized clinical trials of breast cancer screening. Analysis of each of these trials confirms the independence of the clinical breast and mammography examinations. Based on these three large clinical trials, we conclude that a clinical breast exam considerably increases the overall sensitivity relative to screening with mammography alone and should be routinely included in early breast cancer detection programs.     

Partial AUC estimation and regression

Accurate diagnosis of disease is a critical part of health care. New diagnostic and screening tests must be evaluated based on their abilities to discriminate diseased from nondiseased states. The partial area under the receiver operating characteristic (ROC) curve is a measure of diagnostic test accuracy. We present an interpretation of the partial area under the curve (AUC), which gives rise to a nonparametric estimator. This estimator is more robust than existing estimators, which make parametric assumptions. We show that the robustness is gained with only a moderate loss in efficiency. We describe a regression modeling framework for making inference about covariate effects on the partial AUC. Such models can refine knowledge about test accuracy. Model parameters can be estimated using binary regression methods. We use the regression framework to compare two prostate-specific antigen biomarkers and to evaluate the dependence of biomarker accuracy on the time prior to clinical diagnosis of prostate cancer.     

Improving the performance of positive selection inference by filtering unreliable alignment regions

Errors in the inferred multiple sequence alignment may lead to false prediction of positive selection. Recently, methods for detecting unreliable alignment regions were developed and were shown to accurately identify incorrectly aligned regions. While removing unreliable alignment regions is expected to increase the accuracy of positive selection inference, such filtering may also significantly decrease the power of the test, as positively selected regions are fast evolving, and those same regions are often those that are difficult to align. Here, we used realistic simulations that mimic sequence evolution of HIV-1 genes to test the hypothesis that the performance of positive selection inference using codon models can be improved by removing unreliable alignment regions. Our study shows that the benefit of removing unreliable regions exceeds the loss of power due to the removal of some of the true positively selected sites.     

Profile-likelihood inference for highly accurate diagnostic tests

We consider profile-likelihood inference based on the multinomial distribution for assessing the accuracy of a diagnostic test. The methods apply to ordinal rating data when accuracy is assessed using the area under the receiver operating characteristic (ROC) curve. Simulation results suggest that the derived confidence intervals have acceptable coverage probabilities, even when sample sizes are small and the diagnostic tests have high accuracies. The methods extend to stratified settings and situations in which the ratings are correlated. We illustrate the methods using data from a clinical trial on the detection of ovarian cancer.     

Consequences of the widespread use of antibiotics

The cancer problem is increasing as life expectancy increases and greater portions of the populace live to the age at which cancer is more likely. Early diagnosis still is difficult. Even with modern methods and with considerable public education with regard to cancer, the disease is often not diagnosed until it is beyond the stage at which cure might be effected. The need for a serodiagnostic test for general screening purposes for cancer detection is tremendous. The major objective of cancer serodiagnostic test methods is to discover a general test that will detect cancer in a high percentage of cases while it is in an early stage; that will give few "false positive" results; that can be done in any laboratory; and that is simple and inexpensive. Many serodiagnostic tests for cancer have been published but none has proven worthy of being a good general test to detect cancer. Yet unless some serodiagnostic test which will be suitable for general screening purposes is developed, it is difficult to see how there can be much improvement in the early diagnosis of cancer, particularly internal cancer. It is hoped that an open-minded attitude will be maintained by physicians on this subject. Recent reports of such a test being developed are encouraging and it is hoped that continued investigations will be confirmatory.     

Urinary analyte screening: a noninvasive detection method for Down syndrome?

Prenatal screening for Down syndrome using maternal serum markers achieves detection rates of 60-80% with a 5% false positive rate. Improvement in the accuracy of screening, as well as its ease and safety, will increase the use of such tests. The most effective of the current serum markers is human chorionic gonadotropin (hCG). Studies on beta core fragment (beta CF), the major urinary metabolite of hCG, have indicated that screening with beta CF and other markers measured in maternal urine might improve the detection of Down syndrome and provide a less expensive and simpler test. However, recent results have been unusually variable. Although it has great potential, the true clinical value of maternal urine screening to detect Down syndrome still remains to be determined.     

The effect of errors of diagnosis and frequency of examination on reported rates of disease

Follow-up studies of certain diseases such as cervical dysplasia and carcinoma in situ may employ repeated screening to determine disease incidence in selected cohorts. Where errors of diagnosis occur, the true cohort experience will be distorted by an amount dependent upon the frequency of screening and the magnitude of the probabilities of incorrect diagnoses. These effects are investigated through a simple model of follow-up which employs three basic assumptions: that the probability of false positive and false negative diagnoses are constant; that individuals diagnosed as positive are treated and removed from the study, irrespective of whether or not they actually have the disease; and that fixed intervals occur between follow-up exams. The analysis permits one to assess the amount of bias introduced, and numerical examples are provided.     

Diagnosis of Alzheimer's disease based on disease-specific autoantibody profiles in human sera

After decades of Alzheimer's disease (AD) research, the development of a definitive diagnostic test for this disease has remained elusive. The discovery of blood-borne biomarkers yielding an accurate and relatively non-invasive test has been a primary goal. Using human protein microarrays to characterize the differential expression of serum autoantibodies in AD and non-demented control (NDC) groups, we identified potential diagnostic biomarkers for AD. The differential significance of each biomarker was evaluated, resulting in the selection of only 10 autoantibody biomarkers that can effectively differentiate AD sera from NDC sera with a sensitivity of 96.0% and specificity of 92.5%. AD sera were also distinguishable from sera obtained from patients with Parkinson's disease and breast cancer with accuracies of 86% and 92%, respectively. Results demonstrate that serum autoantibodies can be used effectively as highly-specific and accurate biomarkers to diagnose AD throughout the course of the disease.     

Prevalence screening for ovarian cancer in postmenopausal women by CA 125 measurement and ultrasonography.

OBJECTIVE--To assess the performance of the sequential combination of serum CA 125 measurement and ultrasonography in screening for ovarian cancer. DESIGN--The serum CA 125 concentration of each subject was determined and those with a concentration > or = 30 U/ml were recalled for abdominal ultrasonography. If ultrasonography gave abnormal results surgical investigation was arranged. Volunteers were followed up by annual postal questionnaire. SETTING--General practice, occupational health departments, ovarian cancer screening clinic. SUBJECTS--22,000 women volunteers who were postmenopausal and aged over 45 years. MAIN OUTCOME MEASURES--Apparent sensitivity, specificity, positive predictive value, years of cancer detected. RESULTS--41 women had a positive screening result and were investigated surgically. 11 had ovarian cancer (true positive result) and 30 had other disorders or no abnormality (false positive result). Of the 21,959 volunteers with a negative screening result, eight subsequently presented clinically with ovarian cancer (false negative result) and 21,951 had not developed ovarian cancer during follow up (apparent true negative result). The screening protocol achieved a specificity of 99.9%, a positive predictive value of 26.8%, and an apparent sensitivity of 78.6% and 57.9% at one year and two year follow up respectively. The estimated number of years of cancer detected by the prevalence screen was 1.4 years. CONCLUSIONS--This screening protocol is highly specific for ovarian cancer and can detect a substantial proportion of cases at a preclinical stage. Further investigation is required to determine the effect of the screening protocol on the ratio of early to late stage disease detected and on mortality from ovarian cancer.     

A model‐based solution for observational errors in laboratory studies

Molecular techniques for detecting microorganisms, macroorganisms and infectious agents are susceptible to false‐negative and false‐positive errors. If left unaddressed, these observational errors may yield misleading inference concerning occurrence, prevalence, sensitivity, specificity and covariate relationships. Occupancy models are widely used to account for false‐negative errors and more recently have even been used to address false‐positive errors, too. Current modelling options assume false‐positive errors only occur in truly negative samples, an assumption that yields biased inference concerning detection because a positive sample could be classified as such not because the target agent was successfully detected, but rather due to a false‐positive test result. We present an extension to the occupancy modelling framework that allows false‐positive errors in both negative and positive samples, thereby providing unbiased inference concerning occurrence and detection, as well as reliable conclusions about the efficacy of sampling designs, handling protocols and diagnostic tests. We apply the model to simulated data, showing that it recovers known parameters and outperforms other approaches that are commonly used when confronted with observation errors. We then apply the model to an experimental data set on Batrachochytrium dendrobatidis, a pathogenic fungus that is implicated in the global decline or extinction of hundreds of amphibian species. The model‐based approach we present is not only useful for obtaining reliable inference when data are contaminated with observational errors, but also eliminates the need for establishing arbitrary thresholds or decision rules that have hidden and unintended consequences.     

Screening for ovarian,prostatic, and testicular cancers.

Screening for cancer should not be offered routinely to a symptomatic people on a population basis unless it has been shown to be effective in reducing mortality in randomised controlled trials. A suitable screening test should have high sensitivity and specificity and a high positive predictive value. There is an ethical imperative to ensure that the benefit to each person from screening is likely to outweight the possible harm. Preliminary studies have identified suitable screening tests for ovarian cancer, and a randomised controlled trail is about to start. There is considerable controversy about whether to screen for prostatic cancer. Likewise, there is uncertainty about the best means of treating localised prostatic cancer. Screening for prostatic cancer raises important ethical considerations which should not be ignored. Testicular self examination is of unproved benefit. Although there is a need for education about the early signs and symptoms of testicular cancer to reduce delay at presentation, a case cannot be made for screening.     

Venous volume displacement plethysmography: Its diagnostic value in deep venous thrombosis as determined by receiver operator characteristic curves

The pitfall of several reviews of noninvasive venous assessment has been the expression of the test data solely in terms of diagnostic accuracy (the number of correct tests in ratio to all tests performed), where results of a test will vary according to disease prevalence. The advantages of receiver operator characteristic curve analysis are twofold: (1) it describes the dynamic relationship between sensitivity (the ratio of the number of true positive tests to the patients with deep venous thrombosis) and specificity (the ratio of true negative tests to the number of patients with no deep venous thrombosis) independent of disease prevalence; and (2) the threshold criteria that defines a positive test can be set by the best balance between sensitivity and specificity and then applied to a given patient population for its diagnostic accuracy. Venous volume plethysmography is a widely used, simple and rapid method. It was compared to the "gold standard" of phlebography in a prospective blind study of 70 limbs that were clinically suspect of having deep venous thrombosis (DVT). Venous volume displacement plethysmography was defined objectively by three quantitative parameters: (1) maximum venous outflow, (2) integer ratio, and (3) segmental venous capacitance ratio. The DVT (22 to 70 positive phlebograms) was divided by anatomic location into either calf vein DVT or proximal DVT (popliteal vein or above). By combining these three parameters, a balance between sensitivity and specificity was obtained to provide a rapid, objective method for screening patients with suspected DVT.     

Evaluation of five hepatitis C virus screening tests and two supplemental assays: performance when testing sera from sexually transmitted diseases clinic attendees in the USA

The performances of five screening tests (recombinant peptide-based first and second generation tests from Abbott and Ortho, and a synthetic peptide-based test from Biochem Immunosystems) and two supplemental tests: recombinant peptide- based, Abbott neutralization test and Chiron second generation recombinant immunoblot assay (RIBA 2), were evaluated for their ability to detect hepatitis C virus (HCV) antibodies in a population of 276 individuals attending a sexually transmitted diseases (STD) clinic in the USA. Although the five screening tests produced a variable number (35-62) of repeatedly reactive samples, only 13% (36/276) were classified as true positives by the supplemental tests. Thirty-four of the 36 were reactive by all screening tests and 32 of the true positives were reactive by both supplemental tests, while 2 did not neutralize but were reactive in the RIBA 2 test. Of the remaining 2 of the true positives which were discordant by several of the screening assays, 1 was confirmed by both supplemental assays but the other required a chemiluminescent enhancement technique to show positivity in RIBA 2. The sensitivities of the first and second generation Abbott and Ortho tests ranged from 97% to 100% and that of the Biochem test was 94%. The specificities of these tests ranged from 89.2% to 99.6%. The second generation Ortho test presented 9.4% (26/276) false positives. The use of second generation Ortho as a screening test would lead to an excessive number of confirmatory false positives. the positive predictive values of the screening tests ranged from 58.1% to 97.1%. Although the synthetic peptide based Biochem test exhibited the best overall indices, the presence of 2 false negative results would prevent its use as a singular screening test. Nevertheless its high specificity may lend itself to be used as a second screening test before confirmatory testing with RIBA 2.     

The early detection of cervical cancer. The current and changing landscape of cervical disease detection

Cervical cancer prevention has undergone dramatic changes over the past decade. With the introduction of human papillomavirus (HPV) vaccination, some countries have seen a dramatic decline in HPV‐mediated cervical disease. However, widespread implementation has been limited by economic considerations and the varying healthcare priorities of different countries, as well as by vaccine availability and, in some instances, vaccine hesitancy amongst the population/government. In this environment, it is clear that cervical screening will retain a critical role in the prevention of cervical cancer and will in due course need to adapt to the changing incidence of HPV‐associated neoplasia. Cervical screening has, for many years, been performed using Papanicolaou staining of cytology samples. As our understanding of the role of HPV in cervical cancer progression has advanced, and with the availability of sensitive detection systems, cervical screening now incorporates HPV testing. Although such tests improve disease detection, they are not specific, and cannot discriminate high‐grade from low‐grade disease. This has necessitated the development of effective triage approaches to stratify HPV‐positive women according to their risk of cancer progression. Although cytology triage remains the mainstay of screening, novel strategies under evaluation include DNA methylation, biomarker detection and the incorporation of artificial intelligence systems to detect cervical abnormalities. These tests, which can be partially anchored in a molecular understanding of HPV pathogenesis, will enhance the sensitivity of disease detection and improve patient outcomes. This review will provide insight on these innovative methodologies while explaining their scientific basis drawing from our understanding of HPV tumour biology.     

The effect of conditional dependence on the evaluation of diagnostic tests

The accuracy of a new diagnostic test is often determined by comparison with a reference test which also has unknown error rates. Maximum likelihood estimation of the error rates of both tests is possible if they are simultaneously applied to two populations with different disease prevalences. The estimation procedure assumes that the two tests are independent, conditional on a subject's true diagnostic status. If the tests are conditionally dependent, error rates for both tests can be substantially underestimated. Estimators for the prevalence rates in the two populations can be positively or negatively biased, depending on the relative magnitude of the two conditional covariances and the value of the prevalence parameter.     

Accuracy of haplotype reconstruction from haplotype-tagging single-nucleotide polymorphisms

Many investigators are now using haplotype-tagging single-nucleotide polymorphism (htSNPs) as a way of screening regions of the genome for association with disease. A common approach is to genotype htSNPs in a study population and to use this information to draw inferences about each individual's haplotypic makeup, including SNPs that were not directly genotyped. To test the validity of this approach, we simulated the exercise of typing htSNPs in a large sample of individuals and compared the true and inferred haplotypes. The accuracy of haplotype inference varied, depending on the method of selecting htSNPs, the linkage-disequilibrium structure of the region, and the amount of missing data. At the stage of selection of htSNPs, haplotype-block-based methods required a larger number of htSNPs than did unstructured methods but gave lower levels of error in haplotype inference, particularly when there was a significant amount of missing data. We present a Web-based utility that allows investigators to compare the likely error rates of different sets of htSNPs and to arrive at an economical set of htSNPs that provides acceptable levels of accuracy in haplotype inference.     

Screening for possible human carcinogens and mutagens. False positives, false negatives: statistical implications

A screening method aimed at identifying potential human carcinogens using either animal cancer bioassays or short-term genotoxic assays has 4 possible results: true positive, true negative, false positive and false negative. Such a categorisation is superficially similar to the results of hypothesis testing in a statistical analysis. In this latter case the false positive rate is determined by the significance level of the test and the false negative rate by the statistical power of the test. Although the two types of categorisation appear somewhat similar, different statistical issues are involved in their interpretation. Statistical methods appropriate for the analysis of the results of a series of assays include the use of Bayes' theorem and multivariate methods such as clustering techniques for the selection of batteries of short-term test capable of a better prediction of potential carcinogens. The conclusions drawn from such studies are dependent upon the estimates of values of sensitivity and specificity used, the choice of statistical method and the nature of the data set. The statistical issues resulting from the analysis of specific genotoxicity experiments involve the choice of suitable experimental designs and appropriate analyses together with the relationship of statistical significance to biological importance. The purpose of statistical analysis should increasingly be to estimate and explore effects rather than for formal hypothesis testing.