Diagnosis of tuberculosis in macaques, using whole-blood in vitro interferon-gamma (PRIMAGAM) testing

During the fall of 2001, a tuberculosis outbreak caused by Mycobacterium bovis occurred in a conditioned colony of rhesus (Macaca mulatta) and cynomolgus (Macaca fascicularis) macaques at Stanford University School of Medicine. During this outbreak, we evaluated the diagnostic performance of a new in vitro tuberculosis screening test (PRIMAGAM). The PRIMAGAM test measures the interferon-gamma (IFNgamma) response to purified protein derivatives (PPDs) of M. bovis and M. avium. On the basis of the results of the last test administered before necropsy, the PRIMAGAM test had good sensitivity (68%) and excellent specificity (97%), compared with the disease status, as determined by the presence or absence of gross and/or histologic lesions indicative of tuberculosis. By contrast, sensitivity and specificity of the tuberculin skin test (TST) was 84 and 87%, respectively. Both tests suffered from intermittent positive and negative reactions on repeat testing. Overall, however, there was no significant difference (P = 0.09, McNemar's chi2-test) and moderate agreement (kappa = 0.52) between these two tests. Lastly, the IFNgamma response to bovine PPD was significantly lower in infected cynomolgus macaques. Moreover, each test failed to detect tuberculosis in three cynomolgus macaques. Fortunately, they were different animals; therefore, we recommend the parallel use of the TST and PRIMAGAM test for maximal overall sensitivity in a tuberculosis screening program, especially for cynomolgus macaques.     

Bayesian sample size determination for prevalence and diagnostic test studies in the absence of a gold standard test

Planning studies involving diagnostic tests is complicated by the fact that virtually no test provides perfectly accurate results. The misclassification induced by imperfect sensitivities and specificities of diagnostic tests must be taken into account, whether the primary goal of the study is to estimate the prevalence of a disease in a population or to investigate the properties of a new diagnostic test. Previous work on sample size requirements for estimating the prevalence of disease in the case of a single imperfect test showed very large discrepancies in size when compared to methods that assume a perfect test. In this article we extend these methods to include two conditionally independent imperfect tests, and apply several different criteria for Bayesian sample size determination to the design of such studies. We consider both disease prevalence studies and studies designed to estimate the sensitivity and specificity of diagnostic tests. As the problem is typically nonidentifiable, we investigate the limits on the accuracy of parameter estimation as the sample size approaches infinity. Through two examples from infectious diseases, we illustrate the changes in sample sizes that arise when two tests are applied to individuals in a study rather than a single test. Although smaller sample sizes are often found in the two-test situation, they can still be prohibitively large unless accurate information is available about the sensitivities and specificities of the tests being used.     

Bayesian Receiver Operating Characteristic Estimation of Multiple Tests for Diagnosis of Bovine Tuberculosis in Chadian Cattle

Background

Bovine tuberculosis (BTB) today primarily affects developing countries. In Africa, the disease is present essentially on the whole continent; however, little accurate information on its distribution and prevalence is available. Also, attempts to evaluate diagnostic tests for BTB in naturally infected cattle are scarce and mostly complicated by the absence of knowledge of the true disease status of the tested animals. However, diagnostic test evaluation in a given setting is a prerequisite for the implementation of local surveillance schemes and control measures.

Methodology/Principal Findings

We subjected a slaughterhouse population of 954 Chadian cattle to single intra-dermal comparative cervical tuberculin (SICCT) testing and two recently developed fluorescence polarization assays (FPA). Using a Bayesian modeling approach we computed the receiver operating characteristic (ROC) curve of each diagnostic test, the true disease prevalence in the sampled population and the disease status of all sampled animals in the absence of knowledge of the true disease status of the sampled animals. In our Chadian setting, SICCT performed better if the cut-off for positive test interpretation was lowered from >4 mm (OIE standard cut-off) to >2 mm. Using this cut-off, SICCT showed a sensitivity and specificity of 66% and 89%, respectively. Both FPA tests showed sensitivities below 50% but specificities above 90%. The true disease prevalence was estimated at 8%. Altogether, 11% of the sampled animals showed gross visible tuberculous lesions. However, modeling of the BTB disease status of the sampled animals indicated that 72% of the suspected tuberculosis lesions detected during standard meat inspections were due to other pathogens than Mycobacterium bovis.

Conclusions/Significance

Our results have important implications for BTB diagnosis in a high incidence sub-Saharan African setting and demonstrate the practicability of our Bayesian approach for diagnostic test evaluation.     

CXCL10/IP-10 release is induced by incubation of whole blood from tuberculosis patients with ESAT-6, CFP10 and TB7.7

IFN-gamma responses to Mycobacterium tuberculosis specific antigens are used as in vitro diagnostic tests for tuberculosis infection. The tests are sensitive and specific for latent and active tuberculosis disease, but sensitivity may be reduced during immunosuppression. The objective of the study was to explore new ways to improve the diagnosis of tuberculosis infection using CXCL10 and IL-2 as alternative markers to IFN-gamma. CXCL10, IL-2, and IFN-gamma responses to stimulation with ESAT-6/CFP10/TB7.7 were assessed in 12 Quantiferon positive, 8 Quantiferon negative tuberculosis patients and 11 Quantiferon negative controls. CXCL10 and IL-2 were determined by multiplex and IFN-gamma by the Quantiferon ELISA. The median antigen specific CXCL10, IFN-gamma, and IL-2 responses in patients with tuberculosis were 870 pg/ml (range 261-1576 pg/ml), 217 pg/ml (81-1273 pg/ml), 59 pg/ml (14-276 pg/ml) respectively, and the CXCL10 responses were significantly higher than any of the other cytokines measured (p=0.001). In 4/7 individuals with a negative (n=6) or indeterminate (n=1) Quantiferon test, antigen specific CXCL10 responses were detectable at high levels ranging from 196-532 pg/ml. In conclusion CXCL10 was strongly induced after M. tuberculosis specific stimulation and sensitivity appeared superior to the Quantiferon test. Our findings suggest that CXCL10 may serve as an alternative or additional marker for the immunodiagnosis of tuberculosis.     

Optimal Triage Test Characteristics to Improve the Cost-Effectiveness of the Xpert MTB/RIF Assay for TB Diagnosis: A Decision Analysis

Background

High costs are a limitation to scaling up the Xpert MTB/RIF assay (Xpert) for the diagnosis of tuberculosis in resource-constrained settings. A triaging strategy in which a sensitive but not necessarily highly specific rapid test is used to select patients for Xpert may result in a more affordable diagnostic algorithm. To inform the selection and development of particular diagnostics as a triage test we explored combinations of sensitivity, specificity and cost at which a hypothetical triage test will improve affordability of the Xpert assay.

Methods

In a decision analytical model parameterized for Uganda, India and South Africa, we compared a diagnostic algorithm in which a cohort of patients with presumptive TB received Xpert to a triage algorithm whereby only those with a positive triage test were tested by Xpert.

Findings

A triage test with sensitivity equal to Xpert, 75% specificity, and costs of US$5 per patient tested reduced total diagnostic costs by 42% in the Uganda setting, and by 34% and 39% respectively in the India and South Africa settings. When exploring triage algorithms with lower sensitivity, the use of an example triage test with 95% sensitivity relative to Xpert, 75% specificity and test costs $5 resulted in similar cost reduction, and was cost-effective by the WHO willingness-to-pay threshold compared to Xpert for all in Uganda, but not in India and South Africa. The gain in affordability of the examined triage algorithms increased with decreasing prevalence of tuberculosis among the cohort.

Conclusions

A triage test strategy could potentially improve the affordability of Xpert for TB diagnosis, particularly in low-income countries and with enhanced case-finding. Tests and markers with lower accuracy than desired of a diagnostic test may fall within the ranges of sensitivity, specificity and cost required for triage tests and be developed as such.     

A parametric comparison of diagnostic accuracy with three ordinal diagnostic groups

Many medical diagnostic studies involve three ordinal diagnostic groups in which the diagnostic accuracy can be summarized by the volume or partial volume under a Receiver Operating Characteristic (ROC) surface. We study in this paper the statistical comparison of diagnostic accuracy from multiple diagnostic tests when three ordinal diagnostic groups are involved. Under the assumption that the multiple diagnostic tests follow a multivariate normal distribution within each diagnostic group, we provide the asymptotic variance and covariance for the maximum likelihood estimates of the volumes under the ROC surfaces from multiple diagnostic tests and propose statistical tests to test whether the diagnostic accuracy as measured by the volume under the ROC surface is the same for multiple diagnostic tests. We also propose a confidence interval estimate to the difference of two volumes under two ROC surfaces. Our approach depends crucially on the assumptions of normal distributions on diagnostic tests, which might not be robust when such assumptions are violated. Finally, we apply our proposed methodology to a real data set of 118 subjects to compare the diagnostic accuracy of early stage Alzheimer's disease (AD) from multiple neuropsychological tests.     

Diagnostic testing for Legionnaires’ disease

Legionnaires’ disease is commonly diagnosed clinically using a urinary antigen test. The urinary antigen test is highly accurate for L. pneumophila serogroup 1, however other diagnostic tests should also be utilized in conjunction with the urinary antigen as many other Legionella species and serogroups are pathogenic. Culturing of patient specimens remains the gold standard for diagnosis of Legionnaires’ disease. Selective media, BYCE with the addition of antibiotics, allows for a high sensitivity and specificity. Culturing can identify all species and serogroups of Legionella. A major benefit of culturing is that it provides the recovery of a patient isolate, which can be used to find an environmental match. Other diagnostic tests, including DFA and molecular tests such as PCR and LAMP, are useful tests to supplement culturing. Molecular tests provide much more rapid results in comparison to culture, however these tests should not be a primary diagnostic tool given their lower sensitivity and specificity in comparison to culturing. It is recommended that all laboratories develop the ability to culture patient specimens in-house with the selective media.     

Serodiagnosis of Tuberculosis in Asian Elephants (Elephas maximus) in Southern India: A Latent Class Analysis

Background

Mycobacterium tuberculosis, a causative agent of chronic tuberculosis disease, is widespread among some animal species too. There is paucity of information on the distribution, prevalence and true disease status of tuberculosis in Asian elephants (Elephas maximus). The aim of this study was to estimate the sensitivity and specificity of serological tests to diagnose M. tuberculosis infection in captive elephants in southern India while simultaneously estimating sero-prevalence.

Methodology/Principal Findings

Health assessment of 600 elephants was carried out and their sera screened with a commercially available rapid serum test. Trunk wash culture of select rapid serum test positive animals yielded no animal positive for M. tuberculosis isolation. Under Indian field conditions where the true disease status is unknown, we used a latent class model to estimate the diagnostic characteristics of an existing (rapid serum test) and new (four in-house ELISA) tests. One hundred and seventy nine sera were randomly selected for screening in the five tests. Diagnostic sensitivities of the four ELISAs were 91.3–97.6% (95% Credible Interval (CI): 74.8–99.9) and diagnostic specificity were 89.6–98.5% (95% CI: 79.4–99.9) based on the model we assumed. We estimate that 53.6% (95% CI: 44.6–62.8) of the samples tested were free from infection with M. tuberculosis and 15.9% (97.5% CI: 9.8 - to 24.0) tested positive on all five tests.

Conclusions/Significance

Our results provide evidence for high prevalence of asymptomatic M. tuberculosis infection in Asian elephants in a captive Indian setting. Further validation of these tests would be important in formulating area-specific effective surveillance and control measures.     

Comparing disease screening tests when true disease status is ascertained only for screen positives

Disease screening is a fundamental part of health care. To evaluate the accuracy of a new screening modality, ideally the results of the screening test are compared with those of a definitive diagnostic test in a set of study subjects. However, definitive diagnostic tests are often invasive and cannot be applied to subjects whose screening tests are negative for disease. For example, in cancer screening, the assessment of true disease status requires a biopsy sample, which for ethical reasons can only be obtained if a subject's screening test indicates presence of cancer. Although the absolute accuracy of screening tests cannot be evaluated in such circumstances, it is possible to compare the accuracies of screening tests. Specifically, using relative true positive rate (the ratio of the true positive rate of one test to another) and relative false positive rate (the ratio of the false positive rates of two tests) as measures of relative accuracy, we show that inference about relative accuracy can be made from such studies. Analogies with case-control studies can be drawn where inference about absolute risk cannot be made, but inference about relative risk can. In this paper, we develop a marginal regression analysis framework for making inference about relative accuracy when only screen positives are followed for true disease. In this context factors influencing the relative accuracies of tests can be evaluated. It is important to determine such factors in order to understand circumstances in which one test is preferable to another. The methods are applied to two cancer screening studies, one concerning the effect of race on screening for prostate cancer and the other concerning the effect of tumour grade on the detection of cervical cancer with cytology versus cervicography screening.     

How Much Overtesting Is Needed to Safely Exclude a Diagnosis? A Different Perspective on Triage Testing Using Bayes' Theorem

Ruling out disease often requires expensive or potentially harmful confirmation testing. For such testing, a less invasive triage test is often used. Intuitively, few negative confirmatory tests suggest success of this approach. However, if negative confirmation tests become too rare, too many disease cases could have been missed. It is therefore important to know how many negative tests are needed to safely exclude a diagnosis. We quantified this relationship using Bayes’ theorem, and applied this to the example of pulmonary embolism (PE), for which triage is done with a Clinical Decision Rule (CDR) and D-dimer testing, and CT-angiography (CTA) is the confirmation test. For a maximum proportion of missed PEs of 1% in triage-negative patients, we calculate a 67% ''mandatory minimum'' proportion of negative CTA scans. To achieve this, the proportion of patients with PE undergoing triage testing should be appropriately low, in this case no higher than 24%. Pre-test probability, triage test characteristics, the proportion of negative confirmation tests, and the number of missed diagnoses are mathematically entangled. The proportion of negative confirmation tests—not too high, but definitely not too low either—could be a quality benchmark for diagnostic processes.     

Comparisons of predictive values of binary medical diagnostic tests for paired designs

Positive and negative predictive values of a diagnostic test are key clinically relevant measures of test accuracy. Surprisingly, statistical methods for comparing tests with regard to these parameters have not been available for the most common study design in which each test is applied to each study individual. In this paper, we propose a statistic for comparing the predictive values of two diagnostic tests using this paired study design. The proposed statistic is a score statistic derived from a marginal regression model and bears some relation to McNemar's statistic. As McNemar's statistic can be used to compare sensitivities and specificities of diagnostic tests, parameters that condition on disease status, our statistic can be considered as an analog of McNemar's test for the problem of comparing predictive values, parameters that condition on test outcome. We report on the results of a simulation study designed to examine the properties of this test under a variety of conditions. The method is illustrated with data from a study of methods for diagnosis of coronary artery disease.     

The role of laboratory diagnostics in emerging viral infections: the example of the Middle East respiratory syndrome epidemic

Rapidly emerging infectious disease outbreaks place a great strain on laboratories to develop and implement sensitive and specific diagnostic tests for patient management and infection control in a timely manner. Furthermore, laboratories also play a role in real-time zoonotic, environmental, and epidemiological investigations to identify the ultimate source of the epidemic, facilitating measures to eventually control the outbreak. Each assay modality has unique pros and cons; therefore, incorporation of a battery of tests using traditional culture-based, molecular and serological diagnostics into diagnostic algorithms is often required. As such, laboratories face challenges in assay development, test evaluation, and subsequent quality assurance. In this review, we describe the different testing modalities available for the ongoing Middle East respiratory syndrome (MERS) epidemic including cell culture, nucleic acid amplification, antigen detection, and antibody detection assays. Applications of such tests in both acute clinical and epidemiological investigation settings are highlighted. Using the MERS epidemic as an example, we illustrate the various challenges faced by laboratories in test development and implementation in the setting of a rapidly emerging infectious disease. Future directions in the diagnosis of MERS and other emerging infectious disease investigations are also highlighted.     

Bayesian Meta‐Analysis of the Accuracy of a Test for Tuberculous Pleuritis in the Absence of a Gold Standard Reference

Summary Absence of a perfect reference test is an acknowledged source of bias in diagnostic studies. In the case of tuberculous pleuritis, standard reference tests such as smear microscopy, culture and biopsy have poor sensitivity. Yet meta‐analyses of new tests for this disease have always assumed the reference standard is perfect, leading to biased estimates of the new test’s accuracy. We describe a method for joint meta‐analysis of sensitivity and specificity of the diagnostic test under evaluation, while considering the imperfect nature of the reference standard. We use a Bayesian hierarchical model that takes into account within‐ and between‐study variability. We show how to obtain pooled estimates of sensitivity and specificity, and how to plot a hierarchical summary receiver operating characteristic curve. We describe extensions of the model to situations where multiple reference tests are used, and where index and reference tests are conditionally dependent. The performance of the model is evaluated using simulations and illustrated using data from a meta‐analysis of nucleic acid amplification tests (NAATs) for tuberculous pleuritis. The estimate of NAAT specificity was higher and the sensitivity lower compared to a model that assumed that the reference test was perfect.     

Statistical Inference For Risk Difference in an Incomplete Correlated 2 × 2 Table

In some infectious disease studies and 2‐step treatment studies, 2 × 2 table with structural zero could arise in situations where it is theoretically impossible for a particular cell to contain observations or structural void is introduced by design. In this article, we propose a score test of hypotheses pertaining to the marginal and conditional probabilities in a 2 × 2 table with structural zero via the risk/rate difference measure. Score test‐based confidence interval will also be outlined. We evaluate the performance of the score test and the existing likelihood ratio test. Our empirical results evince the similar and satisfactory performance of the two tests (with appropriate adjustments) in terms of coverage probability and expected interval width. Both tests consistently perform well from small‐ to moderate‐sample designs. The score test however has the advantage that it is only undefined in one scenario while the likelihood ratio test can be undefined in many scenarios. We illustrate our method by a real example from a two‐step tuberculosis skin test study.     

Estimating prevalence and test accuracy in disease ecology: How Bayesian latent class analysis can boost or bias imperfect test results

1. Obtaining accurate estimates of disease prevalence is crucial for the monitoring and management of wildlife populations but can be difficult if different diagnostic tests yield conflicting results and if the accuracy of each diagnostic test is unknown. Bayesian latent class analysis (BLCA) modeling offers a potential solution, providing estimates of prevalence levels and diagnostic test accuracy under the realistic assumption that no diagnostic test is perfect.
2. In typical applications of this approach, the specificity of one test is fixed at or close to 100%, allowing the model to simultaneously estimate the sensitivity and specificity of all other tests, in addition to infection prevalence. In wildlife systems, a test with near‐perfect specificity is not always available, so we simulated data to investigate how decreasing this fixed specificity value affects the accuracy of model estimates.
3. We used simulations to explore how the trade‐off between diagnostic test specificity and sensitivity impacts prevalence estimates and found that directional biases depend on pathogen prevalence. Both the precision and accuracy of results depend on the sample size, the diagnostic tests used, and the true infection prevalence, so these factors should be considered when applying BLCA to estimate disease prevalence and diagnostic test accuracy in wildlife systems. A wildlife disease case study, focusing on leptospirosis in California sea lions, demonstrated the potential for Bayesian latent class methods to provide reliable estimates under real‐world conditions.
4. We delineate conditions under which BLCA improves upon the results from a single diagnostic across a range of prevalence levels and sample sizes, demonstrating when this method is preferable for disease ecologists working in a wide variety of pathogen systems.

     

Disease model: pulmonary tuberculosis

In spite of a massive effort to apply the tools currently available for tuberculosis (TB) control, both in this country and abroad, it is clear that complicating factors [for example, HIV co-infection, drug resistance, lack of patient compliance with chemotherapy, variable efficacy of Bacille Calmette-Guerin (BCG) vaccine] will prevent disease control unless new drugs, vaccines and diagnostic tests are developed (1). The publication of the complete genome sequence of Mycobacterium tuberculosis in 1998 (2) has facilitated a directed search for virulence genes, new drug targets, and vaccine antigens. This research effort has been made possible by the availability of highly biologically relevant animal models of pulmonary TB ((3)).     

Molecular diagnostics of medically important bacterial infections

Infectious diseases are common diseases all over the world. A recent World Health Organization report indicated that infectious diseases are now the world's biggest killer of children and young adults. Infectious diseases in non-industrialized countries caused 45% in all and 63% of death in early childhood. In developed countries, the emergence of new, rare or already-forgotten infectious diseases, such as HIV/AIDS, Lyme disease and tuberculosis, has stimulated public interest and inspired commitments to surveillance and control. Recently, it is reported that infectious diseases are responsible for more than 17 million deaths worldwide each year, most of which are associated with bacterial infections. Hence, the control of infectious diseases control is still an important task in the world. The ability to control such bacterial infections is largely dependent on the ability to detect these aetiological agents in the clinical microbiology laboratory. Diagnostic medical bacteriology consists of two main components namely identification and typing. Molecular biology has the potential to revolutionise the way in which diagnostic tests are delivered in order to optimise care of the infected patient, whether they occur in hospital or in the community. Since the discovery of PCR in the late 1980s, there has been an enormous amount of research performed which has enabled the introduction of molecular tests to several areas of routine clinical microbiology. Molecular biology techniques continue to evolve rapidly, so it has been problematic for many laboratories to decide upon which test to introduce before that technology becomes outdated. However the vast majority of diagnostic clinical bacteriology laboratories do not currently employ any form of molecular diagnostics but the use such technology is becoming more widespread in both specialized regional laboratories as well as in national reference laboratories. Presently molecular biology offers a wide repertoire of techniques and permutations of these analytical tools, hence this article wishes to explore the application of these in the diagnostic laboratory setting.     

New approaches to tuberculosis surveillance in nonhuman primates

Despite significant progress in reducing the incidence of tuberculosis in nonhuman primates (NHPs) maintained in captivity, outbreaks continue to occur in established colonies, with potential serious consequences in human exposures, animal losses, disruption of research, and costs related to disease control efforts. The intradermal tuberculin skin test (TST) using mammalian old tuberculin (MOT) has been the mainstay of NHP tuberculosis surveillance and antemortem diagnosis for more than 60 years. But limitations of the TST, particularly its inability to reliably identify animals with latent TB infections, make it unsuitable for use as a single, standalone test for TB surveillance in nonhuman primates in the 21st century. Advances in technology and the availability of Mycobacterium spp. genomic sequence data have facilitated the development and evaluation of new immune-based screening assays as possible adjuncts and alternatives to the TST, including in vitro whole blood assays that measure the release of interferon gamma in response to stimulation with tuberculin or specific mycobacterial antigens, and assays that detect antibodies to highly immunogenic secreted proteins unique to M. tuberculosis, M. bovis, and other species belonging to the M. tuberculosis complex. It is becoming apparent that no single screening test will meet all the requirements for surveillance and diagnosis of tuberculosis in nonhuman primates. Instead, the use of several tests in combination can increase the overall sensitivity and specificity of screening and surveillance programs and likely represents the future of TB testing in nonhuman primates. In this article we describe the characteristics of these newer screening tests and discuss their potential contributions to NHP tuberculosis surveillance programs.     

Interpretation of diagnostic data: 5. How to do it with simple maths.

The use of simple maths with the likelihood ratio strategy fits in nicely with our clinical views. By making the most out of the entire range of diagnostic test results (i.e., several levels, each with its own likelihood ratio, rather than a single cut-off point and a single ratio) and by permitting us to keep track of the likelihood that a patient has the target disorder at each point along the diagnostic sequence, this strategy allows us to place patients at an extremely high or an extremely low likelihood of disease. Thus, the numbers of patients with ultimately false-positive results (who suffer the slings of labelling and the arrows of needless therapy) and of those with ultimately false-negative results (who therefore miss their chance for diagnosis and, possibly, efficacious therapy) will be dramatically reduced. The following guidelines will be useful in interpreting signs, symptoms and laboratory tests with the likelihood ratio strategy: Seek out, and demand from the clinical or laboratory experts who ought to know, the likelihood ratios for key symptoms and signs, and several levels (rather than just the positive and negative results) of diagnostic test results. Identify, when feasible, the logical sequence of diagnostic tests. Estimate the pretest probability of disease for the patient, and, using either the nomogram or the conversion formulas, apply the likelihood ratio that corresponds to the first diagnostic test result. While remembering that the resulting post-test probability or odds from the first test becomes the pretest probability or odds for the next diagnostic test, repeat the process for all the pertinent symptoms, signs and laboratory studies that pertain to the target disorder. However, these combinations may not be independent, and convergent diagnostic tests, if treated as independent, will combine to overestimate the final post-test probability of disease. You are now far more sophisticated in interpreting diagnostic tests than most of your teachers. In the last part of our series we will show you some rather complex strategies that combine diagnosis and therapy, quantify our as yet nonquantified ideas about use, and require the use of at least a hand calculator.