Bayesian inference for prevalence in longitudinal two-phase studies

We consider Bayesian inference and model selection for prevalence estimation using a longitudinal two-phase design in which subjects initially receive a low-cost screening test followed by an expensive diagnostic test conducted on several occasions. The change in the subject's diagnostic probability over time is described using four mixed-effects probit models in which the subject-specific effects are captured by latent variables. The computations are performed using Markov chain Monte Carlo methods. These models are then compared using the deviance information criterion. The methodology is illustrated with an analysis of alcohol and drug use in adolescents using data from the Great Smoky Mountains Study.     

Prospective studies of diagnostic test accuracy when disease prevalence is low

Prospective studies of diagnostic test accuracy have important advantages over retrospective designs. Yet, when the disease being detected by the diagnostic test(s) has a low prevalence rate, a prospective design can require an enormous sample of patients. We consider two strategies to reduce the costs of prospective studies of binary diagnostic tests: stratification and two-phase sampling. Utilizing neither, one, or both of these strategies provides us with four study design options: (1) the conventional design involving a simple random sample (SRS) of patients from the clinical population; (2) a stratified design where patients from higher-prevalence subpopulations are more heavily sampled; (3) a simple two-phase design using a SRS in the first phase and selection for the second phase based on the test results from the first; and (4) a two-phase design with stratification in the first phase. We describe estimators for sensitivity and specificity and their variances for each design, along with sample size estimation. We offer some recommendations for choosing among the various designs. We illustrate the study designs with two examples.     

Optimal surveillance strategy for invasive species management when surveys stop after detection

Invasive species are a cause for concern in natural and economic systems and require both monitoring and management. There is a trade‐off between the amount of resources spent on surveying for the species and conducting early management of occupied sites, and the resources that are ultimately spent in delayed management at sites where the species was present but undetected. Previous work addressed this optimal resource allocation problem assuming that surveys continue despite detection until the initially planned survey effort is consumed. However, a more realistic scenario is often that surveys stop after detection (i.e., follow a “removal” sampling design) and then management begins. Such an approach will indicate a different optimal survey design and can be expected to be more efficient. We analyze this case and compare the expected efficiency of invasive species management programs under both survey methods. We also evaluate the impact of mis‐specifying the type of sampling approach during the program design phase. We derive analytical expressions that optimize resource allocation between monitoring and management in surveillance programs when surveys stop after detection. We do this under a scenario of unconstrained resources and scenarios where survey budget is constrained. The efficiency of surveillance programs is greater if a “removal survey” design is used, with larger gains obtained when savings from early detection are high, occupancy is high, and survey costs are not much lower than early management costs at a site. Designing a surveillance program disregarding that surveys stop after detection can result in an efficiency loss. Our results help guide the design of future surveillance programs for invasive species. Addressing program design within a decision‐theoretic framework can lead to a better use of available resources. We show how species prevalence, its detectability, and the benefits derived from early detection can be considered.     

Comparing disease screening tests when true disease status is ascertained only for screen positives

Disease screening is a fundamental part of health care. To evaluate the accuracy of a new screening modality, ideally the results of the screening test are compared with those of a definitive diagnostic test in a set of study subjects. However, definitive diagnostic tests are often invasive and cannot be applied to subjects whose screening tests are negative for disease. For example, in cancer screening, the assessment of true disease status requires a biopsy sample, which for ethical reasons can only be obtained if a subject's screening test indicates presence of cancer. Although the absolute accuracy of screening tests cannot be evaluated in such circumstances, it is possible to compare the accuracies of screening tests. Specifically, using relative true positive rate (the ratio of the true positive rate of one test to another) and relative false positive rate (the ratio of the false positive rates of two tests) as measures of relative accuracy, we show that inference about relative accuracy can be made from such studies. Analogies with case-control studies can be drawn where inference about absolute risk cannot be made, but inference about relative risk can. In this paper, we develop a marginal regression analysis framework for making inference about relative accuracy when only screen positives are followed for true disease. In this context factors influencing the relative accuracies of tests can be evaluated. It is important to determine such factors in order to understand circumstances in which one test is preferable to another. The methods are applied to two cancer screening studies, one concerning the effect of race on screening for prostate cancer and the other concerning the effect of tumour grade on the detection of cervical cancer with cytology versus cervicography screening.     

Towards more efficient large-scale DNA-based detection of terrestrial mammal predators from scats

The DNA detection of wildlife from environmental samples has the potential to contribute significantly to wildlife management and ecological research. In terrestrial ecosystems, much work has focused on the identification of mammal predators from faecal (scat) samples. However, the relatively high time and financial costs of collecting and analysing scat DNA remain barriers to more widespread implementation of such DNA detection methods, especially for high-throughput surveys. Here, we evaluate methods used for DNA extraction from scats, as applied to detection of the Australian red fox, an introduced predator. We compare the relative costs of two approaches: the method previously used to screen thousands of scat samples in surveys over several years, and a modified version which involves swabbing scats at the time of collection and using a mechanised liquid handling platform to extract DNA from the swabs. We demonstrate that mechanised DNA extraction from swabs is more efficient than manual DNA extraction from whole scats, in terms of both time and resources. This provides a means for rapid, high-throughput screening of scats for the presence of mammal predators, enabling time-effective management responses to non-invasive surveys.     

Design and Analysis of Line Transect Surveys for Primates

Line transect surveys are widely used for estimating abundance of primate populations. The method relies on a small number of key assumptions, and if these are not met, substantial bias may occur. For a variety of reasons, primate surveys often do not follow what is generally considered to be best practice, either in survey design or in analysis. The design often comprises too few lines (sometimes just 1), subjectively placed or placed along trails, so lacks both randomization and adequate replication. Analysis often involves flawed or inefficient models, and often uses biased estimates of the locations of primate groups relative to the line. We outline the standard method, emphasizing the assumptions underlying the approach. We then consider options for when it is difficult or impossible to meet key assumptions. We explore the performance of these options by simulation, focusing particularly on the analysis of primate group sizes, where many of the variations in survey methods have been developed. We also discuss design issues, field methods, analysis, and potential alternative methodologies for when standard line transect sampling cannot deliver reliable abundance estimates.     

Testing the Independence of Two Diagnostic Tests

Consider two diagnostic procedures having binary outcomes. If one of the tests results in a positive finding, a more definitive diagnostic procedure will be administered to establish the presence or absence of a disease. The use of both tests will improve the overall screening sensitivity when the two tests are independent, compared with employing two tests that are positively correlated. We estimate the correlation coefficient of the two tests and derive statistical methods for testing the independence of the two diagnostic procedures conditional on disease status. The statistical tests are used to investigate the independence of mammography and clinical breast exams aimed at establishing the benefit of early detection of breast cancer. The data used in the analysis are obtained from periodic screening examinations of three randomized clinical trials of breast cancer screening. Analysis of each of these trials confirms the independence of the clinical breast and mammography examinations. Based on these three large clinical trials, we conclude that a clinical breast exam considerably increases the overall sensitivity relative to screening with mammography alone and should be routinely included in early breast cancer detection programs.     

Peptostreptococcus vaginalis or Peptostreptococcus prevotii? Identification and clinical importance of a new species of Peptostreptococcus

Peptostreptococcus vaginalis is a recently described species of Gram-positive anaerobic coccus. We report one case in which P. vaginalis was isolated in pure culture from an abscess on the upper arm, and summarise nine further cases where it was isolated in mixed culture from other superficial sites, particularly infected leg ulcers. We suggest that clinical strains of P. vaginalis have probably been described in the past as Peptostreptococcus prevotii, a species which has frequently been reported from clinical surveys of anaerobic infections; their relative importance and appropriate treatment are discussed. Preformed enzyme profiles provide a simple method of identification accessible to routine diagnostic laboratories; when clinically significant isolates of GPAC are isolated in pure growth, they should be identified to the species level by use of preformed enzyme kits.     

Diagnostics to support elimination of lymphatic filariasis—Development of two target product profiles

As lymphatic filariasis (LF) programs move closer to established targets for validation elimination of LF as a public health problem, diagnostic tools capable of supporting the needs of the programs are critical for success. Known limitations of existing diagnostic tools make it challenging to have confidence that program endpoints have been achieved. In 2019, the World Health Organization (WHO) established a Diagnostic Technical Advisory Group (DTAG) for Neglected Tropical Diseases tasked with prioritizing diagnostic needs including defining use-cases and target product profiles (TPPs) for needed tools. Subsequently, disease-specific DTAG subgroups, including one focused on LF, were established to develop TPPs and use-case analyses to be used by product developers. Here, we describe the development of two priority TPPs for LF diagnostics needed for making decisions for stopping mass drug administration (MDA) of a triple drug regimen and surveillance. Utilizing the WHO core TPP development process as the framework, the LF subgroup convened to discuss and determine attributes required for each use case. TPPs considered the following parameters: product use, design, performance, product configuration and cost, and access and equity. Version 1.0 TPPs for two use cases were published by WHO on 12 March 2021 within the WHO Global Observatory on Health Research and Development. A common TPP characteristic that emerged in both use cases was the need to identify new biomarkers that would allow for greater precision in program delivery. As LF diagnostic tests are rarely used for individual clinical diagnosis, it became apparent that reliance on population-based surveys for decision making requires consideration of test performance in the context of such surveys. In low prevalence settings, the number of false positive test results may lead to unnecessary continuation or resumption of MDA, thus wasting valuable resources and time. Therefore, highly specific diagnostic tools are paramount when used to measure low thresholds. The TPP process brought to the forefront the importance of linking use case, program platform and diagnostic performance characteristics when defining required criteria for diagnostic tools.     

Using Resource Selection Functions to Improve Estimation of Elk Population Numbers

Abstract: Stratification is commonly used to improve sampling efficiency of aerial surveys of ungulate populations with strata typically based on a priori information, such as preflight animal observations or vegetation attributes as surrogates for animal densities. We evaluated the usefulness of stratifying survey units for elk (Cervus elaphus) in the Rocky Mountain foothills of Alberta, Canada, using a resource selection function (RSF). We compared precision and design efficiency (DEFF) of population estimates from stratification approaches based on an RSF model to the past approach using amount of forest cover. We used a sample of telemetry relocations taken over a 3-year period from 165 elk, rarified to times of the day and months of the year when aerial surveys are conducted, to develop the RSF. We then used the top RSF model, based on Akaike's Information Criterion, to derive the average RSF value for an 8-km² survey unit. Using survey data from the first year, we evaluated binning schemes to define RSF-oriented strata based on poststratification and showed that Jenks natural breaks in the RSF values provided the greatest improvement in DEFF and increased precision, compared to 2 other stratification schemes. We then used this approach with data from 2 additional surveys to find that stratification by RSF consistently improves relative precision and design efficiency of elk population estimates, whether we employ pre- or poststratification. Where a RSF is available it could be used as a surrogate for animal densities when conducting stratified sampling for population surveys.     

Assessing the diagnostic accuracy of a sequence of tests

We consider the assessment of the overall diagnostic accuracy of a sequence of tests (e.g. repeated screening tests). The complexity of diagnostic choices when two or more continuous tests are used in sequence is illustrated, and different approaches to reducing the dimensionality are presented and evaluated. For instance, in practice, when a single test is used repeatedly in routine screening, the same screening threshold is typically used at each screening visit. One possible alternative is to adjust the threshold at successive visits according to individual-specific characteristics. Such possibilities represent a particular slice of a receiver operating characteristic surface, corresponding to all possible combinations of test thresholds. We focus in the development and examples on the setting where an overall test is defined to be positive if any of the individual tests are positive ('believe the positive'). The ideas developed are illustrated by an example of application to screening for prostate cancer using prostate-specific antigen.     

Two-stage designs for gene-disease association studies

The goal of this article is to describe a two-stage design that maximizes the power to detect gene-disease associations when the principal design constraint is the total cost, represented by the total number of gene evaluations rather than the total number of individuals. In the first stage, all genes of interest are evaluated on a subset of individuals. The most promising genes are then evaluated on additional subjects in the second stage. This will eliminate wastage of resources on genes unlikely to be associated with disease based on the results of the first stage. We consider the case where the genes are correlated and the case where the genes are independent. Using simulation results, it is shown that, as a general guideline when the genes are independent or when the correlation is small, utilizing 75% of the resources in stage 1 to screen all the markers and evaluating the most promising 10% of the markers with the remaining resources provides near-optimal power for a broad range of parametric configurations. This translates to screening all the markers on approximately one quarter of the required sample size in stage 1.     

On repeated screening tests

We point out the connection between quality control and repeated screening tests, and suggest a practical way to design the repeated screening tests so that the composite test achieves the desired level of sensitivity and specificity. We further point out the benefits of using likelihood ratios to measure the performance of a screening test. Then we investigate repeated tests when we have polytomous disease levels and polytomous diagnostic levels.     

Estimating disease prevalence in two-phase studies

Disease prevalence is ideally estimated using a 'gold standard' to ascertain true disease status on all subjects in a population of interest. In practice, however, the gold standard may be too costly or invasive to be applied to all subjects, in which case a two-phase design is often employed. Phase 1 data consisting of inexpensive and non-invasive screening tests on all study subjects are used to determine the subjects that receive the gold standard in the second phase. Naive estimates of prevalence in two-phase studies can be biased (verification bias). Imputation and re-weighting estimators are often used to avoid this bias. We contrast the forms and attributes of the various prevalence estimators. Distribution theory and simulation studies are used to investigate their bias and efficiency. We conclude that the semiparametric efficient approach is the preferred method for prevalence estimation in two-phase studies. It is more robust and comparable in its efficiency to imputation and other re-weighting estimators. It is also easy to implement. We use this approach to examine the prevalence of depression in adolescents with data from the Great Smoky Mountain Study.     

The optimal ratio of cases to controls for estimating the classification accuracy of a biomarker

The case-control design is frequently used to study the discriminatory accuracy of a screening or diagnostic biomarker. Yet, the appropriate ratio in which to sample cases and controls has never been determined. It is common for researchers to sample equal numbers of cases and controls, a strategy that can be optimal for studies of association. However, considerations are quite different when the biomarker is to be used for classification. In this paper, we provide an expression for the optimal case-control ratio, when the accuracy of the biomarker is quantified by the receiver operating characteristic (ROC) curve. We show how it can be integrated with choosing the overall sample size to yield an efficient study design with specified power and type-I error. We also derive the optimal case-control ratios for estimating the area under the ROC curve and the area under part of the ROC curve. Our methods are applied to a study of a new marker for adenocarcinoma in patients with Barrett's esophagus.     

Group screening for rare events based on incomplete block designs

Fields such as, diagnostic testing, biotherapeutics, drug development, and toxicology among others, center on the premise of searching through many specimens for a rare event. Scientists in the business of “searching for a needle in a haystack” may greatly benefit from the use of group screening design strategies. Group screening, where specimens are composited into pools with each pool being tested for the presence of the event, can be much more cost-efficient than testing each individual specimen. A number of group screening designs have been proposed in the literature. Incomplete block screening designs are described here and compared with other group screening designs. It is shown under certain conditions, that incomplete block screening designs can provide nearly a 90% cost saving compared to other group screening designs such as when prevalence is 0.001 and screening 3876 specimens with an ICB-sequential design vs. a Dorfman design. In other cases, previous group screening designs are shown to be most efficient. Overall, when prevalence is small (≤0.05) group screening designs are shown to be quite cost effective at screening a large number of specimens and in general there is no one design that is best in all situations. © 2018 American Institute of Chemical Engineers Biotechnol Progress, 35: e2770, 2019.     

Diagnostic and economic evaluation of new biomarkersfor Alzheimer¿s disease: the research protocol of a prospective cohort study

ABSTRACT: BACKGROUND: New research criteria for the diagnosis of Alzheimer's disease (AD) have recently been developed to enable an early diagnosis of AD pathophysiologyby relying on emerging biomarkers. To enable efficient allocation of health care resources, evidence is needed to support decision makers on the adoption of emerging biomarkers in clinical practice. The research goals are to 1) assess the diagnostic test accuracy of current clinical diagnostic work-up and emerging biomarkers in MRI, PET and CSF, 2) perform a cost-consequence analysis and 3) assess long-term cost-effectiveness by an economic model.Methods/designIn a cohort design 223 consecutive patients suspected of having a primary neurodegenerative disease are approached in four academic memory clinics and followed for two years. Clinical data and data on quality of life, costs and emerging biomarkers are gathered.Diagnostic test accuracy is determined by relating the clinical practice and new research criteria diagnoses to the reference diagnosis. The clinical practice diagnosis at baseline is reflected by a consensus procedure among experts using clinical information only (no biomarkers). The diagnosis based on the new research criteria is reflected by decision rules that combine clinical and biomarker information. The reference diagnosis is determined by a consensus procedure among experts based on clinical information on the course of symptoms over a two-year time period.A decision analytic model is built combining available evidence from different resources among which (accuracy) results from the study, literature and expert opinion to assess long-term cost-effectiveness of the emerging biomarkers. DISCUSSION: Several other multi-centre trials study the relative value of new biomarkers for early evaluation of AD and related disorders. The uniqueness of this study is the assessment of resource utilization and quality of life to enable an economic evaluation. The study results are generalizable to a population of patients who are referred to a memory clinic due to their memory problems.Trial registrationNCT01450891.     

Design and analysis of two-phase experiments for gene expression microarrays-part I.

Gene expression microarray experiments are intrinsically two-phase experiments. Messenger RNA (mRNA), required for the microarray experiment, must first be derived from plants or animals that are exposed to a set of treatments in a previous experiment (Phase 1). The mRNA is then used in the subsequent laboratory-based microarray experiment (Phase 2) from which gene expression is measured and ultimately analyzed. We show that obtaining a valid test for the effects of treatments on gene expression depends on the design of both the Phase 1 and Phase 2 experiments. Examples show that the multiple dye-swap design at Phase 2 is more robust than the alternating loop design in the absence of prior knowledge of the relative size of variation in the Phase 1 and Phase 2 experiments.     

Design and Analysis of Two-Phase Experiments for Gene Expression Microarrays—Part I

Summary .   Gene expression microarray experiments are intrinsically two-phase experiments. Messenger RNA (mRNA), required for the microarray experiment, must first be derived from plants or animals that are exposed to a set of treatments in a previous experiment (Phase 1). The mRNA is then used in the subsequent laboratory-based microarray experiment (Phase 2) from which gene expression is measured and ultimately analyzed. We show that obtaining a valid test for the effects of treatments on gene expression depends on the design of both the Phase 1 and Phase 2 experiments. Examples show that the multiple dye-swap design at Phase 2 is more robust than the alternating loop design in the absence of prior knowledge of the relative size of variation in the Phase 1 and Phase 2 experiments.     

The use of biomarkers in surveillance, medical screening, and intervention

Building on mechanistic information, much of molecular epidemiologic research has focused on validating biomarkers, that is, assessing their ability to accurately indicate exposure, effect, disease, or susceptibility. To be of use in surveillance, medical screening, or interventions, biomarkers must already be validated so that they can be used as outcomes or indicators that can serve a particular function. In surveillance, biomarkers can be used as indicators of hazard, exposure, disease, and population risk. However, to obtain rates for these measures, the population at risk will need to be assessed. In medical screening, biomarkers can serve as early indicators of disease in asymptomatic people. This allows for the identification of those who should receive diagnostic confirmation and early treatment. In intervention (which includes risk assessment and communication, risk management, and various prevention efforts), biomarkers can be used to assess the effectiveness of a prevention or control strategy as well as help determine whether the appropriate individuals are assigned to the correct intervention category. Biomarkers can be used to provide group and individual risk assessments that can be the basis for marshalling resources. Critical for using biomarkers in surveillance, medical screening, and intervention is the justification that the biomarkers can provide information not otherwise accessible by a less expensive and easier-to-obtain source of information, such as medical records, surveys, or vital statistics. The ability to use validated biomarkers in surveillance, medical screening, and intervention will depend on the extent to which a strategy for evidence-based procedures for biomarker knowledge transfer can be developed and implemented. This will require the interaction of researchers and decision-makers to collaborate on public health and medical issues.