ROC curve estimation when covariates affect the verification process

A receiver operating characteristic (ROC) curve is commonly used to measure the accuracy of a medical test. It is a plot of the true positive fraction (sensitivity) against the false positive fraction (1-specificity) for increasingly stringent positivity criterion. Bias can occur in estimation of an ROC curve if only some of the tested patients are selected for disease verification and if analysis is restricted only to the verified cases. This bias is known as verification bias. In this paper, we address the problem of correcting for verification bias in estimation of an ROC curve when the verification process and efficacy of the diagnostic test depend on covariates. Our method applies the EM algorithm to ordinal regression models to derive ML estimates for ROC curves as a function of covariates, adjusted for covariates affecting the likelihood of being verified. Asymptotic variance estimates are obtained using the observed information matrix of the observed data. These estimates are derived under the missing-at-random assumption, which means that selection for disease verification depends only on the observed data, i.e., the test result and the observed covariates. We also address the issues of model selection and model checking. Finally, we illustrate the proposed method on data from a two-phase study of dementia disorders, where selection for verification depends on the screening test result and age.     

Inverse probability weighting estimation of the volume under the ROC surface in the presence of verification bias

In diagnostic medicine, the volume under the receiver operating characteristic (ROC) surface (VUS) is a commonly used index to quantify the ability of a continuous diagnostic test to discriminate between three disease states. In practice, verification of the true disease status may be performed only for a subset of subjects under study since the verification procedure is invasive, risky, or expensive. The selection for disease examination might depend on the results of the diagnostic test and other clinical characteristics of the patients, which in turn can cause bias in estimates of the VUS. This bias is referred to as verification bias. Existing verification bias correction in three‐way ROC analysis focuses on ordinal tests. We propose verification bias‐correction methods to construct ROC surface and estimate the VUS for a continuous diagnostic test, based on inverse probability weighting. By applying U‐statistics theory, we develop asymptotic properties for the estimator. A Jackknife estimator of variance is also derived. Extensive simulation studies are performed to evaluate the performance of the new estimators in terms of bias correction and variance. The proposed methods are used to assess the ability of a biomarker to accurately identify stages of Alzheimer's disease.     

Estimation of disease prevalence, true positive rate, and false positive rate of two screening tests when disease verification is applied on only screen-positives: a hierarchical model using multi-center data

Objectives: A model is proposed to estimate and compare cervical cancer screening test properties for third world populations when only subjects with a positive screen receive the gold standard test. Two fallible screening tests are compared, VIA and VILI. Methods: We extend the model of Berry et al. [1] to the multi-site case in order to pool information across sites and form better estimates for prevalences of cervical cancer, the true positive rates (TPRs), and false positive rates (FPRs). For 10 centers in five African countries and India involving more than 52,000 women, Bayesian methods were applied when gold standard results for subjects who screened negative on both tests were treated as missing. The Bayesian methods employed suitably correct for the missing screen negative subjects. The study included gold standard verification for all cases, making it possible to validate model-based estimation of accuracy using only outcomes of women with positive VIA or VILI result (ignoring verification of double negative screening test results) with the observed full data outcomes. Results: Across the sites, estimates for the sensitivity of VIA ranged from 0.792 to 0.917 while for VILI sensitivities ranged from 0.929 to 0.977. False positive estimates ranged from 0.056 to 0.256 for VIA and 0.085 to 0.269 for VILI. The pooled estimates for the TPR of VIA and VILI are 0.871 and 0.968, respectively, compared to the full data values of 0.816 and 0.918. Similarly, the pooled estimates for the FPR of VIA and VILI are 0.134 and 0.146, respectively, compared to the full data values of 0.144 and 0.146. Globally, we found VILI had a statistically significant higher sensitivity but no statistical difference for the false positive rates could be determined. Conclusion: Hierarchical Bayesian methods provide a straight forward approach to estimate screening test properties, prevalences, and to perform comparisons for screening studies where screen negative subjects do not receive the gold standard test. The hierarchical model with random effects used to analyze the sites simultaneously resulted in improved estimates compared to the single-site analyses with improved TPR estimates and nearly identical FPR estimates to the full data outcomes. Furthermore, higher TPRs but similar FPRs were observed for VILI compared to VIA.     

Accounting for nonignorable verification bias in assessment of diagnostic tests

A "gold" standard test, providing definitive verification of disease status, may be quite invasive or expensive. Current technological advances provide less invasive, or less expensive, diagnostic tests. Ideally, a diagnostic test is evaluated by comparing it with a definitive gold standard test. However, the decision to perform the gold standard test to establish the presence or absence of disease is often influenced by the results of the diagnostic test, along with other measured, or not measured, risk factors. If only data from patients who received the gold standard test were used to assess the test performance, the commonly used measures of diagnostic test performance--sensitivity and specificity--are likely to be biased. Sensitivity would often be higher, and specificity would be lower, than the true values. This bias is called verification bias. Without adjustment for verification bias, one may possibly introduce into the medical practice a diagnostic test with apparent, but not truly, high sensitivity. In this article, verification bias is treated as a missing covariate problem. We propose a flexible modeling and computational framework for evaluating the performance of a diagnostic test, with adjustment for nonignorable verification bias. The presented computational method can be utilized with any software that can repetitively use a logistic regression module. The approach is likelihood-based, and allows use of categorical or continuous covariates. An explicit formula for the observed information matrix is presented, so that one can easily compute standard errors of estimated parameters. The methodology is illustrated with a cardiology data example. We perform a sensitivity analysis of the dependency of verification selection process on disease.     

Development of a hand tool to test the degree of retting of flax straw

To determine the optimum conditions for inducing retting of flax, by the application of glyphosate herbicide, it is necessary to have an easily portable tool which will give a rapid, accurate assessment of the stage that retting has reached. Such a tool, which can be used to test individual stems, is described and the estimates of retting obtained compared with those from other methods.     

The clinical application of serial bone mass measurements

Although the short-term precision of various bone mineral content (BMC) measurements is known, questions about the clinical use of serial BMC measurements remain: how frequently should BMC be measured? When is it appropriate to calculate bone loss rates? How are estimates of loss rate interpreted? This paper discusses both biological and technical sources of uncertainty, and the estimation of confidence limits for measured bone loss rates. For many, possibly most, patients, calculation of bone loss rate may not be necessary; however, repeated measures of BMC can still be useful for re-evaluating fracture risk. Indications for repeating BMC measurements may include low initial BMC (moderate to high fracture risk), anticipation of rapid bone loss (e.g., menopause, estrogen discontinuation), and verification of treatment efficacy.     

The chloride-water balance sheet; an aid in the management of difficult fluid balance problems

In cases in which there is massive loss of fluids or electrolytes or prolonged decreased urinary output, an absolutely accurate knowledge of all water and chloride intake and output by all routes is necessary to avert large and often fatal errors in water and electrolyte therapy based on estimates made from nurses' notes alone. By use of the Scribner water-chloride balance sheet method of recording data, it is possible to determine with the necessary accuracy how much fluid and what kind of electrolytes to administer to achieve and maintain balance. The method is simple and can be put into effect in any hospital with the cooperation of an educated nursing and laboratory staff.     

Adjusting for selection bias in retrospective, case-control studies

Retrospective case–control studies are more susceptibleto selection bias than other epidemiologic studies as by designthey require that both cases and controls are representativeof the same population. However, as cases and control recruitmentprocesses are often different, it is not always obvious thatthe necessary exchangeability conditions hold. Selection biastypically arises when the selection criteria are associatedwith the risk factor under investigation. We develop a methodwhich produces bias-adjusted estimates for the odds ratio. Ourmethod hinges on 2 conditions. The first is that a variablethat separates the risk factor from the selection criteria canbe identified. This is termed the "bias breaking" variable.The second condition is that data can be found such that a bias-correctedestimate of the distribution of the bias breaking variable canbe obtained. We show by means of a set of examples that suchbias breaking variables are not uncommon in epidemiologic settings.We demonstrate using simulations that the estimates of the oddsratios produced by our method are consistently closer to thetrue odds ratio than standard odds ratio estimates using logisticregression. Further, by applying it to a case–controlstudy, we show that our method can help to determine whetherselection bias is present and thus confirm the validity of studyconclusions when no evidence of selection bias can be found.     

Estimation of rates-across-sites distributions in phylogenetic substitution models

Previous work has shown that it is often essential to account for the variation in rates at different sites in phylogenetic models in order to avoid phylogenetic artifacts such as long branch attraction. In most current models, the gamma distribution is used for the rates-across-sites distributions and is implemented as an equal-probability discrete gamma. In this article, we introduce discrete distribution estimates with large numbers of equally spaced rate categories allowing us to investigate the appropriateness of the gamma model. With large numbers of rate categories, these discrete estimates are flexible enough to approximate the shape of almost any distribution. Likelihood ratio statistical tests and a nonparametric bootstrap confidence-bound estimation procedure based on the discrete estimates are presented that can be used to test the fit of a parametric family. We applied the methodology to several different protein data sets, and found that although the gamma model often provides a good parametric model for this type of data, rate estimates from an equal-probability discrete gamma model with a small number of categories will tend to underestimate the largest rates. In cases when the gamma model assumption is in doubt, rate estimates coming from the discrete rate distribution estimate with a large number of rate categories provide a robust alternative to gamma estimates. An alternative implementation of the gamma distribution is proposed that, for equal numbers of rate categories, is computationally more efficient during optimization than the standard gamma implementation and can provide more accurate estimates of site rates.     

THE CHLORIDE-WATER BALANCE SHEET—An Aid in the Management of Difficult Fluid Balance Problems

In cases in which there is massive loss of fluids or electrolytes or prolonged decreased urinary output, an absolutely accurate knowledge of all water and chloride intake and output by all routes is necessary to avert large and often fatal errors in water and electrolyte therapy based on estimates made from nurses'' notes alone.By use of the Scribner water-chloride balance sheet method of recording data, it is possible to determine with the necessary accuracy how much fluid and what kind of electrolytes to administer to achieve and maintain balance. The method is simple and can be put into effect in any hospital with the cooperation of an educated nursing and laboratory staff.     

Insights into latent class analysis of diagnostic test performance

Latent class analysis is used to assess diagnostic test accuracy when a gold standard assessment of disease is not available but results of multiple imperfect tests are. We consider the simplest setting, where 3 tests are observed and conditional independence (CI) is assumed. Closed-form expressions for maximum likelihood parameter estimates are derived. They show explicitly how observed 2- and 3-way associations between test results are used to infer disease prevalence and test true- and false-positive rates. Although interesting and reasonable under CI, the estimators clearly have no basis when it fails. Intuition for bias induced by conditional dependence follows from the analytic expressions. Further intuition derives from an Expectation Maximization (EM) approach to calculating the estimates. We discuss implications of our results and related work for settings where more than 3 tests are available. We conclude that careful justification of assumptions about the dependence between tests in diseased and nondiseased subjects is necessary in order to ensure unbiased estimates of prevalence and test operating characteristics and to provide these estimates clinical interpretations. Such justification must be based in part on a clear clinical definition of disease and biological knowledge about mechanisms giving rise to test results.     

Exploiting Hardy-Weinberg equilibrium for efficient screening of single SNP associations from case-control studies

In case-control studies, the assessment of the association between a binary disease outcome and a single nucleotide polymorphism (SNP) is often based on comparing the observed genotype distribution for the cases against that for the controls. In this article, we investigate an alternative analytic strategy in which the observed genotype frequencies of cases are compared against the expected genotype frequencies of controls assuming Hardy-Weinberg Equilibrium (HWE). Assuming HWE for controls, we derive closed-form expressions for maximum likelihood estimates of the genotype-specific disease odds ratio (OR) parameters and related variance-covariances. Based on these estimates and their variance-covariance structure, we then propose a two-degree-of-freedom test for disease-SNP association. We show that the proposed test can have substantially higher power than a variety of existing methods, especially when the true effect of the SNP is recessive. We also obtain analytic expressions for the bias of the OR estimates when the underlying HWE assumption is violated. We conclude that the novel test would be particularly useful for analyzing data from the initial 'screening' stages of contemporary multi-stage association studies.     

Genome scanning by composite likelihood

Ambitious programs have recently been advocated or launched to create genomewide databases for meta-analysis of association between DNA markers and phenotypes of medical and/or social concern. A necessary but not sufficient condition for success in association mapping is that the data give accurate estimates of both genomic location and its standard error, which are provided for multifactorial phenotypes by composite likelihood. That class includes the Malecot model, which we here apply with an illustrative example. This preliminary analysis leads to five inferences: permutation of cases and controls provides a test of association free of autocorrelation; two hypotheses give similar estimates, but one is consistently more accurate; estimation of the false-discovery rate is extended to causal genes in a small proportion of regions; the minimal data for successful meta-analysis are inferred; and power is robust for all genomic factors except minor-allele frequency. An extension to meta-analysis is proposed. Other approaches to genome scanning and meta-analysis should, if possible, be similarly extended so that their operating characteristics can be compared.     

Coping with convection in the ultracentrifuge

If cell orientation and temperature control are good, convective flow in the analytical ultracentrifuge cell occurs due to vibrations transmitted from a faulty drive to the rotor itself. Minor vibrations are usually “ironed out” at high speeds but, at low speeds, (<15,000 rpm) can result in incorrectly low molecular weights as well as the appearance of false heterogeneity. In severe cases, a lack of sedimentation of the macromolecule under study may occur. Since drive replacement is sometimes time consuming, we describe here simple procedures which the non-expert can use to test for convection and “eliminate” it, allowing the investigator to rapidly make reliable molecular weight estimates.     

Analysis of allelic association: estimation of the criteria power of the TDT

Analysis of allelic associations is an increasingly more widely used approach to fine mapping of genes of various diseases. To interpret the results correctly, it is necessary to estimate the power of the statistical test used. The principle of the analysis of associations and testing of hypothesis are described, and analytically obtained estimates of the power of the transmission disequilibrium test (TDT), one of the most popular methods of analysis of allelic associations, are presented. These estimates are applicable to arbitrary models of inheritance formulated in terms of relative genotypic risk. The proposed method is illustrated by analysis of the associations of idiopathic scoliosis and aggrecan gene alleles.     

Analysis of Allelic Association: Estimation of the Power of the TDT

Analysis of allelic associations is an increasingly more widely used approach to fine mapping of genes of various diseases. To interpret the results correctly, it is necessary to estimate the power of the statistical test used. The principle of the analysis of associations and testing of hypothesis are described, and analytically obtained estimates of the power of the transmission disequilibrium test (TDT), one of the most popular methods of analysis of allelic associations, are presented. These estimates are applicable to arbitrary models of inheritance formulated in terms of genotypic relative risk. The proposed method is illustrated by analysis of the associations of idiopathic scoliosis and aggrecan gene alleles.     

Models for Estimating Abundance from Repeated Counts of an Open Metapopulation

Summary Using only spatially and temporally replicated point counts, Royle (2004b, Biometrics 60, 108–115) developed an N ‐mixture model to estimate the abundance of an animal population when individual animal detection probability is unknown. One assumption inherent in this model is that the animal populations at each sampled location are closed with respect to migration, births, and deaths throughout the study. In the past this has been verified solely by biological arguments related to the study design as no statistical verification was available. In this article, we propose a generalization of the N ‐mixture model that can be used to formally test the closure assumption. Additionally, when applied to an open metapopulation, the generalized model provides estimates of population dynamics parameters and yields abundance estimates that account for imperfect detection probability and do not require the closure assumption. A simulation study shows these abundance estimates are less biased than the abundance estimate obtained from the original N ‐mixture model. The proposed model is then applied to two data sets of avian point counts. The first example demonstrates the closure test on a single‐season study of Mallards (Anas platyrhynchos), and the second uses the proposed model to estimate the population dynamics parameters and yearly abundance of American robins (Turdus migratorius) from a multi‐year study.     

The Role of Light in the Emergence of Weeds: Using Camelina microcarpa as an Example

When modelling the emergence of weeds, two main factors are considered that condition this process: temperature and soil moisture. Optimum temperature is necessary for metabolic processes that generate energy for growth, while turgor pressure is necessary for root and shoot elongation which eventually leads to seedling emergence from the soil. Most emergence models do not usually consider light as a residual factor, but it could have an important role as it can alter directly or indirectly the dormancy and germination of seeds. In this paper, inclusion of light as an additional factor to photoperiod and radiation in emergence models is explored and compared with the classical hydrothermal time (HTT) model using Camelina microcarpa as an example. HTT based on hourly estimates is also compared with that based on daily estimates. Results suggest that, although HTT based models are accurate enough for local applications, the precision of these models is improved when HTT is estimated hourly and solar radiation is included as a factor.     

Mitogenome phylogenetics: the impact of using single regions and partitioning schemes on topology, substitution rate and divergence time estimation

The availability of mitochondrial genome sequences is growing as a result of recent technological advances in molecular biology. In phylogenetic analyses, the complete mitogenome is increasingly becoming the marker of choice, usually providing better phylogenetic resolution and precision relative to traditional markers such as cytochrome b (CYTB) and the control region (CR). In some cases, the differences in phylogenetic estimates between mitogenomic and single-gene markers have yielded incongruent conclusions. By comparing phylogenetic estimates made from different genes, we identified the most informative mitochondrial regions and evaluated the minimum amount of data necessary to reproduce the same results as the mitogenome. We compared results among individual genes and the mitogenome for recently published complete mitogenome datasets of selected delphinids (Delphinidae) and killer whales (genus Orcinus). Using Bayesian phylogenetic methods, we investigated differences in estimation of topologies, divergence dates, and clock-like behavior among genes for both datasets. Although the most informative regions were not the same for each taxonomic group (COX1, CYTB, ND3 and ATP6 for Orcinus, and ND1, COX1 and ND4 for Delphinidae), in both cases they were equivalent to less than a quarter of the complete mitogenome. This suggests that gene information content can vary among groups, but can be adequately represented by a portion of the complete sequence. Although our results indicate that complete mitogenomes provide the highest phylogenetic resolution and most precise date estimates, a minimum amount of data can be selected using our approach when the complete sequence is unavailable. Studies based on single genes can benefit from the addition of a few more mitochondrial markers, producing topologies and date estimates similar to those obtained using the entire mitogenome.