Statistical inference for net benefit measures in biomarker validation studies

Referral strategies based on risk scores and medical tests are commonly proposed. Direct assessment of their clinical utility requires implementing the strategy and is not possible in the early phases of biomarker research. Prior to late-phase studies, net benefit measures can be used to assess the potential clinical impact of a proposed strategy. Validation studies, in which the biomarker defines a prespecified referral strategy, are a gold standard approach to evaluating biomarker potential. Uncertainty, quantified by a confidence interval, is important to consider when deciding whether a biomarker warrants an impact study, does not demonstrate clinical potential, or that more data are needed. We establish distribution theory for empirical estimators of net benefit and propose empirical estimators of variance. The primary results are for the most commonly employed estimators of net benefit: from cohort and unmatched case-control samples, and for point estimates and net benefit curves. Novel estimators of net benefit under stratified two-phase and categorically matched case-control sampling are proposed and distribution theory developed. Results for common variants of net benefit and for estimation from right-censored outcomes are also presented. We motivate and demonstrate the methodology with examples from lung cancer research and highlight its application to study design.     

Modeling and optimization in early detection programs with a single exam

The choice of timing of screening examinations is an important element in determining the efficacy of strategies for the early detection of occult disease. In this article, we describe a flexible decision-making framework for the design of early detection programs, and we investigate the choice of timing when each individual in the screening program is examined only once. We focus on the theoretical relation between the optimal examination time and the distributions of sojourn times in health-related states. Specifically, we derive closed-form solutions of the optimal age using two specifications of utility functions, discuss the effects of natural history and utility specifications on the optimal solution, and present an application to early detection of colorectal cancer by once-only sigmoidoscopy or colonoscopy.     

Design Aspects and Odds Ratio Estimation in Multicenter Case-Control Studies

In this paper we propose a method to be used in the planning stage of a case-control study. An allocation rule for controls in multicenter case-control studies is proposed which would assure a simple, efficient and unbiased estimation of the odds ratio in the pooled data. It is shown that the efficiency of the design increases with increasing correlation between study center and risk factor. Sources of bias and their implications for relative risk estimation are discussed. The method is demonstrated with data from a case-control study.     

The Effectiveness of Teleglaucoma versus In-Patient Examination for Glaucoma Screening: A Systematic Review and Meta-Analysis

BackgroundGlaucoma is the leading cause of irreversible visual impairment in the world affecting 60.5 million people worldwide in 2010, which is expected to increase to approximately 79.6 million by 2020. Therefore, glaucoma screening is important to detect, diagnose, and treat patients at the earlier stages to prevent disease progression and vision loss. Teleglaucoma uses stereoscopic digital imaging to take ocular images, which are transmitted electronically to an ocular specialist. The purpose is to synthesize literature to evaluate teleglaucoma, its diagnostic accuracy, healthcare system benefits, and cost-effectiveness.MethodsA systematic search was conducted to help locate published and unpublished studies. Studies which evaluate teleglaucoma as a screening device for glaucoma were included. A meta-analysis was conducted to provide estimates of diagnostic accuracy, diagnostic odds ratio, and the relative percentage of glaucoma cases detected. The improvements to healthcare service quality and cost data were assessed.ResultsOf 11237 studies reviewed, 45 were included. Our results indicated that, teleglaucoma is more specific and less sensitive than in-person examination. The pooled estimates of sensitivity was 0.832 [95% CI 0.770, 0.881] and specificity was 0.790 [95% CI 0.668, 0.876]. The relative odds of a positive screen test in glaucoma cases are 18.7 times more likely than a negative screen test in a non-glaucoma cases. Additionally, the mean cost for every case of glaucoma detected was $1098.67 US and of teleglaucoma per patient screened was $922.77 US.ConclusionTeleglaucoma can accurately discriminate between screen test results with greater odds for positive cases. It detects more cases of glaucoma than in-person examination. Both patients and the healthcare systems benefit from early detection, reduction in wait and travel times, increased specialist referral rates, and cost savings. Teleglaucoma is an effective screening tool for glaucoma specifically for remote and under-services communities.     

Testing the Independence of Two Diagnostic Tests

Consider two diagnostic procedures having binary outcomes. If one of the tests results in a positive finding, a more definitive diagnostic procedure will be administered to establish the presence or absence of a disease. The use of both tests will improve the overall screening sensitivity when the two tests are independent, compared with employing two tests that are positively correlated. We estimate the correlation coefficient of the two tests and derive statistical methods for testing the independence of the two diagnostic procedures conditional on disease status. The statistical tests are used to investigate the independence of mammography and clinical breast exams aimed at establishing the benefit of early detection of breast cancer. The data used in the analysis are obtained from periodic screening examinations of three randomized clinical trials of breast cancer screening. Analysis of each of these trials confirms the independence of the clinical breast and mammography examinations. Based on these three large clinical trials, we conclude that a clinical breast exam considerably increases the overall sensitivity relative to screening with mammography alone and should be routinely included in early breast cancer detection programs.     

Family-based versus unrelated case-control designs for genetic associations

The most simple and commonly used approach for genetic associations is the case-control study design of unrelated people. This design is susceptible to population stratification. This problem is obviated in family-based studies, but it is usually difficult to accumulate large enough samples of well-characterized families. We addressed empirically whether the two designs give similar estimates of association in 93 investigations where both unrelated case-control and family-based designs had been employed. Estimated odds ratios differed beyond chance between the two designs in only four instances (4%). The summary relative odds ratio (ROR) (the ratio of odds ratios obtained from unrelated case-control and family-based studies) was close to unity (0.96 [95% confidence interval, 0.91-1.01]). There was no heterogeneity in the ROR across studies (amount of heterogeneity beyond chance I(2) = 0%). Differences on whether results were nominally statistically significant (p < 0.05) or not with the two designs were common (opposite classification rates 14% and 17%); this reflected largely differences in power. Conclusions were largely similar in diverse subgroup analyses. Unrelated case-control and family-based designs give overall similar estimates of association. We cannot rule out rare large biases or common small biases.     

Comparing disease screening tests when true disease status is ascertained only for screen positives

Disease screening is a fundamental part of health care. To evaluate the accuracy of a new screening modality, ideally the results of the screening test are compared with those of a definitive diagnostic test in a set of study subjects. However, definitive diagnostic tests are often invasive and cannot be applied to subjects whose screening tests are negative for disease. For example, in cancer screening, the assessment of true disease status requires a biopsy sample, which for ethical reasons can only be obtained if a subject's screening test indicates presence of cancer. Although the absolute accuracy of screening tests cannot be evaluated in such circumstances, it is possible to compare the accuracies of screening tests. Specifically, using relative true positive rate (the ratio of the true positive rate of one test to another) and relative false positive rate (the ratio of the false positive rates of two tests) as measures of relative accuracy, we show that inference about relative accuracy can be made from such studies. Analogies with case-control studies can be drawn where inference about absolute risk cannot be made, but inference about relative risk can. In this paper, we develop a marginal regression analysis framework for making inference about relative accuracy when only screen positives are followed for true disease. In this context factors influencing the relative accuracies of tests can be evaluated. It is important to determine such factors in order to understand circumstances in which one test is preferable to another. The methods are applied to two cancer screening studies, one concerning the effect of race on screening for prostate cancer and the other concerning the effect of tumour grade on the detection of cervical cancer with cytology versus cervicography screening.     

Comparison of Pooled Risk Estimates for Adverse Effects from Different Observational Study Designs: Methodological Overview

Background

A diverse range of study designs (e.g. case-control or cohort) are used in the evaluation of adverse effects. We aimed to ascertain whether the risk estimates from meta-analyses of case-control studies differ from that of other study designs.

Methods

Searches were carried out in 10 databases in addition to reference checking, contacting experts, and handsearching key journals and conference proceedings. Studies were included where a pooled relative measure of an adverse effect (odds ratio or risk ratio) from case-control studies could be directly compared with the pooled estimate for the same adverse effect arising from other types of observational studies.

Results

We included 82 meta-analyses. Pooled estimates of harm from the different study designs had 95% confidence intervals that overlapped in 78/82 instances (95%). Of the 23 cases of discrepant findings (significant harm identified in meta-analysis of one type of study design, but not with the other study design), 16 (70%) stemmed from significantly elevated pooled estimates from case-control studies. There was associated evidence of funnel plot asymmetry consistent with higher risk estimates from case-control studies. On average, cohort or cross-sectional studies yielded pooled odds ratios 0.94 (95% CI 0.88–1.00) times lower than that from case-control studies.

Interpretation

Empirical evidence from this overview indicates that meta-analysis of case-control studies tend to give slightly higher estimates of harm as compared to meta-analyses of other observational studies. However it is impossible to rule out potential confounding from differences in drug dose, duration and populations when comparing between study designs.     

Negative and Non-Positive Epidemiological Studies

Aim. To identify and discuss validity aspects on so called negative and non-positive studies. Methods. Arguments and examples are drawn from experiences in occupational health epidemiology regarding the interpretation of more or less equivocal study results. Results and conclusions. A negative study may be defined as showing a result that goes against the investigated hypothesis of an increased (or prevented) risk. Traditionally, studies with a risk estimate (relative risk or odds ratio) above but close to unity are also referred to as negative, given a narrow confidence interval (CI) that includes unity. A risk estimate above unity with the CI including unity is non-positive, however, but an estimate below unity with upper CI bond exceeding unity might be seen as possibly negative or non-negative. A weaker “significance” than usually required should perhaps be accepted when evaluating serious hazards. In contrast to positive studies, the negative and non-positive studies tend to escape criticism in spite of questionable validity that may have obscured existing risks (or preventive effects). Even stronger arguments can be made in criticising negative and non-positive studies than positive studies, for example, regarding selection phenomena, and observational problems regarding exposure and outcome. Negative confounding should be considered although usually weak. In case-control studies, so-called over-matching may obscure an existing risk as could the “healthy worker effect” in cohort studies. Small scale non-positive studies should be made available for meta-analyses and when considering studies that do not convincingly show a risk; those who are exposed should be given the “benefit of the doubt.”     

Estimation of Odds Ratios from Matched Case-Control Studies with Incomplete Data

Matched case-control studies often include pairs with incomplete exposure information. This work presents and compares two estimators for the odds ratio that can be used when the exposures of some of the cases and controls are missing. A simulation study shows that the estimator that uses the marginal exposure frequencies is usually more efficient than the estimator based on discordant pairs.     

The optimal ratio of cases to controls for estimating the classification accuracy of a biomarker

The case-control design is frequently used to study the discriminatory accuracy of a screening or diagnostic biomarker. Yet, the appropriate ratio in which to sample cases and controls has never been determined. It is common for researchers to sample equal numbers of cases and controls, a strategy that can be optimal for studies of association. However, considerations are quite different when the biomarker is to be used for classification. In this paper, we provide an expression for the optimal case-control ratio, when the accuracy of the biomarker is quantified by the receiver operating characteristic (ROC) curve. We show how it can be integrated with choosing the overall sample size to yield an efficient study design with specified power and type-I error. We also derive the optimal case-control ratios for estimating the area under the ROC curve and the area under part of the ROC curve. Our methods are applied to a study of a new marker for adenocarcinoma in patients with Barrett's esophagus.     

Mortality modeling of early detection programs

Summary .   Consider a group of subjects who are offered an opportunity to receive a sequence of periodic special examinations for the purpose of diagnosing a chronic disease earlier relative to usual care. The mortality for the early detection group is to be compared with a group receiving usual care. Benefit is reflected in a potential reduction in mortality. This article develops a general probability model that can be used to predict cumulative mortality for each of these groups. The elements of the model assume (i) a four-state progressive disease model in which a subject may be in a disease-free state (or a disease state that cannot be detected), preclinical disease state (capable of being diagnosed by a special exam), clinical state (diagnosis by usual care), and a death state; (ii) age-dependent transitions into the states; (iii) age-dependent examination sensitivity; (iv) age-dependent sojourn time in each state; and (v) the distribution of disease stages on diagnosis conditional on modality of detection. The model may be used to (i) compare mortality rates for different screening schedules; (ii) explore potential benefit of subpopulations; and (iii) compare relative reductions in disease-specific mortality due to advances and dissemination of both treatment and early detection screening programs.     

Overdiagnosis in early detection programs

Overdiagnosis refers to the situation where a screening exam detects a disease that would have otherwise been undetected in a person's lifetime. The disease would have not have been diagnosed because the individual would have died of other causes prior to its clinical onset. Although the probability of overdiagnosis is an important quantity for understanding early detection programs it has not been rigorously studied. We analyze an idealized early detection program and derive the mathematical expression for the probability of overdiagnosis. The results are studied numerically for prostate cancer and applied to a variety of screening schedules. Our investigation indicates that the probability of overdiagnosis is remarkably high.     

High-throughput determination of mode of inhibition in lead identification and optimization

After finishing the primary high-throughput screening, the screening team is often faced with thousands of hits to be evaluated further. Effective filtering of these hits is crucial in identifying leads. Mode of inhibition (MOI) study is extremely useful in validating whether the observed compound activity is specific to the biological target. In this article, the authors describe a high-throughput MOI determination method for evaluating thousands of compounds using an existing screening infrastructure. Based on enzyme or receptor kinetics theory, the authors developed the method by measuring the ratio of IC(50) or percent inhibition at 2 carefully chosen substrate or ligand concentrations to define an inhibitor as competitive, uncompetitive, or noncompetitive. This not only facilitates binning of HTS hits according to their MOI but also greatly expands HTS utility in support of the medicinal chemistry team's lead optimization practice. Three case studies are presented to demonstrate how the method was applied successfully in 3 discovery programs targeting either an enzyme or a G-protein-coupled receptor.     

Cellular Differences Between True Negative and False Negative Papanicolaou Smears

A matched case-control study of 123 false negative Papanicolaou smears and 488 true negative Papanicolaou smears was undertaken to determine the association between the types of cells present on the smear and the correctness of the cytology report. the false negative slides were significantly more likely to include endocervical columnar cells than the true negative slides (odds ratio 1.90, 95% confidence interval (CI) 1.21–3.01). No statistically significant difference in metaplastic cell status was evident (odds ratio 1.48, 95% CI 0.95–2.30). When considered together, metaplastic and/or columnar cells were significantly more likely to be present in false negative smears than in true negative smears (odds ratio 1.87, 95% CI 1.13–3.08). the implications of these findings for improving the accuracy of cervical cytology for the detection of precancerous lesions are discussed.     

Semiparametric maximum likelihood methods for analyzing genetic and environmental effects with case-control mother-child pair data

Summary Case-control mother-child pair design represents a unique advantage for dissecting genetic susceptibility of complex traits because it allows the assessment of both maternal and offspring genetic compositions. This design has been widely adopted in studies of obstetric complications and neonatal outcomes. In this work, we developed an efficient statistical method for evaluating joint genetic and environmental effects on a binary phenotype. Using a logistic regression model to describe the relationship between the phenotype and maternal and offspring genetic and environmental risk factors, we developed a semiparametric maximum likelihood method for the estimation of odds ratio association parameters. Our method is novel because it exploits two unique features of the study data for the parameter estimation. First, the correlation between maternal and offspring SNP genotypes can be specified under the assumptions of random mating, Hardy-Weinberg equilibrium, and Mendelian inheritance. Second, environmental exposures are often not affected by offspring genes conditional on maternal genes. Our method yields more efficient estimates compared with the standard prospective method for fitting logistic regression models to case-control data. We demonstrated the performance of our method through extensive simulation studies and the analysis of data from the Jerusalem Perinatal Study.     

The Aetiological Role of Human Papillomavirus in Oesophageal Squamous Cell Carcinoma: A Meta-Analysis

Background

The aetiological role of human papillomavirus (HPV) in oesophageal squamous cell carcinoma (OSCC) has been widely researched for more than three decades, with conflicting findings. In the absence of a large, adequately powered single case-control study, a meta-analysis of all available case-control studies is the most rigorous way of identifying any potential association between HPV and OSCC. We present the first global meta-analysis of case-control studies investigating the role of HPV in OSCC.

Methods

Case-control studies investigating OSCC tissue for presence of HPV DNA were identified. 21 case-control studies analyzing a total of 1223 cases and 1415 controls, met our inclusion criteria. HPV detection rates were tabulated for each study and all studies were assessed for quality. The random effects method was used to pool the odds ratios (OR).

Results

From all OSCC specimens included in this meta-analysis, 35% (426/1223) were positive for HPV DNA. The pooled OR for an HPV-OSCC association was 3.04 (95% CI 2.20 to 4.20). Meta-regression analysis did not find a significant association between OR and any of the quality domains. Influence analysis was non-significant for the effect of individual studies on the pooled estimate. Studies conducted in countries with low to medium OSCC incidence showed a stronger relationship (OR 4.65, 95% CI 2.47 to 8.76) than regions of high OSCC incidence (OR 2.65, 95% CI 1.80 to 3.91).

Conclusions

Uncertainty around the aetiological role of HPV in OSCC is due largely to the small number and scale of appropriately designed studies. Our meta-analysis of these studies suggests that HPV increases the risk of OSCC three-fold. This study provides the strongest evidence to date of an HPV-OSCC association. The importance of these findings is that prophylactic vaccination could be of public health benefit in prevention of OSCC in countries with high OSCC incidence.     

Reducing mean squared error in the analysis of stratified epidemiologic studies

Kalish (1990, Biometrics 46, 493-499) proposed an approximately optimal estimator of a common odds ratio for pair-matched case-control studies. His approach is easily extended to general stratified studies. For unmatched stratified studies, the form of the approximately optimal estimator is very simple, and may often correspond to no more than a small correction to the ordinary stratified estimator.     

Excess Relative Risk as an Effect Measure in Case-Control Studies of Rare Diseases

Epidemiologists often use ratio-type indices (rate ratio, risk ratio and odds ratio) to quantify the association between exposure and disease. By comparison, less attention has been paid to effect measures on a difference scale (excess rate or excess risk). The excess relative risk (ERR) used primarily by radiation epidemiologists is of peculiar interest here, in that it involves both difference and ratio operations. The ERR index (but not the difference-type indices) is estimable in case-control studies. Using the theory of sufficient component cause model, the author shows that when there is no mechanistic interaction (no synergism in the sufficient cause sense) between the exposure under study and the stratifying variable, the ERR index (but not the ratio-type indices) in a rare-disease case-control setting should remain constant across strata and can therefore be regarded as a common effect parameter. By exploiting this homogeneity property, the related attributable fraction indices can also be estimated with greater precision. The author demonstrates the methodology (SAS codes provided) using a case-control dataset, and shows that ERR preserves the logical properties of the ratio-type indices. In light of the many desirable properties of the ERR index, the author advocates its use as an effect measure in case-control studies of rare diseases.     

The design and analysis of case-control studies with biased sampling

A design is proposed for case-control studies in which selection of subjects for full variable ascertainment is based jointly on disease status and on easily obtained "screening" variables that may be related to the disease. Recruitment of subjects follows an independent Bernoulli sampling scheme, with recruitment probabilities set by the investigator in advance. In particular, the sampling can be set up to achieve, on average, frequency matching, provided prior estimates of the disease rates or odds ratios associated with screening variables such as age and sex are available. Alternatively--for example, when studying a rare exposure--one can enrich the sample with certain categories of subject. Following such a design, there are two valid approaches to logistic regression analysis, both of which allow for efficient estimation of effects associated with the screening variables that were allowed to bias the recruitment. The statistical properties of the estimators are compared, both for large samples, based on asymptotics, and for small samples, based on simulations.