Potential misinterpretation of treatment effects due to use of odds ratios and logistic regression in randomized controlled trials

Background

In randomized controlled trials (RCTs), the odds ratio (OR) can substantially overestimate the risk ratio (RR) if the incidence of the outcome is over 10%. This study determined the frequency of use of ORs, the frequency of overestimation of the OR as compared with its accompanying RR in published RCTs, and we assessed how often regression models that calculate RRs were used.

Methods

We included 288 RCTs published in 2008 in five major general medical journals (Annals of Internal Medicine, British Medical Journal, Journal of the American Medical Association, Lancet, New England Journal of Medicine). If an OR was reported, we calculated the corresponding RR, and we calculated the percentage of overestimation by using the formula .

Results

Of 193 RCTs with a dichotomous primary outcome, 24 (12.4%) presented a crude and/or adjusted OR for the primary outcome. In five RCTs (2.6%), the OR differed more than 100% from its accompanying RR on the log scale. Forty-one of all included RCTs (n = 288; 14.2%) presented ORs for other outcomes, or for subgroup analyses. Nineteen of these RCTs (6.6%) had at least one OR that deviated more than 100% from its accompanying RR on the log scale. Of 53 RCTs that adjusted for baseline variables, 15 used logistic regression. Alternative methods to estimate RRs were only used in four RCTs.

Conclusion

ORs and logistic regression are often used in RCTs and in many articles the OR did not approximate the RR. Although the authors did not explicitly misinterpret these ORs as RRs, misinterpretation by readers can seriously affect treatment decisions and policy making.     

Optimal approximate conversions of odds ratios and hazard ratios to risk ratios

Odds ratios approximate risk ratios when the outcome under consideration is rare but can diverge substantially from risk ratios when the outcome is common. In this paper, we derive optimal analytic conversions of odds ratios and hazard ratios to risk ratios that are minimax for the bias ratio when outcome probabilities are specified to fall in any fixed interval. The results for hazard ratios are derived under a proportional hazard assumption for the exposure. For outcome probabilities specified to lie in symmetric intervals centered around 0.5, it is shown that the square-root transformation of the odds ratio is the optimal minimax conversion for the risk ratio. General results for any nonsymmetric interval are given both for odds ratio and for hazard ratio conversions. The results are principally useful when odds ratios or hazard ratios are reported in papers, and the reader does not have access to the data or to information about the overall outcome prevalence.     

Using conditional logistic regression to fit proportional odds models to interval censored data

An easily implemented approach to fitting the proportional odds regression model to interval-censored data is presented. The approach is based on using conditional logistic regression routines in standard statistical packages. Using conditional logistic regression allows the practitioner to sidestep complications that attend estimation of the baseline odds ratio function. The approach is applicable both for interval-censored data in settings in which examinations continue regardless of whether the event of interest has occurred and for current status data. The methodology is illustrated through an application to data from an AIDS study of the effect of treatment with ZDV+ddC versus ZDV alone on 50% drop in CD4 cell count from baseline level. Simulations are presented to assess the accuracy of the procedure.     

Non‐proportional odds multivariate logistic regression of ordinal family data

Methods to examine whether genetic and/or environmental sources can account for the residual variation in ordinal family data usually assume proportional odds. However, standard software to fit the non‐proportional odds model to ordinal family data is limited because the correlation structure of family data is more complex than for other types of clustered data. To perform these analyses we propose the non‐proportional odds multivariate logistic regression model and take a simulation‐based approach to model fitting using Markov chain Monte Carlo methods, such as partially collapsed Gibbs sampling and the Metropolis algorithm. We applied the proposed methodology to male pattern baldness data from the Victorian Family Heart Study.     

Misuse of odds ratios in obesity literature: an empirical analysis of published studies

Odds ratios (ORs) are widely used in scientific research to demonstrate the associations between outcome variables and covariates (risk factors) of interest, and are often described in language suitable for risks or probabilities, but odds and probabilities are related, not equivalent. In situations where the outcome is not rare (e.g., obesity), ORs no longer approximate the relative risk ratio (RR) and may be misinterpreted. Our study examines the extent of misinterpretation of ORs in Obesity and International Journal of Obesity. We reviewed all 2010 issues of these journals to identify all articles that presented ORs. Included articles were then primarily reviewed for correct presentation and interpretation of ORs; and secondarily reviewed for article characteristics that may have been associated with how ORs are presented and interpreted. Of the 855 articles examined, 62 (7.3%) presented ORs. ORs were presented incorrectly in 23.2% of these articles. Clinical articles were more likely to present ORs correctly than social science or basic science articles. Studies with outcome variables that had higher relative prevalence were less likely to present ORs correctly. Overall, almost one-quarter of the studies presenting ORs in two leading journals on obesity misinterpreted them. Furthermore, even when researchers present ORs correctly, the lay media may misinterpret them as relative RRs. Therefore, we suggest that when the magnitude of associations is of interest, researchers should carefully and accurately present interpretable measures of association--including RRs and risk differences--to minimize confusion and misrepresentation of research results.     

Bias-reduced estimators and confidence intervals for odds ratios in genome-wide association studies

Genome-wide association studies (GWAS) provide an important approach to identifying common genetic variants that predispose to human disease. A typical GWAS may genotype hundreds of thousands of single nucleotide polymorphisms (SNPs) located throughout the human genome in a set of cases and controls. Logistic regression is often used to test for association between a SNP genotype and case versus control status, with corresponding odds ratios (ORs) typically reported only for those SNPs meeting selection criteria. However, when these estimates are based on the original data used to detect the variant, the results are affected by a selection bias sometimes referred to the "winner's curse" (Capen and others, 1971). The actual genetic association is typically overestimated. We show that such selection bias may be severe in the sense that the conditional expectation of the standard OR estimator may be quite far away from the underlying parameter. Also standard confidence intervals (CIs) may have far from the desired coverage rate for the selected ORs. We propose and evaluate 3 bias-reduced estimators, and also corresponding weighted estimators that combine corrected and uncorrected estimators, to reduce selection bias. Their corresponding CIs are also proposed. We study the performance of these estimators using simulated data sets and show that they reduce the bias and give CI coverage close to the desired level under various scenarios, even for associations having only small statistical power.     

Bayesian dose-finding in phase I/II clinical trials using toxicity and efficacy odds ratios

A Bayesian adaptive design is proposed for dose-finding in phase I/II clinical trials to incorporate the bivariate outcomes, toxicity and efficacy, of a new treatment. Without specifying any parametric functional form for the drug dose-response curve, we jointly model the bivariate binary data to account for the correlation between toxicity and efficacy. After observing all the responses of each cohort of patients, the dosage for the next cohort is escalated, deescalated, or unchanged according to the proposed odds ratio criteria constructed from the posterior toxicity and efficacy probabilities. A novel class of prior distributions is proposed through logit transformations which implicitly imposes a monotonic constraint on dose toxicity probabilities and correlates the probabilities of the bivariate outcomes. We conduct simulation studies to evaluate the operating characteristics of the proposed method. Under various scenarios, the new Bayesian design based on the toxicity-efficacy odds ratio trade-offs exhibits good properties and treats most patients at the desirable dose levels. The method is illustrated with a real trial design for a breast medical oncology study.     

Correlating observed odds ratios from lung cancer case-control studies to SNP functional scores predicted by bioinformatic tools

Bioinformatic tools are widely utilized to predict functional single nucleotide polymorphisms (SNPs) for genotyping in molecular epidemiological studies. However, the extent to which these approaches are mirrored by epidemiological findings has not been fully explored. In this study, we first surveyed SNPs examined in case-control studies of lung cancer, the most extensively studied cancer type. We then computed SNP functional scores using four popular bioinformatics tools: SIFT, PolyPhen, SNPs3D, and PMut, and determined their predictive potential using the odds ratios (ORs) reported. Spearman's correlation coefficient (r) for the association with SNP score from SIFT, PolyPhen, SNPs3D, and PMut, and the summary ORs were r=-0.36 (p=0.007), r=0.25 (p=0.068), r=-0.20 (p=0.205), and r=-0.12 (p=0.370), respectively. By creating a combined score using information from all four tools we were able to achieve a correlation coefficient of r=0.51 (p<0.001). These results indicate that scores of predicted functionality could explain a certain fraction of the lung cancer risk detected in genetic association studies and more accurate predictions may be obtained by combining information from a variety of tools. Our findings suggest that bioinformatic tools are useful in predicting SNP functionality and may facilitate future genetic epidemiological studies.     

Application of logistic regression to case-control association studies involving two causative loci

Models in which two susceptibility loci jointly influence the risk of developing disease can be explored using logistic regression analysis. Comparison of likelihoods of models incorporating different sets of disease model parameters allows inferences to be drawn regarding the nature of the joint effect of the loci. We have simulated case-control samples generated assuming different two-locus models and then analysed them using logistic regression. We show that this method is practicable and that, for the models we have used, it can be expected to allow useful inferences to be drawn from sample sizes consisting of hundreds of subjects. Interactions between loci can be explored, but interactive effects do not exactly correspond with classical definitions of epistasis. We have particularly examined the issue of the extent to which it is helpful to utilise information from a previously identified locus when investigating a second, unknown locus. We show that for some models conditional analysis can have substantially greater power while for others unconditional analysis can be more powerful. Hence we conclude that in general both conditional and unconditional analyses should be performed when searching for additional loci.     

Bias-reduced estimators of conditional odds ratios in matched case-control studies with unmatched confounding

We study bias-reduced estimators of exponentially transformed parameters in general linear models (GLMs) and show how they can be used to obtain bias-reduced conditional (or unconditional) odds ratios in matched case-control studies. Two options are considered and compared: the explicit approach and the implicit approach. The implicit approach is based on the modified score function where bias-reduced estimates are obtained by using iterative procedures to solve the modified score equations. The explicit approach is shown to be a one-step approximation of this iterative procedure. To apply these approaches for the conditional analysis of matched case-control studies, with potentially unmatched confounding and with several exposures, we utilize the relation between the conditional likelihood and the likelihood of the unconditional logit binomial GLM for matched pairs and Cox partial likelihood for matched sets with appropriately setup data. The properties of the estimators are evaluated by using a large Monte Carlo simulation study and an illustration of a real dataset is shown. Researchers reporting the results on the exponentiated scale should use bias-reduced estimators since otherwise the effects can be under or overestimated, where the magnitude of the bias is especially large in studies with smaller sample sizes.     

Causal mediation analysis in nested case-control studies using conditional logistic regression

The paper proposes an approach to causal mediation analysis in nested case-control study designs, often incorporated with countermatching schemes using conditional likelihood, and we compare the method's performance to that of mediation analysis using the Cox model for the full cohort with a continuous or dichotomous mediator. Simulation studies are conducted to assess our proposed method and investigate the efficiency relative to the cohort. We illustrate the method using actual data from two studies of potential mediation of radiation risk conducted within the Adult Health Study cohort of atomic-bomb survivors. The performance becomes comparable to that based on the full cohort, illustrating the potential for valid mediation analysis based on the reduced data obtained through the nested case-control design.     

A note on estimating crude odds ratios in case-control studies with differentially misclassified exposure

Morrissey and Spiegelman (1999, Biometrics 55, 338 344) provided a comparative study of adjustment methods for exposure misclassification in case-control studies equipped with an internal validation sample. In addition to the maximum likelihood (ML) approach, they considered two intuitive procedures based on proposals in the literature. Despite appealing ease of computation associated with the latter two methods, efficiency calculations suggested that ML was often to be recommended for the analyst with access to a numerical routine to facilitate it. Here, a reparameterization of the likelihood reveals that one of the intuitive approaches, the inverse matrix method, is in fact ML under differential misclassification. This correction is intended to alert readers to the existence of a simple closed-form ML estimator for the odds ratio in this setting so that they may avoid assuming that a commercially inaccessible optimization routine must be sought to implement ML.     

Classification of gene microarrays by penalized logistic regression

Classification of patient samples is an important aspect of cancer diagnosis and treatment. The support vector machine (SVM) has been successfully applied to microarray cancer diagnosis problems. However, one weakness of the SVM is that given a tumor sample, it only predicts a cancer class label but does not provide any estimate of the underlying probability. We propose penalized logistic regression (PLR) as an alternative to the SVM for the microarray cancer diagnosis problem. We show that when using the same set of genes, PLR and the SVM perform similarly in cancer classification, but PLR has the advantage of additionally providing an estimate of the underlying probability. Often a primary goal in microarray cancer diagnosis is to identify the genes responsible for the classification, rather than class prediction. We consider two gene selection methods in this paper, univariate ranking (UR) and recursive feature elimination (RFE). Empirical results indicate that PLR combined with RFE tends to select fewer genes than other methods and also performs well in both cross-validation and test samples. A fast algorithm for solving PLR is also described.     

Comparison of logistic regression and linear regression in modeling percentage data

Percentage is widely used to describe different results in food microbiology, e.g., probability of microbial growth, percent inactivated, and percent of positive samples. Four sets of percentage data, percent-growth-positive, germination extent, probability for one cell to grow, and maximum fraction of positive tubes, were obtained from our own experiments and the literature. These data were modeled using linear and logistic regression. Five methods were used to compare the goodness of fit of the two models: percentage of predictions closer to observations, range of the differences (predicted value minus observed value), deviation of the model, linear regression between the observed and predicted values, and bias and accuracy factors. Logistic regression was a better predictor of at least 78% of the observations in all four data sets. In all cases, the deviation of logistic models was much smaller. The linear correlation between observations and logistic predictions was always stronger. Validation (accomplished using part of one data set) also demonstrated that the logistic model was more accurate in predicting new data points. Bias and accuracy factors were found to be less informative when evaluating models developed for percentage data, since neither of these indices can compare predictions at zero. Model simplification for the logistic model was demonstrated with one data set. The simplified model was as powerful in making predictions as the full linear model, and it also gave clearer insight in determining the key experimental factors.     

Estimates of absolute cause-specific risk in cohort studies

In this paper we study methods for estimating the absolute risk of an event c1 in a time interval [t1, t2], given that the individual is at risk at t1 and given the presence of competing risks. We discuss some advantages of absolute risk for measuring the prognosis of an individual patient and some difficulties of interpretation for comparing two treatment groups. We also discuss the importance of the concept of absolute risk in evaluating public health measures to prevent disease. Variance calculations permit one to gauge the relative importance of random and systematic errors in estimating absolute risk. Efficiency calculations were also performed to determine how much precision is lost in estimating absolute risk with a nonparametric approach or with a flexible piecewise exponential model rather than a simple exponential model, and other calculations indicate the extent of bias that arises with the simple exponential model when that model is invalid. Such calculations suggest that the more flexible models will be useful in practice. Simulations confirm that asymptotic methods yield reliable variance estimates and confidence interval coverages in samples of practical size.     

Estimating odds ratios in genome scans: an approximate conditional likelihood approach

In modern whole-genome scans, the use of stringent thresholds to control the genome-wide testing error distorts the estimation process, producing estimated effect sizes that may be on average far greater in magnitude than the true effect sizes. We introduce a method, based on the estimate of genetic effect and its standard error as reported by standard statistical software, to correct for this bias in case-control association studies. Our approach is widely applicable, is far easier to implement than competing approaches, and may often be applied to published studies without access to the original data. We evaluate the performance of our approach via extensive simulations for a range of genetic models, minor allele frequencies, and genetic effect sizes. Compared to the naive estimation procedure, our approach reduces the bias and the mean squared error, especially for modest effect sizes. We also develop a principled method to construct confidence intervals for the genetic effect that acknowledges the conditioning on statistical significance. Our approach is described in the specific context of odds ratios and logistic modeling but is more widely applicable. Application to recently published data sets demonstrates the relevance of our approach to modern genome scans.     

Purposeful selection of variables in logistic regression

Background  

The main problem in many model-building situations is to choose from a large set of covariates those that should be included in the "best" model. A decision to keep a variable in the model might be based on the clinical or statistical significance. There are several variable selection algorithms in existence. Those methods are mechanical and as such carry some limitations. Hosmer and Lemeshow describe a purposeful selection of covariates within which an analyst makes a variable selection decision at each step of the modeling process.