The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies

Much biomedical research is observational. The reporting of such research is often inadequate, which hampers the assessment of its strengths and weaknesses and of a study's generalisability. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Initiative developed recommendations on what should be included in an accurate and complete report of an observational study. We defined the scope of the recommendations to cover three main study designs: cohort, case-control, and cross-sectional studies. We convened a 2-day workshop in September 2004, with methodologists, researchers, and journal editors to draft a checklist of items. This list was subsequently revised during several meetings of the coordinating group and in e-mail discussions with the larger group of STROBE contributors, taking into account empirical evidence and methodological considerations. The workshop and the subsequent iterative process of consultation and revision resulted in a checklist of 22 items (the STROBE Statement) that relate to the title, abstract, introduction, methods, results, and discussion sections of articles. 18 items are common to all three study designs and four are specific for cohort, case-control, or cross-sectional studies. A detailed Explanation and Elaboration document is published separately and is freely available on the Web sites of PLoS Medicine, Annals of Internal Medicine, and Epidemiology. We hope that the STROBE Statement will contribute to improving the quality of reporting of observational studies.     

Internal pilots for observational studies

Study planning often involves selecting an appropriate sample size. Power calculations require specifying an effect size and estimating “nuisance” parameters, e.g. the overall incidence of the outcome. For observational studies, an additional source of randomness must be estimated: the rate of the exposure. A poor estimate of any of these parameters will produce an erroneous sample size. Internal pilot (IP) designs reduce the risk of this error ‐ leading to better resource utilization ‐ by using revised estimates of the nuisance parameters at an interim stage to adjust the final sample size. In the clinical trials setting, where allocation to treatment groups is pre‐determined, IP designs have been shown to achieve the targeted power without introducing substantial inflation of the type I error rate. It has not been demonstrated whether the same general conclusions hold in observational studies, where exposure‐group membership cannot be controlled by the investigator. We extend the IP to observational settings. We demonstrate through simulations that implementing an IP, in which prevalence of the exposure can be re‐estimated at an interim stage, helps ensure optimal power for observational research with little inflation of the type I error associated with the final data analysis.     

Improving trial generalizability using observational studies

Complementary features of randomized controlled trials (RCTs) and observational studies (OSs) can be used jointly to estimate the average treatment effect of a target population. We propose a calibration weighting estimator that enforces the covariate balance between the RCT and OS, therefore improving the trial-based estimator's generalizability. Exploiting semiparametric efficiency theory, we propose a doubly robust augmented calibration weighting estimator that achieves the efficiency bound derived under the identification assumptions. A nonparametric sieve method is provided as an alternative to the parametric approach, which enables the robust approximation of the nuisance functions and data-adaptive selection of outcome predictors for calibration. We establish asymptotic results and confirm the finite sample performances of the proposed estimators by simulation experiments and an application on the estimation of the treatment effect of adjuvant chemotherapy for early-stage non-small-cell lung patients after surgery.     

Testing weak nulls in matched observational studies

We develop sensitivity analyses for the sample average treatment effect in matched observational studies while allowing unit-level treatment effects to vary. The methods may be applied to studies using any optimal without-replacement matching algorithm. In contrast to randomized experiments and to paired observational studies, we show for general matched designs that over a large class of test statistics, any procedure bounding the worst-case expectation while allowing for arbitrary effect heterogeneity must be unnecessarily conservative if treatment effects are actually constant across individuals. We present a sensitivity analysis which bounds the worst-case expectation while allowing for effect heterogeneity, and illustrate why it is generally conservative if effects are constant. An alternative procedure is presented that is asymptotically sharp if treatment effects are constant, and that is valid for testing the sample average effect under additional restrictions which may be deemed benign by practitioners. Simulations demonstrate that this alternative procedure results in a valid sensitivity analysis for the weak null hypothesis under a host of reasonable data-generating processes. The procedures allow practitioners to assess robustness of estimated sample average treatment effects to hidden bias while allowing for effect heterogeneity in matched observational studies.     

Directional penalties for optimal matching in observational studies

Multivariate matching in observational studies tends to view covariate differences symmetrically: a difference in age of 10 years is thought equally problematic whether the treated subject is older or younger than the matched control. If matching is correcting an imbalance in age, such that treated subjects are typically older than controls, then the situation in need of correction is asymmetric: a matched pair with a difference in age of 10 years is much more likely to have an older treated subject and a younger control than the opposite. Correcting the bias may be easier if matching tries to avoid the typical case that creates the bias. We describe several easily used, asymmetric, directional penalties and illustrate how they can improve covariate balance in a matched sample. The investigator starts with a matched sample built in a conventional way, then diagnoses residual covariate imbalances in need of reduction, and achieves the needed reduction by slightly altering the distance matrix with directional penalties, creating a new matched sample. Unlike penalties commonly used in matching, a directional penalty can go too far, reversing the direction of the bias rather than reducing the bias, so the magnitude of the directional penalty matters and may need adjustment. Our experience is that two or three adjustments, guided by balance diagnostics, can substantially improve covariate balance, perhaps requiring fifteen minutes effort sitting at the computer. We also explore the connection between directional penalties and a widely used technique in integer programming, namely Lagrangian relaxation of problematic linear side constraints in a minimum cost flow problem. In effect, many directional penalties are Lagrange multipliers, pushing a matched sample in the direction of satisfying a linear constraint that would not be satisfied without penalization. The method and example are in an R package DiPs at CRAN .     

Considerations for reporting finite element analysis studies in biomechanics

Simulation-based medicine and the development of complex computer models of biological structures is becoming ubiquitous for advancing biomedical engineering and clinical research. Finite element analysis (FEA) has been widely used in the last few decades to understand and predict biomechanical phenomena. Modeling and simulation approaches in biomechanics are highly interdisciplinary, involving novice and skilled developers in all areas of biomedical engineering and biology. While recent advances in model development and simulation platforms offer a wide range of tools to investigators, the decision making process during modeling and simulation has become more opaque. Hence, reliability of such models used for medical decision making and for driving multiscale analysis comes into question. Establishing guidelines for model development and dissemination is a daunting task, particularly with the complex and convoluted models used in FEA. Nonetheless, if better reporting can be established, researchers will have a better understanding of a model's value and the potential for reusability through sharing will be bolstered. Thus, the goal of this document is to identify resources and considerate reporting parameters for FEA studies in biomechanics. These entail various levels of reporting parameters for model identification, model structure, simulation structure, verification, validation, and availability. While we recognize that it may not be possible to provide and detail all of the reporting considerations presented, it is possible to establish a level of confidence with selective use of these parameters. More detailed reporting, however, can establish an explicit outline of the decision-making process in simulation-based analysis for enhanced reproducibility, reusability, and sharing.     

Quality control for plant metabolomics: reporting MSI-compliant studies

The Metabolomics Standards Initiative (MSI) has recently released documents describing minimum parameters for reporting metabolomics experiments, in order to validate metabolomic studies and to facilitate data exchange. The reporting parameters encompassed by MSI include the biological study design, sample preparation, data acquisition, data processing, data analysis and interpretation relative to the biological hypotheses being evaluated. Herein we exemplify how such metadata can be reported by using a small case study – the metabolite profiling by GC-TOF mass spectrometry of Arabidopsis thaliana leaves from a knockout allele of the gene At1g08510 in the Wassilewskija ecotype. Pitfalls in quality control are highlighted that can invalidate results even if MSI reporting standards are fulfilled, including reliable compound identification and integration of unknown metabolites. Standardized data processing methods are proposed for consistent data storage and dissemination via databases.     

Proposals to enhance the quality of observational cohort studies

For herbal medicinal products the methodology of observational cohort studies (observational studies, drug monitoring studies, Anwendungsbeobachtung) represents a useful addition to clinical trials. The key objectives are the documentation of efficacy in particular under conditions of everyday medical practice in authentic patients and the documentation of the safety profile. Supplementary to earlier activities, members of the "Clinical Trials of Herbal Medicinal Products" Working Party of the German Society for Phytotherapy have therefore again addressed the issue of observational cohort studies for to enhance the informative value and importance of this clinical research methodology. Recommendations were developed on quality aspects, methodological approaches of observation parameters, and for the reporting of the study's results. Properly planned and conducted observational cohort studies may contribute to the documentation and proof of well-established medicinal use according the EU Directive 2001/83/EC.     

Guidelines for measuring and reporting environmental conditions in controlled-environment studies

Guidelines for measuring and reporting environmental conditions in plant growth chambers are presented in tabular form. These guidelines are recommended by the North Central Region (NCR-101) Technical Committee on Growth Chamber Use, a committee formed under the Cooperative Research Program of the State Agricultural Experiment Stations (SAES) and the United States Department of Agriculture (USDA). Recommendations on location and frequency of measurements as well as suggested format and units of measurement are listed for each environmental parameter. The adoption of these standardized guidelines should greatly improve the uniformity of research conducted in controlled environments and facilitate comparison of experimental results obtained in studies conducted in different laboratories on a world-wide basis.     

Sensitivity analyses informed by tests for bias in observational studies

In an observational study, the treatment received and the outcome exhibited may be associated in the absence of an effect caused by the treatment, even after controlling for observed covariates. Two tactics are common: (i) a test for unmeasured bias may be obtained using a secondary outcome for which the effect is known and (ii) a sensitivity analysis may explore the magnitude of unmeasured bias that would need to be present to explain the observed association as something other than an effect caused by the treatment. Can such a test for unmeasured bias inform the sensitivity analysis? If the test for bias does not discover evidence of unmeasured bias, then ask: Are conclusions therefore insensitive to larger unmeasured biases? Conversely, if the test for bias does find evidence of bias, then ask: What does that imply about sensitivity to biases? This problem is formulated in a new way as a convex quadratically constrained quadratic program and solved on a large scale using interior point methods by a modern solver. That is, a convex quadratic function of N variables is minimized subject to constraints on linear and convex quadratic functions of these variables. The quadratic function that is minimized is a statistic for the primary outcome that is a function of the unknown treatment assignment probabilities. The quadratic function that constrains this minimization is a statistic for subsidiary outcome that is also a function of these same unknown treatment assignment probabilities. In effect, the first statistic is minimized over a confidence set for the unknown treatment assignment probabilities supplied by the unaffected outcome. This process avoids the mistake of interpreting the failure to reject a hypothesis as support for the truth of that hypothesis. The method is illustrated by a study of the effects of light daily alcohol consumption on high-density lipoprotein (HDL) cholesterol levels. In this study, the method quickly optimizes a nonlinear function of $N=800$ variables subject to linear and quadratic constraints. In the example, strong evidence of unmeasured bias is found using the subsidiary outcome, but, perhaps surprisingly, this finding makes the primary comparison insensitive to larger biases.