Statistical strategies for avoiding false discoveries in metabolomics and related experiments |
| |
Authors: | David I Broadhurst Douglas B Kell |
| |
Institution: | (1) School of Chemistry, The University of Manchester, Faraday Building, Sackville St, Manchester, M60 1QD, UK;(2) Manchester Centre for Integrative Systems Biology, The Manchester Interdisciplinary Biocentre, The University of Manchester, 131 Princess St, Manchester, M1 7DN, UK |
| |
Abstract: | Many metabolomics, and other high-content or high-throughput, experiments are set up such that the primary aim is the discovery
of biomarker metabolites that can discriminate, with a certain level of certainty, between nominally matched ‘case’ and ‘control’
samples. However, it is unfortunately very easy to find markers that are apparently persuasive but that are in fact entirely
spurious, and there are well-known examples in the proteomics literature. The main types of danger are not entirely independent
of each other, but include bias, inadequate sample size (especially relative to the number of metabolite variables and to
the required statistical power to prove that a biomarker is discriminant), excessive false discovery rate due to multiple
hypothesis testing, inappropriate choice of particular numerical methods, and overfitting (generally caused by the failure
to perform adequate validation and cross-validation). Many studies fail to take these into account, and thereby fail to discover
anything of true significance (despite their claims). We summarise these problems, and provide pointers to a substantial existing
literature that should assist in the improved design and evaluation of metabolomics experiments, thereby allowing robust scientific
conclusions to be drawn from the available data. We provide a list of some of the simpler checks that might improve one’s
confidence that a candidate biomarker is not simply a statistical artefact, and suggest a series of preferred tests and visualisation
tools that can assist readers and authors in assessing papers. These tools can be applied to individual metabolites by using
multiple univariate tests performed in parallel across all metabolite peaks. They may also be applied to the validation of
multivariate models. We stress in particular that classical p-values such as “p < 0.05”, that are often used in biomedicine, are far too optimistic when multiple tests are done simultaneously (as in metabolomics).
Ultimately it is desirable that all data and metadata are available electronically, as this allows the entire community to
assess conclusions drawn from them. These analyses apply to all high-dimensional ‘omics’ datasets. |
| |
Keywords: | statistics machine learning false discovery receiver– operator characteristic hypothesis testing statistical power Bonferroni correction bias overfitting cross validiation credit assignment visualisation |
本文献已被 SpringerLink 等数据库收录! |
|