On the False Discovery Rate and Expected Type I Errors

The paper is concerned with expected type I errors of some stepwise multiple test procedures based on independent p‐values controlling the so‐called false discovery rate (FDR). We derive an asymptotic result for the supremum of the expected type I error rate(EER) when the number of hypotheses tends to infinity. Among others, it will be shown that when the original Benjamini‐Hochberg step‐up procedure controls the FDR at level α, its EER may approach a value being slightly larger than α/4 when the number of hypotheses increases. Moreover, we derive some least favourable parameter configuration results, some bounds for the FDR and the EER as well as easily computable formulae for the familywise error rate (FWER) of two FDR‐controlling procedures. Finally, we discuss some undesirable properties of the FDR concept, especially the problem of cheating.     

Statistically designing microarrays and microarray experiments to enhance sensitivity and specificity

Gene expression signatures from microarray experiments promise to provide important prognostic tools for predicting disease outcome or response to treatment. A number of microarray studies in various cancers have reported such gene signatures. However, the overlap of gene signatures in the same disease has been limited so far, and some reported signatures have not been reproduced in other populations. Clearly, the methods used for verifying novel gene signatures need improvement. In this article, we describe an experiment in which microarrays and sample hybridization are designed according to the statistical principles of randomization, replication and blocking. Our results show that such designs provide unbiased estimation of differential expression levels as well as powerful tests for them.     

Trimmed Weighted Simes' Test for Two One‐Sided Hypotheses With Arbitrarily Correlated Test Statistics

The two‐sided Simes test is known to control the type I error rate with bivariate normal test statistics. For one‐sided hypotheses, control of the type I error rate requires that the correlation between the bivariate normal test statistics is non‐negative. In this article, we introduce a trimmed version of the one‐sided weighted Simes test for two hypotheses which rejects if (i) the one‐sided weighted Simes test rejects and (ii) both p‐values are below one minus the respective weighted Bonferroni adjusted level. We show that the trimmed version controls the type I error rate at nominal significance level α if (i) the common distribution of test statistics is point symmetric and (ii) the two‐sided weighted Simes test at level 2α controls the level. These assumptions apply, for instance, to bivariate normal test statistics with arbitrary correlation. In a simulation study, we compare the power of the trimmed weighted Simes test with the power of the weighted Bonferroni test and the untrimmed weighted Simes test. An additional result of this article ensures type I error rate control of the usual weighted Simes test under a weak version of the positive regression dependence condition for the case of two hypotheses. This condition is shown to apply to the two‐sided p‐values of one‐ or two‐sample t‐tests for bivariate normal endpoints with arbitrary correlation and to the corresponding one‐sided p‐values if the correlation is non‐negative. The Simes test for such types of bivariate t‐tests has not been considered before. According to our main result, the trimmed version of the weighted Simes test then also applies to the one‐sided bivariate t‐test with arbitrary correlation.     

6-10× pyrosequencing is a practical approach for whole prokaryote genome studies

Next generation 454 pyrosequencing technology for whole bacterial genome sequencing involves a deep sequencing strategy with at least 15-20 × in depth proposed by official protocols but usually done with over 20 × in practices. In this study, we carried out a comprehensive evaluation of quality of the de novo assemblies based on realistic pyrosequencing simulated data from 1480 prokaryote genomes and 7 runs of machine-generated data. Our results demonstrated that for most of the prokaryote genomes, 6-10 × sequencing in qualified runs with 400 bp reads could produce high quality draft assembly (> 98% genome coverage, < 100 contigs with N50 size > 100 kb, single base accuracy > 99.99, indel error rate < 0.01%, false gene loss/duplication rate < 0.5%). Our study proves the power of low depth pyrosequencing strategy, which provides a cost-effective way for sequencing whole prokaryote genomes in a short time and enables further studies in microbial population diversity and comparative genomics.     

Multiple hypothesis testing to detect lineages under positive selection that affects only a few sites

Detection of positive Darwinian selection has become ever more important with the rapid growth of genomic data sets. Recent branch-site models of codon substitution account for variation of selective pressure over branches on the tree and across sites in the sequence and provide a means to detect short episodes of molecular adaptation affecting just a few sites. In likelihood ratio tests based on such models, the branches to be tested for positive selection have to be specified a priori. In the absence of a biological hypothesis to designate so-called foreground branches, one may test many branches, but a correction for multiple testing becomes necessary. In this paper, we employ computer simulation to evaluate the performance of 6 multiple test correction procedures when the branch-site models are used to test every branch on the phylogeny for positive selection. Four of the methods control the familywise error rates (FWERs), whereas the other 2 control the false discovery rate (FDR). We found that all correction procedures achieved acceptable FWER except for extremely divergent sequences and serious model violations, when the test may become unreliable. The power of the test to detect positive selection is influenced by the strength of selection and the sequence divergence, with the highest power observed at intermediate divergences. The 4 correction procedures that control the FWER had similar power. We recommend Rom's procedure for its slightly higher power, but the simple Bonferroni correction is useable as well. The 2 correction procedures that control the FDR had slightly more power and also higher FWER. We demonstrate the multiple test procedures by analyzing gene sequences from the extracellular domain of the cluster of differentiation 2 (CD2) gene from 10 mammalian species. Both our simulation and real data analysis suggest that the multiple test procedures are useful when multiple branches have to be tested on the same data set.     

Significance and importance: some common misapprehensions about statistics

There are many common misapprehensions about statistics that occur in the literature. We are sure that the three misapprehensions we deal with in this short review are widespread. They concern:

• 1) what P values mean;
• 2) what an insignificant result means, and what it does not mean; the question of the ‘power’ of a statistical test;
• 3) the difference between importance and statistical significance.

We produce no formulae or recipes for dealing with particular situations, instead we concentrate on the commonsense use of simple statistics. We emphasise that if the use of any but the simplest statistics is intended, it is much better to get proper statistical help before starting experiments, rather than afterwards. Copyright © 2009 John Wiley & Sons, Ltd.     

A Comparative Analysis of Graphical Interaction and Logistic Regression Modelling: Self‐care and Coping with a Chronic Illness in Later Life

Quantitative research especially in the social, but also in the biological sciences has been limited by the availability and applicability of analytic techniques that elaborate interactions among behaviours, treatment effects, and mediating variables. This gap has been filled by a newly developed statistical technique, known as graphical interaction modelling. The merit of graphical models for analyzing highly structured data is explored in this paper by an empirical study on coping with a chronic condition as a function of interrelationships between three sets of factors. These include background factors, illness context factors, and four self‐care practices. Based on a graphical chain model, the direct and indirect dependencies are revealed and discussed in comparison to the results obtained from a simple logistic regression model ignoring possible interaction effects. Both techniques are introduced from a more tutorial point of view instead of going far into technical details.