The Benjamini-Hochberg method with infinitely many contrasts in linear models

Benjamini and Hochberg's method for controlling the false discoveryrate is applied to the problem of testing infinitely many contrastsin linear models. Exact, easily calculated critical values arederived, defining a new multiple comparisons method for testingcontrasts in linear models. The method is adaptive, dependingon the data through the F-statistic, like the Waller–DuncanBayesian multiple comparisons method. Comparisons with Scheffé'smethod are given, and the method is extended to the simultaneousconfidence intervals of Benjamini and Yekutieli.     

Estimating the false discovery rate using the stochastic approximation algorithm

Testing of multiple hypotheses involves statistics that arestrongly dependent in some applications, but most work on thissubject is based on the assumption of independence. We proposea new method for estimating the false discovery rate of multiplehypothesis tests, in which the density of test scores is estimatedparametrically by minimizing the Kullback–Leibler distancebetween the unknown density and its estimator using the stochasticapproximation algorithm, and the false discovery rate is estimatedusing the ensemble averaging method. Our method is applicableunder general dependence between test statistics. Numericalcomparisons between our method and several competitors, conductedon simulated and real data examples, show that our method achievesmore accurate control of the false discovery rate in almostall scenarios.     

Genomic outlier profile analysis: mixture models, null hypotheses, and nonparametric estimation

In most analyses of large-scale genomic data sets, differentialexpression analysis is typically assessed by testing for differencesin the mean of the distributions between 2 groups. A recentfinding by Tomlins and others (2005) is of a different typeof pattern of differential expression in which a fraction ofsamples in one group have overexpression relative to samplesin the other group. In this work, we describe a general mixturemodel framework for the assessment of this type of expression,called outlier profile analysis. We start by considering thesingle-gene situation and establishing results on identifiability.We propose 2 nonparametric estimation procedures that have naturallinks to familiar multiple testing procedures. We then developmultivariate extensions of this methodology to handle genome-widemeasurements. The proposed methodologies are compared usingsimulation studies as well as data from a prostate cancer geneexpression study.     

Work efficiency: a new criterion for comprehensive comparison and evaluation of statistical methods in large-scale identification of differentially expressed genes

Receiver operating characteristic (ROC) has been widely used to evaluate statistical methods, but a fatal problem is that ROC cannot evaluate estimation of the false discovery rate (FDR) of a statistical method and hence the area under of curve as a criterion cannot tell us if a statistical method is conservative. To address this issue, we propose an alternative criterion, work efficiency. Work efficiency is defined as the product of the power and degree of conservativeness of a statistical method. We conducted large-scale simulation comparisons among the optimizing discovery procedure (ODP), the Bonferroni (B-) procedure, Local FDR (Localfdr), ranking analysis of the F-statistics (RAF), the Benjamini-Hochberg (BH-) procedure, and significance analysis of microarray data (SAM). The results show that ODP, SAM, and the B-procedure perform with low efficiencies while the BH-procedure, RAF, and Localfdr work with higher efficiency. ODP and SAM have the same ROC curves but their efficiencies are significantly different.     

Ranking analysis of microarray data: a powerful method for identifying differentially expressed genes

Microarray technology provides a powerful tool for the expression profile of thousands of genes simultaneously, which makes it possible to explore the molecular and metabolic etiology of the development of a complex disease under study. However, classical statistical methods and technologies fail to be applicable to microarray data. Therefore, it is necessary and motivating to develop powerful methods for large-scale statistical analyses. In this paper, we described a novel method, called Ranking Analysis of Microarray Data (RAM). RAM, which is a large-scale two-sample t-test method, is based on comparisons between a set of ranked T statistics and a set of ranked Z values (a set of ranked estimated null scores) yielded by a "randomly splitting" approach instead of a "permutation" approach and a two-simulation strategy for estimating the proportion of genes identified by chance, i.e., the false discovery rate (FDR). The results obtained from the simulated and observed microarray data show that RAM is more efficient in identification of genes differentially expressed and estimation of FDR under undesirable conditions such as a large fudge factor, small sample size, or mixture distribution of noises than Significance Analysis of Microarrays.     

Wavelet thresholding with bayesian false discovery rate control

The false discovery rate (FDR) procedure has become a popular method for handling multiplicity in high-dimensional data. The definition of FDR has a natural Bayesian interpretation; it is the expected proportion of null hypotheses mistakenly rejected given a measure of evidence for their truth. In this article, we propose controlling the positive FDR using a Bayesian approach where the rejection rule is based on the posterior probabilities of the null hypotheses. Correspondence between Bayesian and frequentist measures of evidence in hypothesis testing has been studied in several contexts. Here we extend the comparison to multiple testing with control of the FDR and illustrate the procedure with an application to wavelet thresholding. The problem consists of recovering signal from noisy measurements. This involves extracting wavelet coefficients that result from true signal and can be formulated as a multiple hypotheses-testing problem. We use simulated examples to compare the performance of our approach to the Benjamini and Hochberg (1995, Journal of the Royal Statistical Society, Series B57, 289-300) procedure. We also illustrate the method with nuclear magnetic resonance spectral data from human brain.     

The Use of Two‐Way Linear Mixed Models in Multitreatment Meta‐Analysis

Summary Meta‐analysis summarizes the results of a series of trials. When more than two treatments are included in the trials and when the set of treatments tested differs between trials, the combination of results across trials requires some care. Several methods have been proposed for this purpose, which feature under different labels, such as network meta‐analysis or mixed treatment comparisons. Two types of linear mixed model can be used for meta‐analysis. The one expresses the expected outcome of treatments as a contrast to a baseline treatment. The other uses a classical two‐way linear predictor with main effects for treatment and trial. In this article, we compare both types of model and explore under which conditions they give equivalent results. We illustrate practical advantages of the two‐way model using two published datasets. In particular, it is shown that between‐trial heterogeneity as well as inconsistency between different types of trial is straightforward to account for.