Local-pooled-error test for identifying differentially expressed genes with a small number of replicated microarrays

MOTIVATION: In microarray studies gene discovery based on fold-change values is often misleading because error variability for each gene is heterogeneous under different biological conditions and intensity ranges. Several statistical testing methods for differential gene expression have been suggested, but some of these approaches are underpowered and result in high false positive rates because within-gene variance estimates are based on a small number of replicated arrays. RESULTS: We propose to use local-pooled-error (LPE) estimates and robust statistical tests for evaluating significance of each gene's differential expression. Our LPE estimation is based on pooling errors within genes and between replicate arrays for genes in which expression values are similar. We have applied our LPE method to compare gene expression in na?ve and activated CD8+ T-cells. Our results show that the LPE method effectively identifies significant differential-expression patterns with a small number of replicated arrays. AVAILABILITY: The methodology is implemented with S-PLUS and R functions available at http://hesweb1.med.virginia.edu/bioinformatics     

Rosetta error model for gene expression analysis

MOTIVATION: In microarray gene expression studies, the number of replicated microarrays is usually small because of cost and sample availability, resulting in unreliable variance estimation and thus unreliable statistical hypothesis tests. The unreliable variance estimation is further complicated by the fact that the technology-specific variance is intrinsically intensity-dependent. RESULTS: The Rosetta error model captures the variance-intensity relationship for various types of microarray technologies, such as single-color arrays and two-color arrays. This error model conservatively estimates intensity error and uses this value to stabilize the variance estimation. We present two commonly used error models: the intensity error-model for single-color microarrays and the ratio error model for two-color microarrays or ratios built from two single-color arrays. We present examples to demonstrate the strength of our error models in improving statistical power of microarray data analysis, particularly, in increasing expression detection sensitivity and specificity when the number of replicates is limited.     

Some comments on instability of false discovery rate estimation

Some extended false discovery rate (FDR) controlling multiple testing procedures rely heavily on empirical estimates of the FDR constructed from gene expression data. Such estimates are also used as performance indicators when comparing different methods for microarray data analysis. The present communication shows that the variance of the proposed estimators may be intolerably high, the correlation structure of microarray data being the main cause of their instability.     

Analyzing population growth curves

Assessing animal population growth curves is an essential feature of field studies in ecology and wildlife management. We used five models to assess population growth rates with a number of sets of population growth rate data. A 'generalized' logistic curve provides a better model than do four other popular models. Use of difference equations for fitting was checked by a comparison of that method and direct fitting of the analytical (integrated) solution for three of the models. Fits to field data indicate that estimates of the asymptote, K, from the 'generalized logistic' and the ordinary logistic agree well enough to support use of estimates of K from the ordinary logistic on data that cannot be satisfactorily fitted with the generalized logistic. Akaike's information criterion is widely used, often with a small sample version AIC_c. Our study of five models indicated a bias in the AIC_c criterion, so we recommend checking results with estimates of variance about regression for fitted models. Fitting growth curves provides a valuable supplement to, and check on computer models of populations.     

Multidimensional local false discovery rate for microarray studies

MOTIVATION: The false discovery rate (fdr) is a key tool for statistical assessment of differential expression (DE) in microarray studies. Overall control of the fdr alone, however, is not sufficient to address the problem of genes with small variance, which generally suffer from a disproportionally high rate of false positives. It is desirable to have an fdr-controlling procedure that automatically accounts for gene variability. METHODS: We generalize the local fdr as a function of multiple statistics, combining a common test statistic for assessing DE with its standard error information. We use a non-parametric mixture model for DE and non-DE genes to describe the observed multi-dimensional statistics, and estimate the distribution for non-DE genes via the permutation method. We demonstrate this fdr2d approach for simulated and real microarray data. RESULTS: The fdr2d allows objective assessment of DE as a function of gene variability. We also show that the fdr2d performs better than commonly used modified test statistics. AVAILABILITY: An R-package OCplus containing functions for computing fdr2d() and other operating characteristics of microarray data is available at http://www.meb.ki.se/~yudpaw.     

Generalized method for probability-based peptide and protein identification from tandem mass spectrometry data and sequence database searching

Tandem mass spectrometry-based proteomics is currently in great demand of computational methods that facilitate the elimination of likely false positives in peptide and protein identification. In the last few years, a number of new peptide identification programs have been described, but scores or other significance measures reported by these programs cannot always be directly translated into an easy to interpret error rate measurement such as the false discovery rate. In this work we used generalized lambda distributions to model frequency distributions of database search scores computed by MASCOT, X!TANDEM with k-score plug-in, OMSSA, and InsPecT. From these distributions, we could successfully estimate p values and false discovery rates with high accuracy. From the set of peptide assignments reported by any of these engines, we also defined a generic protein scoring scheme that enabled accurate estimation of protein-level p values by simulation of random score distributions that was also found to yield good estimates of protein-level false discovery rate. The performance of these methods was evaluated by searching four freely available data sets ranging from 40,000 to 285,000 MS/MS spectra.     

Comparing three methods for variance estimation with duplicated high density oligonucleotide arrays

Microarray experiments are being increasingly used in molecular biology. A common task is to detect genes with differential expression across two experimental conditions, such as two different tissues or the same tissue at two time points of biological development. To take proper account of statistical variability, some statistical approaches based on the t-statistic have been proposed. In constructing the t-statistic, one needs to estimate the variance of gene expression levels. With a small number of replicated array experiments, the variance estimation can be challenging. For instance, although the sample variance is unbiased, it may have large variability, leading to a large mean squared error. For duplicated array experiments, a new approach based on simple averaging has recently been proposed in the literature. Here we consider two more general approaches based on nonparametric smoothing. Our goal is to assess the performance of each method empirically. The three methods are applied to a colon cancer data set containing 2,000 genes. Using two arrays, we compare the variance estimates obtained from the three methods. We also consider their impact on the t-statistics. Our results indicate that the three methods give variance estimates close to each other. Due to its simplicity and generality, we recommend the use of the smoothed sample variance for data with a small number of replicates. Electronic Publication     

A random variance model for detection of differential gene expression in small microarray experiments

MOTIVATION: Microarray techniques provide a valuable way of characterizing the molecular nature of disease. Unfortunately expense and limited specimen availability often lead to studies with small sample sizes. This makes accurate estimation of variability difficult, since variance estimates made on a gene by gene basis will have few degrees of freedom, and the assumption that all genes share equal variance is unlikely to be true. RESULTS: We propose a model by which the within gene variances are drawn from an inverse gamma distribution, whose parameters are estimated across all genes. This results in a test statistic that is a minor variation of those used in standard linear models. We demonstrate that the model assumptions are valid on experimental data, and that the model has more power than standard tests to pick up large changes in expression, while not increasing the rate of false positives. AVAILABILITY: This method is incorporated into BRB-ArrayTools version 3.0 (http://linus.nci.nih.gov/BRB-ArrayTools.html). SUPPLEMENTARY MATERIAL: ftp://linus.nci.nih.gov/pub/techreport/RVM_supplement.pdf     

Variance Component Estimation for Mixed Model Analysis of cDNA Microarray Data

Microarrays provide a valuable tool for the quantification of gene expression. Usually, however, there is a limited number of replicates leading to unsatisfying variance estimates in a gene‐wise mixed model analysis. As thousands of genes are available, it is desirable to combine information across genes. When more than two tissue types or treatments are to be compared it might be advisable to consider the array effect as random. Then information between arrays may be recovered, which can increase accuracy in estimation. We propose a method of variance component estimation across genes for a linear mixed model with two random effects. The method may be extended to models with more than two random effects. We assume that the variance components follow a log‐normal distribution. Assuming that the sums of squares from the gene‐wise analysis, given the true variance components, follow a scaled χ²‐distribution, we adopt an empirical Bayes approach. The variance components are estimated by the expectation of their posterior distribution. The new method is evaluated in a simulation study. Differentially expressed genes are more likely to be detected by tests based on these variance estimates than by tests based on gene‐wise variance estimates. This effect is most visible in studies with small array numbers. Analyzing a real data set on maize endosperm the method is shown to work well. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Surrogate variable analysis using partial least squares (SVA-PLS) in gene expression studies

MOTIVATION: In a typical gene expression profiling study, our prime objective is to identify the genes that are differentially expressed between the samples from two different tissue types. Commonly, standard analysis of variance (ANOVA)/regression is implemented to identify the relative effects of these genes over the two types of samples from their respective arrays of expression levels. But, this technique becomes fundamentally flawed when there are unaccounted sources of variability in these arrays (latent variables attributable to different biological, environmental or other factors relevant in the context). These factors distort the true picture of differential gene expression between the two tissue types and introduce spurious signals of expression heterogeneity. As a result, many genes which are actually differentially expressed are not detected, whereas many others are falsely identified as positives. Moreover, these distortions can be different for different genes. Thus, it is also not possible to get rid of these variations by simple array normalizations. This both-way error can lead to a serious loss in sensitivity and specificity, thereby causing a severe inefficiency in the underlying multiple testing problem. In this work, we attempt to identify the hidden effects of the underlying latent factors in a gene expression profiling study by partial least squares (PLS) and apply ANCOVA technique with the PLS-identified signatures of these hidden effects as covariates, in order to identify the genes that are truly differentially expressed between the two concerned tissue types. RESULTS: We compare the performance of our method SVA-PLS with standard ANOVA and a relatively recent technique of surrogate variable analysis (SVA), on a wide variety of simulation settings (incorporating different effects of the hidden variable, under situations with varying signal intensities and gene groupings). In all settings, our method yields the highest sensitivity while maintaining relatively reasonable values for the specificity, false discovery rate and false non-discovery rate. Application of our method to gene expression profiling for acute megakaryoblastic leukemia shows that our method detects an additional six genes, that are missed by both the standard ANOVA method as well as SVA, but may be relevant to this disease, as can be seen from mining the existing literature.     

Simultaneous Discovery,Estimation and Prediction Analysis of Complex Traits Using a Bayesian Mixture Model

Gene discovery, estimation of heritability captured by SNP arrays, inference on genetic architecture and prediction analyses of complex traits are usually performed using different statistical models and methods, leading to inefficiency and loss of power. Here we use a Bayesian mixture model that simultaneously allows variant discovery, estimation of genetic variance explained by all variants and prediction of unobserved phenotypes in new samples. We apply the method to simulated data of quantitative traits and Welcome Trust Case Control Consortium (WTCCC) data on disease and show that it provides accurate estimates of SNP-based heritability, produces unbiased estimators of risk in new samples, and that it can estimate genetic architecture by partitioning variation across hundreds to thousands of SNPs. We estimated that, depending on the trait, 2,633 to 9,411 SNPs explain all of the SNP-based heritability in the WTCCC diseases. The majority of those SNPs (>96%) had small effects, confirming a substantial polygenic component to common diseases. The proportion of the SNP-based variance explained by large effects (each SNP explaining 1% of the variance) varied markedly between diseases, ranging from almost zero for bipolar disorder to 72% for type 1 diabetes. Prediction analyses demonstrate that for diseases with major loci, such as type 1 diabetes and rheumatoid arthritis, Bayesian methods outperform profile scoring or mixed model approaches.     

Extended Bayesian information criteria for model selection with large model spaces

The ordinary Bayesian information criterion is too liberal formodel selection when the model space is large. In this paper,we re-examine the Bayesian paradigm for model selection andpropose an extended family of Bayesian information criteria,which take into account both the number of unknown parametersand the complexity of the model space. Their consistency isestablished, in particular allowing the number of covariatesto increase to infinity with the sample size. Their performancein various situations is evaluated by simulation studies. Itis demonstrated that the extended Bayesian information criteriaincur a small loss in the positive selection rate but tightlycontrol the false discovery rate, a desirable property in manyapplications. The extended Bayesian information criteria areextremely useful for variable selection in problems with a moderatesample size but with a huge number of covariates, especiallyin genome-wide association studies, which are now an activearea in genetics research.     

Calibrating the performance of SNP arrays for whole-genome association studies

To facilitate whole-genome association studies (WGAS), several high-density SNP genotyping arrays have been developed. Genetic coverage and statistical power are the primary benchmark metrics in evaluating the performance of SNP arrays. Ideally, such evaluations would be done on a SNP set and a cohort of individuals that are both independently sampled from the original SNPs and individuals used in developing the arrays. Without utilization of an independent test set, previous estimates of genetic coverage and statistical power may be subject to an overfitting bias. Additionally, the SNP arrays' statistical power in WGAS has not been systematically assessed on real traits. One robust setting for doing so is to evaluate statistical power on thousands of traits measured from a single set of individuals. In this study, 359 newly sampled Americans of European descent were genotyped using both Affymetrix 500K (Affx500K) and Illumina 650Y (Ilmn650K) SNP arrays. From these data, we were able to obtain estimates of genetic coverage, which are robust to overfitting, by constructing an independent test set from among these genotypes and individuals. Furthermore, we collected liver tissue RNA from the participants and profiled these samples on a comprehensive gene expression microarray. The RNA levels were used as a large-scale set of quantitative traits to calibrate the relative statistical power of the commercial arrays. Our genetic coverage estimates are lower than previous reports, providing evidence that previous estimates may be inflated due to overfitting. The Ilmn650K platform showed reasonable power (50% or greater) to detect SNPs associated with quantitative traits when the signal-to-noise ratio (SNR) is greater than or equal to 0.5 and the causal SNP's minor allele frequency (MAF) is greater than or equal to 20% (N=359). In testing each of the more than 40,000 gene expression traits for association to each of the SNPs on the Ilmn650K and Affx500K arrays, we found that the Ilmn650K yielded 15% times more discoveries than the Affx500K at the same false discovery rate (FDR) level.     

Mathematical algorithm for discovering states of expression from direct genetic comparison by microarrays

Highly specific direct genome-scale expression discovery from two biological samples facilitates functional discovery of molecular systems. Here, expression data from cDNA arrays are ranked and curve-fitted. The algorithm uses filters based on the derivatives (slopes) of the curve fits. The rules are set to (i) filter the largest number of artifactual ratios from same-to-same datasets and (ii) maximize discovery from direct comparisons of different samples. The unsupervised discovery is optimized without lowering specificity. The false discovery rates are significantly lower than other methods. The discovered states of genetic expression facilitate functional discovery and are validated by real-time RT–PCR. Better quality improves sensitivity.     

NuGO PPS1 mouse study 1: preliminary statistical analysis

Time course experiments are aimed at characterizing the dynamic regulation of gene expression in biological systems. Data are collected at different time points to monitor the dynamic behaviour of gene expression. The NuGO PPS Mouse Study 1 investigates the development of high fat-induced insulin resistance (IR) over time in APOE*3Leiden (E3L) mice. The study consists in a series of analyses at time points, which are crucial in the development of central and peripheral IR. Affymetrix arrays have been made on critical organs. We present the results of the preliminary statistical analysis on these microarray data. We used a non-parametric approach to identify genes the expression of which changed over time, separately for three tissues: liver, muscle and white adipose tissue. We specified for each gene a basic ANOVA model, in order to check the null hypothesis that gene expression did not vary over time. We addressed the multiple tests problem calculating positive false discovery rate and q values for the F test statistics. The appropriateness of the hypothesis of homogeneous variances over time was investigated by mean of the Bartlett’s test for homoschedasticity. This is a relevant point because heteroschedasticity could be indicative of outlying behaviour of some individuals at specific time points. The necessity to use a moderated F test was evaluated. We found that a considerable part of the genes varied expression over time. For part of the genes, the variance of the response was not homogeneous over time. Response differed by tissue.     

Linear Mixed Model Selection for False Discovery Rate Control in Microarray Data Analysis

Summary In a microarray experiment, one experimental design is used to obtain expression measures for all genes. One popular analysis method involves fitting the same linear mixed model for each gene, obtaining gene‐specific p‐values for tests of interest involving fixed effects, and then choosing a threshold for significance that is intended to control false discovery rate (FDR) at a desired level. When one or more random factors have zero variance components for some genes, the standard practice of fitting the same full linear mixed model for all genes can result in failure to control FDR. We propose a new method that combines results from the fit of full and selected linear mixed models to identify differentially expressed genes and provide FDR control at target levels when the true underlying random effects structure varies across genes.     

Statistical methods for analyzing microarray feature data with replications.

Expression levels in oligonucleotide microarray experiments depend on a potentially large number of factors, for example, treatment conditions, different probes, different arrays, and so on. To dissect the effects of these factors on expression levels, fixed-effects ANOVA methods have previously been proposed. Because we are not necessarily interested in estimating the specific effects of different probes and arrays, we propose to treat these as random effects. Then we only need to estimate their means and variances but not the effect of each of their levels; that is, we can work with a much reduced number of parameters and, consequently, higher precision for estimating expression levels. Thus, we developed a mixed-effects ANOVA model with some random and some fixed effects. It automatically accounts for local normalization between different arrays and for background correction. The method was applied to each of the 6,584 genes investigated in a microarray experiment on two mouse cell lines, PA6/S and PA6/8, where PA6/S enhances proliferation of Pre B cells in vitro but PA6/8 does not. To detect a set of differentially expressed genes (multiple testing problem), we applied the method of controlling the false discovery rate (FDR), which successfully identified 207 genes with significantly different expression levels.     

Effects of filtering by Present call on analysis of microarray experiments

Background

Affymetrix GeneChips^® are widely used for expression profiling of tens of thousands of genes. The large number of comparisons can lead to false positives. Various methods have been used to reduce false positives, but they have rarely been compared or quantitatively evaluated. Here we describe and evaluate a simple method that uses the detection (Present/Absent) call generated by the Affymetrix microarray suite version 5 software (MAS5) to remove data that is not reliably detected before further analysis, and compare this with filtering by expression level. We explore the effects of various thresholds for removing data in experiments of different size (from 3 to 10 arrays per treatment), as well as their relative power to detect significant differences in expression.

Results

Our approach sets a threshold for the fraction of arrays called Present in at least one treatment group. This method removes a large percentage of probe sets called Absent before carrying out the comparisons, while retaining most of the probe sets called Present. It preferentially retains the more significant probe sets (p ≤ 0.001) and those probe sets that are turned on or off, and improves the false discovery rate. Permutations to estimate false positives indicate that probe sets removed by the filter contribute a disproportionate number of false positives. Filtering by fraction Present is effective when applied to data generated either by the MAS5 algorithm or by other probe-level algorithms, for example RMA (robust multichip average). Experiment size greatly affects the ability to reproducibly detect significant differences, and also impacts the effect of filtering; smaller experiments (3–5 samples per treatment group) benefit from more restrictive filtering (≥50% Present).

Conclusion

Use of a threshold fraction of Present detection calls (derived by MAS5) provided a simple method that effectively eliminated from analysis probe sets that are unlikely to be reliable while preserving the most significant probe sets and those turned on or off; it thereby increased the ratio of true positives to false positives.