Microarray data quality analysis: lessons from the AFGC project

Genome-wide expression profiling with DNA microarrays has and will provide a great deal of data to the plant scientific community. However, reliability concerns have required the development data quality tests for common systematic biases. Fortunately, most large-scale systematic biases are detectable and some are correctable by normalization. Technical replication experiments and statistical surveys indicate that these biases vary widely in severity and appearance. As a result, no single normalization or correction method currently available is able to address all the issues. However, careful sequence selection, array design, experimental design and experimental annotation can substantially improve the quality and biological of microarray data. In this review, we discuss these issues with reference to examples from the Arabidopsis Functional Genomics Consortium (AFGC) microarray project.     

Detection and normalization of biases present in spotted cDNA microarray data: a composite method addressing dye, intensity-dependent, spatially-dependent, and print-order biases.

Microarrays are often used to identify target genes that trigger specific diseases, to elucidate the mechanisms of drug effects, and to check SNPs. However, data from microarray experiments are well known to contain biases resulting from the experimental protocols. Therefore, in order to elucidate biological knowledge from the data, systematic biases arising from their protocols must be removed prior to any data analysis. To remove these biases, many normalization methods are used by researchers. However, not all biases are eliminated from the microarray data because not all types of errors from experimental protocols are known. In this paper, we report an effective way of removing various types of biases by treating each microarray dataset independently to detect biases present in the dataset. After the biases contained in each dataset were identified, a combination of normalization methods specifically made for each dataset was applied to remove biases one at a time.     

Systematic variation normalization in microarray data to get gene expression comparison unbiased

Normalization removes or minimizes the biases of systematic variation that exists in experimental data sets. This study presents a systematic variation normalization (SVN) procedure for removing systematic variation in two channel microarray gene expression data. Based on an analysis of how systematic variation contributes to variability in microarray data sets, our normalization procedure includes background subtraction determined from the distribution of pixel intensity values from each data acquisition channel and log conversion, linear or non-linear regression, restoration or transformation, and multiarray normalization. In the case when a non-linear regression is required, an empirical polynomial approximation approach is used. Either the high terminated points or their averaged values in the distributions of the pixel intensity values observed in control channels may be used for rescaling multiarray datasets. These pre-processing steps remove systematic variation in the data attributable to variability in microarray slides, assay-batches, the array process, or experimenters. Biologically meaningful comparisons of gene expression patterns between control and test channels or among multiple arrays are therefore unbiased using normalized but not unnormalized datasets.     

Using Generalized Procrustes Analysis (GPA) for normalization of cDNA microarray data

Background  

Normalization is essential in dual-labelled microarray data analysis to remove non-biological variations and systematic biases. Many normalization methods have been used to remove such biases within slides (Global, Lowess) and across slides (Scale, Quantile and VSN). However, all these popular approaches have critical assumptions about data distribution, which is often not valid in practice.     

Normalization of boutique two-color microarrays with a high proportion of differentially expressed probes

Normalization is critical for removing systematic variation from microarray data. For two-color microarray platforms, intensity-dependent lowess normalization is commonly used to correct relative gene expression values for biases. Here we outline a normalization method for use when the assumptions of lowess normalization fail. Specifically, this can occur when specialized boutique arrays are constructed that contain a subset of genes selected to test particular biological functions.     

A robust neural networks approach for spatial and intensity-dependent normalization of cDNA microarray data

MOTIVATION: Microarray experiments are affected by numerous sources of non-biological variation that contribute systematic bias to the resulting data. In a dual-label (two-color) cDNA or long-oligonucleotide microarray, these systematic biases are often manifested as an imbalance of measured fluorescent intensities corresponding to Sample A versus those corresponding to Sample B. Systematic biases also affect between-slide comparisons. Making effective corrections for these systematic biases is a requisite for detecting the underlying biological variation between samples. Effective data normalization is therefore an essential step in the confident identification of biologically relevant differences in gene expression profiles. Several normalization methods for the correction of systemic bias have been described. While many of these methods have addressed intensity-dependent bias, few have addressed both intensity-dependent and spatiality-dependent bias. RESULTS: We present a neural network-based normalization method for correcting the intensity- and spatiality-dependent bias in cDNA microarray datasets. In this normalization method, the dependence of the log-intensity ratio (M) on the average log-intensity (A) as well as on the spatial coordinates (X,Y) of spots is approximated with a feed-forward neural network function. Resistance to outliers is provided by assigning weights to each spot based on how distant their M values is from the median over the spots whose A values are similar, as well as by using pseudospatial coordinates instead of spot row and column indices. A comparison of the robust neural network method with other published methods demonstrates its potential in reducing both intensity-dependent bias and spatial-dependent bias, which translates to more reliable identification of truly regulated genes.     

Evaluation of normalization methods for cDNA microarray data by <Emphasis Type="Italic">k</Emphasis>-NN classification

Background  

Non-biological factors give rise to unwanted variations in cDNA microarray data. There are many normalization methods designed to remove such variations. However, to date there have been few published systematic evaluations of these techniques for removing variations arising from dye biases in the context of downstream, higher-order analytical tasks such as classification.     

Adjustment of systematic microarray data biases

MOTIVATION: Systematic differences due to experimental features of microarray experiments are present in most large microarray data sets. Many different experimental features can cause biases including different sources of RNA, different production lots of microarrays or different microarray platforms. These systematic effects present a substantial hurdle to the analysis of microarray data. RESULTS: We present here a new method for the identification and adjustment of systematic biases that are present within microarray data sets. Our approach is based on modern statistical discrimination methods and is shown to be very effective in removing systematic biases present in a previously published breast tumor cDNA microarray data set. The new method of 'Distance Weighted Discrimination (DWD)' is shown to be better than Support Vector Machines and Singular Value Decomposition for the adjustment of systematic microarray effects. In addition, it is shown to be of general use as a tool for the discrimination of systematic problems present in microarray data sets, including the merging of two breast tumor data sets completed on different microarray platforms. AVAILABILITY: Matlab software to perform DWD can be retrieved from https://genome.unc.edu/pubsup/dwd/     

Normalization for cDNA microarray data: a robust composite method addressing single and multiple slide systematic variation

There are many sources of systematic variation in cDNA microarray experiments which affect the measured gene expression levels (e.g. differences in labeling efficiency between the two fluorescent dyes). The term normalization refers to the process of removing such variation. A constant adjustment is often used to force the distribution of the intensity log ratios to have a median of zero for each slide. However, such global normalization approaches are not adequate in situations where dye biases can depend on spot overall intensity and/or spatial location within the array. This article proposes normalization methods that are based on robust local regression and account for intensity and spatial dependence in dye biases for different types of cDNA microarray experiments. The selection of appropriate controls for normalization is discussed and a novel set of controls (microarray sample pool, MSP) is introduced to aid in intensity-dependent normalization. Lastly, to allow for comparisons of expression levels across slides, a robust method based on maximum likelihood estimation is proposed to adjust for scale differences among slides.     

Quantitative quality control in microarray experiments and the application in data filtering,normalization and false positive rate prediction

Data preprocessing including proper normalization and adequate quality control before complex data mining is crucial for studies using the cDNA microarray technology. We have developed a simple procedure that integrates data filtering and normalization with quantitative quality control of microarray experiments. Previously we have shown that data variability in a microarray experiment can be very well captured by a quality score q(com) that is defined for every spot, and the ratio distribution depends on q(com). Utilizing this knowledge, our data-filtering scheme allows the investigator to decide on the filtering stringency according to desired data variability, and our normalization procedure corrects the q(com)-dependent dye biases in terms of both the location and the spread of the ratio distribution. In addition, we propose a statistical model for false positive rate determination based on the design and the quality of a microarray experiment. The model predicts that a lower limit of 0.5 for the replicate concordance rate is needed in order to be certain of true positives. Our work demonstrates the importance and advantages of having a quantitative quality control scheme for microarrays.     

Normalization of two-channel microarray experiments: a semiparametric approach

MOTIVATION: An important underlying assumption of any experiment is that the experimental subjects are similar across levels of the treatment variable, so that changes in the response variable can be attributed to exposure to the treatment under study. This assumption is often not valid in the analysis of a microarray experiment due to systematic biases in the measured expression levels related to experimental factors such as spot location (often referred to as a print-tip effect), arrays, dyes, and various interactions of these effects. Thus, normalization is a critical initial step in the analysis of a microarray experiment, where the objective is to balance the individual signal intensity levels across the experimental factors, while maintaining the effect due to the treatment under investigation. RESULTS: Various normalization strategies have been developed including log-median centering, analysis of variance modeling, and local regression smoothing methods for removing linear and/or intensity-dependent systematic effects in two-channel microarray experiments. We describe a method that incorporates many of these into a single strategy, referred to as two-channel fastlo, and is derived from a normalization procedure that was developed for single-channel arrays. The proposed normalization procedure is applied to a two-channel dose-response experiment.     

Selection and validation of normalization methods for c-DNA microarrays using within-array replications

MOTIVATION: Normalization of microarray data is essential for multiple-array analyses. Several normalization protocols have been proposed based on different biological or statistical assumptions. A fundamental problem arises whether they have effectively normalized arrays. In addition, for a given array, the question arises how to choose a method to most effectively normalize the microarray data. RESULTS: We propose several techniques to compare the effectiveness of different normalization methods. We approach the problem by constructing statistics to test whether there are any systematic biases in the expression profiles among duplicated spots within an array. The test statistics involve estimating the genewise variances. This is accomplished by using several novel methods, including empirical Bayes methods for moderating the genewise variances and the smoothing methods for aggregating variance information. P-values are estimated based on a normal or chi approximation. With estimated P-values, we can choose a most appropriate method to normalize a specific array and assess the extent to which the systematic biases due to the variations of experimental conditions have been removed. The effectiveness and validity of the proposed methods are convincingly illustrated by a carefully designed simulation study. The method is further illustrated by an application to human placenta cDNAs comprising a large number of clones with replications, a customized microarray experiment carrying just a few hundred genes on the study of the molecular roles of Interferons on tumor, and the Agilent microarrays carrying tens of thousands of total RNA samples in the MAQC project on the study of reproducibility, sensitivity and specificity of the data. AVAILABILITY: Code to implement the method in the statistical package R is available from the authors.     

OLIN: optimized normalization, visualization and quality testing of two-channel microarray data

SUMMARY: Microarray data are generated in complex experiments and frequently compromised by a variety of systematic errors. Subsequent data normalization aims to correct these errors. Although several normalization methods have recently been proposed, they frequently fail to account for the variability of systematic errors within and between microarray experiments. However, optimal adjustment of normalization procedures to the underlying data structure is crucial for the efficiency of normalization. To overcome this restriction of current methods, we have developed two normalization schemes based on iterative local regression combined with model selection. The schemes have been demonstrated to improve considerably the quality of normalization. They are implemented in a freely available R package. Additionally, functions for visualization and detection of systematic errors in microarray data have been incorporated in the software package. A graphical user interface is also available. AVAILABILITY: The R package can be downloaded from http://itb.biologie.hu-berlin.de/~futschik/software/R/OLIN. It underlies the GPL version 2. CONTACT: m.futschik@biologie.hu-berlin.de SUPPLEMENTARY INFORMATION: Further information about the methods used in the OLIN software package can be found at http://itb.biologie.hu-berlin.de/~futschik/software/R/OLIN.     

Amplification protocols introduce systematic but reproducible errors into gene expression studies

The desire to perform microarray experiments with small starting amounts of RNA has led to the development of a variety of protocols for preparing and amplifying mRNA. This has consequences not only for the standardization of experimental design, but also for reproducibility and comparability between experiments. Here we investigate the differences between the Affymetrix standard and small sample protocols and address the data analysis issues that arise when comparing samples and experiments that have been processed in different ways. We show that data generated on the same platform using different protocols are not directly comparable. Further, protocols introduce systematic biases that can be largely accounted for by using the correct data analysis techniques.     

基因芯片表达谱数据的预处理分析

基因芯片数据的预处理是一个十分关键的步骤,通过数据过滤获取需要的数据、数据转换满足正态分布的分析要求、缺失值的估计弥补不完整的数据、数据归一化纠正系统误差等处理为后续分析工作做准备,预处理分析的重要性并不亚于基因芯片的后续分析,它将直接影响后续分析是否能得到预期的结果.本文重点综述了cDNA芯片的数据预处理,简要地概述寡核苷酸芯片的数据预处理.     

ArrayNorm: comprehensive normalization and analysis of microarray data

SUMMARY: ArrayNorm is a user-friendly, versatile and platform-independent Java application for the visualization, normalization and analysis of two-color microarray data. A variety of normalization options were implemented to remove the systematic and random errors in the data, taking into account the experimental design and the particularities of every slide. In addition, ArrayNorm provides a module for statistical identification of genes with significant changes in expression. AVAILABILITY: The package is freely available for academic and non-profit institutions from http://genome.tugraz.at     

A stepwise framework for the normalization of array CGH data

Background  

In two-channel competitive genomic hybridization microarray experiments, the ratio of the two fluorescent signal intensities at each spot on the microarray is commonly used to infer the relative amounts of the test and reference sample DNA levels. This ratio may be influenced by systematic measurement effects from non-biological sources that can introduce biases in the estimated ratios. These biases should be removed before drawing conclusions about the relative levels of DNA. The performance of existing gene expression microarray normalization strategies has not been evaluated for removing systematic biases encountered in array-based comparative genomic hybridization (CGH), which aims to detect single copy gains and losses typically in samples with heterogeneous cell populations resulting in only slight shifts in signal ratios. The purpose of this work is to establish a framework for correcting the systematic sources of variation in high density CGH array images, while maintaining the true biological variations.     

Comparison of algorithms for the analysis of Affymetrix microarray data as evaluated by co-expression of genes in known operons

Oligonucleotide microarrays are an informative tool to elucidate gene regulatory networks. In order for gene expression levels to be comparable across microarrays, normalization procedures have to be invoked. A large number of methods have been described to correct for systematic biases in microarray experiments. The performance of these methods has been tested only to a limited extend. Here, we evaluate two different types of microarray analyses: (i) the same gene in replicate samples and (ii) different, but co-expressed genes in the same sample. The reliability of the latter analysis needs to be determined for the analysis of regulatory networks and our report is the first attempt to evaluate for the accuracy of different microarray normalization methods in this respect. Consistent with previous results we observed a large effect of the normalization method on the outcome of the expression analyses. Our analyses indicate that different normalization methods should be performed depending on whether a study is aiming to detect differential gene expression between independent samples or whether co-expressed genes should be identified. We make recommendations about the most appropriate method to use.     

Two-stage normalization using background intensities in cDNA microarray data

Background  

In the microarray experiment, many undesirable systematic variations are commonly observed. Normalization is the process of removing such variation that affects the measured gene expression levels. Normalization plays an important role in the earlier stage of microarray data analysis. The subsequent analysis results are highly dependent on normalization. One major source of variation is the background intensities. Recently, some methods have been employed for correcting the background intensities. However, all these methods focus on defining signal intensities appropriately from foreground and background intensities in the image analysis. Although a number of normalization methods have been proposed, no systematic methods have been proposed using the background intensities in the normalization process.