Hit selection with false discovery rate control in genome-scale RNAi screens

RNA interference (RNAi) is a modality in which small double-stranded RNA molecules (siRNAs) designed to lead to the degradation of specific mRNAs are introduced into cells or organisms. siRNA libraries have been developed in which siRNAs targeting virtually every gene in the human genome are designed, synthesized and are presented for introduction into cells by transfection in a microtiter plate array. These siRNAs can then be transfected into cells using high-throughput screening (HTS) methodologies. The goal of RNAi HTS is to identify a set of siRNAs that inhibit or activate defined cellular phenotypes. The commonly used analysis methods including median ± kMAD have issues about error rates in multiple hypothesis testing and plate-wise versus experiment-wise analysis. We propose a methodology based on a Bayesian framework to address these issues. Our approach allows for sharing of information across plates in a plate-wise analysis, which obviates the need for choosing either a plate-wise or experimental-wise analysis. The proposed approach incorporates information from reliable controls to achieve a higher power and a balance between the contribution from the samples and control wells. Our approach provides false discovery rate (FDR) control to address multiple testing issues and it is robust to outliers.     

Median absolute deviation to improve hit selection for genome-scale RNAi screens

High-throughput screening (HTS) of large-scale RNA interference (RNAi) libraries has become an increasingly popular method of functional genomics in recent years. Cell-based assays used for RNAi screening often produce small dynamic ranges and significant variability because of the combination of cellular heterogeneity, transfection efficiency, and the intrinsic nature of the genes being targeted. These properties make reliable hit selection in the RNAi screen a difficult task. The use of robust methods based on median and median absolute deviation (MAD) has been suggested to improve hit selection in such cases, but mean and standard deviation (SD)-based methods are still predominantly used in many RNAi HTS. In an experimental approach to compare these 2 methods, a genome-scale small interfering RNA (siRNA) screen was performed, in which the identification of novel targets increasing the therapeutic index of the chemotherapeutic agent mitomycin C (MMC) was sought. MAD values were resistant to the presence of outliers, and the hits selected by the MAD-based method included all the hits that would be selected by SD-based method as well as a significant number of additional hits. When retested in triplicate, a similar percentage of these siRNAs were shown to genuinely sensitize cells to MMC compared with the hits shared between SD- and MAD-based methods. Confirmed hits were enriched with the genes involved in the DNA damage response and cell cycle regulation, validating the overall hit selection strategy. Finally, computer simulations showed the superiority and generality of the MAD-based method in various RNAi HTS data models. In conclusion, the authors demonstrate that the MAD-based hit selection method rescued physiologically relevant false negatives that would have been missed in the SD-based method, and they believe it to be the desirable 1st-choice hit selection method for RNAi screen results.     

A genome-scale shRNA resource for transgenic RNAi in Drosophila

Existing transgenic RNAi resources in Drosophila melanogaster based on long double-stranded hairpin RNAs are powerful tools for functional studies, but they are ineffective in gene knockdown during oogenesis, an important model system for the study of many biological questions. We show that shRNAs, modeled on an endogenous microRNA, are extremely effective at silencing gene expression during oogenesis. We also describe our progress toward building a genome-wide shRNA resource.     

Novel analytic criteria and effective plate designs for quality control in genome-scale RNAi screens

One of the most fundamental challenges in genome-wide RNA interference (RNAi) screens is to glean biological significance from mounds of data, which relies on the development and adoption of appropriate analytic methods and designs for quality control (QC) and hit selection. Currently, a Z-factor-based QC criterion is widely used to evaluate data quality. However, this criterion cannot take into account the fact that different positive controls may have different effect sizes and leads to inconsistent QC results in experiments with 2 or more positive controls with different effect sizes. In this study, based on a recently proposed parameter, strictly standardized mean difference (SSMD), novel QC criteria are constructed for evaluating data quality in genome-wide RNAi screens. Two good features of these novel criteria are: (1) SSMD has both clear original and probability meanings for evaluating the differentiation between positive and negative controls and hence the SSMD-based QC criteria have a solid probabilistic and statistical basis, and (2) these QC criteria obtain consistent QC results for multiple positive controls with different effect sizes. In addition, I propose multiple plate designs and the guidelines for using them in genome-wide RNAi screens. Finally, I provide strategies for using the SSMD-based QC criteria and effective plate design together to improve data quality. The novel SSMD-based QC criteria, effective plate designs, and related guidelines and strategies may greatly help to obtain high quality of data in genome-wide RNAi screens.     

RNAi living-cell microarrays for loss-of-function screens in Drosophila melanogaster cells

RNA interference (RNAi)-mediated loss-of-function screening in Drosophila melanogaster tissue culture cells is a powerful method for identifying the genes underlying cell biological functions and for annotating the fly genome. Here we describe the development of living-cell microarrays for screening large collections of RNAi-inducing double-stranded RNAs (dsRNAs) in Drosophila cells. The features of the microarrays consist of clusters of cells 200 mum in diameter, each with an RNAi-mediated depletion of a specific gene product. Because of the small size of the features, thousands of distinct dsRNAs can be screened on a single chip. The microarrays are suitable for quantitative and high-content cellular phenotyping and, in combination screens, for the identification of genetic suppressors, enhancers and synthetic lethal interactions. We used a prototype cell microarray with 384 different dsRNAs to identify previously unknown genes that affect cell proliferation and morphology, and, in a combination screen, that regulate dAkt/dPKB phosphorylation in the absence of dPTEN expression.     

High-throughput reverse genetics: RNAi screens in Caenorhabditis elegans

Two recent chromosome-wide screens for phenotypes caused by RNA-mediated interference (RNAi) in Caenorhabditis elegans have increased our understanding of essential genes in nematodes. These papers represent a major advance in functional genomics.     

High-throughput RNAi in Caenorhabditis elegans: genome-wide screens and functional genomics

The phenomenon of RNA-mediated interference (RNAi) was first discovered in the nematode Caenorhabditis elegans, in which introduction of double-stranded RNA causes specific inactivation of genes with corresponding sequences. Technical advances in RNAi methodology and the availability of the complete genome sequence have enabled the high-throughput, genome-wide RNAi analysis of this organism. Several groups have used large-scale RNAi to systematically examine every C. elegans gene for knock-down phenotypes, providing basal information to be mined in more detailed studies. Now, in addition to functional genomic RNAi analyses, high-throughput RNAi is also routinely used for rapid, genome-wide screens for genes involved in specific biological processes. The integration of high-throughput RNAi experiments with other large-scale data, such as DNA microarrays and protein-protein interaction maps, enhances the speed and reliability of such screens. The accumulation of RNAi phenotype data dramatically accelerates our understanding of this organism at the genetic level.     

Lessons from genetics: interpreting complex phenotypes in RNAi screens

Mammalian cell biology is witnessing a new era in which cellular processes are explained through dynamic networks of interacting cellular components. In this fast-pacing field, where image-based RNAi screening is taking a central role, there is a strong need to improve ways to capture such interactions in space and time. Cell biologists traditionally depict these events by confining themselves to the level of a single cell, or to many population-averaged cells. Similarly, classical geneticists observe and interpret phenotypes in a single organism to delineate signaling processes, but have also described genetic phenomena in populations of organisms. The analogy in the two approaches inspired us to draw parallels with, and take lessons from concepts in classical genetics.     

An analysis of normalization methods for Drosophila RNAi genomic screens and development of a robust validation scheme

Genome-wide RNA interference (RNAi) screening allows investigation of the role of individual genes in a process of choice. Most RNAi screens identify a large number of genes with a continuous gradient in the assessed phenotype. Screeners must decide whether to examine genes with the most robust phenotype or the full gradient of genes that cause an effect and how to identify candidate genes. The authors have used RNAi in Drosophila cells to examine viability in a 384-well plate format and compare 2 screens, untreated control and treatment. They compare multiple normalization methods, which take advantage of different features within the data, including quantile normalization, background subtraction, scaling, cellHTS2 (Boutros et al. 2006), and interquartile range measurement. Considering the false-positive potential that arises from RNAi technology, a robust validation method was designed for the purpose of gene selection for future investigations. In a retrospective analysis, the authors describe the use of validation data to evaluate each normalization method. Although no method worked ideally, a combination of 2 methods, background subtraction followed by quantile normalization and cellHTS2, at different thresholds, captures the most dependable and diverse candidate genes. Thresholds are suggested depending on whether a few candidate genes are desired or a more extensive systems-level analysis is sought. The normalization approaches and experimental design to perform validation experiments are likely to apply to those high-throughput screening systems attempting to identify genes for systems-level analysis.     

An efficient and fully automated high-throughput transfection method for genome-scale siRNA screens

RNA interference (RNAi), combined with the availability of genome sequences, provides an unprecedented opportunity for the massive and parallel investigations of gene function. Small interfering RNA (siRNA) represents a popular and quick approach of RNAi for in vitro loss-of-function genetic screens. Efficient transfection of siRNA is critical for unambiguous interpretation of screen results and thus overall success of any siRNA screen. A high-throughput, lipid-based transfection method for siRNA was developed that can process eighty 384-well microplates in triplicate (for a total of 30,720 unique transfections) in 8 h. Transfection throughput was limited only by the speed of robotics, whereas the cost of screening was reduced. As a proof of principle, a genome-scale screen with a library of 22,108 siRNAs was performed to identify the genes sensitizing cells to mitomycin C at concentrations of 0, 20, and 60 nM. Transfection efficiency, performances of control siRNAs, and other quality metrics were monitored and demonstrated that the new, optimized transfection protocol produced high-quality results throughout the screen.     

The art and design of genetic screens: Drosophila melanogaster

The success of Drosophila melanogaster as a model organism is largely due to the power of forward genetic screens to identify the genes that are involved in a biological process. Traditional screens, such as the Nobel-prize-winning screen for embryonic-patterning mutants, can only identify the earliest phenotype of a mutation. This review describes the ingenious approaches that have been devised to circumvent this problem: modifier screens, for example, have been invaluable for elucidating signal-transduction pathways, whereas clonal screens now make it possible to screen for almost any phenotype in any cell at any stage of development.     

Minimizing the risk of reporting false positives in large-scale RNAi screens

Large-scale RNA interference (RNAi)-based analyses, very much as other 'omic' approaches, have inherent rates of false positives and negatives. The variability in the standards of care applied to validate results from these studies, if left unchecked, could eventually begin to undermine the credibility of RNAi as a powerful functional approach. This Commentary is an invitation to an open discussion started among various users of RNAi to set forth accepted standards that would insure the quality and accuracy of information in the large datasets coming out of genome-scale screens.     

The FLIGHT Drosophila RNAi database: 2010 update

FLIGHT (http://flight.icr.ac.uk/) is an online resource compiling data from high-throughput Drosophila in vivo and in vitro RNAi screens. FLIGHT includes details of RNAi reagents and their predicted off-target effects, alongside RNAi screen hits, scores and phenotypes, including images from high-content screens. The latest release of FLIGHT is designed to enable users to upload, analyze, integrate and share their own RNAi screens. Users can perform multiple normalizations, view quality control plots, detect and assign screen hits and compare hits from multiple screens using a variety of methods including hierarchical clustering. FLIGHT integrates RNAi screen data with microarray gene expression as well as genomic annotations and genetic/physical interaction datasets to provide a single interface for RNAi screen analysis and datamining in Drosophila.Key words: RNAi, database, integration, bioinformatics, phenotype     

Genome-wide RNAi screens identify genes required for Ricin and PE intoxications

Protein toxins such as Ricin and Pseudomonas exotoxin (PE) pose major public health challenges. Both toxins depend on host cell machinery for internalization, retrograde trafficking from endosomes to?the ER, and translocation to cytosol. Although both toxins follow a similar intracellular route, it is unknown how much they rely on the same genes. Here we conducted two genome-wide RNAi screens identifying genes required for intoxication and demonstrating that requirements are strikingly different between PE and Ricin, with only 13% overlap. Yet factors required by both toxins are present from the endosomes to the ER, and, at the morphological level, the toxins colocalize in multiple structures. Interestingly, Ricin, but not PE, depends on Golgi complex integrity and colocalizes significantly with a medial Golgi marker. Our data are consistent with two intertwined pathways converging and diverging at multiple points and reveal the complexity of retrograde membrane trafficking in mammalian cells.     

Integrating experimental and analytic approaches to improve data quality in genome-wide RNAi screens

RNA interference (RNAi) not only plays an important role in drug discovery but can also be developed directly into drugs. RNAi high-throughput screening (HTS) biotechnology allows us to conduct genome-wide RNAi research. A central challenge in genome-wide RNAi research is to integrate both experimental and computational approaches to obtain high quality RNAi HTS assays. Based on our daily practice in RNAi HTS experiments, we propose the implementation of 3 experimental and analytic processes to improve the quality of data from RNAi HTS biotechnology: (1) select effective biological controls; (2) adopt appropriate plate designs to display and/or adjust for systematic errors of measurement; and (3) use effective analytic metrics to assess data quality. The applications in 5 real RNAi HTS experiments demonstrate the effectiveness of integrating these processes to improve data quality. Due to the effectiveness in improving data quality in RNAi HTS experiments, the methods and guidelines contained in the 3 experimental and analytic processes are likely to have broad utility in genome-wide RNAi research.     

Cross-Species RNAi Rescue Platform in Drosophila melanogaster

RNAi-mediated gene knockdown in Drosophila melanogaster is a powerful method to analyze loss-of-function phenotypes both in cell culture and in vivo. However, it has also become clear that false positives caused by off-target effects are prevalent, requiring careful validation of RNAi-induced phenotypes. The most rigorous proof that an RNAi-induced phenotype is due to loss of its intended target is to rescue the phenotype by a transgene impervious to RNAi. For large-scale validations in the mouse and Caenorhabditis elegans, this has been accomplished by using bacterial artificial chromosomes (BACs) of related species. However, in Drosophila, this approach is not feasible because transformation of large BACs is inefficient. We have therefore developed a general RNAi rescue approach for Drosophila that employs Cre/loxP-mediated recombination to rapidly retrofit existing fosmid clones into rescue constructs. Retrofitted fosmid clones carry a selection marker and a phiC31 attB site, which facilitates the production of transgenic animals. Here, we describe our approach and demonstrate proof-of-principle experiments showing that D. pseudoobscura fosmids can successfully rescue RNAi-induced phenotypes in D. melanogaster, both in cell culture and in vivo. Altogether, the tools and method that we have developed provide a gold standard for validation of Drosophila RNAi experiments.RNAi-mediated gene knockdown, whereby an exogenous double stranded RNA (dsRNA) is used to trigger homology-dependent suppression of the target gene, is an effective loss-of-function method to interrogate gene function. The RNAi technology in Drosophila melanogaster is widely used for genomewide RNAi screens in cell culture (see review by Perrimon and Mathey-Prevot 2007a), and more recently has been extended to large scale in vivo studies (Dietzl et al. 2007; Ni et al. 2009; Mummery-Widmer et al. 2009). Gene knockdown by RNAi is achieved by the introduction of dsRNAs into cultured cells or by inducible overexpression of “hairpin” dsRNAs in transgenic flies. In the context of in vivo RNAi screening, the combination of a tissue-specific GAL4 driver with a GAL4-responsive hairpin dsRNA transgene allows knockdown of the target gene only in the desired cells, thus providing a powerful way of probing biological processes that have been so far difficult to investigate.Analysis of the specificity of long dsRNAs in Drosophila cells has revealed that these reagents, depending on their sequences and levels of expression, can knock down genes others than the intended target (Kulkarni et al. 2006; Ma et al. 2006). This phenomenon is not specific to long dsRNAs and has also been commonly observed with 21-nt long siRNAs and shRNAs used in mammalian RNAi screens. In fact the rate of false positives associated with off-target effects observed in mammalian screens is usually higher than those observed with long dsRNAs (Echeverri and Perrimon 2006). Unwanted false positives created by off-target effects are a major problem in RNAi screens and require lengthy secondary validation tests (Echeverri and Perrimon 2006; Perrimon and Mathey-Prevot 2007b; Ramadan et al. 2007). Further, false positives associated with RNAi reagents are not limited to tissue culture experiments, as they have also been reported in the context of transgenic RNAi. For example, ∼25% of the hairpins targeting nonessential genes cause lethality when driven by the constitutively expressed Act5C-GAL4 driver (Dietzl et al. 2007; Ni et al. 2009).A number of approaches can be used to validate the specificity of RNAi-induced phenotypes (Echeverri and Perrimon 2006). These include validation by multiple dsRNAs that target the same gene but that do not overlap in sequence, comparison of knockdown efficiencies of multiple dsRNAs and the phenotypic strengths, and rescue of the phenotype by either cDNAs or genomic DNAs. Rescue of RNAi phenotypes constitutes the gold standard in the field as it provides unambiguous proof that the targeted gene is indeed responsible for the phenotype observed. In Drosophila cell culture experiments, cDNAs that lack the original 3′-untranslated region (UTR) have been used to rescue phenotypes induced by dsRNAs targeting the 3′-UTR (Yokokura et al. 2004; Stielow et al. 2008). In mammalian cell culture experiments, cDNAs that have a silent point mutation in the region targeted by an siRNA are commonly used (Lassus et al. 2002). The intrinsic problem of these approaches, however, is that overexpression of cDNAs alone can evoke abnormal cellular responses on their own, complicating interpretation of the results. A cleaner method is based on cross-species rescue that uses genomic DNA from a different species whose sequence is divergent enough from the host species to make it refractory to the RNAi reagent directed against the host gene. This approach effectively addresses the issue of overexpression artifact, as the rescue transgene is expressed from its endogenous promoter, ensuring proper levels and precise spatiotemporal regulation of gene expression. Cross-species rescue methods that use bacterial artificial chromosome (BACs) retrofitted with an appropriate selection marker have been described for mammals and C. elegans (Kittler et al. 2005; Sarov et al. 2006). However, the BAC strategies are not realistic for large-scale studies, because transformation of BACs, which are typically larger than 100 kb, is inefficient, albeit not impossible, in Drosophila (Venken et al. 2006).To provide a feasible way to validate large-scale RNAi screening results, we decided to develop a universal method for cross-species RNAi rescue in Drosophila. We chose to use fosmids, which are single-copy bacterial vectors with a cloning capacity of ∼40 kb, rather than BACs because (1) transformation of plasmids around this size is relatively efficient (Venken et al. 2006) and (2) end-sequenced fosmid clones for 11 different Drosophila species generated by the Drosophila species genome project are now publicly available (Richards et al. 2005; Drosophila 12 Genomes Consortium 2007).