A pair of new statistical parameters for quality control in RNA interference high-throughput screening assays

RNA interference (RNAi) high-throughput screening (HTS) enables massive parallel gene silencing and is increasingly being used to reveal novel connections between genes and disease-relevant phenotypes. The application of genome-scale RNAi relies on the development of high-quality RNAi HTS assays. To obtain high-quality HTS assays, there is a strong need for an easily interpretable and theoretically based quality control (QC) metric. Signal-to-noise ratio (S/N), signal-to-background ratio (S/B), and Z-factor have been adopted as QC metrics in HTS assays. In this paper, I proposed a pair of new parameters, strictly standardized mean difference (SSMD) and coefficient of variability in difference (CVD), as QC metrics in RNAi HTS assays. Compared to S/B and S/N, SSMD and CVD capture the variabilities in both compared populations. Compared to Z-factor, SSMD and CVD have a clear probability interpretation and a solid statistical basis. Accordingly, the cutoff criteria of using SSMD or CVD as a QC metric in HTS assays are fully theoretically based. In addition, I discuss the relationship between the SSMD-based criterion and the popular Z-factor-based criterion and elucidate why p-value from t-test of testing mean difference fails to serve as a QC metric.     

GUItars: A GUI Tool for Analysis of High-Throughput RNA Interference Screening Data

Background

High-throughput RNA interference (RNAi) screening has become a widely used approach to elucidating gene functions. However, analysis and annotation of large data sets generated from these screens has been a challenge for researchers without a programming background. Over the years, numerous data analysis methods were produced for plate quality control and hit selection and implemented by a few open-access software packages. Recently, strictly standardized mean difference (SSMD) has become a widely used method for RNAi screening analysis mainly due to its better control of false negative and false positive rates and its ability to quantify RNAi effects with a statistical basis. We have developed GUItars to enable researchers without a programming background to use SSMD as both a plate quality and a hit selection metric to analyze large data sets.

Results

The software is accompanied by an intuitive graphical user interface for easy and rapid analysis workflow. SSMD analysis methods have been provided to the users along with traditionally-used z-score, normalized percent activity, and t-test methods for hit selection. GUItars is capable of analyzing large-scale data sets from screens with or without replicates. The software is designed to automatically generate and save numerous graphical outputs known to be among the most informative high-throughput data visualization tools capturing plate-wise and screen-wise performances. Graphical outputs are also written in HTML format for easy access, and a comprehensive summary of screening results is written into tab-delimited output files.

Conclusion

With GUItars, we demonstrated robust SSMD-based analysis workflow on a 3840-gene small interfering RNA (siRNA) library and identified 200 siRNAs that increased and 150 siRNAs that decreased the assay activities with moderate to stronger effects. GUItars enables rapid analysis and illustration of data from large- or small-scale RNAi screens using SSMD and other traditional analysis methods. The software is freely available at http://sourceforge.net/projects/guitars/.     

Integrating experimental and analytic approaches to improve data quality in genome-wide RNAi screens

RNA interference (RNAi) not only plays an important role in drug discovery but can also be developed directly into drugs. RNAi high-throughput screening (HTS) biotechnology allows us to conduct genome-wide RNAi research. A central challenge in genome-wide RNAi research is to integrate both experimental and computational approaches to obtain high quality RNAi HTS assays. Based on our daily practice in RNAi HTS experiments, we propose the implementation of 3 experimental and analytic processes to improve the quality of data from RNAi HTS biotechnology: (1) select effective biological controls; (2) adopt appropriate plate designs to display and/or adjust for systematic errors of measurement; and (3) use effective analytic metrics to assess data quality. The applications in 5 real RNAi HTS experiments demonstrate the effectiveness of integrating these processes to improve data quality. Due to the effectiveness in improving data quality in RNAi HTS experiments, the methods and guidelines contained in the 3 experimental and analytic processes are likely to have broad utility in genome-wide RNAi research.     

The use of strictly standardized mean difference for hit selection in primary RNA interference high-throughput screening experiments

RNA interference (RNAi) high-throughput screening (HTS) has been hailed as the 2nd genomics wave following the 1st genomics wave of gene expression microarrays and single-nucleotide polymorphism discovery platforms. Following an RNAi HTS, the authors are interested in identifying short interfering RNA (siRNA) hits with large inhibition/activation effects. For hit selection, the z-score method and its variants are commonly used in primary RNAi HTS experiments. Recently, strictly standardized mean difference (SSMD) has been proposed to measure the siRNA effect represented by the magnitude of difference between an siRNA and a negative reference group. The links between SSMD and d+-probability offer a clear interpretation of siRNA effects from a probability perspective. Hence, SSMD can be used as a ranking metric for hit selection. In this article, the authors investigated both the SSMD-based testing process and the use of SSMD as a ranking metric for hit selection in 2 primary siRNA HTS experiments. The analysis results showed that, as a ranking metric, SSMD was more stable and reliable than percentage inhibition and led to more robust hit selection results. Using the SSMD -based testing method, the false-negative rate can more readily be obtained. More important, the use of the SSMD-based method can result in a reduction in both the false-negative and false-positive rates. The applications presented in this article demonstrate that the SSMD method addresses scientific questions and fills scientific needs better than both percentage inhibition and the commonly used z-score method for hit selection.     

A new method with flexible and balanced control of false negatives and false positives for hit selection in RNA interference high-throughput screening assays

The z-score method and its variants for testing mean difference are commonly used for hit selection in high-throughput screening (HTS) assays. Strictly standardized mean difference (SSMD) offers a way to measure and classify the short interfering RNA (siRNA) effects. In this article, based on SSMD, the authors propose a new testing method for hit selection in RNA interference (RNAi) HTS assays. This SSMD-based method allows the differentiation between siRNAs with large and small effects on the assay output and maintains flexible and balanced control of both the false-negative rate, in which the siRNAs with strong effects are not selected as hits, and the restricted false-positive rate, in which the siRNAs with weak or no effects are selected as hits. This method directly addresses the size of siRNA effects represented by the strength of difference between an siRNA and a negative reference, whereas the classic z-score method and t-test of testing no mean difference address whether the mean of an siRNA is exactly the same as the mean of a negative reference. This method can readily control the false-negative rate, whereas it is nontrivial for the classic z-score method and t-test to control the false-negative rate. Therefore, theoretically, the SSMD-based method offers better control of the sizes of siRNA effects and the associated false-positive and false-negative rates than the commonly used z-score method and t-test for hit selection in HTS assays. The SSMD-based method should generally be applicable to any assay in which the end point is a difference in signal compared to a reference sample, including those for RNAi, receptor, enzyme, and cellular function.     

High-throughput RNAi screening to dissect cellular pathways: A how-to guide

RNA interference (RNAi) has become a powerful tool to dissect cellular pathways and characterize gene functions. The availability of genome-wide RNAi libraries for various model organisms and mammalian cells has enabled high-throughput RNAi screenings. These RNAi screens successfully identified key components that had previously been missed in classical forward genetic screening approaches and allowed the assessment of combined loss-of-function phenotypes. Crucially, the quality of RNAi screening results depends on quantitative assays and the choice of the right biological context. In this review, we provide an overview on the design and application of high-throughput RNAi screens as well as data analysis and candidate validation strategies.     

High-Throughput Screening by RNA Interference: Control of Two Distinct Types of Variance

The availability of genome-wide RNAi libraries has enabled researchers to rapidly assess the functions of thousands of genes; however the fact that these screens are run in living biological systems add complications above and beyond that normally seen in high-throughput screening (HTS). Specifically, error due to variance in both measurement and biology are large in such screens, leading to the conclusion that the majority of "hits" are expected to be false positives. Here, we outline basic guidelines for screen development that will help the researcher to control these forms of variance. By running a large number of positive and negative control genes, error of measurement can be accurately estimated and false negatives reduced. Likewise, by using a complex readout for the screen which is not easily mimicked by other biological pathways and phenomena, false positives can be minimized. By controlling variance in these ways, the researcher can maximize the utility of genome-wide RNAi screening.     

Technique review: how to use RNA interference.

RNA interference (RNAi) has been rapidly adopted as a general method for inhibiting gene expression in most laboratory organisms. This paper discusses how libraries of RNAi reagents are being used to perform genome-wide reverse genetic screens in both model organisms and mammalian cells. Guidelines for designing effective small interfering RNAs and appropriate controls for mammalian RNAi experiments will also be discussed.     

High-throughput RNAi in Caenorhabditis elegans: genome-wide screens and functional genomics

The phenomenon of RNA-mediated interference (RNAi) was first discovered in the nematode Caenorhabditis elegans, in which introduction of double-stranded RNA causes specific inactivation of genes with corresponding sequences. Technical advances in RNAi methodology and the availability of the complete genome sequence have enabled the high-throughput, genome-wide RNAi analysis of this organism. Several groups have used large-scale RNAi to systematically examine every C. elegans gene for knock-down phenotypes, providing basal information to be mined in more detailed studies. Now, in addition to functional genomic RNAi analyses, high-throughput RNAi is also routinely used for rapid, genome-wide screens for genes involved in specific biological processes. The integration of high-throughput RNAi experiments with other large-scale data, such as DNA microarrays and protein-protein interaction maps, enhances the speed and reliability of such screens. The accumulation of RNAi phenotype data dramatically accelerates our understanding of this organism at the genetic level.     

Evaluating the influence of quality control decisions and software algorithms on SNP calling for the affymetrix 6.0 SNP array platform

Objective: Our goal was to evaluate the influence of quality control (QC) decisions using two genotype calling algorithms, CRLMM and Birdseed, designed for the Affymetrix SNP Array 6.0. Methods: Various QC options were tried using the two algorithms and comparisons were made on subject and call rate and on association results using two data sets. Results: For Birdseed, we recommend using the contrast QC instead of QC call rate for sample QC. For CRLMM, we recommend using the signal-to-noise rate ≥4 for sample QC and a posterior probability of 90% for genotype accuracy. For both algorithms, we recommend calling the genotype separately for each plate, and dropping SNPs with a lower call rate (<95%) before evaluating samples with lower call rates. To investigate whether the genotype calls from the two algorithms impacted the genome-wide association results, we performed association analysis using data from the GENOA cohort; we observed that the number of significant SNPs were similar using either CRLMM or Birdseed. Conclusions: Using our suggested workflow both algorithms performed similarly; however, fewer samples were removed and CRLMM took half the time to run our 854 study samples (4.2 h) compared to Birdseed (8.4 h).     

Genome-wide screening for gene function using RNAi in mammalian cells

Mammalian genome sequencing has identified numerous genes requiring functional annotation. The discovery that dsRNA can direct gene-specific silencing in both model organisms and mammalian cells through RNA interference (RNAi) has provided a platform for dissecting the function of independent genes. The generation of large-scale RNAi libraries targeting all predicted genes within mouse, rat and human cells, combined with the large number of cell-based assays, provides a unique opportunity to perform high-throughput genetics in these complex cell systems. Many different formats exist for the generation of genome-wide RNAi libraries for use in mammalian cells. Furthermore, the use of these libraries in either genetic screens or genetic selections allows for the identification of known and novel genes involved in complex cellular phenotypes and biological processes, some of which underpin human disease. In this review, we examine genome-wide RNAi libraries used in model organisms and mammalian cells and provide examples of how these information rich reagents can be used for determining gene function, discovering novel therapeutic targets and dissecting signalling pathways, cellular processes and complex phenotypes.     

High-throughput RNA interference screening using pooled shRNA libraries and next generation sequencing

RNA interference (RNAi) screening is a state-of-the-art technology that enables the dissection of biological processes and disease-related phenotypes. The commercial availability of genome-wide, short hairpin RNA (shRNA) libraries has fueled interest in this area but the generation and analysis of these complex data remain a challenge. Here, we describe complete experimental protocols and novel open source computational methodologies, shALIGN and shRNAseq, that allow RNAi screens to be rapidly deconvoluted using next generation sequencing. Our computational pipeline offers efficient screen analysis and the flexibility and scalability to quickly incorporate future developments in shRNA library technology.     

Design and evaluation of genome-wide libraries for RNA interference screens

RNA interference (RNAi) screens have enabled the systematic analysis of many biological processes in cultured cells and whole organisms. The success of such screens and the interpretation of the data depend on the stringent design of RNAi libraries. We describe and validate NEXT-RNAi, a software for the automated design and evaluation of RNAi sequences on a genome-wide scale. NEXT-RNAi is implemented as open-source software and is accessible at http://www.nextrnai.org/     

Computational detection and suppression of sequence-specific off-target phenotypes from whole genome RNAi screens

A challenge for large-scale siRNA loss-of-function studies is the biological pleiotropy resulting from multiple modes of action of siRNA reagents. A major confounding feature of these reagents is the microRNA-like translational quelling resulting from short regions of oligonucleotide complementarity to many different messenger RNAs. We developed a computational approach, deconvolution analysis of RNAi screening data, for automated quantitation of off-target effects in RNAi screening data sets. Substantial reduction of off-target rates was experimentally validated in five distinct biological screens across different genome-wide siRNA libraries. A public-access graphical-user-interface has been constructed to facilitate application of this algorithm.     

Genome-wide RNAi of C. elegans using the hypersensitive rrf-3 strain reveals novel gene functions

RNA-mediated interference (RNAi) is a method to inhibit gene function by introduction of double-stranded RNA (dsRNA). Recently, an RNAi library was constructed that consists of bacterial clones expressing dsRNA, corresponding to nearly 90% of the 19,427 predicted genes of C. elegans. Feeding of this RNAi library to the standard wild-type laboratory strain Bristol N2 detected phenotypes for approximately 10% of the corresponding genes. To increase the number of genes for which a loss-of-function phenotype can be detected, we undertook a genome-wide RNAi screen using the rrf-3 mutant strain, which we found to be hypersensitive to RNAi. Feeding of the RNAi library to rrf-3 mutants resulted in additional loss-of-function phenotypes for 393 genes, increasing the number of genes with a phenotype by 23%. These additional phenotypes are distributed over different phenotypic classes. We also studied interexperimental variability in RNAi results and found persistent levels of false negatives. In addition, we used the RNAi phenotypes obtained with the genome-wide screens to systematically clone seven existing genetic mutants with visible phenotypes. The genome-wide RNAi screen using rrf-3 significantly increased the functional data on the C. elegans genome. The resulting dataset will be valuable in conjunction with other functional genomics approaches, as well as in other model organisms.     

Loss of the putative RNA-directed RNA polymerase RRF-3 makes C. elegans hypersensitive to RNAi

RNA interference (RNAi) is a broadly used reverse genetics method in C. elegans. Unfortunately, RNAi does not inhibit all genes. We show that loss of function of a putative RNA-directed RNA polymerase (RdRP) of C. elegans, RRF-3, results in a substantial enhancement of sensitivity to RNAi in diverse tissues. This is particularly striking in the nervous system; neurons that are generally refractory to RNAi in a wild-type genetic background can respond effectively to interference in an rrf-3 mutant background. These data provide the first indication of physiological negative modulation of the RNAi response and implicate an RdRP-related factor in this effect. The rrf-3 strain can be useful to study genes that, in wild-type, do not show a phenotype after RNAi, and it is probably the strain of choice for genome-wide RNAi screens.     

A primer on using pooled shRNA libraries for functional genomic screens

The discovery of RNA interference (RNAi) has revolutionized genetic analysis in mammalian cells. Loss-of-function RNAi screens enable rapid, functional annotation of the genome. Of the various RNAi approaches, pooled shRNA libraries have received considerable attention because of their versatility. A number of genome-wide shRNA libraries have been constructed against the human and mouse genomes, and these libraries can be readily applied to a variety of screens to interrogate the function of human and mouse genes in an unbiased fashion. We provide an introduction to the technical aspects of using pooled shRNA libraries for genetic screens.     

A Genome-Wide CRISPR Library for High-Throughput Genetic Screening in Drosophila Cells

The simplicity of the CRISPR/Cas9 system of genome engineering has opened up the possibility of performing genome-wide targeted mutagenesis in cell lines,enabling screening for cellular phenotypes resulting from genetic aberrations.Drosophila cells have proven to be highly effective in identifying genes involved in cellular processes through similar screens using partial knockdown by RNAi.This is in part due to the lower degree of redundancy between genes in this organism,whilst still maintaining highly conserved gene networks and orthologs of many human disease-causing genes.The ability of CRISPR to generate genetic loss of function mutations not only increases the magnitude of any effect over currently employed RNAi techniques,but allows analysis over longer periods of time which can be critical for certain phenotypes.In this study,we have designed and built a genome-wide CRISPR library covering 13,501 genes,among which 8989 genes are targeted by three or more independent single guide RNAs(sg RNAs).Moreover,we describe strategies to monitor the population of guide RNAs by high throughput sequencing(HTS).We hope that this library will provide an invaluable resource for the community to screen loss of function mutations for cellular phenotypes,and as a source of guide RNA designs for future studies.