A pair of new statistical parameters for quality control in RNA interference high-throughput screening assays

RNA interference (RNAi) high-throughput screening (HTS) enables massive parallel gene silencing and is increasingly being used to reveal novel connections between genes and disease-relevant phenotypes. The application of genome-scale RNAi relies on the development of high-quality RNAi HTS assays. To obtain high-quality HTS assays, there is a strong need for an easily interpretable and theoretically based quality control (QC) metric. Signal-to-noise ratio (S/N), signal-to-background ratio (S/B), and Z-factor have been adopted as QC metrics in HTS assays. In this paper, I proposed a pair of new parameters, strictly standardized mean difference (SSMD) and coefficient of variability in difference (CVD), as QC metrics in RNAi HTS assays. Compared to S/B and S/N, SSMD and CVD capture the variabilities in both compared populations. Compared to Z-factor, SSMD and CVD have a clear probability interpretation and a solid statistical basis. Accordingly, the cutoff criteria of using SSMD or CVD as a QC metric in HTS assays are fully theoretically based. In addition, I discuss the relationship between the SSMD-based criterion and the popular Z-factor-based criterion and elucidate why p-value from t-test of testing mean difference fails to serve as a QC metric.     

Identifying actives from HTS data sets: practical approaches for the selection of an appropriate HTS data-processing method and quality control review

High-throughput screening (HTS) has achieved a dominant role in drug discovery over the past 2 decades. The goal of HTS is to identify active compounds (hits) by screening large numbers of diverse chemical compounds against selected targets and/or cellular phenotypes. The HTS process consists of multiple automated steps involving compound handling, liquid transfers, and assay signal capture, all of which unavoidably contribute to systematic variation in the screening data. The challenge is to distinguish biologically active compounds from assay variability. Traditional plate controls-based and non-controls-based statistical methods have been widely used for HTS data processing and active identification by both the pharmaceutical industry and academic sectors. More recently, improved robust statistical methods have been introduced, reducing the impact of systematic row/column effects in HTS data. To apply such robust methods effectively and properly, we need to understand their necessity and functionality. Data from 6 HTS case histories are presented to illustrate that robust statistical methods may sometimes be misleading and can result in more, rather than less, false positives or false negatives. In practice, no single method is the best hit detection method for every HTS data set. However, to aid the selection of the most appropriate HTS data-processing and active identification methods, the authors developed a 3-step statistical decision methodology. Step 1 is to determine the most appropriate HTS data-processing method and establish criteria for quality control review and active identification from 3-day assay signal window and DMSO validation tests. Step 2 is to perform a multilevel statistical and graphical review of the screening data to exclude data that fall outside the quality control criteria. Step 3 is to apply the established active criterion to the quality-assured data to identify the active compounds.     

Three mechanistically distinct kinase assays compared: Measurement of intrinsic ATPase activity identified the most comprehensive set of ITK inhibitors

Numerous assay methods have been developed to identify small-molecule effectors of protein kinases, but no single method can be applied to all isolated kinases. The authors developed a set of 3 high-throughput screening (HTS)-compatible biochemical assays that can measure 3 mechanistically distinct properties of a kinase active site, with the goal that at least 1 of the 3 would be applicable to any kinase selected as a target for drug discovery efforts. Two assays measure catalytically active enzyme: A dissociation-enhanced lanthanide fluoroimmuno assay (DELFIA) uses an antibody to quantitate the generation of phosphorylated substrate; a second assay uses luciferase to measure the consumption of adenosine triphosphate (ATP) during either phosphoryl-transfer to a peptide substrate or to water (intrinsic ATPase activity). A third assay, which is not dependent on a catalytically active enzyme, measures the competition for binding to kinase between an inhibitor and a fluorescent ATP binding site probe. To evaluate the suitability of these assays for drug discovery, the authors compared their ability to identify inhibitors of a nonreceptor protein tyrosine kinase from the Tec family, interleukin-2-inducible T cell kinase (ITK). The 3 assays agreed on 57% of the combined confirmed hit set identified from screening a 10,208-compound library enriched with known kinase inhibitors and molecules that were structurally similar. Among the 3 assays, the one measuring intrinsic ATPase activity produced the largest number of unique hits, the fewest unique misses, and the most comprehensive hit set, missing only 2.7% of the confirmed inhibitors identified by the other 2 assays combined. Based on these data, all 3 assay formats are viable for screening and together provide greater options for assay design depending on the targeted kinase.     

High throughput screening for small molecule therapy for Gaucher disease using patient tissue as the source of mutant glucocerebrosidase

Gaucher disease (GD), the most common lysosomal storage disorder, results from the inherited deficiency of the lysosomal enzyme glucocerebrosidase (GCase). Previously, wildtype GCase was used for high throughput screening (HTS) of large collections of compounds to identify small molecule chaperones that could be developed as new therapies for GD. However, the compounds identified from HTS usually showed reduced potency later in confirmatory cell-based assays. An alternate strategy is to perform HTS on mutant enzyme to identify different lead compounds, including those enhancing mutant enzyme activities. We developed a new screening assay using enzyme extract prepared from the spleen of a patient with Gaucher disease with genotype N370S/N370S. In tissue extracts, GCase is in a more native physiological environment, and is present with the native activator saposin C and other potential cofactors. Using this assay, we screened a library of 250,000 compounds and identified novel modulators of mutant GCase including 14 new lead inhibitors and 30 lead activators. The activities of some of the primary hits were confirmed in subsequent cell-based assays using patient-derived fibroblasts. These results suggest that primary screening assays using enzyme extracted from tissues is an alternative approach to identify high quality, physiologically relevant lead compounds for drug development.     

Using clustering techniques to improve hit selection in high-throughput screening

A typical modern high-throughput screening (HTS) operation consists of testing thousands of chemical compounds to select active ones for future detailed examination. The authors describe 3 clustering techniques that can be used to improve the selection of active compounds (i.e., hits). They are designed to identify quality hits in the observed HTS measurements. The considered clustering techniques were first tested on simulated data and then applied to analyze the assay inhibiting Escherichia coli dihydrofo-late reductase produced at the HTS laboratory of McMaster University.     

High-throughput identification of inhibitors of human mitochondrial peptide deformylase

The human mitochondrial peptide deformylase (HsPDF) provides a potential new target for broadly acting antiproliferative agents. To identify novel nonpeptidomimetic and nonhydroxamic acid-based inhibitors of HsPDF, the authors have developed a high-throughput screening (HTS) strategy using a fluorescence polarization (FP)-based binding assay as the primary assay for screening chemical libraries, followed by an enzymatic-based assay to confirm hits, prior to characterization of their antiproliferative activity against established tumor cell lines. The authors present the results and performance of the established strategy tested in a pilot screen of 2880 compounds and the identification of the 1st inhibitors. Two common scaffolds were identified within the hits. Furthermore, cytotoxicity studies revealed that most of the confirmed hits have antiproliferative activity. These findings demonstrate that the designed strategy can identify novel functional inhibitors and provide a powerful alternative to the use of functional assays in HTS and support the hypothesis that HsPDF inhibitors may constitute a new class of antiproliferative agent.     

A novel platform for accelerated pharmacodynamic profiling for lead optimization of anticancer drug candidates

Oncology drug discovery is, by definition, a target-rich enterprise. High-throughput screening (HTS) laboratories have supported a wide array of molecularly targeted and chemical genomic approaches for anticancer lead generation, and the number of hits emerging from such campaigns has increased dramatically. Although automation of HTS processes has eliminated primary screening as a bottleneck, the demands on secondary screening in appropriate cell-based assays have increased concomitantly with the numbers of hits delivered to therapeutic area laboratories. The authors describe herein the implementation of a novel platform using off-the-shelf solutions that have allowed them to efficiently characterize hundreds of HTS hits using a palette of Western blot-based pharmacodynamic assays. The platform employs a combination of a flatbed bufferless SDS-PAGE system, a dry ultra-rapid electroblotting apparatus, and a highly sensitive and quantitative infrared imaging system. Cumulatively, this platform has significantly reduced the cycle time for HTS hit evaluation. In addition, the routine use of this platform has resulted in higher quality data that have allowed the development of structure-activity databases that have tangibly improved lead optimization. The authors describe in detail the application of this platform, designated the Accelerated Pharmaco-Dynamic Profiler (APDP), to the annotation of inhibitors of 2 attractive oncology targets, BRAF kinase and Hsp90.     

HTS promiscuity analyses for accelerating decision making

Frequent hitters are compounds that are detected as a "hit" in multiple high-throughput screening (HTS) assays. Such behavior is specific (e.g., target family related) or unspecific (e.g., reactive compounds) or can result from a combination of such behaviors. Detecting such hits while predicting the underlying reason behind their promiscuous behavior is desirable because it provides valuable information not only about the compounds themselves but also about the assay methodology and target classes at hand. This information can also greatly reduce cost and time during HTS hit profiling. The present study exemplifies how to mine large HTS data repositories, such as the one at Boehringer Ingelheim, to identify frequent hitters, gain further insights into the causes of promiscuous behavior, and generate models for predicting promiscuous compounds. Applications of this approach are demonstrated using two recent large-scale HTS assays. The authors believe this analysis and its concrete applications are valuable tools for streamlining and accelerating decision-making processes during the course of hit discovery.     

An Improved Method for P2X7R Antagonist Screening

ATP physiologically activates the P2X7 receptor (P2X7R), a member of the P2X ionotropic receptor family. When activated by high concentrations of ATP (i.e., at inflammation sites), this receptor is capable of forming a pore that allows molecules of up to 900 Da to pass through. This receptor is upregulated in several diseases, particularly leukemia, rheumatoid arthritis and Alzheimer''s disease. A selective antagonist of this receptor could be useful in the treatment of P2X7R activation-related diseases. In the present study, we have evaluated several parameters using in vitro protocols to validate a high-throughput screening (HTS) method to identify P2X7R antagonists. We generated dose-response curves to determine the EC₅₀ value of the known agonist ATP and the IC_s50 values for the known antagonists Brilliant Blue G (BBG) and oxidized ATP (OATP). The values obtained were consistent with those found in the literature (0.7 ± 0.07 mM, 1.3-2.6 mM and 173-285 μM for ATP, BBG and OATP, respectively). The Z-factor, an important statistical tool that can be used to validate the robustness and suitability of an HTS assay, was 0.635 for PI uptake and 0.867 for LY uptake. No inter-operator variation was observed, and the results obtained using our improved method were reproducible. Our data indicate that our assay is suitable for the selective and reliable evaluation of P2X7 activity in multiwell plates using spectrophotometry-based methodology. This method might improve the high-throughput screening of conventional chemical or natural product libraries for possible candidate P2X7R antagonist or agonist     

Assay concordance between SPA and TR-FRET in high-throughput screening

High-throughput screening (HTS) of large chemical libraries has become the main source of new lead compounds for drug development. Several specialized detection technologies have been developed to facilitate the cost- and time-efficient screening of millions of compounds. However, concerns have been raised, claiming that different HTS technologies may produce different hits, thus limiting trust in the reliability of HTS data. This study was aimed to investigate the reliability of the authors most frequently used assay techniques: scintillation proximity assay (SPA) and homogeneous time-resolved fluorescence resonance energy transfer (TR-FRET). To investigate the data concordance between these 2 detection technologies, the authors screened a large subset of the Schering compound library consisting of 300,000 compounds for inhibitors of a nonreceptor tyrosine kinase. They chose to set up this study in realistic HTS scale to ensure statistical significance of the results. The findings clearly demonstrate that the choice of detection technology has no significant impact on hit finding, provided that assays are biochemically equivalent. Data concordance is up to 90%. The little differences in hit findings are caused by threshold setting but not by systematic differences between the technologies. The most significant difference between the compared techniques is that in the SPA format, more false-positive primary hits were obtained.     

A simple, no-wash cell adhesion-based high-throughput assay for the discovery of small-molecule regulators of the integrin CD11b/CD18

The leukocyte-specific integrin CD11b/CD18 plays a key role in the biological function of these cells and represents a validated therapeutic target for inflammatory diseases. Currently, the low affinity interaction between CD11b/CD18 integrin and its respective ligand poses a challenge in the development of cell-based adhesion assays for the high-throughput screening (HTS) environment. Here the authors describe a simple cell-based adhesion assay that can be readily used for HTS for the discovery of functional regulators of CD11b/CD18. The assay consistently produces acceptable Z' values (> 0.5) for HTS. After testing the assay using 2 established blocking antibodies as reference biologicals, the authors performed a proof-of-concept primary screen using a library of 6612 compounds and identified both agonist and antagonist hits.     

TR-FRET-based high-throughput screening assay for identification of UBC13 inhibitors

UBC13 is a noncanonical ubiquitin conjugating enzyme (E2) that has been implicated in a variety of cellular signaling processes due to its ability to catalyze formation of lysine 63-linked polyubiquitin chains on various substrates. In particular, UBC13 is required for signaling by a variety of receptors important in immune regulation, making it a candidate target for inflammatory diseases. UBC13 is also critical for double-strand DNA repair and thus a potential radiosensitizer and chemosensitizer target for oncology. The authors developed a high-throughput screening (HTS) assay for UBC13 based on the method of time-resolved fluorescence resonance energy transfer (TR-FRET). The TR-FRET assay combines fluorochrome (Fl)-conjugated ubiquitin (fluorescence acceptor) with terbium (Tb)-conjugated ubiquitin (fluorescence donor), such that the assembly of mixed chains of Fl- and Tb-ubiquitin creates a robust TR-FRET signal. The authors defined conditions for optimized performance of the TR-FRET assay in both 384- and 1536-well formats. Chemical library screens (total 456 865 compounds) were conducted in high-throughput mode using various compound collections, affording superb Z' scores (typically >0.7) and thus validating the performance of the assays. Altogether, the HTS assays described here are suitable for large-scale, automated screening of chemical libraries in search of compounds with inhibitory activity against UBC13.     

Use of mixture distributions to deconvolute the behavior of "hits" and controls in high-throughput screening data

The stochastic nature of high-throughput screening (HTS) data indicates that information may be gleaned by applying statistical methods to HTS data. A foundation of parametric statistics is the study and elucidation of population distributions, which can be modeled using modern spreadsheet software. The methods and results described here use fundamental concepts of statistical population distributions analyzed using a spreadsheet to provide tools in a developing armamentarium for extracting information from HTS data. Specific examples using two HTS kinase assays are analyzed. The analyses use normal and gamma distributions, which combine to form mixture distributions. HTS data were found to be described well using such mixture distributions, and deconvolution of the mixtures to the constituent gamma and normal parts provided insight into how the assays performed. In particular, the proportion of hits confirmed was predicted from the original HTS data and used to assess screening assay performance. The analyses also provide a method for determining how hit thresholds--values used to separate active from inactive compounds--affect the proportion of compounds verified as active and how the threshold can be chosen to optimize the selection process.     

A new method with flexible and balanced control of false negatives and false positives for hit selection in RNA interference high-throughput screening assays

The z-score method and its variants for testing mean difference are commonly used for hit selection in high-throughput screening (HTS) assays. Strictly standardized mean difference (SSMD) offers a way to measure and classify the short interfering RNA (siRNA) effects. In this article, based on SSMD, the authors propose a new testing method for hit selection in RNA interference (RNAi) HTS assays. This SSMD-based method allows the differentiation between siRNAs with large and small effects on the assay output and maintains flexible and balanced control of both the false-negative rate, in which the siRNAs with strong effects are not selected as hits, and the restricted false-positive rate, in which the siRNAs with weak or no effects are selected as hits. This method directly addresses the size of siRNA effects represented by the strength of difference between an siRNA and a negative reference, whereas the classic z-score method and t-test of testing no mean difference address whether the mean of an siRNA is exactly the same as the mean of a negative reference. This method can readily control the false-negative rate, whereas it is nontrivial for the classic z-score method and t-test to control the false-negative rate. Therefore, theoretically, the SSMD-based method offers better control of the sizes of siRNA effects and the associated false-positive and false-negative rates than the commonly used z-score method and t-test for hit selection in HTS assays. The SSMD-based method should generally be applicable to any assay in which the end point is a difference in signal compared to a reference sample, including those for RNAi, receptor, enzyme, and cellular function.     

Development of a High-Throughput Three-Dimensional Invasion Assay for Anti-Cancer Drug Discovery

The lack of three-dimensional (3-D) high-throughput (HT) screening assays designed to identify anti-cancer invasion drugs is a major hurdle in reducing cancer-related mortality, with the key challenge being assay standardization. Presented is the development of a novel 3-D invasion assay with HT potential that involves surrounding cell-collagen spheres within collagen to create a 3-D environment through which cells can invade. Standardization was achieved by designing a tooled 96-well plate to create a precisely designated location for the cell-collagen spheres and by using dialdehyde dextran to inhibit collagen contraction, maintaining uniform size and shape. This permits automated readout for determination of the effect of inhibitory compounds on cancer cell invasion. Sensitivity was demonstrated by the ability to distinguish varying levels of invasiveness of cancer cell lines, and robustness was determined by calculating the Z-factor. A Z-factor of 0.65 was obtained by comparing the effects of DMSO and anti-β1-integrin antibody, an inhibitory reagent, on the invasion of Du145 cancer cells, suggesting this novel assay is suitable for large scale drug discovery. As proof of principle, the NCI Diversity Compound Library was screened against human invasive cancer cells. Nine compounds exhibiting high potency and low toxicity were identified, including DX-52-1, a compound previously reported to inhibit cell migration, a critical determinant of cancer invasion. The results indicate that this innovative HT platform is a simple, precise, and easy to replicate 3-D invasion assay for anti-cancer drug discovery.     

Label Free Fragment Screening Using Surface Plasmon Resonance as a Tool for Fragment Finding – Analyzing Parkin,a Difficult CNS Target

Surface Plasmon Resonance (SPR) is rarely used as a primary High-throughput Screening (HTS) tool in fragment-based approaches. With SPR instruments becoming increasingly high-throughput it is now possible to use SPR as a primary tool for fragment finding. SPR becomes, therefore, a valuable tool in the screening of difficult targets such as the ubiquitin E3 ligase Parkin. As a prerequisite for the screen, a large number of SPR tests were performed to characterize and validate the active form of Parkin. A set of compounds was designed and used to define optimal SPR assay conditions for this fragment screen. Using these conditions, more than 5000 pre-selected fragments from our in-house library were screened for binding to Parkin. Additionally, all fragments were simultaneously screened for binding to two off target proteins to exclude promiscuous binding compounds. A low hit rate was observed that is in line with hit rates usually obtained by other HTS screening assays. All hits were further tested in dose responses on the target protein by SPR for confirmation before channeling the hits into Nuclear Magnetic Resonance (NMR) and other hit-confirmation assays.     

High-throughput screening assays for the identification of chemical probes

High-throughput screening (HTS) assays enable the testing of large numbers of chemical substances for activity in diverse areas of biology. The biological responses measured in HTS assays span isolated biochemical systems containing purified receptors or enzymes to signal transduction pathways and complex networks functioning in cellular environments. This Review addresses factors that need to be considered when implementing assays for HTS and is aimed particularly at investigators new to this field. We discuss assay design strategies, the major detection technologies and examples of HTS assays for common target classes, cellular pathways and simple cellular phenotypes. We conclude with special considerations for configuring sensitive, robust, informative and economically feasible HTS assays.     

Automated multiple ligand screening by frontal affinity chromatography-mass spectrometry (FAC-MS)

High-throughput screening (HTS) efforts to discover "hits" typically rely on the large-scale parallel screening of individual compounds with attempts to screen mixtures of compounds typically and, unfortunately, giving rise to false positives and false negatives due to the nature of the HTS readout (% inhibition/activation above a defined threshold) that makes deconvolution virtually intractable. Bioaffinity screening methods have emerged as an alternative or orthogonal method to classic HTS. One of these methods, frontal affinity chromatography coupled to mass spectrometry detection (FAC-MS), although still a relatively new technique, is turning out to be a viable screening tool. However, to push FAC-MS more to the forefront as a moderate primary HTS system (or a secondary screening assay), automation needs to be addressed. An automated FAC-MS system is described using 2 columns containing immobilized hERbeta, whereby while 1 column is being regenerated, the other is being used. The authors are extrapolating that in a continuous 24-h operation, the number of ligands screened could potentially approach 10,000. In addition, preliminary structure-activity relationship binding information (typically not seen in early primary HTS) can be obtained by observing the rank order of the library members in the various mixtures.     

Virtual screening to enrich hit lists from high-throughput screening: a case study on small-molecule inhibitors of angiogenin

"Hit lists" generated by high-throughput screening (HTS) typically contain a large percentage of false positives, making follow-up assays necessary to distinguish active from inactive substances. Here we present a method for improving the accuracy of HTS hit lists by computationally based virtual screening (VS) of the corresponding chemical libraries and selecting hits by HTS/VS consensus. This approach was applied in a case study on the target-enzyme angiogenin, a potent inducer of angiogenesis. In conjunction with HTS of the National Cancer Institute Diversity Set and ChemBridge DIVERSet E (approximately 18,000 compounds total), VS was performed with two flexible library docking/scoring methods, DockVision/Ludi and GOLD. Analysis of the results reveals that dramatic enrichment of the HTS hit rate can be achieved by selecting compounds in consensus with one or both of the VS functions. For example, HTS hits ranked in the top 2% by GOLD included 42% of the true hits, but only 8% of the false positives; this represents a sixfold enrichment over the HTS hit rate. Notably, the HTS/VS method was effective in selecting out inhibitors with midmicromolar dissociation constants typical of leads commonly obtained in primary screens.     

Design and Implementation of High Throughput Screening Assays

High throughput screening (HTS) is at the core of the drug discovery process, and so it is critical to design and implement HTS assays in a comprehensive fashion involving scientists from the disciplines of biology, chemistry, engineering, and informatics. This requires careful analysis of many variables, starting with the choice of assay target and ending with the discovery of lead compounds. At every step in this process, there are decisions to be made that can greatly impact the outcome of the HTS effort, to the point of making it a success or a failure. Although specific guidelines should be established to insure that the screening assay reaches an acceptable level of quality, many choices require pragmatism and the ability to compromise opposing forces.