On the prediction of statistical parameters in high-throughput screening using resampling techniques

A severe drawback in the high-throughput screening (HTS) process is the unintentional (random) presence of false positives and negatives. Their rates depend, among others, on the screening process being applied and the target class. Although false positives can be sorted out in subsequent process steps, their occurrence can lead to increased project cost. More fundamentally, it is not possible to rescue false nonhits. In this article, we investigate the prediction of the primary hit rate, hit confirmation rate, and false-positive and false-negative rates. Results for approximately 2800 compounds are considered that are tested as a pilot screen ahead of the primary screening work. This pilot screen is done at several concentrations and in replicates. The rates are predicted as a function of the proposed hit threshold by having the replicates serve as each other's confirmers, and confidence limits to the prediction are attached by means of a resampling scheme. A comparison of the rates resulting from the resampling with the primary hit rate and the confirmation rates obtained during the screening campaign shows how accurate this method is. Hence, the "optimal" compound concentration for the screen as well as the optimal hit threshold corresponding to low false rates can be determined prior to starting the subsequent screening campaign.     

Hypothesis testing in high-throughput screening for drug discovery

Following the success of small-molecule high-throughput screening (HTS) in drug discovery, other large-scale screening techniques are currently revolutionizing the biological sciences. Powerful new statistical tools have been developed to analyze the vast amounts of data in DNA chip studies, but have not yet found their way into compound screening. In HTS, characterization of single-point hit lists is often done only in retrospect after the results of confirmation experiments are available. However, for prioritization, for optimal use of resources, for quality control, and for comparison of screens it would be extremely valuable to predict the rates of false positives and false negatives directly from the primary screening results. Making full use of the available information about compounds and controls contained in HTS results and replicated pilot runs, the Z score and from it the p value can be estimated for each measurement. Based on this consideration, we have applied the concept of p-value distribution analysis (PVDA), which was originally developed for gene expression studies, to HTS data. PVDA allowed prediction of all relevant error rates as well as the rate of true inactives, and excellent agreement with confirmation experiments was found.     

Automated multiple ligand screening by frontal affinity chromatography-mass spectrometry (FAC-MS)

High-throughput screening (HTS) efforts to discover "hits" typically rely on the large-scale parallel screening of individual compounds with attempts to screen mixtures of compounds typically and, unfortunately, giving rise to false positives and false negatives due to the nature of the HTS readout (% inhibition/activation above a defined threshold) that makes deconvolution virtually intractable. Bioaffinity screening methods have emerged as an alternative or orthogonal method to classic HTS. One of these methods, frontal affinity chromatography coupled to mass spectrometry detection (FAC-MS), although still a relatively new technique, is turning out to be a viable screening tool. However, to push FAC-MS more to the forefront as a moderate primary HTS system (or a secondary screening assay), automation needs to be addressed. An automated FAC-MS system is described using 2 columns containing immobilized hERbeta, whereby while 1 column is being regenerated, the other is being used. The authors are extrapolating that in a continuous 24-h operation, the number of ligands screened could potentially approach 10,000. In addition, preliminary structure-activity relationship binding information (typically not seen in early primary HTS) can be obtained by observing the rank order of the library members in the various mixtures.     

Identification of novel drug scaffolds for inhibition of SARS-CoV 3-Chymotrypsin-like protease using virtual and high-throughput screenings

We have used a combination of virtual screening (VS) and high-throughput screening (HTS) techniques to identify novel, non-peptidic small molecule inhibitors against human SARS-CoV 3CLpro. A structure-based VS approach integrating docking and pharmacophore based methods was employed to computationally screen 621,000 compounds from the ZINC library. The screening protocol was validated using known 3CLpro inhibitors and was optimized for speed, improved selectivity, and for accommodating receptor flexibility. Subsequently, a fluorescence-based enzymatic HTS assay was developed and optimized to experimentally screen approximately 41,000 compounds from four structurally diverse libraries chosen mainly based on the VS results. False positives from initial HTS hits were eliminated by a secondary orthogonal binding analysis using surface plasmon resonance (SPR). The campaign identified a reversible small molecule inhibitor exhibiting mixed-type inhibition with a K_i value of 11.1 μM. Together, these results validate our protocols as suitable approaches to screen virtual and chemical libraries, and the newly identified compound reported in our study represents a promising structural scaffold to pursue for further SARS-CoV 3CLpro inhibitor development.     

Identification of free fatty acid receptor 2 agonists using virtual screening

Structure- and ligand-based virtual-screening methods (docking, 2D- and 3D-similarity searching) were analyzed for their effectiveness in virtual screening against FFAR2. To evaluate the performance of these methods, retrospective virtual screening was performed. Statistical quality of the methods was evaluated by BEDROC and RIE. The results revealed that electrostatic similarity search protocol using EON (ET combo) outperformed all other protocols with outstanding enrichment of >95% in top 1% and 2% of the dataset with an AUC of 0.958. Interestingly, the hit lists that are obtained from different virtual-screening methods are generally highly complementary to hits found from electrostatic similarity searching. These results suggest that considering electrostatic similarity searching first increases the chance of identifying more (and more diverse) active compounds from a virtual-screening campaign. Accordingly, prospective virtual screening using electrostatic similarity searching was used to identify novel FFAR2 ligands. The discovered compounds provide new chemical matter starting points for the initiation of a medicinal chemistry campaign.     

Quantitative assessment of hit detection and confirmation in single and duplicate high-throughput screenings

The process of identifying active targets (hits) in high-throughput screening (HTS) usually involves 2 steps: first, removing or adjusting for systematic variation in the measurement process so that extreme values represent strong biological activity instead of systematic biases such as plate effect or edge effect and, second, choosing a meaningful cutoff on the calculated statistic to declare positive compounds. Both false-positive and false-negative errors are inevitable in this process. Common control or estimation of error rates is often based on an assumption of normal distribution of the noise. The error rates in hit detection, especially false-negative rates, are hard to verify because in most assays, only compounds selected in primary screening are followed up in confirmation experiments. In this article, the authors take advantage of a quantitative HTS experiment in which all compounds are tested 42 times over a wide range of 14 concentrations so true positives can be found through a dose-response curve. Using the activity status defined by dose curve, the authors analyzed the effect of various data-processing procedures on the sensitivity and specificity of hit detection, the control of error rate, and hit confirmation. A new summary score is proposed and demonstrated to perform well in hit detection and useful in confirmation rate estimation. In general, adjusting for positional effects is beneficial, but a robust test can prevent overadjustment. Error rates estimated based on normal assumption do not agree with actual error rates, for the tails of noise distribution deviate from normal distribution. However, false discovery rate based on empirically estimated null distribution is very close to observed false discovery proportion.     

Discovery of novel mGluR1 antagonists: a multistep virtual screening approach based on an SVM model and a pharmacophore hypothesis significantly increases the hit rate and enrichment factor

Development of glutamate non-competitive antagonists of mGluR1 (Metabotropic glutamate receptor subtype 1) has increasingly attracted much attention in recent years due to their potential therapeutic application for various nervous disorders. Since there is no crystal structure reported for mGluR1, ligand-based virtual screening (VS) methods, typically pharmacophore-based VS (PB-VS), are often used for the discovery of mGluR1 antagonists. Nevertheless, PB-VS usually suffers a lower hit rate and enrichment factor. In this investigation, we established a multistep ligand-based VS approach that is based on a support vector machine (SVM) classification model and a pharmacophore model. Performance evaluation of these methods in virtual screening against a large independent test set, M-MDDR, show that the multistep VS approach significantly increases the hit rate and enrichment factor compared with the individual SB-VS and PB-VS methods. The multistep VS approach was then used to screen several large chemical libraries including PubChem, Specs, and Enamine. Finally a total of 20 compounds were selected from the top ranking compounds, and shifted to the subsequent in vitro and in vivo studies, which results will be reported in the near future.     

Uncovering false positives on a virtual screening search for cruzain inhibitors

Some unexpected promiscuous inhibitors were observed in a virtual screening protocol applied to select cruzain inhibitors from the ZINC database. Physical-chemical and pharmacophore model filters were used to reduce the database size. The selected compounds were docked into the cruzain active site. Six hit compounds were tested as inhibitors. Although the compounds were designed to be nucleophilically attacked by the catalytic cysteine of cruzain, three of them showed typical promiscuous behavior, revealing that false positives are a prevalent concern in VS programs.     

Assay concordance between SPA and TR-FRET in high-throughput screening

High-throughput screening (HTS) of large chemical libraries has become the main source of new lead compounds for drug development. Several specialized detection technologies have been developed to facilitate the cost- and time-efficient screening of millions of compounds. However, concerns have been raised, claiming that different HTS technologies may produce different hits, thus limiting trust in the reliability of HTS data. This study was aimed to investigate the reliability of the authors most frequently used assay techniques: scintillation proximity assay (SPA) and homogeneous time-resolved fluorescence resonance energy transfer (TR-FRET). To investigate the data concordance between these 2 detection technologies, the authors screened a large subset of the Schering compound library consisting of 300,000 compounds for inhibitors of a nonreceptor tyrosine kinase. They chose to set up this study in realistic HTS scale to ensure statistical significance of the results. The findings clearly demonstrate that the choice of detection technology has no significant impact on hit finding, provided that assays are biochemically equivalent. Data concordance is up to 90%. The little differences in hit findings are caused by threshold setting but not by systematic differences between the technologies. The most significant difference between the compared techniques is that in the SPA format, more false-positive primary hits were obtained.     

A multifaceted approach to identify non-specific enzyme inhibition: Application to Mycobacterium tuberculosis shikimate kinase

Single dose high-throughput screening (HTS) followed by dose-response evaluations is a common strategy for the identification of initial hits for further development. Early identification and exclusion of false positives is a cost-saving and essential step in early drug discovery. One of the mechanisms of false positive compounds is the formation of aggregates in assays. This study evaluates the mechanism(s) of inhibition of a set of 14 compounds identified previously as actives in Mycobacterium tuberculosis (Mt) cell culture screening and in vitro actives in Mt shikimate kinase (MtSK) assay. Aggregation of hit compounds was characterized using multiple experimental methods, LC-MS, ¹HNMR, dynamic light scattering (DLS), transmission electron microscopy (TEM), and visual inspection after centrifugation for orthogonal confirmation. Our results suggest that the investigated compounds containing oxadiazole-amide and aminobenzothiazole moieties are false positive hits and non-specific inhibitors of MtSK through aggregate formation.     

Identifying actives from HTS data sets: practical approaches for the selection of an appropriate HTS data-processing method and quality control review

High-throughput screening (HTS) has achieved a dominant role in drug discovery over the past 2 decades. The goal of HTS is to identify active compounds (hits) by screening large numbers of diverse chemical compounds against selected targets and/or cellular phenotypes. The HTS process consists of multiple automated steps involving compound handling, liquid transfers, and assay signal capture, all of which unavoidably contribute to systematic variation in the screening data. The challenge is to distinguish biologically active compounds from assay variability. Traditional plate controls-based and non-controls-based statistical methods have been widely used for HTS data processing and active identification by both the pharmaceutical industry and academic sectors. More recently, improved robust statistical methods have been introduced, reducing the impact of systematic row/column effects in HTS data. To apply such robust methods effectively and properly, we need to understand their necessity and functionality. Data from 6 HTS case histories are presented to illustrate that robust statistical methods may sometimes be misleading and can result in more, rather than less, false positives or false negatives. In practice, no single method is the best hit detection method for every HTS data set. However, to aid the selection of the most appropriate HTS data-processing and active identification methods, the authors developed a 3-step statistical decision methodology. Step 1 is to determine the most appropriate HTS data-processing method and establish criteria for quality control review and active identification from 3-day assay signal window and DMSO validation tests. Step 2 is to perform a multilevel statistical and graphical review of the screening data to exclude data that fall outside the quality control criteria. Step 3 is to apply the established active criterion to the quality-assured data to identify the active compounds.     

Use of mixture distributions to deconvolute the behavior of "hits" and controls in high-throughput screening data

The stochastic nature of high-throughput screening (HTS) data indicates that information may be gleaned by applying statistical methods to HTS data. A foundation of parametric statistics is the study and elucidation of population distributions, which can be modeled using modern spreadsheet software. The methods and results described here use fundamental concepts of statistical population distributions analyzed using a spreadsheet to provide tools in a developing armamentarium for extracting information from HTS data. Specific examples using two HTS kinase assays are analyzed. The analyses use normal and gamma distributions, which combine to form mixture distributions. HTS data were found to be described well using such mixture distributions, and deconvolution of the mixtures to the constituent gamma and normal parts provided insight into how the assays performed. In particular, the proportion of hits confirmed was predicted from the original HTS data and used to assess screening assay performance. The analyses also provide a method for determining how hit thresholds--values used to separate active from inactive compounds--affect the proportion of compounds verified as active and how the threshold can be chosen to optimize the selection process.     

Probing the primary screening efficiency by multiple replicate testing: a quantitative analysis of hit confirmation and false screening results of a biochemical assay

Despite a large body of references on assay development, assay optimization, strategies, and methodologies for high-throughput screening (HTS), there have been few reports on investigations of the efficiency of primary screening in a systematic and quantitative manner for a typical HTS process. Recently, the authors investigated the primary hit comparison and the effect of measurement variability by screening a library of approximately 25,000 random compounds in multiple replicate tests in a nuclear receptor recruitment assay with 2 different assay detection technologies. In this report, we utilized these sets of multiple replicate screening data from a different perspective and conducted a systematic data analysis in order to gain some insights into the hit-finding efficiency of a typical primary screening process. Specifically, hit confirmation, false-positive (declaration) rates, and false-negative rates at different hit cutoff limits were explored and calculated from the 2 different assay formats. Results and analyses provided some quantitative estimation regarding the reliability and efficiency of the primary screening process. For the 2 assay formats tested in this report, the confirmation rate (activity repeated at or above a certain hit limit) was found to be 65% or above. It was also suggested that, at least in this case, applying some hit-selection strategies, it is possible to decrease the number of false-negative or false-positive hits without significantly increasing the efforts in primary screening.     

Minimizing false positives in kinase virtual screens

In spite of recent improvements in docking and scoring methods, high false-positive rates remain a common issue in structure-based virtual screening. In this study, the distinctive features of false positives in kinase virtual screens were investigated. A series of retrospective virtual screens on kinase targets was performed on specifically designed test sets, each combining true ligands and experimentally confirmed inactive compounds. A systematic analysis of the docking poses generated for the top-ranking compounds highlighted key aspects differentiating true hits from false positives. The most recurring feature in the poses of false positives was the absence of certain key interactions known to be required for kinase binding. A systematic analysis of 444 crystal structures of ligand-bound kinases showed that at least two hydrogen bonds between the ligand and the backbone protein atoms in the kinase hinge region are present in 90% of the complexes, with very little variability across targets. Closer inspection showed that when the two hydrogen bonds are present, one of three preferred hinge-binding motifs is involved in 96.5% of the cases. Less than 10% of the false positives satisfied these two criteria in the minimized docking poses generated by our standard protocol. Ligand conformational artifacts were also shown to contribute to the occurrence of false positives in a number of cases. Application of this knowledge in the form of docking constraints and post-processing filters provided consistent improvements in virtual screening performance on all systems. The false-positive rates were significantly reduced and the enrichment factors increased by an average of twofold. On the basis of these results, a generalized two-step protocol for virtual screening on kinase targets is suggested.     

A Combination of Receptor-Based Pharmacophore Modeling & QM Techniques for Identification of Human Chymase Inhibitors

Inhibition of chymase is likely to divulge therapeutic ways for the treatment of cardiovascular diseases, and fibrotic disorders. To find novel and potent chymase inhibitors and to provide a new idea for drug design, we used both ligand-based and structure-based methods to perform the virtual screening(VS) of commercially available databases. Different pharmacophore models generated from various crystal structures of enzyme may depict diverse inhibitor binding modes. Therefore, multiple pharmacophore-based approach is applied in this study. X-ray crystallographic data of chymase in complex with different inhibitors were used to generate four structure–based pharmacophore models. One ligand–based pharmacophore model was also developed from experimentally known inhibitors. After successful validation, all pharmacophore models were employed in database screening to retrieve hits with novel chemical scaffolds. Drug-like hit compounds were subjected to molecular docking using GOLD and AutoDock. Finally four structurally diverse compounds with high GOLD score and binding affinity for several crystal structures of chymase were selected as final hits. Identification of final hits by three different pharmacophore models necessitates the use of multiple pharmacophore-based approach in VS process. Quantum mechanical calculation is also conducted for analysis of electrostatic characteristics of compounds which illustrates their significant role in driving the inhibitor to adopt a suitable bioactive conformation oriented in the active site of enzyme. In general, this study is used as example to illustrate how multiple pharmacophore approach can be useful in identifying structurally diverse hits which may bind to all possible bioactive conformations available in the active site of enzyme. The strategy used in the current study could be appropriate to design drugs for other enzymes as well.     

Evaluation of the InteraX system technology in a high-throughput screening environment

The authors have developed a cell-based high-throughput screening (HTS)-compatible assay to measure EGFR dimerization using the InteraX enzyme complementation technology of Applied Biosystems. The cells contain 2 chimeric proteins with complementing deletion mutants of the beta galactosidase enzyme, each fused to the extracellular and transmembrane part of EGFR. On binding of EGF, EGF receptor dimerizes and an active beta galactosidase is built. The authors used this homogeneous 384-well assay to screen about 20,000 diverse compounds. From 2 independent primary screen runs 239 hits were identified. For run 1, a mean S/B ratio of 4.26 and a mean Z' factor of 0.74 were obtained, for run 2 a mean S/B ratio of 3.88 and a mean Z' factor of 0.71 were obtained. After hit confirmation, repeated 4 times, 112 hits remained with a confirmation rate of 48.9%. Thirty of the 112 could be identified as cytotoxic. Fifty-one of the remaining 82 compounds could be shown to be inhibitors of the beta galactosidase enzyme itself. In summary, 31 compounds remained as potential EGFR dimerization or EGF stimulation inhibitors. The authors conclude that the InteraX system technology is HTS capable and can detect small molecule inhibitors capable of inhibiting protein-protein interactions.     

High-throughput screening of a 100,000-compound library for inhibitors of influenza A virus (H3N2)

Using a highly reproducible and robust cell-based high-throughput screening (HTS) assay, the authors screened a 100,000-compound library at 14- and 114-microM compound concentration against influenza strain A/Udorn/72 (H3N2). The "hit" rates (>50% inhibition of the viral cytopathic effect) from the 14- and 114-microM screens were 0.022% and 0.38%, respectively. The hits were evaluated for their antiviral activity, cell toxicity, and selectivity in dose-response experiments. The screen at the lower concentration yielded 3 compounds, which displayed moderate activity (SI(50) = 10-49). Intriguingly, the screen at the higher concentration revealed several additional hits. Two of these hits were highly active with an SI(50) > 50. Time of addition experiments revealed 1 compound that inhibited early and 4 other compounds that inhibited late in the virus life cycle, suggesting they affect entry and replication, respectively. The active compounds represent several different classes of molecules such as carboxanilides, 1-benzoyl-3-arylthioureas, sulfonamides, and benzothiazinones, which have not been previously identified as having antiviral/anti-influenza activity.     

Post-high-throughput screening analysis: an empirical compound prioritization scheme

An empirical scheme to evaluate and prioritize screening hits from high-throughput screening (HTS) is proposed. Negative scores are given when chemotypes found in the HTS hits are present in annotated databases such as MDDR and WOMBAT or for testing positive in toxicity-related experiments reported in TOXNET. Positive scores were given for higher measured biological activities, for testing negative in toxicity-related literature, and for good overlap when profiled against drug-related properties. Particular emphasis is placed on estimating aqueous solubility to prioritize in vivo experiments. This empirical scheme is given as an illustration to assist the decision-making process in selecting chemotypes and individual compounds for further experimentation, when confronted with multiple hits from high-throughput experiments. The decision-making process is discussed for a set of G-protein coupled receptor antagonists and validated on a literature example for dihydrofolate reductase inhibition.     

Development of a scintillation proximity binding assay for high-throughput screening of hematopoietic prostaglandin D2 synthase

Prostaglandin D₂ synthase (PGDS) catalyzes the isomerization of prostaglandin H₂ (PGH₂) to prostaglandin D₂ (PGD₂). PGD₂ produced by hematopoietic prostaglandin D₂ synthase (H-PGDS) in mast cells and Th2 cells is proposed to be a mediator of allergic and inflammatory responses. Consequently, inhibitors of H-PGDS represent potential therapeutic agents for the treatment of inflammatory diseases such as asthma. Due to the instability of the PGDS substrate PGH₂, an in-vitro enzymatic assay is not feasible for large-scale screening of H-PGDS inhibitors. Herein, we report the development of a competition binding assay amenable to high-throughput screening (HTS) in a scintillation proximity assay (SPA) format. This assay was used to screen an in-house compound library of approximately 280,000 compounds for novel H-PGDS inhibitors. The hit rate of the H-PGDS primary screen was found to be 4%. This high hit rate suggests that the active site of H-PGDS can accommodate a large diversity of chemical scaffolds. For hit prioritization, these initial hits were rescreened at a lower concentration in SPA and tested in the LAD2 cell assay. 116 compounds were active in both assays with IC₅₀s ranging from 6 to 807 nM in SPA and 82 nM to 10 μM in the LAD2 cell assay.