Where have all the interactions gone? Estimating the coverage of two-hybrid protein interaction maps

Yeast two-hybrid screens are an important method for mapping pairwise physical interactions between proteins. The fraction of interactions detected in independent screens can be very small, and an outstanding challenge is to determine the reason for the low overlap. Low overlap can arise from either a high false-discovery rate (interaction sets have low overlap because each set is contaminated by a large number of stochastic false-positive interactions) or a high false-negative rate (interaction sets have low overlap because each misses many true interactions). We extend capture-recapture theory to provide the first unified model for false-positive and false-negative rates for two-hybrid screens. Analysis of yeast, worm, and fly data indicates that 25% to 45% of the reported interactions are likely false positives. Membrane proteins have higher false-discovery rates on average, and signal transduction proteins have lower rates. The overall false-negative rate ranges from 75% for worm to 90% for fly, which arises from a roughly 50% false-negative rate due to statistical undersampling and a 55% to 85% false-negative rate due to proteins that appear to be systematically lost from the assays. Finally, statistical model selection conclusively rejects the Erd?s-Rényi network model in favor of the power law model for yeast and the truncated power law for worm and fly degree distributions. Much as genome sequencing coverage estimates were essential for planning the human genome sequencing project, the coverage estimates developed here will be valuable for guiding future proteomic screens. All software and datasets are available in and , -, and -, and are also available from our Web site, http://www.baderzone.org.     

Probabilistic prediction of unknown metabolic and signal-transduction networks.

Regulatory networks provide control over complex cell behavior in all kingdoms of life. Here we describe a statistical model, based on representing proteins as collections of domains or motifs, which predicts unknown molecular interactions within these biological networks. Using known protein-protein interactions of Saccharomyces cerevisiae as training data, we were able to predict the links within this network with only 7% false-negative and 10% false-positive error rates. We also use Markov chain Monte Carlo simulation for the prediction of networks with maximum probability under our model. This model can be applied across species, where interaction data from one (or several) species can be used to infer interactions in another. In addition, the model is extensible and can be analogously applied to other molecular data (e.g., DNA sequences).     

Emission and transmission brain tomography.

Comparative emission and transmission brain tomograms were obtained in 209 patients to establish the diagnostic accuracy of a new emission tomographic scanner in detecting space-occupying disease in the brain. Concordant results were obtained in 169 patients (81%). Computed transmission tomography (transmission CT) yielded an overall rate of false-positive results of 0.48% and a false-negative rate of 6%. Emission CT yielded a false-positive rate of 0% and false-negative rates of 2.4% for malignant disease and 10% for vascular disease. The higher rate of false-negative results for vascular disease with emission CT occurs because transmission CT can detect old infarction. The rates of detection of recent vascular disease with emission and trnasmission CT are identical. Thus emission CT is highly sensitive in detecting space-occupying disease in the brain. It represents an ideal screening procedure.     

Vertebrate gene predictions and the problem of large genes

To find unknown protein-coding genes, annotation pipelines use a combination of ab initio gene prediction and similarity to experimentally confirmed genes or proteins. Here, we show that although the ab initio predictions have an intrinsically high false-positive rate, they also have a consistently low false-negative rate. The incorporation of similarity information is meant to reduce the false-positive rate, but in doing so it increases the false-negative rate. The crucial variable is gene size (including introns)--genes of the most extreme sizes, especially very large genes, are most likely to be incorrectly predicted.     

A new method with flexible and balanced control of false negatives and false positives for hit selection in RNA interference high-throughput screening assays

The z-score method and its variants for testing mean difference are commonly used for hit selection in high-throughput screening (HTS) assays. Strictly standardized mean difference (SSMD) offers a way to measure and classify the short interfering RNA (siRNA) effects. In this article, based on SSMD, the authors propose a new testing method for hit selection in RNA interference (RNAi) HTS assays. This SSMD-based method allows the differentiation between siRNAs with large and small effects on the assay output and maintains flexible and balanced control of both the false-negative rate, in which the siRNAs with strong effects are not selected as hits, and the restricted false-positive rate, in which the siRNAs with weak or no effects are selected as hits. This method directly addresses the size of siRNA effects represented by the strength of difference between an siRNA and a negative reference, whereas the classic z-score method and t-test of testing no mean difference address whether the mean of an siRNA is exactly the same as the mean of a negative reference. This method can readily control the false-negative rate, whereas it is nontrivial for the classic z-score method and t-test to control the false-negative rate. Therefore, theoretically, the SSMD-based method offers better control of the sizes of siRNA effects and the associated false-positive and false-negative rates than the commonly used z-score method and t-test for hit selection in HTS assays. The SSMD-based method should generally be applicable to any assay in which the end point is a difference in signal compared to a reference sample, including those for RNAi, receptor, enzyme, and cellular function.     

Simple duplex fecal PCR assay that allows identification of false-negative results in Helicobacter sp.-infected mice

We designed a simple and sensitive duplex polymerase chain reaction (PCR) assay for detection of false-negative results during routine Helicobacter sp. feces analysis. We took advantage of the various Lactobacillus species that form part of the normal intestinal flora of laboratory rodents to improve our PCR diagnostic assays. Using this one-step PCR assay, we were able to rule out false-negative results without the need of adding internal standard molecules. This is an important quality control for PCR diagnostic tests, since the presence of inhibitors in feces can prevent detection of Helicobacter infections using PCR analysis. Use of this Lactobacillus group-specific PCR assay can be extended to other feces tests used in mouse quality-control programs.     

Quantitative assessment of hit detection and confirmation in single and duplicate high-throughput screenings

The process of identifying active targets (hits) in high-throughput screening (HTS) usually involves 2 steps: first, removing or adjusting for systematic variation in the measurement process so that extreme values represent strong biological activity instead of systematic biases such as plate effect or edge effect and, second, choosing a meaningful cutoff on the calculated statistic to declare positive compounds. Both false-positive and false-negative errors are inevitable in this process. Common control or estimation of error rates is often based on an assumption of normal distribution of the noise. The error rates in hit detection, especially false-negative rates, are hard to verify because in most assays, only compounds selected in primary screening are followed up in confirmation experiments. In this article, the authors take advantage of a quantitative HTS experiment in which all compounds are tested 42 times over a wide range of 14 concentrations so true positives can be found through a dose-response curve. Using the activity status defined by dose curve, the authors analyzed the effect of various data-processing procedures on the sensitivity and specificity of hit detection, the control of error rate, and hit confirmation. A new summary score is proposed and demonstrated to perform well in hit detection and useful in confirmation rate estimation. In general, adjusting for positional effects is beneficial, but a robust test can prevent overadjustment. Error rates estimated based on normal assumption do not agree with actual error rates, for the tails of noise distribution deviate from normal distribution. However, false discovery rate based on empirically estimated null distribution is very close to observed false discovery proportion.     

A new MSPQC system for rapid detection of pathogens in clinical samples

A new multi-channel series piezoelectric quartz crystal (MSPQC) system for detection of pathogens in clinical sample was proposed. Some factors, which affect the detection of pathogens by using MSPQC, were all investigated. A total of 650 clinical samples were detected by MSPQC and compared with licensed BACTEC 9120 system (Becton Dickinson Diagnostic Instrument Systems, Sparks, MD, USA) simultaneously in the Third Xiangya Hospital of Central South University, China. When the incubation period was 5 days, two systems had similar detected results: the MSPQC system detected 123 growth of 650 (18.92%) bottles while the BACTEC 9120 detected 125 growth of 650 (19.23%) bottles. The MSPQC had 2 false-positive signals and 2 false-negative signals. However, BACTEC 9120 had 3 false-positive signals and 0 false-negative signals. Further identifications of bacteria were run by VITEK-2 (bioMérieux China Ltd.), 5% sheep blood trypticase soy agar (SBA) and chocolate agar (CA). Comparing with BACTEC 9120, MSPQC system possesses following advantages: shorter average detection time, less blood volume needed, less false-positive results and low cost. It can also provide information in real time. So MSPQC has a wonderful perspective in clinical application.     

Statistical approach to fine needle aspiration diagnosis of breast masses

A statistical algorithm was used for recursively partitioning a consecutive series of 37 benign and 69 malignant fine needle aspirates to produce a decision tree for diagnosing breast masses. Optimal separation between benign and malignant cytology was accomplished by evaluating clump characteristics when clumps were present and evaluating cell integrity when clumps were absent. The 1.5% false-negative and 9.7% false-positive rates obtained through this scheme are better than those reported for most series.     

Genome-Wide Association Study Heterogeneous Cohort Homogenization via Subject Weight Knock-Down

Population structure can be a source of both false-positive and false-negative findings in a genome-wide association study. This article proposes an approach that helps to reduce the false-positives. It consists of homogenizing the diseased/healthy phenotype ratio across the cohort, by decreasing the statistical weight of selected individuals. After homogenization, the cohort is statistically handled as if originating from a single well-mixed population. The method was applied to homogenize a Parkinson''s disease genome-wide association study cohort.     

Evaluation of a Semiautomated System for Direct Fluorescent Antibody Detection of Salmonellae

A semi-automatic system under development by Aerojet Medical and Biological Systems for the direct fluorescent antibody detection of salmonellae was evaluated with various food, feed, and environmental samples. All samples were simultaneously examined by Automated Bioassay System (ABS), manual direct fluorescent antibody procedures and cultural procedures. The ABS gave satisfactory results with the processed samples. It detected all of the culturally positive powdered egg and candy samples with no false negative results and gave only 6.6 and 5.3% false positive rates, respectively. With meatmeal samples the ABS failed to detect one culturally positive specimen that was also positive by manual fluorescent antibody and gave one (1.1%) false-positive result. A high rate of false-negative results was obtained by ABS on unprocessed samples of creek water, poultry, and sausage. Adding another enrichment step to the protocol reduced the false-negative rate considerably but severely increased the false-positive rate. The instruments worked reasonably well, but research is needed to improve enrichment procedures for samples to be processed by the system.     

Diagnosis of benign and malignant cervical specimens by multifiberoptic microspectrophometry and by flow cytometry

Two techniques for the automation of mass screening for cervical cancer were studied. Microspectrophotometry was tried first, using a novel multifiberoptic scanning system that measured the nuclear size and DNA content of cells in routine smears restained by the Feulgen technique. Specimen diagnoses were based on the percentages of cell types present, as determined by thresholds set for the two parameters. While this method gave good results in the automated detection of severe dysplasias and carcinomas, with only 3 of 72 cases misdiagnosed as negative (4.2%), it had a 22.9% false-positive rate (misdiagnosing 24 of 105 "benign" cases) and a 30.3% false-negative rate for adenocarcinomas (10 of 33 cases misclassified). The second approach involved flow cytometric measurements of specimens that were double stained for the assessment of both the DNA and RNA content, with the results analyzed by preset windows in a two-dimensional plane. This technique gave a 6.1% false-negative rate in 49 positive specimens and a 32.3% false-positive rate in 102 benign specimens, with an overall correct classification rate of 76.2%, including adenocarcinomas.     

Reliability of non-lethal surveillance methods for detecting ranavirus infection

Ranaviruses have been identified as the etiologic agent in many amphibian die-offs across the globe. Polymerase chain reaction (PCR) is commonly used to detect ranavirus infection in amphibian hosts, but the test results may vary between tissue samples obtained by lethal and non-lethal procedures. Testing liver samples for infection is a common lethal sampling technique to estimate ranavirus prevalence because the pathogen often targets this organ and the liver is easy to identify and collect. However, tail clips or swabs may be more practicable for ranavirus surveillance programs compared with collecting and euthanizing animals, especially for uncommon species. Using PCR results from liver samples for comparison, we defined false-positive test results as occurrences when a non-lethal technique indicated positive but the liver sample was negative. Similarly, we defined false-negative test results as occurrences when a non-lethal technique was negative but the liver sample was positive. Using these decision rules, we estimated false-negative and false-positive rates for tail clips and swabs. Our study was conducted in a controlled facility using American bullfrog Lithobates catesbeianus tadpoles; false-positive and false-negative rates were estimated after different periods of time following exposure to ranavirus. False-negative and false-positive rates were 20 and 6%, respectively, for tail samples, and 22 and 12%, respectively, for swabs. False-negative rates were constant over time, but false-positive rates decreased with post-exposure duration. Our results suggest that non-lethal sampling techniques can be useful for ranavirus surveillance, although the prevalence of infection may be underestimated when compared to results obtained with liver samples.     

Performance of the LARC classifier in clinical laboratories.

As part of the installation procedure of the LARC leukocyte differential classifier in a clinical laboratory, a 100-slide protocol is carried out to establish the performance of the classifier in the laboratory. The detailed make-up of this protocol and its relationship to key performance parameters for the leukocyte differential are described in detail. Data from the first ten of these protocols are presented which establish the (a) normal ranges, (b) reproducibility, (c) accuracy, (d) false-positive/false-negative rates for the detection of left shifts and (e) false-positive/false-negative rates for the detection of bloods with abnormal cells.     

Rapid detection of rectal cancer by gloved digital-scrape cytology

To evaluate the use of cytology for the rapid detection of rectal cancer, 39 suspected cases of carcinoma of the rectum and anal canal and 7 age- and sex-matched control cases were studied by the gloved digital-scrape method. The cytologic findings were then correlated with the histopathologic diagnoses. Of 30 evaluable cases with follow-up, a correct cytologic diagnosis was made in 28 cases (93.4%), with 1 false-negative diagnosis (3.3%) and 1 false-positive diagnosis (3.3%). Gloved digital-scrape cytology is very simple, quick and inexpensive and can be carried out an out-patient basis without any complications.     

Statistical measurements of accuracy in cervical cytology

Statistical measures that can be used to monitor the level of accuracy of cervical cytologic screening are examined: sensitivity, the false-negative rate, the interval cancer rate among women with negative reports and the predictive value of a positive test. Sensitivity and the false-negative rate are subject to biases and problems of determination that may make them less useful as measures of accuracy. The interval cancer rate and the positive predictive value may be better methods of assessing the frequency of serious abnormalities following negative cytologic reports and the accuracy of cytologic reports of serious abnormalities, respectively. It is important to recognize that no single measure of accuracy can adequately define a laboratory's performance. The use of statistical analyses can be invaluable in placing in context the criticisms of cervical cytology as an appropriate screening test for the prevention of cervical cancer; statistical analysis should be used as one component of quality control.     

Increasing sensitivity of Ca2+ spark detection in noisy images by application of a matched-filter object detection algorithm

Microscopic calcium (Ca²⁺) events (such as Ca²⁺ sparks) are an important area for study, as they help clarify the mechanism(s) underlying intracellular signaling. In the heart, Ca²⁺ sparks occur as a result of Ca²⁺ release from the sarcoendoplasmic reticulum, via ryanodine receptor channels. Measurement of Ca²⁺ spark properties can provide valuable information about the control of ryanodine receptor channel gating in situ, but requires high spatiotemporal resolution imaging, which produces noisy datasets that are problematic for spark detection. Automated detection algorithms may overcome visual detection bias, but missed and false-positive events can distort the distribution of measured Ca²⁺ spark properties. We present a sensitive and reliable method for the automated detection of Ca²⁺ sparks in datasets obtained using confocal line-scanning or total internal reflection fluorescence microscopy. This matched-filter detection algorithm (MFDA) employs a user-defined object, chosen to mimic Ca²⁺ spark properties, and the experimental dataset is searched for instances of the object. Detection certainty is provided by nonparametric statistical testing. The supplied codes can also refine the search object on the basis of those detected to further increase detection sensitivity. In comparison to a commonly used, intensity-thresholding algorithm, the MFDA is more sensitive and reliable, particularly at low signal/noise ratios. The MFDA can also be easily adapted to other signal-detection problems in noisy datasets.     

Probing the primary screening efficiency by multiple replicate testing: a quantitative analysis of hit confirmation and false screening results of a biochemical assay

Despite a large body of references on assay development, assay optimization, strategies, and methodologies for high-throughput screening (HTS), there have been few reports on investigations of the efficiency of primary screening in a systematic and quantitative manner for a typical HTS process. Recently, the authors investigated the primary hit comparison and the effect of measurement variability by screening a library of approximately 25,000 random compounds in multiple replicate tests in a nuclear receptor recruitment assay with 2 different assay detection technologies. In this report, we utilized these sets of multiple replicate screening data from a different perspective and conducted a systematic data analysis in order to gain some insights into the hit-finding efficiency of a typical primary screening process. Specifically, hit confirmation, false-positive (declaration) rates, and false-negative rates at different hit cutoff limits were explored and calculated from the 2 different assay formats. Results and analyses provided some quantitative estimation regarding the reliability and efficiency of the primary screening process. For the 2 assay formats tested in this report, the confirmation rate (activity repeated at or above a certain hit limit) was found to be 65% or above. It was also suggested that, at least in this case, applying some hit-selection strategies, it is possible to decrease the number of false-negative or false-positive hits without significantly increasing the efforts in primary screening.     

Field test and experimental use of CYBEST model 2 for practical gynecologic mass screening

CYBEST is an automated cytologic screening system for uterine cancer utilizing a pattern-recognition image-analysis system. The prototype was developed in 1972 following fundamental studies of feature extraction, feature evaluation using ambiguity differential functions and segmentation of cell and nuclear images. Model 2 was developed in 1974 with an improved mechanism and function. The parameters employed are nuclear size, nuclear optical density, N/C ratio and nuclear shape. The data of field tests using 220 samples containing three cases of dysplasia, 110 cases of carcinoma and 107 nonmalignant cases were as follows: two false-negative cases (1.8%), 13 false positives (12.1%) and one reject (0.9%). This system was experimentally tested for practical mass population screening with 1,829 cases including 17 atypical cases (four epidermoid carcinomas). The data were as follows: no false-negative cases and 581 false-positive cases (32.1%). Of the latter, 311 cases (17.2%) were pathologic samples, such as severe cervicitis, senile colpitis, Trichomonas infestation, etc., and the remaining 270 cases (14.9%) were within physiologic limits, corresponding to true false-positive samples.