A robust linear regression based algorithm for automated evaluation of peptide identifications from shotgun proteomics by use of reversed-phase liquid chromatography retention time

Background  

Rejection of false positive peptide matches in database searches of shotgun proteomic experimental data is highly desirable. Several methods have been developed to use the peptide retention time as to refine and improve peptide identifications from database search algorithms. This report describes the implementation of an automated approach to reduce false positives and validate peptide matches.     

A Dynamic Noise Level Algorithm for Spectral Screening of Peptide MS/MS Spectra

Background  

High-throughput shotgun proteomics data contain a significant number of spectra from non-peptide ions or spectra of too poor quality to obtain highly confident peptide identifications. These spectra cannot be identified with any positive peptide matches in some database search programs or are identified with false positives in others. Removing these spectra can improve the database search results and lower computational expense.     

Rank-invariant resampling based estimation of false discovery rate for analysis of small sample microarray data

Background  

The evaluation of statistical significance has become a critical process in identifying differentially expressed genes in microarray studies. Classical p-value adjustment methods for multiple comparisons such as family-wise error rate (FWER) have been found to be too conservative in analyzing large-screening microarray data, and the False Discovery Rate (FDR), the expected proportion of false positives among all positives, has been recently suggested as an alternative for controlling false positives. Several statistical approaches have been used to estimate and control FDR, but these may not provide reliable FDR estimation when applied to microarray data sets with a small number of replicates.     

A New Method for Assessing How Sensitivity and Specificity of Linkage Studies Affects Estimation

Background

While the importance of record linkage is widely recognised, few studies have attempted to quantify how linkage errors may have impacted on their own findings and outcomes. Even where authors of linkage studies have attempted to estimate sensitivity and specificity based on subjects with known status, the effects of false negatives and positives on event rates and estimates of effect are not often described.

Methods

We present quantification of the effect of sensitivity and specificity of the linkage process on event rates and incidence, as well as the resultant effect on relative risks. Formulae to estimate the true number of events and estimated relative risk adjusted for given linkage sensitivity and specificity are then derived and applied to data from a prisoner mortality study. The implications of false positive and false negative matches are also discussed.

Discussion

Comparisons of the effect of sensitivity and specificity on incidence and relative risks indicate that it is more important for linkages to be highly specific than sensitive, particularly if true incidence rates are low. We would recommend that, where possible, some quantitative estimates of the sensitivity and specificity of the linkage process be performed, allowing the effect of these quantities on observed results to be assessed.     

Improving the yeast two-hybrid system with permutated fusions proteins: the Varicella Zoster Virus interactome

Background  

Yeast two-hybrid (Y2H) screens have been among the most powerful methods to detect and analyze protein-protein interactions. However, they suffer from a significant degree of false negatives, i.e. true interactions that are not detected, and to a certain degree from false positives, i.e. interactions that appear to take place only in the context of the Y2H assay. While the fraction of false positives remains difficult to estimate, the fraction of false negatives in typical Y2H screens is on the order of 70-90%. Here we present novel Y2H vectors that significantly decrease the number of false negatives and help to mitigate the false positive problem.     

Dissecting systems-wide data using mixture models: application to identify affected cellular processes

Background  

Functional analysis of data from genome-scale experiments, such as microarrays, requires an extensive selection of differentially expressed genes. Under many conditions, the proportion of differentially expressed genes is considerable, making the selection criteria a balance between the inclusion of false positives and the exclusion of false negatives.     

Frequency of False Positive Rapid HIV Serologic Tests in African Men and Women Receiving PrEP for HIV Prevention: Implications for Programmatic Roll-Out of Biomedical Interventions

Background

Rapid HIV assays are the mainstay of HIV testing globally. Delivery of effective biomedical HIV prevention strategies such as antiretroviral pre-exposure prophylaxis (PrEP) requires periodic HIV testing. Because rapid tests have high (>95%) but imperfect specificity, they are expected to generate some false positive results.

Methods

We assessed the frequency of true and false positive rapid results in the Partners PrEP Study, a randomized, placebo-controlled trial of PrEP. HIV testing was performed monthly using 2 rapid tests done in parallel with HIV enzyme immunoassay (EIA) confirmation following all positive rapid tests.

Results

A total of 99,009 monthly HIV tests were performed; 98,743 (99.7%) were dual-rapid HIV negative. Of the 266 visits with ≥1 positive rapid result, 99 (37.2%) had confirmatory positive EIA results (true positives), 155 (58.3%) had negative EIA results (false positives), and 12 (4.5%) had discordant EIA results. In the active PrEP arms, over two-thirds of visits with positive rapid test results were false positive results (69.2%, 110 of 159), although false positive results occurred at <1% (110/65,945) of total visits.

Conclusions

When HIV prevalence or incidence is low due to effective HIV prevention interventions, rapid HIV tests result in a high number of false relative to true positive results, although the absolute number of false results will be low. Program roll-out for effective interventions should plan for quality assurance of HIV testing, mechanisms for confirmatory HIV testing, and counseling strategies for persons with positive rapid test results.     

A structural study for the optimisation of functional motifs encoded in protein sequences

Background  

A large number of PROSITE patterns select false positives and/or miss known true positives. It is possible that – at least in some cases – the weak specificity and/or sensitivity of a pattern is due to the fact that one, or maybe more, functional and/or structural key residues are not represented in the pattern. Multiple sequence alignments are commonly used to build functional sequence patterns. If residues structurally conserved in proteins sharing a function cannot be aligned in a multiple sequence alignment, they are likely to be missed in a standard pattern construction procedure.     

A novel approach to denoising ion trap tandem mass spectra

Background  

Mass spectrometers can produce a large number of tandem mass spectra. They are unfortunately noise-contaminated. Noises can affect the quality of tandem mass spectra and thus increase the false positives and false negatives in the peptide identification. Therefore, it is appealing to develop an approach to denoising tandem mass spectra.     

Information assessment on predicting protein-protein interactions

Background  

Identifying protein-protein interactions is fundamental for understanding the molecular machinery of the cell. Proteome-wide studies of protein-protein interactions are of significant value, but the high-throughput experimental technologies suffer from high rates of both false positive and false negative predictions. In addition to high-throughput experimental data, many diverse types of genomic data can help predict protein-protein interactions, such as mRNA expression, localization, essentiality, and functional annotation. Evaluations of the information contributions from different evidences help to establish more parsimonious models with comparable or better prediction accuracy, and to obtain biological insights of the relationships between protein-protein interactions and other genomic information.     

PPIRank - an advanced method for ranking protein-protein interations in TAP/MS data

Background

Tandem affinity purification coupled with mass-spectrometry (TAP/MS) analysis is a popular method for the identification of novel endogenous protein-protein interactions (PPIs) in large-scale. Computational analysis of TAP/MS data is a critical step, particularly for high-throughput datasets, yet it remains challenging due to the noisy nature of TAP/MS data.

Results

We investigated several major TAP/MS data analysis methods for identifying PPIs, and developed an advanced method, which incorporates an improved statistical method to filter out false positives from the negative controls. Our method is named PPIRank that stands for PPI rank ing in TAP/MS data. We compared PPIRank with several other existing methods in analyzing two pathway-specific TAP/MS PPI datasets from Drosophila.

Conclusion

Experimental results show that PPIRank is more capable than other approaches in terms of identifying known interactions collected in the BioGRID PPI database. Specifically, PPIRank is able to capture more true interactions and simultaneously less false positives in both Insulin and Hippo pathways of Drosophila Melanogaster.

     

Effects of filtering by Present call on analysis of microarray experiments

Background

Affymetrix GeneChips^® are widely used for expression profiling of tens of thousands of genes. The large number of comparisons can lead to false positives. Various methods have been used to reduce false positives, but they have rarely been compared or quantitatively evaluated. Here we describe and evaluate a simple method that uses the detection (Present/Absent) call generated by the Affymetrix microarray suite version 5 software (MAS5) to remove data that is not reliably detected before further analysis, and compare this with filtering by expression level. We explore the effects of various thresholds for removing data in experiments of different size (from 3 to 10 arrays per treatment), as well as their relative power to detect significant differences in expression.

Results

Our approach sets a threshold for the fraction of arrays called Present in at least one treatment group. This method removes a large percentage of probe sets called Absent before carrying out the comparisons, while retaining most of the probe sets called Present. It preferentially retains the more significant probe sets (p ≤ 0.001) and those probe sets that are turned on or off, and improves the false discovery rate. Permutations to estimate false positives indicate that probe sets removed by the filter contribute a disproportionate number of false positives. Filtering by fraction Present is effective when applied to data generated either by the MAS5 algorithm or by other probe-level algorithms, for example RMA (robust multichip average). Experiment size greatly affects the ability to reproducibly detect significant differences, and also impacts the effect of filtering; smaller experiments (3–5 samples per treatment group) benefit from more restrictive filtering (≥50% Present).

Conclusion

Use of a threshold fraction of Present detection calls (derived by MAS5) provided a simple method that effectively eliminated from analysis probe sets that are unlikely to be reliable while preserving the most significant probe sets and those turned on or off; it thereby increased the ratio of true positives to false positives.     

CrossHybDetector: detection of cross-hybridization events in DNA microarray experiments

Background  

DNA microarrays contain thousands of different probe sequences represented on their surface. These are designed in such a way that potential cross-hybridization reactions with non-target sequences are minimized. However, given the large number of probes, the occurrence of cross hybridization events cannot be excluded. This problem can dramatically affect the data quality and cause false positive/false negative results.     

Implications of False Negative and False Positive Diagnosis in Lymph Node Staging of NSCLC by Means of 18F-FDG PET/CT

Background

Integrated ¹⁸F-fluorodeoxyglucose positron emission tomography/computed tomography (¹⁸F-FDG PET/CT) is widely performed in hilar and mediastinal lymph node (HMLN) staging of non-small cell lung cancer (NSCLC). However, the diagnostic efficiency of PET/CT remains controversial. This retrospective study is to evaluate the accuracy of PET/CT and the characteristics of false negatives and false positives to improve specificity and sensitivity.

Methods

219 NSCLC patients with systematic lymph node dissection or sampling underwent preoperative PET/CT scan. Nodal uptake with a maximum standardized uptake value (SUVmax) >2.5 was interpreted as PET/CT positive. The results of PET/CT were compared with the histopathological findings. The receiver operating characteristic (ROC) curve was generated to determine the diagnostic efficiency of PET/CT. Univariate and multivariate analysis were conducted to detect risk factors of false negatives and false positives.

Results

The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of PET/ CT in detecting HMLN metastases were 74.2% (49/66), 73.2% (112/153), 54.4% (49/90), 86.8% (112/129), and 73.5% (161/219). The ROC curve had an area under curve (AUC) of 0.791 (95% CI 0.723-0.860). The incidence of false negative HMLN metastases was 13.2% (17 of 129 patients). Factors that are significantly associated with false negatives are: concurrent lung disease or diabetes (p<0.001), non-adenocarcinoma (p<0.001), and SUVmax of primary tumor >4.0 (p=0.009). Postoperatively, 45.5% (41/90) patients were confirmed as false positive cases. The univariate analysis indicated age > 65 years old (p=0.009), well differentiation (p=0.002), and SUVmax of primary tumor ≦4.0 (p=0.007) as risk factors for false positive uptake.

Conclusion

The SUVmax of HMLN is a predictor of malignancy. Lymph node staging using PET/CT is far from equal to pathological staging account of some risk factors. This study may provide some aids to pre-therapy evaluation and decision-making.     

Predicting the sensitivity and specificity of published real-time PCR assays

Background

In recent years real-time PCR has become a leading technique for nucleic acid detection and quantification. These assays have the potential to greatly enhance efficiency in the clinical laboratory. Choice of primer and probe sequences is critical for accurate diagnosis in the clinic, yet current primer/probe signature design strategies are limited, and signature evaluation methods are lacking.

Methods

We assessed the quality of a signature by predicting the number of true positive, false positive and false negative hits against all available public sequence data. We found real-time PCR signatures described in recent literature and used a BLAST search based approach to collect all hits to the primer-probe combinations that should be amplified by real-time PCR chemistry. We then compared our hits with the sequences in the NCBI taxonomy tree that the signature was designed to detect.

Results

We found that many published signatures have high specificity (almost no false positives) but low sensitivity (high false negative rate). Where high sensitivity is needed, we offer a revised methodology for signature design which may designate that multiple signatures are required to detect all sequenced strains. We use this methodology to produce new signatures that are predicted to have higher sensitivity and specificity.

Conclusion

We show that current methods for real-time PCR assay design have unacceptably low sensitivities for most clinical applications. Additionally, as new sequence data becomes available, old assays must be reassessed and redesigned. A standard protocol for both generating and assessing the quality of these assays is therefore of great value. Real-time PCR has the capacity to greatly improve clinical diagnostics. The improved assay design and evaluation methods presented herein will expedite adoption of this technique in the clinical lab.     

Validation of an Automated Cough Detection Algorithm for Tracking Recovery of Pulmonary Tuberculosis Patients

Background

A laboratory-free test for assessing recovery from pulmonary tuberculosis (TB) would be extremely beneficial in regions of the world where laboratory facilities are lacking. Our hypothesis is that analysis of cough sound recordings may provide such a test. In the current paper, we present validation of a cough analysis tool.

Methodology/Principal Findings

Cough data was collected from a cohort of TB patients in Lima, Peru and 25.5 hours of recordings were manually annotated by clinical staff. Analysis software was developed and validated by comparison to manual scoring. Because many patients cough in bursts, coughing was characterized in terms of cough epochs. Our software correctly detects 75.5% of cough episodes with a specificity of 99.6% (comparable to past results using the same definition) and a median false positive rate of 4 false positives/hour, due to the noisy, real-world nature of our dataset. We then manually review detected coughs to eliminate false positives, in effect using the algorithm as a pre-screening tool that reduces reviewing time to roughly 5% of the recording length. This cough analysis approach provides a foundation to support larger-scale studies of coughing rates over time for TB patients undergoing treatment.     

HMM-ModE – Improved classification using profile hidden Markov models by optimising the discrimination threshold and modifying emission probabilities with negative training sequences

Background  

Profile Hidden Markov Models (HMM) are statistical representations of protein families derived from patterns of sequence conservation in multiple alignments and have been used in identifying remote homologues with considerable success. These conservation patterns arise from fold specific signals, shared across multiple families, and function specific signals unique to the families. The availability of sequences pre-classified according to their function permits the use of negative training sequences to improve the specificity of the HMM, both by optimizing the threshold cutoff and by modifying emission probabilities to minimize the influence of fold-specific signals. A protocol to generate family specific HMMs is described that first constructs a profile HMM from an alignment of the family's sequences and then uses this model to identify sequences belonging to other classes that score above the default threshold (false positives). Ten-fold cross validation is used to optimise the discrimination threshold score for the model. The advent of fast multiple alignment methods enables the use of the profile alignments to align the true and false positive sequences, and the resulting alignments are used to modify the emission probabilities in the original model.