iBench: A ground truth approach for advanced validation of mass spectrometry identification method

The discovery of many noncanonical peptides detectable with sensitive mass spectrometry inside, outside, and on cells shepherded the development of novel methods for their identification, often not supported by a systematic benchmarking with other methods. We here propose iBench, a bioinformatic tool that can construct ground truth proteomics datasets and cognate databases, thereby generating a training court wherein methods, search engines, and proteomics strategies can be tested, and their performances estimated by the same tool. iBench can be coupled to the main database search engines, allows the selection of customized features of mass spectrometry spectra and peptides, provides standard benchmarking outputs, and is open source. The proof-of-concept application to tryptic proteome digestions, immunopeptidomes, and synthetic peptide libraries dissected the impact that noncanonical peptides could have on the identification of canonical peptides by Mascot search with rescoring via Percolator (Mascot+Percolator).     

Improving sensitivity in proteome studies by analysis of false discovery rates for multiple search engines

LC‐MS experiments can generate large quantities of data, for which a variety of database search engines are available to make peptide and protein identifications. Decoy databases are becoming widely used to place statistical confidence in result sets, allowing the false discovery rate (FDR) to be estimated. Different search engines produce different identification sets so employing more than one search engine could result in an increased number of peptides (and proteins) being identified, if an appropriate mechanism for combining data can be defined. We have developed a search engine independent score, based on FDR, which allows peptide identifications from different search engines to be combined, called the FDR Score. The results demonstrate that the observed FDR is significantly different when analysing the set of identifications made by all three search engines, by each pair of search engines or by a single search engine. Our algorithm assigns identifications to groups according to the set of search engines that have made the identification, and re‐assigns the score (combined FDR Score). The combined FDR Score can differentiate between correct and incorrect peptide identifications with high accuracy, allowing on average 35% more peptide identifications to be made at a fixed FDR than using a single search engine.     

FDRAnalysis: a tool for the integrated analysis of tandem mass spectrometry identification results from multiple search engines

Confident identification of peptides via tandem mass spectrometry underpins modern high-throughput proteomics. This has motivated considerable recent interest in the postprocessing of search engine results to increase confidence and calculate robust statistical measures, for example through the use of decoy databases to calculate false discovery rates (FDR). FDR-based analyses allow for multiple testing and can assign a single confidence value for both sets and individual peptide spectrum matches (PSMs). We recently developed an algorithm for combining the results from multiple search engines, integrating FDRs for sets of PSMs made by different search engine combinations. Here we describe a web-server and a downloadable application that makes this routinely available to the proteomics community. The web server offers a range of outputs including informative graphics to assess the confidence of the PSMs and any potential biases. The underlying pipeline also provides a basic protein inference step, integrating PSMs into protein ambiguity groups where peptides can be matched to more than one protein. Importantly, we have also implemented full support for the mzIdentML data standard, recently released by the Proteomics Standards Initiative, providing users with the ability to convert native formats to mzIdentML files, which are available to download.     

Implementing Data Standards: A report on the HUPOPSI Workshop September 2009, Toronto,Canada

The plenary session of the Proteomics Standards Initiative of the Human Proteome Organisation at the 8th Annual HUPO World Congress updated the delegates on the current status of the ongoing work of this group. The mass spectrometry group reviewed the progress of mzML since its release last year and detailed new work on providing a common format for SRM/MRM transition lists (TraML). The implementation of mzIdentML, for describing the output of proteomics search engines, was outlined and the release of a new web service PSICQUIC, which allows users to simultaneously search multiple interaction databases, was announced. Finally, the audience participated in a lively debate, discussing both the benefits of these standard formats and issues with their adoption and use in a research environment.     

OVNIp: An open source application facilitating the interpretation,the validation and the edition of proteomics data generated by MS analyses and de novo sequencing

Several academic software are available to help the validation and reporting of proteomics data generated by MS analyses. However, to our knowledge, none of them have been conceived to meet the particular needs generated by the study of organisms whose genomes are not sequenced. In that context, we have developed OVNIp, an open‐source application which facilitates the whole process of proteomics results interpretation. One of its unique attributes is its capacity to compile multiple results (from several search engines and/or several databank searches) with a resolution of conflicting interpretations. Moreover, OVNIp enables automated exploitation of de novo sequences generated from unassigned MS/MS spectra leading to higher sequence coverage and enhancing confidence in the identified proteins. The exploitation of these additional spectra might also identify novel proteins through a MS‐BLAST search, which can be easily ran from the OVNIp interface. Beyond this primary scope, OVNIp can also benefit to users who look for a simple standalone application to both visualize and confirm MS/MS result interpretations through a simple graphical interface and generate reports according to user‐defined forms which may integrate the prerequisites for publication. Sources, documentation and a stable release for Windows are available at http://wwwappli.nantes.inra.fr:8180/OVNIp .     

SearchGUI: An open-source graphical user interface for simultaneous OMSSA and X!Tandem searches

The identification of proteins by mass spectrometry is a standard technique in the field of proteomics, relying on search engines to perform the identifications of the acquired spectra. Here, we present a user-friendly, lightweight and open-source graphical user interface called SearchGUI (http://searchgui.googlecode.com), for configuring and running the freely available OMSSA (open mass spectrometry search algorithm) and X!Tandem search engines simultaneously. Freely available under the permissible Apache2 license, SearchGUI is supported on Windows, Linux and OSX.     

Comparison of search engine contributions in protein mass fingerprinting for protein identification

Peptide mass fingerprinting (PMF) is a valuable method for rapid and high-throughput protein identification using the proteomics approach. Automated search engines, such as Ms-Fit, Mascot, ProFound, and Peptldent, have facilitated protein identification through PMF. The potential to obtain a true MS protein identification result depends on the choice of algorithm as well as experimental factors that influence the information content in MS data. When mass spectral data are incomplete and/or have low mass accuracy, the “number of matches” approach may be inadequate for a useful identification. Several studies have evaluated factors influencing the quality of mass spectrometry (MS) experiments. Missed cleavages, posttranslational modifications of peptides and contaminants (e.g., keratin) are important factors that can affect the results of MS analyses by influencing the identification process as well as the quality of the MS spectra. We compared search engines frequently used to identify proteins fromHomo sapiens andHalobacterium salinarum by evaluating factors, including data-based and mass tolerance to develop an improved search engine for PMF. This study may provide information to help develop a more effective algorithm for protein identification in each species through PMF.     

PAS‐cal: A repetitive peptide sequence calibration standard for MALDI mass spectrometry

Mass spectrometers equipped with matrix‐assisted laser desorption/ionization (MALDI‐MS) require frequent multipoint calibration to obtain good mass accuracy over a wide mass range and across large numbers of samples. In this study, we introduce a new synthetic peptide mass calibration standard termed PAS‐cal tailored for MALDI‐MS based bottom‐up proteomics. This standard consists of 30 peptides between 8 and 37 amino acids long and each constructed to contain repetitive sequences of Pro, Ala and Ser as well as one C‐terminal arginine residue. MALDI spectra thus cover a mass range between 750 and 3200 m/z in MS mode and between 100 and 3200 m/z in MS/MS mode. Our results show that multipoint calibration of MS spectra using PAS‐cal peptides compares well to current commercial reagents for protein identification by PMF. Calibration of tandem mass spectra from LC‐MALDI experiments using the longest peptide, PAS‐cal37, resulted in smaller fragment ion mass errors, more matching fragment ions and more protein and peptide identifications compared to commercial standards, making the PAS‐cal standard generically useful for bottom‐up proteomics.     

Pepitome: evaluating improved spectral library search for identification complementarity and quality assessment

Spectral libraries have emerged as a viable alternative to protein sequence databases for peptide identification. These libraries contain previously detected peptide sequences and their corresponding tandem mass spectra (MS/MS). Search engines can then identify peptides by comparing experimental MS/MS scans to those in the library. Many of these algorithms employ the dot product score for measuring the quality of a spectrum-spectrum match (SSM). This scoring system does not offer a clear statistical interpretation and ignores fragment ion m/z discrepancies in the scoring. We developed a new spectral library search engine, Pepitome, which employs statistical systems for scoring SSMs. Pepitome outperformed the leading library search tool, SpectraST, when analyzing data sets acquired on three different mass spectrometry platforms. We characterized the reliability of spectral library searches by confirming shotgun proteomics identifications through RNA-Seq data. Applying spectral library and database searches on the same sample revealed their complementary nature. Pepitome identifications enabled the automation of quality analysis and quality control (QA/QC) for shotgun proteomics data acquisition pipelines.     

Algorithms for the de novo sequencing of peptides from tandem mass spectra

Proteomics is the study of proteins, their time- and location-dependent expression profiles, as well as their modifications and interactions. Mass spectrometry is useful to investigate many of the questions asked in proteomics. Database search methods are typically employed to identify proteins from complex mixtures. However, databases are not often available or, despite their availability, some sequences are not readily found therein. To overcome this problem, de novo sequencing can be used to directly assign a peptide sequence to a tandem mass spectrometry spectrum. Many algorithms have been proposed for de novo sequencing and a selection of them are detailed in this article. Although a standard accuracy measure has not been agreed upon in the field, relative algorithm performance is discussed. The current state of the de novo sequencing is assessed thereafter and, finally, examples are used to construct possible future perspectives of the field.     

BuildSummary: using a group-based approach to improve the sensitivity of peptide/protein identification in shotgun proteomics

The target-decoy database search strategy is widely accepted as a standard method for estimating the false discovery rate (FDR) of peptide identification, based on which peptide-spectrum matches (PSMs) from the target database are filtered. To improve the sensitivity of protein identification given a fixed accuracy (frequently defined by a protein FDR threshold), a postprocessing procedure is often used that integrates results from different peptide search engines that had assayed the same data set. In this work, we show that PSMs that are grouped by the precursor charge, the number of missed internal cleavage sites, the modification state, and the numbers of protease termini and that the proteins grouped by their unique peptide count should be filtered separately according to the given FDR. We also develop an iterative procedure to filter the PSMs and proteins simultaneously, according to the given FDR. Finally, we present a general framework to integrate the results from different peptide search engines using the same FDR threshold. Our method was tested with several shotgun proteomics data sets that were acquired by multiple LC/MS instruments from two different biological samples. The results showed a satisfactory performance. We implemented the method in a user-friendly software package called BuildSummary, which can be downloaded for free from http://www.proteomics.ac.cn/software/proteomicstools/index.htm as part of the software suite ProteomicsTools.     

Sociomics: CESAGen Multidisciplinary Workshop on the Transformation of Knowledge Production in the Biosciences, and its Consequences. 23-24 July, 2007 Wellcome Trust Genome Campus, Hinxton, UK

On 23 and 24 July, 2007, the Centre for Economic and Social Aspects of Genomics (CESAGen) held its first sociomics workshop at the Wellcome Trust Genome Campus, Hinxton, UK. The topic was transformation of knowledge production. Participants included social scientists together with those working on different elements of the proteomics knowledge production-line, including core facilities, data repositories, large-scale projects, MS, search engines, reference databases, standardisation and public funding. Recurrent motifs included gear-heads, black boxes, uncertainty and getting back to biology.     

Proteomics‐based discovery of biomarkers for paediatric acute lymphoblastic leukaemia: challenges and opportunities

There are important breakthroughs in the treatment of paediatric acute lymphoblastic leukaemia (ALL) since 1950, by which the prognosis of the child majority suffered from ALL has been improved. However, there are urgent needs to have disease‐specific biomarkers to monitor the therapeutic efficacy and predict the patient prognosis. The present study overviewed proteomics‐based research on paediatric ALL to discuss important advances to combat cancer cells and search novel and real protein biomarkers of resistance or sensitivity to drugs which target the signalling networks. We highlighted the importance and significance of a proper phospho‐quantitative design and strategy for paediatric ALL between relapse and remission, when human body fluids from cerebrospinal, peripheral blood, or bone‐marrow were applied. The present article also assessed the schedule for the analysis of body fluids from patients at different states, importance of proteomics‐based tools to discover ALL‐specific and sensitive biomarkers, to stimulate paediatric ALL research via proteomics to ‘build’ the reference map of the signalling networks from leukemic cells at relapse, and to monitor significant clinical therapies for ALL‐relapse.     

Identification of dominant signaling pathways from proteomics expression data

The availability of the results of high-throughput analyses coming from ‘omic’ technologies has been one of the major driving forces of pathway biology. Analytical pathway biology strives to design a ‘pathway search engine’, where the input is the ‘omic’ data and the output is the list of activated or dominant pathways in a given sample. Here we describe the first attempt to design and validate such a pathway search engine using as input expression proteomics data. The engine represents a specific workflow in computational tools developed originally for mRNA analysis (BMC Bioinformatics 2006, 7 (Suppl 2), S13). Using our own datasets as well as data from recent proteomics literature we demonstrate that different dominant pathways (EGF, TGF_β, stress, and Fas pathways) can be correctly identified even from limited datasets. Pathway search engines can find application in a variety of proteomics-related fields, from fundamental molecular biology to search for novel types of disease biomarkers.     

A bioinformatics workflow for variant peptide detection in shotgun proteomics

Shotgun proteomics data analysis usually relies on database search. However, commonly used protein sequence databases do not contain information on protein variants and thus prevent variant peptides and proteins from been identified. Including known coding variations into protein sequence databases could help alleviate this problem. Based on our recently published human Cancer Proteome Variation Database, we have created a protein sequence database that comprehensively annotates thousands of cancer-related coding variants collected in the Cancer Proteome Variation Database as well as noncancer-specific ones from the Single Nucleotide Polymorphism Database (dbSNP). Using this database, we then developed a data analysis workflow for variant peptide identification in shotgun proteomics. The high risk of false positive variant identifications was addressed by a modified false discovery rate estimation method. Analysis of colorectal cancer cell lines SW480, RKO, and HCT-116 revealed a total of 81 peptides that contain either noncancer-specific or cancer-related variations. Twenty-three out of 26 variants randomly selected from the 81 were confirmed by genomic sequencing. We further applied the workflow on data sets from three individual colorectal tumor specimens. A total of 204 distinct variant peptides were detected, and five carried known cancer-related mutations. Each individual showed a specific pattern of cancer-related mutations, suggesting potential use of this type of information for personalized medicine. Compatibility of the workflow has been tested with four popular database search engines including Sequest, Mascot, X!Tandem, and MyriMatch. In summary, we have developed a workflow that effectively uses existing genomic data to enable variant peptide detection in proteomics.     

An easy-to-use Decoy Database Builder software tool, implementing different decoy strategies for false discovery rate calculation in automated MS/MS protein identifications

One of the major challenges for large scale proteomics research is the quality evaluation of results. Protein identification from complex biological samples or experimental setups is often a manual and subjective task which lacks profound statistical evaluation. This is not feasible for high-throughput proteomic experiments which result in large datasets of thousands of peptides and proteins and their corresponding mass spectra. To improve the quality, reliability and comparability of scientific results, an estimation of the rate of erroneously identified proteins is advisable. Moreover, scientific journals increasingly stipulate that articles containing considerable MS data should be subject to stringent statistical evaluation. We present a newly developed easy-to-use software tool enabling quality evaluation by generating composite target-decoy databases usable with all relevant protein search engines. This tool, when used in conjunction with relevant statistical quality criteria, enables to reliably determine peptides and proteins of high quality, even for nonexperienced users (e.g. laboratory staff, researchers without programming knowledge). Different strategies for building decoy databases are implemented and the resulting databases are characterized and compared. The quality of protein identification in high-throughput proteomics is usually measured by the false positive rate (FPR), but it is shown that the false discovery rate (FDR) delivers a more meaningful, robust and comparable value.     

A collection of open source applications for mass spectrometry data mining

We present several bioinformatics applications for the identification and quantification of phosphoproteome components by MS. These applications include a front‐end graphical user interface that combines several Thermo RAW formats to MASCOT? Generic Format extractors (EasierMgf), two graphical user interfaces for search engines OMSSA and SEQUEST (OmssaGui and SequestGui), and three applications, one for the management of databases in FASTA format (FastaTools), another for the integration of search results from up to three search engines (Integrator), and another one for the visualization of mass spectra and their corresponding database search results (JsonVisor). These applications were developed to solve some of the common problems found in proteomic and phosphoproteomic data analysis and were integrated in the workflow for data processing and feeding on our LymPHOS database. Applications were designed modularly and can be used standalone. These tools are written in Perl and Python programming languages and are supported on Windows platforms. They are all released under an Open Source Software license and can be freely downloaded from our software repository hosted at GoogleCode.     

Scavager: A Versatile Postsearch Validation Algorithm for Shotgun Proteomics Based on Gradient Boosting

Shotgun proteomics workflows for database protein identification typically include a combination of search engines and postsearch validation software based mostly on machine learning algorithms. Here, a new postsearch validation tool called Scavager employing CatBoost, an open‐source gradient boosting library, which shows improved efficiency compared with the other popular algorithms, such as Percolator, PeptideProphet, and Q‐ranker, is presented. The comparison is done using multiple data sets and search engines, including MSGF+, MSFragger, X!Tandem, Comet, and recently introduced IdentiPy. Implemented in Python programming language, Scavager is open‐source and freely available at https://bitbucket.org/markmipt/scavager .