iBench: A ground truth approach for advanced validation of mass spectrometry identification method

Abstract

The discovery of many noncanonical peptides detectable with sensitive mass spectrometry inside, outside, and on cells shepherded the development of novel methods for their identification, often not supported by a systematic benchmarking with other methods. We here propose iBench, a bioinformatic tool that can construct ground truth proteomics datasets and cognate databases, thereby generating a training court wherein methods, search engines, and proteomics strategies can be tested, and their performances estimated by the same tool. iBench can be coupled to the main database search engines, allows the selection of customized features of mass spectrometry spectra and peptides, provides standard benchmarking outputs, and is open source. The proof‐of‐concept application to tryptic proteome digestions, immunopeptidomes, and synthetic peptide libraries dissected the impact that noncanonical peptides could have on the identification of canonical peptides by Mascot search with rescoring via Percolator (Mascot+Percolator).

Protein degradation by human 20S proteasomes elucidates the interplay between peptide hydrolysis and splicing

“…We then compared DN1 and DN5 to inSPIRE 1.5 using iBench 2.0, which is the latest update of our iBench benchmarking software 100 . iBench 2.0 takes high confidence peptide identifications from previously measured MS data and: (i) generates a constructed dataset containing only the PSMs originally assigned and (ii) embeds the peptide sequences in an in silico-only proteome as spliced or non-spliced peptides, which is used as constructed reference database.…”

“…As a further test of inSPIRE 1.5 performance in spliced peptide identification, we generated a spliced peptide-free ground-truth dataset by applying iBench 2.0 to a large dataset of synthetic peptides 72 , 100 , 101 . The iBench 2.0 constructed a reference database represented by a protein sequence through the concatenation of 390 overlapping synthetic peptide sequences (Supplementary Fig.…”

inSPIRE: An Open-Source Tool for Increased Mass Spectrometry Identification Rates Using Prosit Spectral Prediction

“…The ground truth dataset construction followed the approach described by Cormican et al. ( 54 ) and is explained in the Experimental Procedures section. The pipelines’ benchmarking on a ground truth dataset containing PSMs with characteristics similar to HLA-I immunopeptidomes could let us estimate the precision—that is, number of correctly identified peptides over the number of identified peptides—and recall—that is number of correctly identified peptides over the number of correct peptides—of a given method.…”

“…As a validation of the increased number of PSMs for the inSPIRE pipeline, we benchmarked all rescoring methods using “ground truth” datasets, in line with the benchmarking tool iBench (QSB lab) ( 54 ). In the approach applied in this study, we measured synthetic peptides via MS and selected MS2 scans that were identified with 1% FDR using PEAKS search engine.…”

“…Our MS proteomics data have been deposited to the ProteomeXchange Consortium via the PRIDE ( 58 ) partner repository with the dataset identifiers PXD031709 ( 13 ), PXD031812 ( 54 ), and PXD034056.…”