Evaluating Proteomics Imputation Methods with Improved Criteria

Abstract

Quantitative measurements produced by tandem mass spectrometry proteomics experiments typically contain a large proportion of missing values. Missing values hinder reproducibility, reduce statistical power, and make it difficult to compare across samples or experiments. Although many methods exist for imputing missing values, in practice, the most commonly used methods are among the worst performing. Furthermore, previous benchmarking studies have focused on relatively simple measurements of error such as the mean-squared error between imputed and held-out values. Here we evaluate the performance of commonly used imputation methods using three practical, “downstream-centric” criteria. These criteria measure the ability to identify differentially expressed peptides, generate new quantitative peptides, and improve the peptide lower limit of quantification. Our evaluation comprises several experiment types and acquisition strategies, including data-dependent and data-independent acquisition. We find that imputation does not necessarily improve the ability to identify differentially expressed peptides but that it can identify new quantitative peptides and improve the peptide lower limit of quantification. We find that MissForest is generally the best performing method per our downstream-centric criteria. We also argue that existing imputation methods do not properly account for the variance of peptide quantifications and highlight the need for methods that do.

Augmented Doubly Robust Post-Imputation Inference for Proteomic data

“…In the proteomic literature, one of the most commonly applied imputation procedures is a version of k -nearest neighbors (kNN). The standard kNN procedure in use involves imputing the missing values based on the mean of the k closest peptides (Harris et al, 2023). Close peptides are used instead of close samples because it is very difficult to define close samples when there is excessive missingness.…”

“…A substantial ongoing research effort is to search and experiment with numerous imputation methods to determine the optimal one, including sample matching methods (Stuart and Satija, 2019), matrix factorization methods (Hastie et al, 2015), deep learning methods (Yoon et al, 2018; Qiu et al, 2020; Du et al, 2022) and more (Wang et al, 2016; Chen et al, 2017). See Harris et al (2023); Välikangas et al (2018); Liu and Dongre (2021) for a comprehensive review.…”

FLiPPR: A Processor for Limited Proteolysis (LiP) Mass Spectrometry Data Sets Built on FragPipe

“…If one ion count is missing, a ratio is still calculated, but rather than treating the missing value as a zero (as commonly employed), it is dropped, and the ratio and P -value are calculated using one fewer replicate for either the test or control condition (Figure C, case ii ). This treats the single missing value as missing at random (MAR) and avoids artificially decreasing the calculated mean. , The maximum number of allowable missing values for an ion to be further considered can be modified by the user (default is one) and may be recommended in studies with greater than three replicates per condition.…”