RiboReport - benchmarking tools for ribosome profiling-based identification of open reading frames in bacteria

Abstract

Small proteins encoded by short open reading frames (ORFs) with 50 codons or fewer are emerging as an important class of cellular macromolecules in diverse organisms. However, they often evade detection by proteomics or in silico methods. Ribosome profiling (Ribo-seq) has revealed widespread translation in genomic regions previously thought to be non-coding, driving the development of ORF detection tools using Ribo-seq data. However, only a handful of tools have been designed for bacteria, and these have not yet been systematically compared. Here, we aimed to identify tools that use Ribo-seq data to correctly determine the translational status of annotated bacterial ORFs and also discover novel translated regions with high sensitivity. To this end, we generated a large set of annotated ORFs from four diverse bacterial organisms, manually labeled for their translation status based on Ribo-seq data, which are available for future benchmarking studies. This set was used to investigate the predictive performance of seven Ribo-seq-based ORF detection tools (REPARATION_blast, DeepRibo, Ribo-TISH, PRICE, smORFer, ribotricer and SPECtre), as well as IRSOM, which uses coding potential and RNA-seq coverage only. DeepRibo and REPARATION_blast robustly predicted translated ORFs, including sORFs, with no significant difference for ORFs in close proximity to other genes versus stand-alone genes. However, no tool predicted a set of novel, experimentally verified sORFs with high sensitivity. Start codon predictions with smORFer show the value of initiation site profiling data to further improve the sensitivity of ORF prediction tools in bacteria. Overall, we find that bacterial tools perform well for sORF detection, although there is potential for improving their performance, applicability, usability and reproducibility.

Complementary Ribo-seq approaches map the translatome and provide a small protein census in the foodborne pathogenCampylobacter jejuni

“…For predictions using classical Ribo-seq, we used two published tools in our HRIBO pipeline: REPARATION and DeepRibo (Clauwaert et al, 2019; Gelhausen et al, 2021; Ndah et al, 2017). As these tools can generate long lists of candidates with little guidance for ranking and many are likely false positives (Gelhausen et al, 2022), we used our WB/MS-validated small proteins ( Figs. 1E & S2 ) as a ground-truth set to benchmark cutoffs for RNA-seq RPKM, TE, as well as the score generated by DeepRibo ( Fig.…”

“…Our results demonstrate that while using several Ribo-seq approaches and requiring signals in all datasets can increase confidence, more flexible criteria for sORF predictions might reveal additional, bona fide small proteins. Our approach also shows the utility of an experimentally validated sORF set to guide cutoffs to cope with high numbers of tool predictions (Gelhausen et al, 2022), and especially that manual inspection of Ribo-seq coverage and independent validation is an essential part of Ribo-seq.…”

“…Processing of cell pellets and isolation of monosomes and footprints. Processing of cell pellets was performed generally as described previously (Gelhausen et al, 2022;Venturini et al, 2020). Cm was omitted from lysis buffers for TIS and TTS experiments, but used at 1 mM for Ribo-seq(Cm).…”

“…Ribo-seq-based ORF prediction. ORFs were called from Ribo-seq libraries using two prediction tools included in HRIBO: DeepRibo (Clauwaert et al, 2019) and Reparation_blast, an adapted version of REPARATION (Ndah et al, 2017), which uses blast instead of usearch (https://github.com/RickGelhausen/REPARATION_blast) (Gelhausen et al, 2022). Predictions were generated from normal Ribo-seq data from all datasets ((Ribo-seq (Cm), control (no drug) libraries from the TIS/TTS experiments).…”

“…Manual curation of Ribo-seq/TIS/TTS-based sORF predictions. The 421 filtered sORF predictions were inspected based on coverage files for one replicate of each experiment and were inspected using criteria previously used for bacterial Ribo-seq data (Gelhausen et al, 2022). Coverage files were loaded together in IGB (Integrated Genome Browser (Freese et al, 2016), https://www.bioviz.org/) along with ORF, sRNA, and TSS annotations (Dugar et al, 2013;Gundogdu et al, 2007).…”