Gencore: an efficient tool to generate consensus reads for error suppressing and duplicate removing of NGS data

Abstract

BackgroundRemoving duplicates might be considered as a well-resolved problem in next-generation sequencing (NGS) data processing domain. However, as NGS technology gains more recognition in clinical application, researchers start to pay more attention to its sequencing errors, and prefer to remove these errors while performing deduplication operations. Recently, a new technology called unique molecular identifier (UMI) has been developed to better identify sequencing reads derived from different DNA fragments. Most existing duplicate removing tools cannot handle the UMI-integrated data. Some modern tools can work with UMIs, but are usually slow and use too much memory. Furthermore, existing tools rarely report rich statistical results, which are very important for quality control and downstream analysis. These unmet requirements drove us to develop an ultra-fast, simple, little-weighted but powerful tool for duplicate removing and sequence error suppressing, with features of handling UMIs and reporting informative results.ResultsThis paper presents an efficient tool gencore for duplicate removing and sequence error suppressing of NGS data. This tool clusters the mapped sequencing reads and merges reads in each cluster to generate one single consensus read. While the consensus read is generated, the random errors introduced by library construction and sequencing can be removed. This error-suppressing feature makes gencore very suitable for the application of detecting ultra-low frequency mutations from deep sequencing data. When unique molecular identifier (UMI) technology is applied, gencore can use them to identify the reads derived from same original DNA fragment. Gencore reports statistical results in both HTML and JSON formats. The HTML format report contains many interactive figures plotting statistical coverage and duplication information. The JSON format report contains all the statistical results, and is interpretable for downstream programs.ConclusionsComparing to the conventional tools like Picard and SAMtools, gencore greatly reduces the output data’s mapping mismatches, which are mostly caused by errors. Comparing to some new tools like UMI-Reducer and UMI-tools, gencore runs much faster, uses less memory, generates better consensus reads and provides simpler interfaces. To our best knowledge, gencore is the only duplicate removing tool that generates both informative HTML and JSON reports. This tool is available at: https://github.com/OpenGene/gencore

Comprehensive molecular profiling of intrahepatic cholangiocarcinoma in the Chinese population and therapeutic experience

“…FASTP [9] was used to trim adapters and remove low-quality sequences to obtain clean reads, which were aligned to the Ensemble GRCh37/ hg19 reference genome by BWA [10]. PCR duplicates were processed by GenCore [11], and consensus reads were generated. SAMtools [12] was utilized for the detection of single-nucleotide variations (SNVs), insertions and deletions, and Human Genome Variation Society (HGVS) variant descriptions were annotated by ANNO-VAR [13] software.…”

Comprehensive molecular profiling of intrahepatic cholangiocarcinoma in the Chinese population and therapeutic experience

“…FASTP 7 was used to trim adapters and remove low-quality sequences to obtain clean reads, which were aligned to the Ensemble GRCh37/hg19 reference genome by BWA 8 . PCR duplicates were processed by GenCore 9 , and consensus reads were generated. SAMtools 10 was utilized for the detection of single-nucleotide variations (SNVs), insertions and deletions, and Human Genome Variation Society (HGVS) variant descriptions were annotated by ANNOVAR 11 software.…”

Adaptor Template Oligo-Mediated Sequencing (ATOM-Seq): A versatile and ultra-sensitive UMI-based NGS library preparation technology, for use with cfDNA and cfRNA

“…Initially, adaptors were trimmed from reads, 20 bp UMIs were extracted from the beginning of read 2 into read headers, and inserts below The trimmed reads were then mapped to the HG38 reference genome using BWA v0.7.15 [22] with default settings. Mapped reads were passed to gencore v0.15.0 [23] for consensus read generation and error correction using default settings (80% consensus to generate a consensus base) with a minimum family size of two. Consensus reads were parsed by a custom python script which removed any reads not mapping to a primer site and trimmed the primer sequence from those reads which did.…”