Impact of lossy compression of nanopore raw signal data on basecalling and consensus accuracy

Abstract

Motivation: Nanopore sequencing provides a real-time and portable solution to genomic sequencing, with long reads enabling better assembly and structural variant discovery than second generation technologies. The nanopore sequencing process generates huge amounts of data in the form of raw current data, which must be compressed to enable efficient storage and transfer. Since the raw current data is inherently noisy, lossy compression has potential to significantly reduce space requirements without adversely impacting performance of downstream applications. Results: We study the impact of two state-of-the-art lossy time-series compressors applied to nanopore raw current data, evaluating the tradeoff between compressed size and basecalling/consensus accuracy. We test several basecallers and consensus tools on a variety of datasets at varying depths of coverage, and conclude that lossy compression can provide as much as 40-50% reduction in compressed size of raw signal data with negligible impact on basecalling and consensus accuracy. Availability: The code, data and supplementary results are available at https://github.com/ shubhamchandak94/lossy_compression_evaluation.

Beyond sequencing: machine learning algorithms extract biology hidden in Nanopore signal data

“…Besides improvements in accuracy, improvements in data handling will become central as the raw data are multifold larger than those obtained from short read data. Methods that improve compression of fast5 files and more space-efficient alternative file types for storing raw nanopore data are currently being developed [78,79] xi , and graphics processing unit acceleration is used routinely [80]. However, further improvements to reduce file sizes, standardizing file formats, and compute and memory-efficient algorithms will greatly reduce the barrier for larger-scale applications and adaptation.…”

NanoSpring: reference-free lossless compression of nanopore sequencing reads using an approximate assembly approach

“…Given the high sequencing depth, there is much redundancy to be exploited in the reads, and several specialized compressors like SPRING (Chandak et al, 2019) and PgRC (Kowalski and Grabowski, 2019) have been developed for this data. The typical approach used by these compressors is to efficiently build an with the advent of deep learning based basecallers which achieve median error rate close to 5% or better (Chandak et al, 2020), suggesting that a similar approximate assembly approach with some adaptations can be applied to nanopore sequencing reads.…”

“…On the other hand, nanopore reads are much longer (often over hundreds of thousands of bases long), and have a much higher error rate, including substitution, insertion, and deletion errors from the basecalling process that converts the raw current signal to the read sequences (Wick et al ., 2019). However, the error rate has fallen dramatically in the recent years with the advent of deep learning based basecallers which achieve median error rate close to 5% or better (Chandak et al ., 2020), suggesting that a similar approximate assembly approach with some adaptations can be applied to nanopore sequencing reads.…”