Impact of lossy compression of nanopore raw signal data on basecalling and consensus accuracy

Abstract

Motivation Nanopore sequencing provides a real-time and portable solution to genomic sequencing, enabling better assembly, structural variant discovery and modified base detection than second generation technologies. The sequencing process generates a huge amount of data in the form of raw signal contained in fast5 files, which must be compressed to enable efficient storage and transfer. Since the raw data is inherently noisy, lossy compression has potential to significantly reduce space requirements without adversely impacting performance of downstream applications. Results We explore the use of lossy compression for nanopore raw data using two state-of-the-art lossy time-series compressors, and evaluate the tradeoff between compressed size and basecalling/consensus accuracy. We test several basecallers and consensus tools on a variety of datasets at varying depths of coverage, and conclude that lossy compression can provide 35–50% further reduction in compressed size of raw data over the state-of-the-art lossless compressor with negligible impact on basecalling accuracy (≲0.2% reduction) and consensus accuracy (≲0.002% reduction). In addition, we evaluate the impact of lossy compression on methylation calling accuracy and observe that this impact is minimal for similar reductions in compressed size, although further evaluation with improved benchmark datasets is required for reaching a definite conclusion. The results suggest the possibility of using lossy compression, potentially on the nanopore sequencing device itself, to achieve significant reductions in storage and transmission costs while preserving the accuracy of downstream applications. Availabilityand implementation The code is available at https://github.com/shubhamchandak94/lossy_compression_evaluation. Supplementary information Supplementary data are available at Bioinformatics online.

Towards population-scale long-read sequencing

“…Currently, incremental updates to the ONT base-calling algorithm regularly improve the read accuracy 49 , which suggests that repeating the base calling of older data is valuable. This reanalysis requires long-term storage of the fast5 files, which can be up to 1.5 TB for a single PromethION flow cell, although further compression is possible 50 . By contrast, the PacBio base-calling process is highly mature, and BAM files containing unaligned reads are produced directly from the sequencing machine.…”

Current Uses and Future Perspectives of Genomic Technologies in Clinical Microbiology

“…To date, one of the primary challenges associated with ONT's technology has been its relatively high basecalling error rate. Even with the recent introduction of basecallers based on deep learning algorithms, the error rate has decreased to a median value of approximately 5% [33]. Nanopore sequencing can be enhanced through the incorporation of complementary short-read data for error correction, particularly in regions with high GC content or repetitive sequences.…”

Reference-free lossless compression of nanopore sequencing reads using an approximate assembly approach

“…On the other hand, nanopore reads are much longer (often over hundreds of thousands of bases long), and have a much higher error rate, including substitution, insertion, and deletion errors from the basecalling process that converts the raw current signal to the read sequences 4 . However, the error rate has fallen dramatically in the recent years with the advent of deep learning based basecallers which achieve median error rate close to 5% or better 5 , suggesting that a similar approximate assembly approach with some adaptations can be applied to nanopore sequencing reads.…”