Lerna: transformer architectures for configuring error correction tools for short- and long-read genome sequencing

Abstract

Background Sequencing technologies are prone to errors, making error correction (EC) necessary for downstream applications. EC tools need to be manually configured for optimal performance. We find that the optimal parameters (e.g., k-mer size) are both tool- and dataset-dependent. Moreover, evaluating the performance (i.e., Alignment-rate or Gain) of a given tool usually relies on a reference genome, but quality reference genomes are not always available. We introduce Lerna for the automated configuration of k-mer-based EC tools. Lerna first creates a language model (LM) of the uncorrected genomic reads, and then, based on this LM, calculates a metric called the perplexity metric to evaluate the corrected reads for different parameter choices. Next, it finds the one that produces the highest alignment rate without using a reference genome. The fundamental intuition of our approach is that the perplexity metric is inversely correlated with the quality of the assembly after error correction. Therefore, Lerna leverages the perplexity metric for automated tuning of k-mer sizes without needing a reference genome. Results First, we show that the best k-mer value can vary for different datasets, even for the same EC tool. This motivates our design that automates k-mer size selection without using a reference genome. Second, we show the gains of our LM using its component attention-based transformers. We show the model’s estimation of the perplexity metric before and after error correction. The lower the perplexity after correction, the better the k-mer size. We also show that the alignment rate and assembly quality computed for the corrected reads are strongly negatively correlated with the perplexity, enabling the automated selection of k-mer values for better error correction, and hence, improved assembly quality. We validate our approach on both short and long reads. Additionally, we show that our attention-based models have significant runtime improvement for the entire pipeline—18$$\times$$ × faster than previous works, due to parallelizing the attention mechanism and the use of JIT compilation for GPU inferencing. Conclusion Lerna improves de novo genome assembly by optimizing EC tools. Our code is made available in a public repository at: https://github.com/icanforce/lerna-genomics.

CARE 2.0: reducing false-positive sequencing error corrections using machine learning

“…The random forests hereby replace our hand-crafted conditions to decide whether a specific position in a read should be modified. This is in contrast to previous recent machine learning approaches like Athena [ 18 ] and Lerna [ 19 ] which try to find optimal input parameters for existing correction algorithms. Third, the algorithm has been optimized to reduce both runtime and memory consumption on both CPUs and GPUs.…”

MetaTransformer: deep metagenomic sequencing read classification using self-attention models

“…Compared to k -mer-based approaches the performance is drastically lower. Naturally, with longer substrings, the vocabulay size increases allowing for more detailed pattern recognition ( 28 ).…”

“…Taxonomic classification tools are often based on alignment-free mapping approaches enabling the analysis of millions of reads with a high accuracy. Deep learning has shown its potential in a variety of problems on sequence-based data ( 12 , 13 , 28–30 ). Therefore, new classification approaches were explored using deep learning with good results on genus and species prediction even on non-curated databases ( 16 ).…”

“…Lerna ( 28 ), a transformer-based model on short- and long-read data for parameter tuning of error correction tools, generates language models of the uncorrected reads and builds a perplexity metric out of this model. Using the perplexity metric, the corrected reads can be evaluated without the need of reference genomes.…”

MAC-ErrorReads: machine learning-assisted classifier for filtering erroneous NGS reads

“…This information and factors such as quality scores and genomic coverage contribute to the formulation of features used in training machine learning models. Furthermore, other machine learning methods specialize in identifying the optimal k -mer size essential for independent error correction tools [ 31 , 32 ] (see Table 1 ).…”