Shepherd: accurate clustering for correcting DNA barcode errors

Abstract

Motivation DNA barcodes are short, random nucleotide sequences introduced into cell populations to track the relative counts of hundreds of thousands of individual lineages over time. Lineage tracking is widely applied, e.g. to understand evolutionary dynamics in microbial populations and the progression of breast cancer in humans. Barcode sequences are unknown upon insertion and must be identified using next-generation sequencing technology, which is error prone. In this study, we frame the barcode error correction task as a clustering problem with the aim to identify true barcode sequences from noisy sequencing data. We present Shepherd, a novel clustering method that is based on an indexing system of barcode sequences using k-mers, and a Bayesian statistical test incorporating a substitution error rate to distinguish true from error sequences. Results When benchmarking with synthetic data, Shepherd provides barcode count estimates that are significantly more accurate than state-of-the-art methods, producing 10-150 times fewer spurious lineages. For empirical data, Shepherd produces results that are consistent with the improvements seen on synthetic data. These improvements enable higher resolution lineage tracking and more accurate estimates of biologically relevant quantities, e.g. the detection of small effect mutations. Availability A Python implementation of Shepherd is freely available at: https://www.github.com/Nik-Tavakolian/Shepherd. Supplementary information Supplementary data are available at Bioinformatics online.

Best Practices in Designing, Sequencing, and Identifying Random DNA Barcodes

“…Moreover, some errors may be sequence-specific (see “ Structure of the Barcode Locus ” section), such that the naive approach may produce biased lineage frequency estimates. Fortunately, a number of error-correction techniques are available (e.g., Li and Godzik 2006 ; Edgar 2010 , 2016 ; Ghodsi et al 2011 ; James et al 2018 ; Wei et al 2021 ; Dasari and Bhukya 2022 ; Millán Arias et al 2022 ), some of which were developed specifically for barcode data (e.g., Zorita et al 2015 ; Zhao et al 2018 ; Tavakolian et al 2022 ).…”

“…We selected six error-correction software, two developed for generic sequence data, DNAClust (Ghodsi et al 2011 ) and CD-Hit (Li and Godzik 2006 ), and four developed specifically for barcode data, Bartender (Zhao et al 2018 ), Starcode (Zorita et al 2015 ), Shepherd (Tavakolian et al 2022 ), and “Deletion-Correct,” a modified version of the algorithm used in (Johnson et al 2019 ). We tested their accuracy by performing error correction on a dataset of simulated barcode reads with realistic errors (Methods, “ Comparison of Error Correction Methods ” section).…”

Best practices in designing, sequencing and identifying random DNA barcodes

“…Moreover, some errors may be sequence-specific (see Section 2.1), such that the naive approach may produce biased lineage frequency estimates. Fortunately, a number of error-correction techniques are available (e.g., (Li & Godzik, 2006;Edgar, 2010Edgar, , 2016Ghodsi et al, 2011;James et al, 2018;Wei et al, 2021;Dasari & Bhukya, 2022;Millán Arias et al, 2022)), some of which were developed specifically for barcode data (e.g., (Zorita et al, 2015;Zhao et al, 2018;Tavakolian et al, 2022)).…”

“…We selected six error-correction software, two developed for generic sequence data, DNAClust (Ghodsi et al, 2011) and CD-Hit (Li & Godzik, 2006), and four developed specifically for barcode data, Bartender (Zhao et al, 2018), Starcode (Zorita et al, 2015), Shepherd (Tavakolian et al, 2022) and "Deletion-Correct", a modified version of the algorithm used in Johnson et al (2019). We first tested their accuracy by performing error correction on a dataset of simulated barcode reads with realistic errors (Methods).…”