Overcoming High Nanopore Basecaller Error Rates for DNA Storage via Basecaller-Decoder Integration and Convolutional Codes

Chandak, Shubham; Neu, Joachim; Tatwawadi, Kedar; Mardia, Jay; Lau, Billy T.; Kubit, Matthew; Hulett, Reyna; Griffin, P.B.; Wootters, Mary; Weissman, Tsachy; Ji, Hanlee P.

doi:10.1109/icassp40776.2020.9053441

Cited by 35 publications

(25 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As JPEG formats are highly sensitive to errors, they result in poor-quality reconstructions if one does not use a coding overhead that guarantees exact reconstruction. Furthermore, alternative methods based on joint source-channel coding 15,40,41 still require introducing error-control redundancy which we are aiming to dispose of in our learning-based approach. To demonstrate this point, we performed extensive simulations with six combinations of JPEG image compression qualities and matching error-control coding schemes.…”

Section: Sequence Dimension Decoding and Post-processingmentioning

confidence: 99%

Rewritable Two-Dimensional DNA-Based Data Storage with Machine Learning Reconstruction

Pan

Tabatabaei

Yazdi

et al. 2021

Preprint

View full text Add to dashboard Cite

DNA-based data storage platforms traditionally encode information only in the nucleotide sequence of the molecule. Here, we report on a two-dimensional molecular data storage system that records information in both the sequence and the backbone structure of DNA. Our "2DDNA" method efficiently stores high-density images in synthetic DNA and embeds metadata as nicks in the DNA backbone. To avoid costly redundancy used to combat sequencing errors and missing information content that typically requires additional synthesis, specialized machine learning methods are developed for automatic discoloration detection and image inpainting. The 2DDNA platform is experimentally tested on a library of images that show undetectable visual degradation after processing, while the image metadata is erased and rewritten to modify copyright information. Our results show that DNA can serve both as a write-once and rewritable memory for heterogenous data. Moreover, the storage density of the molecules can be increased by using different encoding dimensions and avoiding error-correction redundancy.

show abstract

Section: Sequence Dimension Decoding and Post-processingmentioning

confidence: 99%

Rewritable Two-Dimensional DNA-Based Data Storage with Machine Learning Reconstruction

Pan

Tabatabaei

Yazdi

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…They have observed a 3x improvement in reading cost at the same writing cost. They generated a read/write trade-off similar to Illumina-based DNA storage in their proposed algorithm [31].…”

Section: Chapter 2 Background and Literature Reviewmentioning

confidence: 99%

“…And Nanonet uses Bidirectional RNN with Long Short-Term Memory units [66] basecallers can be integrated and used in DeepSimulator. We have integrated Flappie basecaller for simulator validation since the real nanopore signals and basecalled sequences for comparison are from [31] where the research group used flappie basecaller. Moreover, a 10.5% error rate similar to a real nanopore signal is maintained.…”

Section: Base Callersmentioning

confidence: 99%

“…Chandak.et.al [31] designed coding methodology with the outer code as Reed Solomon and inner code as Convolutional code. The encoded data consisting of varied number of sequences in each experiment and are mapped to oligonucleotides.…”

Section: Detection Of Distributionmentioning

confidence: 99%

“…On doing that, the visual representation of the signal is similar as shown in Figures 23 and 24. [31] and correspondingly four original output nanopore signals for validation. Each of the input sequences was processed in DeepSimulator, producing one simulated output per sequence.…”

Section: Validation Of the Proposed Simulatormentioning

confidence: 99%

See 2 more Smart Citations

Reconstruction of encoded data in DNA storage technology

Subhasiny¹

View full text Add to dashboard Cite

Inevitable biomolecular errors in DNA storage technology could be resolved by designing robust error correction codes or intelligent clustering/decoding algorithms. The first objective of our work is to reconstruct the encoded DNA sequences read from the Illumina sequencer before decoding by studying the efficiencies of the existing clustering tools in the biological domain and then modifying, tuning, and analyzing their applicability in the DNA data storage domain. The investigated tools and algorithms include Starcode, Cooperative Sequence Clustering, Majority nucleotide selection algorithm, Slidesort, and MeShClust. We observed and compared them, Starcode, Majority nucleotide selection algorithm and Cooperative Sequence Clustering yields the highest recovery rate with less sequencing redundancy for three datasets. The benefit of portability using nanopore-based storage leads to the second objective of designing a Nanopore based DNA storage simulator that can serve as a tool for evaluating coding and clustering techniques. We simulated the DNA channel and subsampling of sequenced data using the non-parametric subsampling method by studying the distribution of real nanopore DNA storage data and then integrated it with DeepSimulator. The design is evaluated for its accuracy by comparing it with real nanopore reads. Besides, nanopore reads obtained from the designed simulator are clustered and representatives in each cluster are extracted for reconstructing the encoded data.

show abstract

Rewritable two-dimensional DNA-based data storage with machine learning reconstruction

et al. 2022

View full text Add to dashboard Cite

DNA-based data storage platforms traditionally encode information only in the nucleotide sequence of the molecule. Here we report on a two-dimensional molecular data storage system that records information in both the sequence and the backbone structure of DNA and performs nontrivial joint data encoding, decoding and processing. Our 2DDNA method efficiently stores images in synthetic DNA and embeds pertinent metadata as nicks in the DNA backbone. To avoid costly worst-case redundancy for correcting sequencing/rewriting errors and to mitigate issues associated with mismatched decoding parameters, we develop machine learning techniques for automatic discoloration detection and image inpainting. The 2DDNA platform is experimentally tested by reconstructing a library of images with undetectable or small visual degradation after readout processing, and by erasing and rewriting copyright metadata encoded in nicks. Our results demonstrate that DNA can serve both as a write-once and rewritable memory for heterogenous data and that data can be erased in a permanent, privacy-preserving manner. Moreover, the storage system can be made robust to degrading channel qualities while avoiding global error-correction redundancy.

show abstract

Overcoming High Nanopore Basecaller Error Rates for DNA Storage via Basecaller-Decoder Integration and Convolutional Codes

Cited by 35 publications

References 30 publications

Rewritable Two-Dimensional DNA-Based Data Storage with Machine Learning Reconstruction

Rewritable Two-Dimensional DNA-Based Data Storage with Machine Learning Reconstruction

Reconstruction of encoded data in DNA storage technology

Rewritable two-dimensional DNA-based data storage with machine learning reconstruction

Contact Info

Product

Resources

About