An Evaluation Framework for Lossy Compression of Genome Sequencing Quality Values

Alberti, Claudio; Daniels, Noah M.; Hernáez, Mikel; Voges, Jan; Goldfeder, Rachel L.; Hernandez-Lopez, Ana A.; Mattavelli, Marco; Berger, Bonnie

doi:10.1109/dcc.2016.39

Cited by 13 publications

(15 citation statements)

References 12 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For this individual, the National Institute of Standards and Technology (NIST) released a consensus set of variants which we used for our analyses [24]. Note that similar analyses were conducted in other works [17,1,21]. The selected data sets are shown in Table 2.…”

Section: Resultsmentioning

confidence: 99%

A Two-Level Scheme for Quality Score Compression

Voges

Fotouhi

Ostermann

et al. 2018

Journal of Computational Biology

Self Cite

View full text Add to dashboard Cite

Previous studies on quality score compression can be classified into two main lines: lossy schemes and lossless schemes. Lossy schemes enable a better management of computational resources. Thus, in practice, and for preliminary analyses, bioinformaticians may prefer to work with a lossy quality score representation. However, the original quality scores might be required for a deeper analysis of the data. Hence, it might be necessary to keep them; in addition to lossy compression this requires lossless compression as well. We developed a space-efficient hierarchical representation of quality scores, QScomp, which allows the users to work with lossy quality scores in routine analysis, without sacrificing the capability of reaching the original quality scores when further investigations are required. Each quality score is represented by a tuple through a novel decomposition. The first and second dimensions of these tuples are separately compressed such that the first-level compression is a lossy scheme. The compressed information of the second dimension allows the users to extract the original quality scores. Experiments on real data reveal that the downstream analysis with the lossy part-spending only 0.49 bits per quality score on average-shows a competitive performance, and that the total space usage with the inclusion of the compressed second dimension is comparable to the performance of competing lossless schemes.

show abstract

Section: Resultsmentioning

confidence: 99%

A Two-Level Scheme for Quality Score Compression

Voges

Fotouhi

Ostermann

et al. 2018

Journal of Computational Biology

Self Cite

View full text Add to dashboard Cite

show abstract

“…When it comes to aligned data, an MPEG-G encoder could use a compression method comparable to that of DeeZ [8], which is able to compress a 437 GB H. Sapiens SAM file to about 63 GB, as compared to 75 GB by CRAM (Scramble) or 106 GB by BAM [8]. Regarding quantization of quality values, methods like QVZ [9] and CALQ [10] could be applied yielding overall compression gains of 10x over BAM, while preserving, or even improving, variant calling performance [11].…”

Section: Compression Capabilitiesmentioning

confidence: 99%

“…Due to their higher entropy and larger alphabet, quality values have proven more difficult to compress than the reads [16,11]. In addition, there is evidence that quality values are inherently noisy, and downstream applications that use them do so in varying heuristic manners.…”

Section: Compression Modes For Quality Valuesmentioning

confidence: 99%

“…In addition, there is evidence that quality values are inherently noisy, and downstream applications that use them do so in varying heuristic manners. As a result, quantization of quality values can not only significantly alleviate storage requirements but also provide variant-calling performance comparable-and sometimes superior-to the performance achieved using the uncompressed data (see [16,11] and references therein). This is possible with even 0.5 bits per quality score rather than the 3 bits needed for lossless compression, approximately.…”

Section: Compression Modes For Quality Valuesmentioning

confidence: 99%

See 1 more Smart Citation

An introduction to MPEG-G, the new ISO standard for genomic information representation

Albert

Paridaens²,

Voges³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

The MPEG-G standardization initiative is a coordinated international effort to specify a compressed data format that enables large scale genomic data to be processed, transported and shared. The standard consists of a set of specifications (i.e., a book) describing: i) a normative format syntax, and ii) a normative decoding process to retrieve the information coded in a compliant file or bitstream. Such decoding process enables the use of leading-edge compression technologies that have exhibited significant compression gains over currently used formats for storage of unaligned and aligned sequencing reads. Additionally, the standard provides a wealth of much needed functionality, such as selective access, data aggregation, application programming interfaces to the compressed data, standard interfaces to support data protection mechanisms, support for streaming and a procedure to assess the conformance of implementations. ISO/IEC is engaged in supporting the maintenance and availability of the standard specification, which guarantees the perenniality of applications using MPEG-G. Finally, the standard ensures interoperability and integration with existing genomic information processing pipelines by providing support for conversion from the FASTQ/SAM/BAM file formats.In this paper we provide an overview of the MPEG-G specification, with particular focus on the main advantages and novel functionality it offers. As the standard only specifies the decoding process, encoding performance, both in terms of speed and compression ratio, can vary depending on specific encoder implementations, and will likely improve during the lifetime of MPEG-G. Hence, the performance statistics provided here are only indicative baseline examples of the technologies included in the standard.

show abstract

“…In recent years several research groups have investigated methods to improve the effectiveness of compression technologies for the storage of high-throughput sequencing data. In particular, approaches to lossy or quasi-lossless compression of quality scores have received special attention [10][11][12][13], along with an interest to measure their impact in the calling of genomic variants [14,15], so far the sole downstream application tested for evaluation. In the context of gene expression (section 2) this work sets out to explore the effect of lossy compression of quality scores.…”

Section: Introductionmentioning

confidence: 99%

Lossy Compression of Quality Scores in Differential Gene Expression: A First Assessment and Impact Analysis

Hernandez-Lopez

Voges

Alberti

et al. 2018

2018 Data Compression Conference

Self Cite

View full text Add to dashboard Cite

High-throughput sequencing of RNA molecules has enabled the quantitative analysis of gene expression at the expense of storage space and processing power. To alleviate these problems, lossy compression methods of the quality scores associated to RNA sequencing data have recently been proposed, and the evaluation of their impact on downstream analyses is gaining attention. In this context, this work presents a first assessment of the impact of lossily compressed quality scores in RNA sequencing data on the performance of some of the most recent tools used for differential gene expression.

show abstract

An Evaluation Framework for Lossy Compression of Genome Sequencing Quality Values

Cited by 13 publications

References 12 publications

A Two-Level Scheme for Quality Score Compression

A Two-Level Scheme for Quality Score Compression

An introduction to MPEG-G, the new ISO standard for genomic information representation

Lossy Compression of Quality Scores in Differential Gene Expression: A First Assessment and Impact Analysis

Contact Info

Product

Resources

About