Novel and Generalized Sort-Based Transform for Lossless Data Compression

Inagaki, Kazumasa; Tomizawa, Yoshihiro; Yokoo, Hidetoshi

doi:10.1007/978-3-642-03784-9_10

Cited by 2 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The GRP transform, which is an extension of our previous one [3], converts an input string (1) into another string y [1..n] ∈ A n . The transform has two parameters: the block length and the (context ) order d, where is an integer in 1 ≤ ≤ n and d is an integer in 0 ≤ d ≤ n. The values of these parameters and the string length n are shared by the forward and inverse transformations so that the original string (1) can be uniquely recovered from the transformed string y [1..n].…”

Section: Preliminariesmentioning

confidence: 99%

“…Sorting is performed in a stable manner by using the entire row or its part as a sorting key. In our previous paper [3], we defined the GRP transform on a matrix consisting only of the reference and output parts because other elements have nothing to do with the transform. In fact, the matrix representation is not essential to the transform.…”

Section: Preliminariesmentioning

confidence: 99%

“…The GRP transform, or the generalized radix permutation transform, was proposed by Inagaki, Tomizawa, and the present author in [3] as a parametric generalization of the BWT of the block-sorting data compression algorithm [1], [2]. The BWT and its variations [6], [7], [8] can be derived from the GRP transform as its special cases.…”

Section: Introductionmentioning

confidence: 99%

“…However, it is not obvious whether we can run such a general version in an efficient way. In our previous paper [3], we have shown that the GRP transform can be performed in O(n + bd) = O(n + nd/ ) time. This implies that the transform runs in time linear in the string length, as long as the parameter d stays in the range of 0 ≤ d ≤ .…”

Section: Introductionmentioning

confidence: 99%

“…In order to show this and make clear the relation between the GRP transform and the BWT, we adopt a completely different description on the GRP transform than that given in our previous paper [3]. For example, in [3], the original data string to be compressed is arranged as a column vector of a matrix. In the present paper, on the other hand, we adopt the convention of the BWT, in which the original string is arranged as a row vector of a matrix.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Extension and Faster Implementation of the GRP Transform for Lossless Compression

Yokoo

2010

Combinatorial Pattern Matching

Self Cite

View full text Add to dashboard Cite

Abstract. The GRP transform, or the generalized radix permutation transform was proposed as a parametric generalization of the BWT of the block-sorting data compression algorithm. This paper develops its extension that can be applied with any combination of parameters. By using the technique developed for linear time/space implementation of the sort transform, we propose an efficient implementation for the inverse transformation of the GRP transform. It works for arbitrary parameter values, and can convert the transformed string to the original string in time linear in the string length.

show abstract

Section: Preliminariesmentioning

confidence: 99%

Section: Preliminariesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Extension and Faster Implementation of the GRP Transform for Lossless Compression

Yokoo

2010

Combinatorial Pattern Matching

Self Cite

View full text Add to dashboard Cite

show abstract

Revisiting bounded context block‐sorting transformations

2011

View full text Add to dashboard Cite

The Burrows-Wheeler Transform (BWT) produces a permutation of a string X, denoted X , by sorting the n cyclic rotations of X into full lexicographical order and taking the last column of the resulting n n matrix to be X . The transformation is reversible in O.n/ time. In this paper, we consider an alteration to the process, called k-BWT, where rotations are only sorted to a depth k. We propose new approaches to the forward and reverse transform, and show that the methods are efficient in practice. More than a decade ago, two algorithms were independently discovered for reversing k-BWT, both of which run in O.nk/ time. Two recent algorithms have lowered the bounds for the reverse transformation to O.n log k/ and O.n/, respectively. We examine the practical performance for these reversal algorithms. We find that the original O.nk/ approach is most efficient in practice, and investigates new approaches, aimed at further speeding reversal, which store precomputed context boundaries in the compressed file. By explicitly encoding the context boundaries, we present an O.n/ reversal technique that is both efficient and effective. Finally, our study elucidates an inherently cache-friendly -and hitherto unobserved -behavior in the reverse k-BWT, which could lead to new applications of the k-BWT transform. In contrast to previous empirical studies, we show that the partial transform can be reversed significantly faster than the full transform, without significantly affecting compression effectiveness. approach in which the n rotations are only partially sorted to a fixed prefix depth, k. We refer to this modified transform as k-BWT. By limiting the sort depth to k, sorting can be accomplished in O.nk/ time using radix sort and is very fast in practice. Moreover, Schindler reports nearly identical compression effectiveness to the full transform, even for small values of k. The algorithms developed by Schindler [13] were subsequently made available in the general purpose compression tool szip. However, the simplification of the forward k-BWT transform comes at a cost: the reverse transform becomes more expensive, at least in theory.Our contribution: First, we describe an efficient forward k-BWT algorithm based on induced sorting techniques from suffix array construction [15]. Our second contribution is a practical, O.n/ k-BWT time reversal algorithm that implicitly stores context boundaries. Third, we provide the first thorough empirical analysis of state-of-the-art k-BWT algorithms for the forward and inverse transforms, compression effectiveness, and associated trade-offs. Lastly, we discover a previously undocumented locality of access property inherent to k-BWT algorithms, allowing fast transform reversal for small k. BACKGROUND AND NOTATIONLet X D XOE0..n D XOE0XOE1..XOEn be a string (or text) of n C 1 symbols, where the first n symbols of X are drawn from an alphabet † and comprise the actual input; XOEn D $ is a unique "endof-string" symbol that is defined to be lexicographically smaller than all symbols in †. The stri...

show abstract

Novel and Generalized Sort-Based Transform for Lossless Data Compression

Cited by 2 publications

References 6 publications

Extension and Faster Implementation of the GRP Transform for Lossless Compression

Extension and Faster Implementation of the GRP Transform for Lossless Compression

Revisiting bounded context block‐sorting transformations

Contact Info

Product

Resources

About