A context-tree weighting method for text generating sources

Tjalkens, T.J.; Volf, Petr; Willems, F.M.J.

doi:10.1109/dcc.1997.582140

Cited by 15 publications

(11 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The combination of Beta-Weighting and the KT elementary model refers to the basic CTW variant (coupled with techniques to handle a nonbinary alphabet) as introduced in [101]. If we substitute the KT estimator by the ZR estimator, then we obtain the CTW variant proposed in [88]. The compression rates we measure for those variants are in line with the results reported in [103].…”

supporting

confidence: 63%

“…As we already stated, the most influential work on CTW is [101], although there exists prior work on CTW for an N -ary alphabet [100]. CTW for a binary alphabet, coupled with implementation techniques to handle N -ary alphabets by alphabet decomposition (every letter receives a binary codeword) [103,88], improves the empirical compression over CTW for a N -ary alphabet [8]. Consequently, most CTW research assumes a binary alphabet.…”

Section: Algorithms In Statistical Data Compression 31mentioning

confidence: 99%

“…• the ZR elementary model [88], given by the probability assignment rule 1 ZR(x; x 1:t ) = P (x 1 . .…”

Section: Experiments 145mentioning

confidence: 99%

“…Our implementation operates on a binary alphabet. We treat a file as a sequence of 8-bit letters, thus we apply the following standard-implementation techniques: A flat binary alphabet decomposition, weighting only at letter boundaries and no root weighting [88]. In essence, a letter (over alphabet of size 2 We use 64-bit floating point numbers to store probabilities and weights, and we did neither make any attempt to speed up costly computations (e. g. square roots, logarithms, exponentials) with lookup tables, nor did we introduce specifically tailored data structures (such as the hash tables in [103] or the techniques used in PAQ [52]).…”

Section: Experiments 147mentioning

confidence: 99%

See 3 more Smart Citations

Statistical Data Compression

Mattern¹

SpringerReference

View full text Add to dashboard Cite

The ongoing evolution of hardware leads to a steady increase in the amount of data that is processed, transmitted and stored. Data compression is an essential tool to keep the amount of data manageable. Furthermore, techniques from data compression have many more applications beyond compression, for instance data clustering, classification and time series prediction.In terms of empirical performance statistical data compression algorithms rank among the best. A statistical data compressor processes an input text letter by letter and performs compression in two stages -modeling and coding. During modeling a model estimates a probability distribution on the next letter based on the past input. During coding an encoder translates this probability distribution and the next letter into a codeword. Decoding reverts this process. Note that the model is exchangeable and its actual choice determines a statistical data compression algorithm. All major models use a mixer to combine multiple simple probability estimators, so-called elementary models.In statistical data compression there is an increasing gap between theory and practice. On the one hand, the "theoretician's approach" puts emphasis on models that allow for a mathematical code length analysis to evaluate their performance, but neglects running time and space considerations and empirical improvements. On the other hand the "practitioner's approach" focuses on the very reverse. The family of PAQ statistical compressors demonstrated the superiority of the "practitioner's approach" in terms of empirical compression rates.With this thesis we attempt to bridge the aforementioned gap between theory and practice with special focus on PAQ. To achieve this we apply the theoretician's tools to practitioner's approaches: We provide a code length analysis for several common and practical modeling and mixing techniques. The analysis covers modeling by relative frequencies with frequency discount and modeling by exponential smoothing of probabilities. For mixing we consider linear and geometrically weighted averaging of probabilities with Online Gradient Descent for weight estimation. Our results show that the models and mixers we consider perform nearly as well as idealized competitors that may adapt to the input. Experiments support our analysis. Moreover, our results add a theoretical justification to modeling and mixing from PAQ and generalize methods from PAQ. ii

show abstract

supporting

confidence: 63%

Section: Algorithms In Statistical Data Compression 31mentioning

confidence: 99%

“…• the ZR elementary model [88], given by the probability assignment rule 1 ZR(x; x 1:t ) = P (x 1 . .…”

Section: Experiments 145mentioning

confidence: 99%

Section: Experiments 147mentioning

confidence: 99%

See 2 more Smart Citations

Statistical Data Compression

Mattern¹

SpringerReference

View full text Add to dashboard Cite

show abstract

“…For example, for a binary sequence of length AE, a redundancy is not more than ½ ¾ ÐÓ ¾ AE · ½ and this is optimal according to Rissanen's converse. In particular, the parameter redundancy introduced by the (binary) KT estimator grows logarithmic with the number of zeros (or ones) [12]. This means that the redundancy becomes large if the number of generated zeros (or ones) gets large.…”

Section: Introductionmentioning

confidence: 99%

Theoretical analysis of a zero-redundancy estimator with a finite window for memoryless source

Rashid

Kawabata

2005

IEEE Information Theory Workshop, 2005.

View full text Add to dashboard Cite

A zero-redundancy estimator is defined by a weighted sum of Krichevsky-Trofimov (KT) sequential probability estimators i.e., the minimax bayes of the memoryless process, over all possible alphabets. This estimator is effective for nonbinary sources whose alphabet is embedded in a larger alphabet. We propose a new weighting recursive computation. Next we use the estimator to construct a finite window predictor for lossless data compressor, and we show that its average redundancy for memoryless source has optimal 1st order asymptotics.

show abstract