Switching between two universal source coding algorithms

Volf, Petr; Willems, F.M.J.

doi:10.1109/dcc.1998.672217

Cited by 36 publications

(31 citation statements)

References 4 publications

(3 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, many other data compression techniques suffer from the same problem. For instance, Volf and Willems [11] present a character-wise predict-and-encode technique where not one but two predictors compete to best predict the next character. The encoding of each character consists in a flag that indicates which predictor has been chosen followed by the encoding of the character using that predictor's predictions.…”

Section: Discussion and Future Workmentioning

confidence: 99%

Constructing optimal whole-bit recycling codes

Dube

Beaudoin

2009

2009 IEEE Information Theory Workshop on Networking and Information Theory

View full text Add to dashboard Cite

Bit recycling aims at improving the rates achieved by compression techniques, such as LZ77, that suffer from the redundancy caused by the multiplicity of the encodings. The performance of bit recycling depends crucially on the recycling codes that it uses. A recipe for the construction of optimal recycling codes has been mentioned in previous work. However, no efficient algorithm and proof of optimality were given. We present both here.

show abstract

Section: Discussion and Future Workmentioning

confidence: 99%

Constructing optimal whole-bit recycling codes

Dube

Beaudoin

2009

2009 IEEE Information Theory Workshop on Networking and Information Theory

View full text Add to dashboard Cite

show abstract

“…Instead, one could consider weighting over sequences of models chosen from a fixed base class M. Variants of this fundamental idea have been considered numerous times in the literature, for example by Volf and Willems (1998);Herbster and Warmuth (1998) and Erven et al (2008). We now show how these ideas can be cast into our probabilistic agent setting, by describing an adaptation of the FixedShare algorithm (Herbster and Warmuth, 1998).…”

Section: Switching / Trackingmentioning

confidence: 99%

On Ensemble Techniques for AIXI Approximation

Veness

Sunehag

Hutter

2012

Artificial General Intelligence

View full text Add to dashboard Cite

One of the key challenges in AIXI approximation is model class approximation -i.e. how to meaningfully approximate Solomonoff Induction without requiring an infeasible amount of computation? This paper advocates a bottom-up approach to this problem, by describing a number of principled ensemble techniques for approximate AIXI agents. Each technique works by efficiently combining a set of existing environment models into a single, more powerful model. These techniques have the potential to play an important role in future AIXI approximations.

show abstract

“…The problem of combining arbitrary models in statistical data compression was first investigated in [94,95,96]. 2 However, the setup was rather restrictive, since the main 1…”

Section: Previous Workmentioning

confidence: 99%

“…intention was to combine just two models. This approach turns out to be successful, in particular, when the two models are of opposing nature, e. g. CTW and a LZ77-alike model [95] or CTW and PPM* [96].…”

Section: Previous Workmentioning

confidence: 99%

“…In [95] the authors proposed three algorithms, namely, the Switching Algorithm, the Snake Algorithm and the Reduced Complexity Algorithm. To explain the algorithms consider the t-th step while we process an input sequence x 1:n : We are given a sequence p 1:t of 2-dimensional p-vectors (since we combine two models) and must produce a single prediction, which may be used for coding.…”

Section: Previous Workmentioning

confidence: 99%

See 1 more Smart Citation

Statistical Data Compression

Mattern¹

SpringerReference

View full text Add to dashboard Cite

The ongoing evolution of hardware leads to a steady increase in the amount of data that is processed, transmitted and stored. Data compression is an essential tool to keep the amount of data manageable. Furthermore, techniques from data compression have many more applications beyond compression, for instance data clustering, classification and time series prediction.In terms of empirical performance statistical data compression algorithms rank among the best. A statistical data compressor processes an input text letter by letter and performs compression in two stages -modeling and coding. During modeling a model estimates a probability distribution on the next letter based on the past input. During coding an encoder translates this probability distribution and the next letter into a codeword. Decoding reverts this process. Note that the model is exchangeable and its actual choice determines a statistical data compression algorithm. All major models use a mixer to combine multiple simple probability estimators, so-called elementary models.In statistical data compression there is an increasing gap between theory and practice. On the one hand, the "theoretician's approach" puts emphasis on models that allow for a mathematical code length analysis to evaluate their performance, but neglects running time and space considerations and empirical improvements. On the other hand the "practitioner's approach" focuses on the very reverse. The family of PAQ statistical compressors demonstrated the superiority of the "practitioner's approach" in terms of empirical compression rates.With this thesis we attempt to bridge the aforementioned gap between theory and practice with special focus on PAQ. To achieve this we apply the theoretician's tools to practitioner's approaches: We provide a code length analysis for several common and practical modeling and mixing techniques. The analysis covers modeling by relative frequencies with frequency discount and modeling by exponential smoothing of probabilities. For mixing we consider linear and geometrically weighted averaging of probabilities with Online Gradient Descent for weight estimation. Our results show that the models and mixers we consider perform nearly as well as idealized competitors that may adapt to the input. Experiments support our analysis. Moreover, our results add a theoretical justification to modeling and mixing from PAQ and generalize methods from PAQ. ii

show abstract

Switching between two universal source coding algorithms

Cited by 36 publications

References 4 publications

Constructing optimal whole-bit recycling codes

Constructing optimal whole-bit recycling codes

On Ensemble Techniques for AIXI Approximation

Statistical Data Compression

Contact Info

Product

Resources

About