Asymptotically optimal low complexity sequential lossless coding for regular piecewise stationary memoryless sources

Shamir, Gil I.; Costello, D.J.; Merhav, Neri

doi:10.1109/itcom.1999.781413

Cited by 6 publications

(13 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike sequential estimators based on the standard approach (see, e.g., [3], [4], [5], [9], [10]) which may generate several probability estimators and add them to provide g 3 A , the estimator of Theorem 1 adds but may also subtract a bias from a quantity updated sequentially. The sign of the bias depends on the actual bits in !…”

Section: Priormentioning

confidence: 99%

“…The clean sequence is transformed through a binary channel with (21) can be implemented with a low-complexity sequential algorithm. This can be done using a state transition diagram which resembles those proposed in [4], [5], [10]. A state The reduction of complexity using this method is beyond the scope of this paper, but is studied in future work.…”

Section: Priormentioning

confidence: 99%

See 1 more Smart Citation

Low-complexity sequential probability estimation and universal compression for binary sequences with constrained distributions

Shamir

Tjalkens

Willems

2008

2008 IEEE International Symposium on Information Theory

View full text Add to dashboard Buy / Rent full text

Abstract-Two low-complexity methods are proposed for sequential probability assignment for binary independent and identically distributed (i.i.d.) individual sequences with empirical distributions whose governing parameters are known to be bounded within a limited interval. The methods can be applied to different problems where fast accurate estimation of the maximizing sequence probability is very essential to minimizing some loss. Such applications include applications in finance, learning, channel estimation and decoding, prediction, and universal compression. The application of the new methods to universal compression is studied, and their universal coding redundancies are analyzed. One of the methods is shown to achieve the minimax redundancy within the inner region of the limited parameter interval. The other method achieves better performance on the region boundaries and is more robust numerically to outliers. Simulation results support the analysis of both methods. While non-asymptotically the gains may be significant over standard methods that maximize the probability over the complete parameter simplex, asymptotic gains are in second order. However, these gains translate to meaningful significant factor gains in other applications, such as financial ones. Moreover, the methods proposed generate estimators that are constrained within a given interval throughout the complete estimation process which are essential to applications such as sequential binary channel crossover estimation. The results for the binary case lay the foundation to studying larger alphabets.

show abstract

Section: Priormentioning

confidence: 99%

Section: Priormentioning

confidence: 99%

Low-complexity sequential probability estimation and universal compression for binary sequences with constrained distributions

Shamir

Tjalkens

Willems

2008

2008 IEEE International Symposium on Information Theory

View full text Add to dashboard Buy / Rent full text

show abstract

“…Therefore, the assumption of non time-homogeneous transition probabilities turns the current predictors inefficient and raise some design challenges for any new scheme that will be designed to address this assumption. Although research works exist dealing with piecewise stationarity (e.g., [10]) these works mainly focus on memoryless sources and have not considered Markov sources.…”

Section: Further Researchmentioning

confidence: 99%

Prediction in wireless networks by Markov chains

Katsaros

Manolopoulos²

2009

IEEE Wireless Commun.

View full text Add to dashboard Buy / Rent full text

Discrete sequence modelling and prediction is an important goal and a challenge for pervasive computing. Mobile client's data request forecasting and location tracking in wireless cellular networks are characteristic application areas of sequence prediction in pervasive computing. This article presents information-theoretic techniques for discrete sequence prediction. It surveys, classifies, and compares the state-of-the-art solutions, suggesting routes for further research by discussing the critical issues and challenges of prediction in wireless networks.

show abstract

“…Similar problems have been investigated earlier in the contexts of universal prediction and universal lossless compression of piecewise stationary memoryless sources. The latter problem has been studied by Willems [18], Shamir and Merhav [19], and Shamir and Costello [20]. The efficient sequential algorithms in these papers are based on a two-stage mixture procedure, where mixture estimates are used for the source parameters in each segment, as well as over the possible (or most likely) segmentations of the observed sequence.…”

mentioning

confidence: 99%

Tracking the Best Quantizer

György

Linder

Lugosi

2008

IEEE Trans. Inform. Theory

View full text Add to dashboard Buy / Rent full text

Abstract-An algorithm is presented for online prediction that allows to track the best expert efficiently even when the number of experts is exponentially large, provided that the set of experts has a certain additive structure. As an example, we work out the case where each expert is represented by a path in a directed graph and the loss of each expert is the sum of the weights over the edges in the path. These results are then used to construct universal limiteddelay schemes for lossy coding of individual sequences. In particular, we consider the problem of tracking the best scalar quantizer that is adaptively matched to the source sequence with piecewise different behavior. A randomized algorithm is presented which can perform, on any source sequence, asymptotically as well as the best scalar quantization algorithm that is matched to the sequence and is allowed to change the employed quantizer for a given number of times. The complexity of the algorithm is quadratic in the sequence length, but at the price of some deterioration in performance, the complexity can be made linear. Analogous results are obtained for sequential multiresolution and multiple description scalar quantization of individual sequences.

show abstract

Asymptotically optimal low complexity sequential lossless coding for regular piecewise stationary memoryless sources

Cited by 6 publications

References 5 publications

Low-complexity sequential probability estimation and universal compression for binary sequences with constrained distributions

Low-complexity sequential probability estimation and universal compression for binary sequences with constrained distributions

Prediction in wireless networks by Markov chains

Tracking the Best Quantizer

Contact Info

Product

Resources

About