Variable-Length Source Dispersions Differ Under Maximum and Average Error Criteria

Sakai, Yuta; Tan, Vincent Y. F.

doi:10.1109/tit.2020.3019062

Cited by 9 publications

(16 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…without prefix-free constraints. We [16] then showed that the optimal first-order coding rates (i.e., the n scale) are the same under both average and maximum error criteria, and the optimal second-order coding rates (i.e., the √ n scale) differ under these two error formalisms. This difference is characterized by the law of total variance (see (73)).…”

Section: Discussionmentioning

confidence: 90%

“…In [16], the present authors considered the following limiting case of the cumulant generating function of codeword lengths:…”

Section: Discussionmentioning

confidence: 99%

“…We then characterize asymptotic expansions of fundamental limits of these problems. In the presence of the side-information Y , we show that the first-order terms for the average and maximum error formalisms are different in most non-degenerate cases, in contrast to the main result of [16]. In the absence of the side-information Y , we provide third-order asymptotic expansions of the fundamental limits by applying (1) to our one-shot coding theorems.…”

Section: A Main Contributionsmentioning

confidence: 96%

“…Note that in [3, Lemma 2], Courtade and Verdú provided a unified lemma which can be specialized to Campbell's source coding problem [2] in the absence of prefix-free constraints. Sason and Verdú [27,Lemma 7] established inequalities between the guessing moment and the moment generating function of codeword lengths for a variable-length source code without prefix-free constraints; see [16,Lemma 6] for a similar result to [27, Lemma 7] in the almost lossless regime.…”

Section: A Main Contributionsmentioning

confidence: 99%

“…4) Variable-Length Compression Allowing Errors: While this study examines the cumulant generating function of codeword lengths of a variable-length source code [2], [3] allowing errors as in Kuzuoka's works [11], [26], the (ordinary) expectation of codeword lengths of a variable-length source code allowing errors has been investigated by several researchers [16], [29]- [31]. Specifically, the present authors [16] derived second-order asymptotic expansions of the fundamental limits of variable-length conditional source coding problems under both average and maximum error criteria. We then showed that the difference between the average and maximum error criteria is manifested in the second-order terms in these asymptotic expansions, and this difference can be quantified by the law of total variance for the information variance.…”

Section: A Main Contributionsmentioning

confidence: 99%

See 4 more Smart Citations

Asymptotic Expansions of Smooth Rényi Entropies and Their Applications

Sakai,

Tan

2020

Preprint

Self Cite

View full text Add to dashboard Cite

This study considers the unconditional smooth Rényi entropy, the smooth conditional Rényi entropy proposed by Kuzuoka [IEEE Trans. Inf. Theory, vol. 66, no. 3, pp. 1674-1690, 2020, and a new quantity which we term the conditional smooth Rényi entropy. In particular, we examine asymptotic expansions of these entropies when the underlying source with its side-information are stationary and memoryless. Using these smooth Rényi entropies, we establish one-shot coding theorems of several informationtheoretic problems: Campbell's source coding problems, guessing problems, and task encoding problems, all allowing errors. In each problem, we consider two error formalisms: the average and maximum error criteria, where the averaging and maximization are taken with respect to the side-information of the source. Applying our asymptotic expansions to the one-shot coding theorems, we derive various asymptotic fundamental limits for these problems when their error probabilities are allowed to be non-vanishing. We show that, in non-degenerate settings, the first-order fundamental limits differ under the average and maximum error criteria. This is in contrast to a different but related setting considered by the present authors (for variable-length conditional source coding allowing errors) in which the first-order terms are identical but the second-order terms are different under these criteria.

show abstract

Section: Discussionmentioning

confidence: 90%

“…In [16], the present authors considered the following limiting case of the cumulant generating function of codeword lengths:…”

Section: Discussionmentioning

confidence: 99%

Section: A Main Contributionsmentioning

confidence: 96%