Suboptimality of the Karhunen-Loeve transform for transform coding

Abstract

Abstract-We examine the performance of the Karhunen-Loève transform (KLT) for transform coding applications. The KLT has long been viewed as the best available block transform for a system that orthogonally transforms a vector source, scalar quantizes the components of the transformed vector using optimal bit allocation, and then inverse transforms the vector. This paper treats fixed-rate and variable-rate transform codes of non-Gaussian sources. The fixed-rate approach uses an optimal fixed-rate scalar quantizer to describe the transform coefficients; the variable-rate approach uses a uniform scalar quantizer followed by an optimal entropy code, and each quantized component is encoded separately. Earlier work shows that for the variable-rate case there exist sources on which the KLT is not unique and the optimal quantization and coding stage matched to a "worst" KLT yields performance as much as 1.5 dB worse than the optimal quantization and coding stage matched to a "best" KLT. In this paper, we strengthen that result to show that in both the fixed-rate and the variable-rate coding frameworks there exist sources for which the performance penalty for using a "worst" KLT can be made arbitrarily large. Further, we demonstrate in both frameworks that there exist sources for which even a best KLT gives suboptimal performance. Finally, we show that even for vector sources where the KLT yields independent coefficients, the KLT can be suboptimal for fixed-rate coding.

A necessary and sufficient condition for transform optimality in source coding

“…The theorem also suggests that the optimal transform deviates from KLT whenever second order statistics are not a good representative of the overall dependence. This result also subsumes as a direct corollary the EFZ Theorem [5], and the well known optimality of KLT for jointly Guassian sources at high resolution variable rate coding [2].…”

“…Note: Theorem 1 subsumes Effros-Feng-Zeger theorem [5] as an extreme special case where KLT yields independent coefficients. The proof will make use of a trivial auxiliary lemma, which we state without proof:…”

“…Then a natural question arises: do all these KLTs perform equally? A sufficient condition for optimality of a KLT (that resolves this question if satisfied by one of the contenders) was given in [5] and is reproduced here.…”

“…Effros-Feng-Zeger Theorem (EFZ) [5]: If a KLT produces independent transform coefficients, then it is optimal for variable-rate transform coding at high resolution.…”

“…In [5], the "popular trust" in the optimality of KLT is challenged and it is demonstrated by examples that KLT can be suboptimal for both fixed and variable rate quantization, at asymptotically high rate (with high resolution approximations). A theoretical result is also obtained, namely, a sufficient condition for optimality of KLT: when KLT generates independent coefficients then it is the optimal transform for variable rate coding.…”