On the optimality of operator-like wavelets for sparse AR(1) processes

Abstract

Sinusoidal transforms such as the DCT are known to be optimal-that is, asymptotically equivalent to the KarhunenLoève transform (KLT)-for the representation of Gaussian stationary processes, including the classical AR(1) processes. While the KLT remains applicable for non-Gaussian signals, it loses optimality and, is outperformed by the independentcomponent analysis (ICA), which aims at producing the most-decoupled representation. In this paper, we consider an extension of the classical AR(1) model that is driven by symmetric-alpha-stable (SαS) noise which is either Gaussian (α = 2) or sparse (0 < α < 2). For the sparse (non-Gaussian) regime, we prove that an expansion in a proper wavelet basis (including the Haar transform) is much closer to the optimal orthogonal ICA solution than the classical Fourier-type representations. Our criterion for optimality, which favors independence, is the Kullback-Leibler divergence between the joint pdf of the original signal and the product of the marginals in the transformed domain. We also observe that, for very sparse AR(1) processes (α ≤ 1), the operator-like wavelet transform is indistinguishable from the ICA solution that is determined through numerical optimization.

On the Continuity of Characteristic Functionals and Sparse Stochastic Modeling

“…We can therefore refer to these processes as sparse processes. Second, it has been demonstrated for the case of symmetric α-stable (SαS) AR(1) processes that better decoupling is achieved in a wavelet-like representation than with the traditional sine basis or the KLT [PU13]. The innovation model has already been applied to various fields of image processing such as Bayesian estimation from noisy samples of sparse processes [AKBU13], algorithms for the optimal quadratic estimation of sparse processes [KPAU13], and reconstruction techniques based on sparse and self-similar processes [BFKU13].…”

“…While these examples show that sparse processes are highly relevant for practical applications, the theory currently available is based on too-constraining assumptions. In particular, it excludes some of the sparsest processes such as SαS with α < 1, for which wavelets have been found empirically to be optimal [PU13]. More generally, the compatibility between a linear operator L and an innovation process w, defined as the existence of a process s such that Ls = w, is a crucial question that needs to be addressed.…”

Ultra-Low Bitrate Video Conferencing Using Deep Image Animation

“…A major limitation of the current video compression paradigm comes from employing elementary models of pixel dependencies in order to increase coding efficiency. For instance, the commonly used discrete cosine transform implicitly assumes that pixels are generated by a Gaussian stationary process [3]. Similarly, spatial and temporal prediction exploit low-level pixel dependencies such as spatial directional smoothness and simple translational motion across time.…”

VOW: Variance-optimal wavelets for the steerable pyramid

“…Localization is important from both theoretic and practical points of view. The theory dictates that more localized wavelets tend to decouple and sparsify signals more efficiently [7]. Also in practice, wavelets with better localization result in lesser oscillations and fewer truncation artifacts.…”