Statistically optimum pre- and postfiltering in quantization

Abstract

Abstract-We consider the optimization of pre-and postfilters surrounding a quantization system. The goal is to optimize the filters such that the mean square error is minimized under the key constraint that the quantization noise variance is directly proportional to the variance of the quantization system input. Unlike some previous work, the postfilter is not restricted to be the inverse of the prefilter. With no order constraint on the filters, we present closed-form solutions for the optimum pre-and postfilters when the quantization system is a uniform quantizer. Using these optimum solutions, we obtain a coding gain expression for the system under study. The coding gain expression clearly indicates that, at high bit rates, there is no loss in generality in restricting the postfilter to be the inverse of the prefilter. We then repeat the same analysis with first-order preand postfilters in the form 1+z 01 and 1=(1+ z 01 ). In specific, we study two cases: 1) FIR prefilter, IIR postfilter and 2) IIR prefilter, FIR postfilter. For each case, we obtain a mean square error expression, optimize the coefficients and and provide some examples where we compare the coding gain performance with the case of = . In the last section, we assume that the quantization system is an orthonormal perfect reconstruction filter bank. To apply the optimum pre-and postfilters derived earlier, the output of the filter bank must be wide-sense stationary WSS which, in general, is not true. We provide two theorems, each under a different set of assumptions, that guarantee the wide sense stationarity of the filter bank output. We then propose a suboptimum procedure to increase the coding gain of the orthonormal filter bank.Index Terms-Half-whitening scheme, noise shaping, optimum pre-and postfiltering, subband coding.

Oversampling PCM techniques and optimum noise shapers for quantizing a class of nonbandlimited signals

“…From this point on, the problem under study is very similar to the one analyzed recently in [20], and, in fact becomes exactly the same by setting and to unity in (28). We will therefore omit the proofs of the upcoming theorems and refer to [20]. …”

“…6 can be obtained again as a special case by setting in (14). The optimum prefilter will then have the following magnitude squared response: (20) and can be regarded as a multirate extension of the half whitening filter [15]. Using (20), we can derive an interesting expression for the coding gain of the scheme of Fig.…”

Optimal subband coders of quantized multidimensional signals

“…In [11], it is shown that halfwhitening prefilters and reciprocal postfilters maximize the coding gain of arbitrary orthonormal filter-banks using arbitrary quantizers. The optimization of arbitrary quantizers is also addressed in [12]. Pre-and post-filters optimizing the quantizer performance are determined under the assumption that the quantizer noise is uncorrelated with the quantizer input.…”

Iterative algorithm for the design of optimal FIR analysis/synthesis filters for overdecimated filter banks

“…For simplicity, the analysis and synthesis filters are often chosen to satisfy an orthonormality or biorthogonality condition. If no other restrictions are put on the filters, then the optimal choice for these filters is known (see [5,8,1] for the orthonormal case and [6,9,4] for the biorthogonal case). In the orthonormal case, the optimal solution is an infinite order principal component filter bank (PCFB) [8,1], whereas in the biorthogonal case, the optimal filter bank is a PCFB with a parallel bank of half-whitening filters in the middle of the system [9,4].…”