Statistical results for system identification based on quantized observations

Abstract

System identification based on quantized observations requires either approximations of the quantization noise, leading to suboptimal algorithms, or dedicated algorithms taylored to the quantization noise properties. This contribution studies fundamental issues in estimation that relate directly to the core methods in system identification. As a first contribution, results from statistical quantization theory are surveyed and applied to both moment calculations (mean, variance etc) and the likelihood function of the measured signal. In particular, the role of adding dithering noise at the sensor is studied. The overall message is that taylored dithering noise can considerably simplify the derivation of optimal estimators. The price for this is a decreased signal to noise ratio, and a second contribution is a detailed study of these effects in terms of the Cramér-Rao lower bound. The common additive uniform noise approximation of quantization is discussed, compared, and interpreted in light of the suggested approaches.

Generating dithering noise for maximum likelihood estimation from quantized data

“…Parameter estimation using the ml method for quantization has been discussed in for instance [3,8,11]. A simple but instructive example is the unknown signal mean model z k (θ) = x + e k , where e k denotes white measurement noise and the parameter vector θ = (x, σ 2 e ) contains the mean x and possible also the variance σ 2 e = E e 2 k .…”

“…2, where the method was independently developed in [7,11,22]. The normalization constant and the Alg.…”

“…The normalization constant and the Alg. 2 Band-limited symmetric pdf, [7,22]. Take an integer k ≥ 1 and define the pdf as…”

“…The crlb, [4,12,14], is then given by where J(θ) denotes the Fisher information matrix (fim). Details on how to compute the crlb is given in [8,11,13]. The purpose here is to point out the implications of using band-limited dithering noises.…”

“…They show that quantization adds two kind of errors to the measurement, the first one is a direct effect that can be modeled as additive uniform noise (aun), and the other one is an intrinsic alias like uncertainty, where fast variations in the probability density function (pdf) of the measurement noise are folded to low "frequencies" in the cf domain (see Section 3.2). It is also known, [20], that adding r uniformly distributed samples as dithering noise enables reconstruction of all the first r moments of the unquantized signal, when uniform quantization is considered, where applications to system identification is studied in [7,8]. This is known as Quantization Theorem II (QT II).…”