On the Convergence of an Efficient Algorithm for Kullback–Leibler Approximation of Spectral Densities

Abstract

This paper deals with a method for the approximation of a spectral density function among the solutions of a generalized moment problemà la Byrnes/Georgiou/Lindquist. The approximation is pursued with respect to the Kullback-Leibler pseudo-distance, which gives rise to a convex optimization problem. After developing the variational analysis, we discuss the properties of an efficient algorithm for the solution of the corresponding dual problem, based on the iteration of a nonlinear map in a bounded subset of the dual space. Our main result is the proof of local convergence of the latter, established as a consequence of the Central Manifold Theorem. Supported by numerical evidence, we conjecture that, in the mentioned bounded set, the convergence is actually global.Moment problems have a long history and have been at the heart of many mathematical and engineering problems in the past century, see, e.g., [34], [1] and the references therein. Only with recent developments of the above-mentioned research program, however, the parametrization of solutions in the presence of additional constraints on the complexity have been satisfactorily addressed [12]. This result, that has been possible thanks to a suitable variational formulation, is of key interest in control engineering. In fact, the well-known relation between moment problems and Nevanlinna-Pick interpolation problems allows for solutions of H ∞ control problems that include a bound on the complexity of the controller, which is of paramount practical importance [3], [9]. Similar considerations apply to the covariance extension problem [10], [26]. Among the other applications, we also mention signal and image processing [22], [23], [24], [25], and Biomedical Engineering [30]. These applications are based on a spectral estimation procedure that hinges on optimal approximation of spectral densities with linear integral constraints that may be viewed as constraints on a finite number of moments of the spectrum. The linear integral constraints represent a knowledge on the steady-state state covariance of a bank of filters that is designed in order to estimate the unknown spectral density Φ, while the to-be-approximated spectral density represents a prior knowledge on Φ, see Section II for more details. As discussed in [7], [26], [32], this optimal approximation leads to a tunable spectral estimation algorithm that provides high resolution estimates in prescribed frequency bands even in presence of a short record of observed data. An important feature of the above mentioned optimal approximation method is that the primal optimization problem can be solved in closed form and, as long as the prior spectral density is rational, yields a rational solution with an a priori bound on the complexity (McMillan degree).The numerical challenge, in practical applications, lays with the dual problem. In fact, the dual variable is an Hermitian matrix and, as discussed in [26], the reparametrization in vector form may lead to a loss of convexity. Moreover, the dual functional and it...

Further Results on the Convergence of the Pavon–Ferrante Algorithm for Spectral Estimation

“…The variational analysis outlined in [2] (see also [1], [30] where some additional details are spelled out and [10], [33] for the existence part) leads to the following result.…”

“…The Pavon-Ferrante algorithm is a surprisingly simple and efficient nonlinear fixed-point iteration in the set of positive semi-definite unit trace matrices. Furthermore, the algorithm exhibits very attractive and robust properties from a numerical viewpoint, since it can be implemented via the solution of an algebraic Riccati equation and a Lyapunov equation [30]. On the other hand, despite the huge amount of numerical evidences, proving the convergence of the latter algorithm to a prescribed set of fixed pointswhich provide the solution of the approximation problem-has revealed to be an highly nontrivial challenge [31], [30].…”

“…Furthermore, the algorithm exhibits very attractive and robust properties from a numerical viewpoint, since it can be implemented via the solution of an algebraic Riccati equation and a Lyapunov equation [30]. On the other hand, despite the huge amount of numerical evidences, proving the convergence of the latter algorithm to a prescribed set of fixed pointswhich provide the solution of the approximation problem-has revealed to be an highly nontrivial challenge [31], [30]. In particular, in [30] it has been shown that the Pavon-Ferrante algorithm is locally convergent to the aforesaid set of fixed points, through a rather tortuous yet enlightening proof.…”

“…On the other hand, despite the huge amount of numerical evidences, proving the convergence of the latter algorithm to a prescribed set of fixed pointswhich provide the solution of the approximation problem-has revealed to be an highly nontrivial challenge [31], [30]. In particular, in [30] it has been shown that the Pavon-Ferrante algorithm is locally convergent to the aforesaid set of fixed points, through a rather tortuous yet enlightening proof. Nonetheless, a proof of global convergence of the algorithm, though conjectured and supported by a large number of numerical simulations, has so far been elusive.…”

Uncertainty Bounds for Spectral Estimation

“…In fact, the use of generalized statistics, which relates to beamspace processing, was explored in [7], [15] as a way to improve resolution in power spectral estimation over selected frequency bands. More recent work addresses spectral estimation with priors, computational issues, as well as important multivariate generalizations [3], [5], [9], [10], [11], [17], [18], [20], [21], [38], [40], [43], [44].…”