Convergence Speed of a Dynamical System for Sparse Recovery

Abstract

Abstract-This paper studies the convergence rate of a continuous-time dynamical system for 1-minimization, known as the Locally Competitive Algorithm (LCA). Solving 1-minimization problems efficiently and rapidly is of great interest to the signal processing community, as these programs have been shown to recover sparse solutions to underdetermined systems of linear equations and come with strong performance guarantees. The LCA under study differs from the typical 1-solver in that it operates in continuous time: instead of being specified by discrete iterations, it evolves according to a system of nonlinear ordinary differential equations. The LCA is constructed from simple components, giving it the potential to be implemented as a large-scale analog circuit. The goal of this paper is to give guarantees on the convergence time of the LCA system. To do so, we analyze how the LCA evolves as it is recovering a sparse signal from underdetermined measurements. We show that under appropriate conditions on the measurement matrix and the problem parameters, the path the LCA follows can be described as a sequence of linear differential equations, each with a small number of active variables. This allows us to relate the convergence time of the system to the restricted isometry constant of the matrix. Interesting parallels to sparse-recovery digital solvers emerge from this study. Our analysis covers both the noisy and noiseless settings and is supported by simulation results.

Discrete and Continuous-Time Soft-Thresholding for Dynamic Signal Recovery

“…When P goes to infinity, this additional term disappears. The two conditions (15) and (16) have a similar form to the conditions of Theorem 3 in [17]. If there is no initial guess, then u[0] = 0 and (15) holds.…”

“…In the static setting, The LCA was shown in [14] and [17] to converge exponentially fast to the solution of (1). For the appropriate parameter choices, the 2 -error can be expressed for all time t ≥ 0 as a(t) − a † 2 ≤ C 9 e −vt + C 1 , where v ∈ (0, 1) and C 9 ≥ 0.…”

“…The initial conditions (21) and (22) are similar to the initial conditions of Theorem 3 in [17]. In particular, (21) holds when there is no initial guess (i.e., u(0) = 0) and a similar analysis to that of Section IV.B in [17] shows that the number of measurements for (22) to hold is on the order of S log (N/S) for classic CS matrices for which δ ∼ S/M log (N/S).…”

Dynamic DOA estimation via locally competitive algorithm

“…In [22] introduces the value of τ decides the speed of convergence in some degree, and this exponential convergence is predicted by the mean-squared error between the nodes at time t and the final solution…”

“…where δ is the RIP (Restricted Isometry Property) constant which is defined in [22]. As shown from (6), the speed of convergence has a relationship with the value of τ .…”