Global optimization for low-dimensional switching linear regression and bounded-error estimation

Abstract

The paper provides global optimization algorithms for two particularly difficult nonconvex problems raised by hybrid system identification: switching linear regression and bounded-error estimation. While most works focus on local optimization heuristics without global optimality guarantees or with guarantees valid only under restrictive conditions, the proposed approach always yields a solution with a certificate of global optimality. This approach relies on a branchand-bound strategy for which we devise lower bounds that can be efficiently computed. In order to obtain scalable algorithms with respect to the number of data, we directly optimize the model parameters in a continuous optimization setting without involving integer variables. Numerical experiments show that the proposed algorithms offer a higher accuracy than convex relaxations with a reasonable computational burden for hybrid system identification. In addition, we discuss how bounded-error estimation is related to robust estimation in the presence of outliers and exact recovery under sparse noise, for which we also obtain promising numerical results.arXiv:1707.05533v3 [cs.LG] 23 Nov 2017and the algorithm stops at iteration T . ConvergenceAs for the switching regression case studied in Sect. 3.3, convergence is obtained, for both the 2, and the 0, loss functions, from the tightness of the bounds, under the following assumptions.Assumption 3. For p = 2 (respectively, p = 0), the global optimum of Problem (39) (resp. Problem (40)) is strictly positive:Assumption 4. For any p ∈ {0, 2}, upper bounds in a box B = [u, v] are computed as J(B) = J p BE (u) or such that J(B) ≤ J p BE (u). Theorem 2. Under Assumptions 3-4, the branch-and-bound algorithm described above to minimize J p BE with p ∈ {0, 2} and lower bounds J(B) computed as in Lemma 6 for p = 2 or Lemma 7 for p = 0 converges in a finite number of iterations for any T OL > 0.

Identification of Switched Linear Systems: Persistence of Excitation and Numerical Algorithms

“…Roughly speaking, the proposed approaches can be categorized as 1) Optimization-based approach. This approach formulates the identification problem as a non-convex optimization problem, which is solved by either the global optimization algorithms, e.g., general branch-and-bound scheme [27,15,16], or the local optimization algorithms [14].…”

“…5) Bounded error approach. This approach is inspired by ideas from set-membership identification [20], which enforces that the prediction error for all the samples is bounded by a priori quantity [4,16], where the priori quantity is a tuning parameter and plays a role in the balance between model accuracy and model complexity.…”

“…Example 3. The settings of this example is the same as that given in Section 5.1 of [16]. We simulate the SL system (1) with the cases n = 2, 3, 4, 5, S = 2, 3, and…”

State Observers for Systems Subject to Bounded Disturbances Using Quadratic Boundedness

“…The information available on the boundedness of the disturbances is not usually taken into account, as pointed out in [6]. This has motivated the development of novel optimization approaches such as those presented in [7]. However, the effectiveness of the resulting estimation methods is still a topic under investigation.…”

Classification of dynamics on Riemannian manifold

“…In optimization based methods, [27] and [20] recast the problem into a combinatorial optimization. In the case of bounded noise, [91] proposed to use a branch and bound approach to efficiently solve the optimization problem.…”