Sparsest piecewise-linear regression of one-dimensional data

Sparsest Univariate Learning Models Under Lipschitz Constraint

“…More precisely, we propose a two-step algorithm to reach the sparsest CPWL solution of (4). The first step consists in solving a discrete problem with ∞ regularization, and the second is a sparsification step proposed in [40] that reaches the sparsest solution.…”

“…In the second scenario, we explictly control the Lipschitz constant of the learned mapping by imposing a hard constraint. Inspired by the theoretical insights of the first problem, we add a second-order total-variation (TV) regularization term that is known to promote sparse CPWL functions [41], [40]. This leads to the minimization problem…”

“…Motivated by this, the TV (2) regularization has been exploited to learn activation functions of deep neural networks [41], [75]. In a similar vein, the identification of the sparsest CPWL solutions of TV (2) -regularized problems has been thoroughly studied in [40].…”

“…The proof is given in Appendix D. The proof involves the weak*-closedness of the constraint box L(f ) ≤ L which is essential to prove existence. Once the existence of a minimizer is guaranteed, we can invoke the results of Debarre et al in [40] for TV (2) minimization to deduce the remaining parts. We also remark that the Lipschitz constraint only affects the vector z in (25), which forces its entries to satisfy the inequalities…”

“…, x m . Using this vector, we then deploy the sparsification algorithm of [40], whose use in the present method is motivated by the following theorem.…”