Weighted thresholding homotopy method for sparsity constrained optimization

Sparse Group Feature Selection by Weighted Thresholding Homotopy Method

“…Similar to the proof in [11], we can prove that, if f (x) is a differentiable convex function, then for λ ≥ ∇f (0) ∞ , x = 0 is the optimal solution of problem (15). Thus, λ = ∇f (0) ∞ is big enough such that (x, w) = (0, 1) is the optimal solution of problem (4) with s f = 0.…”

“…Motivated by the weighted thresholding method for individual variable sparsity constrained optimization [11] and the ISTA with sparse group hard thresholding method [1], we adopt the weighted thresholding framework and present the following Algorithm 3 for problem (4). For the convenience of description, we denote the solution in Algorithm 1 by WT λ,L (x k ), then we propose the weighted thresholding method for problem (4) in Algorithm 3.…”

“…First, we introduce an equivalent formulation of the sparsity constraint x ∈ C f , where [11]: x ∈ C f is equivalent to that there exists w ∈ {0, 1} n such that…”

“…And then the function is minimized over the constraints of problem (1), which can be solved optimally by a dynamic programming algorithm. However according to [11], it is restrictive to minimize a proximal function over individual variable sparsity constraint directly. Hence it is interesting to improve the iterative sparse group hard thresholding algorithm [1] for problem (1) using the reformulation technique in [11].…”

“…To achieve this purpose, we reformulate the s f -sparse individual constraint in (1) by a weighted l 1 -norm strategy and get a new problem, which is equivalent to problem (1). This reformulation is similar to that in [11], which does not consider the group sparsity constraint. To solve the reformulated problem, we penalize the side constraint and propose a weighted thresholding method, which is based on a dynamic programming algorithm.…”