Multi-task Learning in vector-valued reproducing kernel Banach spaces with the<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="d1e1008" altimg="si1021.svg"><mml:msup><mml:mrow><mml:mi>ℓ</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msup></mml:math>norm

Lin, Rongrong; Song, Guohui; Zhang, Haizhang

doi:10.1016/j.jco.2020.101514

Cited by 9 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The iteration begins with an initial λ that is large enough to ensure the sparsity level l under the transform B of the solution u of model (1) with this λ does not exceed l * . Proposition 4.16 which shows that the desired λ * and its corresponding u * must satisfy (40) motivates us to update λ by taking Algorithm 1. Parameter choice for the regularization problem (1).…”

Section: Parameter Choices For Sparsity Of Transformed Solutionsmentioning

confidence: 99%

“…Sparse learning models with the ℓ 1 norm, such as ℓ 1 Support vector machine (SVM) classification [38,51,55] and ℓ 1 SVM regression [7,37,55], have received increasing attention in machine learning. Motivated by the need of sparse learning algorithms, the notion of reproducing kernel Banach spaces (RKBSs) was introduced in [76] and further developed in [40,56,57,72]. RKBSs with the ℓ 1 norm [40,56,57] have been proven successful in promoting sparsity in representations for learned functions.…”

Section: Introductionmentioning

confidence: 99%

“…Motivated by the need of sparse learning algorithms, the notion of reproducing kernel Banach spaces (RKBSs) was introduced in [76] and further developed in [40,56,57,72]. RKBSs with the ℓ 1 norm [40,56,57] have been proven successful in promoting sparsity in representations for learned functions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Parameter choices for sparse regularization with the ℓ1 norm ^*

Liu¹,

Wang²,

Xu³

et al. 2023

Inverse Problems

View full text Add to dashboard Cite

We consider a regularization problem whose objective function consists of a convex fidelity term and a regularization term determined by the l1 norm composed with a linear transform. Empirical results show that the regularization with the l1 norm can promote sparsity of a regularized solution. The goal of this paper is to understand theoretically the effect of the regularization parameter on the sparsity of the regularized solutions. We establish a characterization of the sparsity under the transform matrix of the solution. When the objective function is block-separable or an error bound of the regularized solution to a known function is available, the resulting characterization can be taken as a regularization parameter choice strategy with which the regularization problem has a solution having a sparsity of a certain level. When the objective function is not block-separable, we propose an iterative algorithm which simultaneously determines the regularization parameter and its corresponding solution with a prescribed sparsity level. Moreover, we study choices of the regularization parameter so that the regularization term can alleviate the ill-posedness and promote sparsity of the resulting regularized solution. Numerical experiments demonstrate that the proposed algorithm is effective and efficient, and the choices of the regularization parameters can balance the sparsity of the regularized solution and its approximation to the minimizer of the fidelity function.

show abstract

Section: Parameter Choices For Sparsity Of Transformed Solutionsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Parameter choices for sparse regularization with the ℓ1 norm ^*

Liu¹,

Wang²,

Xu³

et al. 2023

Inverse Problems

View full text Add to dashboard Cite

show abstract

“…The more general functional RKHS was studied in [27]. Even vector-valued reproducing kernel Banach spaces [15,29] has been defined and constructed.…”

Section: Introductionmentioning

confidence: 99%

Learning Rates for Multi-task Regularization Networks

Gui¹,

Zhang²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Multi-task learning is an important trend of machine learning in facing the era of artificial intelligence and big data. Despite a large amount of researches on learning rate estimates of various single-task machine learning algorithms, there is little parallel work for multi-task learning. We present mathematical analysis on the learning rate estimate of multi-task learning based on the theory of vector-valued reproducing kernel Hilbert spaces and matrix-valued reproducing kernels. For the typical multi-task regularization networks, an explicit learning rate dependent both on the number of sample data and the number of tasks is obtained. It reveals that the generalization ability of multi-task learning algorithms is indeed affected as the number of tasks increases.

show abstract