L2-norm multiple kernel learning and its application to biomedical data fusion

Abstract

BackgroundThis paper introduces the notion of optimizing different norms in the dual problem of support vector machines with multiple kernels. The selection of norms yields different extensions of multiple kernel learning (MKL) such as L∞, L1, and L2 MKL. In particular, L2 MKL is a novel method that leads to non-sparse optimal kernel coefficients, which is different from the sparse kernel coefficients optimized by the existing L∞ MKL method. In real biomedical applications, L2 MKL may have more advantages over sparse integration method for thoroughly combining complementary information in heterogeneous data sources.ResultsWe provide a theoretical analysis of the relationship between the L2 optimization of kernels in the dual problem with the L2 coefficient regularization in the primal problem. Understanding the dual L2 problem grants a unified view on MKL and enables us to extend the L2 method to a wide range of machine learning problems. We implement L2 MKL for ranking and classification problems and compare its performance with the sparse L∞ and the averaging L1 MKL methods. The experiments are carried out on six real biomedical data sets and two large scale UCI data sets. L2 MKL yields better performance on most of the benchmark data sets. In particular, we propose a novel L2 MKL least squares support vector machine (LSSVM) algorithm, which is shown to be an efficient and promising classifier for large scale data sets processing.ConclusionsThis paper extends the statistical framework of genomic data fusion based on MKL. Allowing non-sparse weights on the data sources is an attractive option in settings where we believe most data sources to be relevant to the problem at hand and want to avoid a "winner-takes-all" effect seen in L∞ MKL, which can be detrimental to the performance in prospective studies. The notion of optimizing L2 kernels can be straightforwardly extended to ranking, classification, regression, and clustering algorithms. To tackle the computational burden of MKL, this paper proposes several novel LSSVM based MKL algorithms. Systematic comparison on real data sets shows that LSSVM MKL has comparable performance as the conventional SVM MKL algorithms. Moreover, large scale numerical experiments indicate that when cast as semi-infinite programming, LSSVM MKL can be solved more efficiently than SVM MKL.AvailabilityThe MATLAB code of algorithms implemented in this paper is downloadable from http://homes.esat.kuleuven.be/~sistawww/bioi/syu/l2lssvm.html.

Improved modeling of clinical data with kernel methods

“…Moreover, the clinical kernel function not only outperformed the linear and polynomial kernel when using the LS-SVM classifier. It also performed well in combination with the regular SVM classifier [42].…”

Data Fusion by Matrix Factorization

“…, where ✓ i are weights of the kernel matrices, is a parameter determining the norm of constraint posed on coe cients (for L 2 , L p -norm MKL, see (Kloft et al, 2009;Yu et al, 2010;) and K i are normalized kernel matrices centered in the Hilbert space. Among other improvements, Yu et al (2010) extended the framework of the MKL in Lanckriet et al (2004) (Lanckriet et al, 2004a) by optimizing various norms in the dual problem of SVMs that allows non-sparse optimal kernel coe cients ✓ ⇤ i .…”

Heterogeneous Visual Features Fusion via Sparse Multimodal Machine

“…We classify the images in the above three data sets using the proposed methods by integrating the six types of image features of each of them. We compare the proposed method against several most recent multiple [20]. Besides, we also compare our method to three most recent multi-model image classification methods published in computer vision community, including Gaussian process (GP) method [7], LPBoost-β method [5] and LPBoost-B method [5], which have demonstrated state-of-the-art object categorization performance.…”

“…We implement the compared MKL methods using the codes published by [20]. Following [20], in LSSVM ∞ and 2 methods, the regularization parameter λ is estimated jointly as the kernel coefficient of an identity matrix; in LSSVM 1 method, λ is set to 1; in all other SVM approaches, the C parameter of the box constraint is fine tuned in the same range as γ. For LPBoost-β and LPBoost-B methods, we use the codes published by the authors 4 .…”