Design of a multiple kernel learning algorithm for LS-SVM by convex programming

Jian, Ling; Xia, Zhonghang; Liang, Xijun; Gao, Chuanhou

doi:10.1016/j.neunet.2011.03.009

Cited by 30 publications

(13 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…G ¼ P M l¼1 a l K l ; a l P 0, [43,44] reconstruct QCQP as the Semi-Infinite Linear Program (SILP) which recycles the standard Support Vector Machine (SVM) implementations [5,11,21]. Moreover, Jian et al [24] address the issue of multiple kernels for Least Squares Support Vector Machine (LSSVM) by formulating a SemiDefinite Programming (SDP). Recently, researchers give some new formulations for multiple kernel optimization [15,45,34].…”

Section: Related Workmentioning

confidence: 99%

Structural multiple empirical kernel learning

Wang

Fan

Sheng

et al. 2015

Information Sciences

View full text Add to dashboard Cite

Section: Related Workmentioning

confidence: 99%

Structural multiple empirical kernel learning

Wang

Fan

Sheng

et al. 2015

Information Sciences

View full text Add to dashboard Cite

“…Substituting by and moving it to the constraint, we get the following QCQP problem: (29) Equations (27) and (29) are the QCQP problems with complexity…”

Section: B Multiple Kernel For Ls-svmmentioning

confidence: 99%

“…Standard software such as MOSEK can solve them effectively. The obtained first dual variables from solving (29) can act as the optimal kernel coefficients [18], [40]. It is thus easily to obtain the optimal regularization parameter .…”

Section: B Multiple Kernel For Ls-svmmentioning

confidence: 99%

Constructing Multiple Kernel Learning Framework for Blast Furnace Automation

Jian

Gao

Xia

2012

IEEE Trans. Automat. Sci. Eng.

Self Cite

View full text Add to dashboard Cite

This paper constructs the framework of the reproducing kernel Hilbert space for multiple kernel learning, which provides clear insights into the reason that multiple kernel support vector machines (SVM) outperform single kernel SVM. These results can serve as a fundamental guide to account for the superiority of multiple kernel to single kernel learning. Subsequently, the constructed multiple kernel learning algorithms are applied to model a nonlinear blast furnace system only based on its input-output signals. The experimental results not only confirm the superiority of multiple kernel learning algorithms, but also indicate that multiple kernel SVM is a kind of highly competitive data-driven modeling method for the blast furnace system and can provide reliable indication for blast furnace operators to take control actions. Note to Practitioners-This paper is motivated by the problem of predicting the silicon content in blast furnace hot metal, whichis an open problem for realizing blast furnace automation. Here, based on the single kernel and multiple kernel SVM, we pay special attention to the silicon trend prediction since it can provide more direct guideline for taking control action in the blast furnace operation. Theoretically, we have given the detailed reasons that multiple kernel SVM is superior to single kernel SVM, which can improve the transparency of multiple kernel learning algorithm. The experimental results, not only confirm the superiority of multiple kernel learning algorithms, but also indicate that multiple kernel SVM is a kind of highly competitive data-driven modeling method for the blast furnace system and can provide reliable indication for blast furnace operators to take control actions.Index Terms-Data-driven, multiple kernel support vector machines (SVM), nonlinear blast furnace system, quadratically constrained quadratic programming, reproducing kernel Hilbert space.

show abstract

“…Many researchers have pointed out that three crucial problems existing in SVR urgently need to be addressed: (1) How to choose or construct an appropriate kernel to complete forecasting problems [8,9]; (2) How to optimize parameters of SVR to improve the quality of prediction [10,11]; (3) How to construct a fast algorithm to operate in presence of large datasets [12,13]. With unsuitable kernel functions or hyperparameter settings, SVR may lead to poor prediction results.…”

Section: Introductionmentioning

confidence: 99%

Global Nonlinear Kernel Prediction for Large Data Set With a Particle Swarm-Optimized Interval Support Vector Regression

Ding

Cheng

Pedrycz

et al. 2015

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Abstract-A new global nonlinear predictor with a particle swarm optimized interval support vector regression (PSO-ISVR) is proposed to address three issues (viz. kernel selection, model optimization, kernel method speed) encountered when applying support vector regression (SVR) in the presence of large datasets. The novel prediction model can reduce the SVR computing overhead by dividing input space and adaptively selecting the optimized kernel functions to obtain optimal SVR parameter by PSO. To quantify the quality of the predictor, its generalization performance and execution speed are investigated based on statistical learning theory. In addition, experiments using synthetic data as well as the stock volume weighted average price (VWAP) are reported to demonstrate the effectiveness of the developed models. The experimental results show that the proposed PSO-ISVR predictor can improve the computational efficiency and the overall prediction accuracy compared with the results produced by the SVR and other regression methods. The proposed PSO-ISVR provides an important tool for nonlinear regression analysis of big data.Index Terms-global nonlinear predictor, interval support vector regression, particle swarm optimization, kernel function, sliding adaptive model, large data I. INTRODUCTIONUPPORT vector regression (SVR) model is constructed based on statistical learning theory [1], which uses a kernel function to map the data from some input space to a highdimensional feature space where the problem becomes amenable for handling by linear regression [2]. Owing to its robustness to noise and its generalization abilities, it has been widely employed in various areas such as adaptive flight identification [3], ore grade estimation [4] , and stock market price forecasting [5][6][7].Many researchers have pointed out that three crucial problems existing in SVR urgently need to be addressed: (1) How to choose or construct an appropriate kernel to complete forecasting problems [8,9]; (2) How to optimize parameters of SVR to improve the quality of prediction [10,11]; (3) How to construct a fast algorithm to operate in presence of large datasets [12,13]. With unsuitable kernel functions or hyperparameter settings, SVR may lead to poor prediction results. In fact, a kernel function forms a certain nonlinear transformation function. Due to data uncertainty in practical regression problems, it is difficult to determine which kernel function is the best one for a specific problem without any prior knowledge [14]. If the adjustable kernel parameters in SVR are not properly selected, it will result in the undesirable phenomena of over-fitting or under-fitting [1]. Furthermore, SVR is typically confronted with a heavy computing overhead due to processing large Gram matrices being associated with the kernels [13]. This computing burden becomes an essential barrier when dealing with massive data, such as those encountered in protein structure prediction and time series prediction [15].During the past several years, various methods h...

show abstract

Design of a multiple kernel learning algorithm for LS-SVM by convex programming

Cited by 30 publications

References 15 publications

Structural multiple empirical kernel learning

Structural multiple empirical kernel learning

Constructing Multiple Kernel Learning Framework for Blast Furnace Automation

Global Nonlinear Kernel Prediction for Large Data Set With a Particle Swarm-Optimized Interval Support Vector Regression

Contact Info

Product

Resources

About