Training Lagrangian twin support vector regression via unconstrained convex minimization

Wang

IEEJ Transactions Elec Engng

2017

Support vector regression (SVR) has unique advantages in system modeling because of the structural risk minimization principle. However, its model generalization ability is subject to excitation signals and kernel parameters. An SVR modeling method excited by sinusoidal signals based on kernel space features is proposed to improve the generalization ability. First, the effect of amplitude of the sinusoidal excitation signals on the generalization ability of SVR model is analyzed. The following conclusions are drawn. The amplitude of the best excitation signal should be in the same order of magnitude as that of actual input signal and not smaller than the amplitude of the actual input signal. Second, based on the considered SVR modeling problem in the classification problem, the kernel space feature named inter-cluster distance (ICD) is employed to determine a more effective searching range of the kernel parameter. Simulation experiments results show less training time and the superior generalization ability of the proposed method. Thus, the proposed method can be used as a guide to select the sinusoidal excitation signals and kernels parameters of SVR models.

“…In this section, four functions are employed to verify the feasibility of the ROGA-SVR [23,24]. The definitions of these functions are listed in Table I.…”

Section: Simulation Examplesmentioning

confidence: 99%

One SVR modeling method based on kernel space feature

Wang

IEEJ Transactions Elec Engng

2017

“…Experimental results show the effectiveness of TWSVM over SVM and GEPSVM [22]. TWSVM takes O 1 4 m 3 operations which is 1/4 of the standard SVM, whereas GEPSVM takes O 1 4 n 3 . Here, m is the number of training examples; n is the dimensionality and m >> n [22,24].…”

Section: Introductionmentioning

confidence: 95%

Linear programming twin support vector regression

Tanveer

2017

Filomat

In this paper, a new linear programming formulation of a 1-norm twin support vector regression is proposed whose solution is obtained by solving a pair of dual exterior penalty problems as unconstrained minimization problems using Newton method. The idea of our formulation is to reformulate TSVR as a strongly convex problem by incorporated regularization technique and then derive a new 1-norm linear programming formulation for TSVR to improve robustness and sparsity. Our approach has the advantage that a pair of matrix equation of order equals to the number of input examples is solved at each iteration of the algorithm. The algorithm converges from any starting point and can be easily implemented in MATLAB without using any optimization packages. The efficiency of the proposed method is demonstrated by experimental results on a number of interesting synthetic and real-world datasets.

“…However, Chapelle [ 47 ] observed that, by comparing the approximate efficiency of SVR in the primal space and the dual space, the approximate dual solution may not produce a good primal approximate solution. Some related work is directly solved in the primal space [ 48 , 49 , 50 , 51 ]. For example, inspired by the twin SVR and Newton methods, Balasundaram et al [ 49 ] proposed a new unconstrained Lagrangian TSVR (ULTSVR) to solve a pair of unconstrained minimization problems, thereby increasing the calculation speed.…”

Section: Introductionmentioning

confidence: 99%

Twin Least Square Support Vector Regression Model Based on Gauss-Laplace Mixed Noise Feature with Its Application in Wind Speed Prediction

Zhang

Liu

Wang

et al. 2020

Entropy

In this article, it was observed that the noise in some real-world applications, such as wind power forecasting and direction of the arrival estimation problem, does not satisfy the single noise distribution, including Gaussian distribution and Laplace distribution, but the mixed distribution. Therefore, combining the twin hyperplanes with the fast speed of Least Squares Support Vector Regression (LS-SVR), and then introducing the Gauss–Laplace mixed noise feature, a new regressor, called Gauss-Laplace Twin Least Squares Support Vector Regression (GL-TLSSVR), for the complex noise. Subsequently, we apply the augmented Lagrangian multiplier method to solve the proposed model. Finally, we apply the short-term wind speed data-set to the proposed model. The results of this experiment confirm the effectiveness of our proposed model.