Weighted least squares support vector machines: robustness and sparse approximation

Suykens, Johan A. K.; Brabanter, Jos De; Lukáš, Luděk; Vandewalle, Joos

doi:10.1016/s0925-2312(01)00644-0

Cited by 1,095 publications

(570 citation statements)

References 22 publications

Supporting

Mentioning

567

Contrasting

Unclassified

Order By: Relevance

“…However, one inherent problem of the image-as-vector representation lies in that the spatial redundancies within each image matrix are not fully utilized, and some of the information about local spatial relationships is lost [7,8]. On the other hand, although computational efficiency in LS-SVM is raised compared to SVM, loss of sparseness in its solution [9] makes every input pattern (e.g. the dimension of an input pattern is d 1 × d 2 and the number of the input patterns is l) contribute to the model, thus leading to LS-SVM have to store the training set for subsequent classification such that the memory overhead is greatly increased.…”

Section: Introductionmentioning

confidence: 99%

New Least Squares Support Vector Machines Based on Matrix Patterns

Wang

Chen

2007

Neural Process Lett

View full text Add to dashboard Cite

Support vector machine (SVM), as an effective method in classification problems, tries to find the optimal hyperplane that maximizes the margin between two classes and can be obtained by solving a constrained optimization criterion using quadratic programming (QP). This QP leads to higher computational cost. Least squares support vector machine (LS-SVM), as a variant of SVM, tries to avoid the above shortcoming and obtain an analytical solution directly from solving a set of linear equations instead of QP. Both SVM and LS-SVM operate directly on patterns represented by vector, i.e., before applying SVM or LS-SVM to a pattern, any non-vector pattern such as an image has to be first vectorized into a vector pattern by some techniques like concatenation. However, some implicit structural or local contextual information may be lost in this transformation. Moreover, as the dimension d of the weight vector in SVM or LS-SVM with the linear kernel is equal to the dimension d 1 × d 2 of the original input pattern, as a result, the higher the dimension of a vector pattern is, the more space is needed for storing it. In this paper, inspired by the method of feature extraction directly based on matrix patterns and the advantages of LS-SVM, we propose a new classifier design method based on matrix patterns, called MatLSSVM, such that the new method can not only directly operate on original matrix patterns, but also efficiently reduce memory for the weight vector (d) fromHowever like LS-SVM, MatLSSVM inherits LS-SVM's existence of unclassifiable regions when extended to multiclass problems. Thus with the fuzzy version of LS-SVM, a corresponding fuzzy version of MatLSSVM (MatFLSSVM) is further proposed to remove unclassifiable regions effectively for multi-class problems. Experimental results on some benchmark datasets show that the proposed method is competitive in classification performance compared to LS-SVM, fuzzy LS-SVM (FLS-SVM), more-recent MatPCA and MatFLDA. In addition, more importantly, the idea used here has a possibility of providing a novel way of constructing learning model.

show abstract

Section: Introductionmentioning

confidence: 99%

New Least Squares Support Vector Machines Based on Matrix Patterns

Wang

Chen

2007

Neural Process Lett

View full text Add to dashboard Cite

show abstract

“…To improve feasibility, SVM has been developed during the last decade and some effective modified SVM versions have been proposed, one of which is the Least Squares SVM (LS-SVM). Rather than solve the problem of convex quadratic programming that is required in standard SVM, LS-SVM provides a simpler solution by solving a linear matrix equation [30,31] …”

Section: Support Vector Regressionmentioning

confidence: 99%

“…Further comparison is carried out by combining the SVM identification and inverse dynamic compensation that is commonly employed to obtain a feedback controller for a certain system (without uncertainties). By observing the inequality (24), the auxiliary controllers u1 and u2 can be designed as Comparing controller (54) with (31), one can find that the controller with inverse dynamic compensation in combination with SVM identification (given by (54)) is more complicated than the controller in which the complicated nonlinear function are indentified by SVM (given by (31)). Figure 5 presents the simulation results using controller (54).…”

Section: Example Studymentioning

confidence: 99%

“…In comparison with the results shown in Figure 3, it can be seen that oscillation of the command rudder angle happens at the initial stage and at the time of instantaneous external disturbance despite rapid convergence. Further remarks can be made as follows: (i) no matter whether inverse dynamic compensation is incorporated, the stability and robustness of the control system can be guaranteed due to the use of Lyapunov theory and L2-gain design; (ii) although the response rate of the controller in the case of inverse dynamic compensation (as described by (54)) is better than the case without compensation (as described by (31)), a moderate helming as shown in Figure 3, which can be achieved by using the controller (31), is preferable from the practical point of view. Moreover, as can be seen, the structure of the controller (31) is simpler than the controller (54), which makes sense in practice.…”

Section: Example Studymentioning

confidence: 99%

See 1 more Smart Citation

Control for Ship Course-Keeping Using Optimized Support Vector Machines

Luo

Cong

2016

Algorithms

View full text Add to dashboard Cite

Support vector machines (SVM) are proposed in order to obtain a robust controller for ship course-keeping. A cascaded system is constructed by combining the dynamics of the rudder actuator with the dynamics of ship motion. Modeling errors and disturbances are taken into account in the plant. A controller with a simple structure is produced by applying an SVM and L2-gain design. The SVM is used to identify the complicated nonlinear functions and the modeling errors in the plant. The Lagrangian factors in the SVM are obtained using on-line tuning algorithms. L2-gain design is applied to suppress the disturbances. To obtain the optimal parameters in the SVM, then particle swarm optimization (PSO) method is incorporated. The stability and robustness of the close-loop system are confirmed by Lyapunov stability analysis. Numerical simulation is performed to demonstrate the validity of the proposed hybrid controller and its superior performance over a conventional PD controller.

show abstract

“…These models can be robustified and sparsified as explained in [7]. Many algorithms for solving the linear system require a positive definite matrix which is not the case here.…”

Section: Nonlinear Function Estimation Using Ls-svmsmentioning

confidence: 99%

Compactly Supported RBF Kernels for Sparsifying the Gram Matrix in LS-SVM Regression Models

Hamers

Suykens

Moor

2002

Artificial Neural Networks — ICANN 2002

View full text Add to dashboard Cite

Abstract. In this paper we investigate the use of compactly supported RBF kernels for nonlinear function estimation with LS-SVMs. The choice of compact kernels recently proposed by Genton may lead to computational improvements and memory reduction. Examples however illustrate that compactly supported RBF kernels may lead to severe loss in generalization performance for some applications, e.g. in chaotic time-series prediction. As a result, the usefulness of such kernels may be much more application dependent than the use of the RBF kernel.

show abstract

Weighted least squares support vector machines: robustness and sparse approximation

Cited by 1,095 publications

References 22 publications

New Least Squares Support Vector Machines Based on Matrix Patterns

New Least Squares Support Vector Machines Based on Matrix Patterns

Control for Ship Course-Keeping Using Optimized Support Vector Machines

Compactly Supported RBF Kernels for Sparsifying the Gram Matrix in LS-SVM Regression Models

Contact Info

Product

Resources

About