Hidden Space Support Vector Machines

Abstract

Abstract-Hidden space support vector machines (HSSVMs) are presented in this paper. The input patterns are mapped into a high-dimensional hidden space by a set of hidden nonlinear functions and then the structural risk is introduced into the hidden space to construct HSSVMs. Moreover, the conditions for the nonlinear kernel function in HSSVMs are more relaxed, and even differentiability is not required. Compared with support vector machines (SVMs), HSSVMs can adopt more kinds of kernel functions because the positive definite property of the kernel function is not a necessary condition. The performance of HSSVMs for pattern recognition and regression estimation is also analyzed. Experiments on artificial and real-world domains confirm the feasibility and the validity of our algorithms.

Feature space versus empirical kernel map and row kernel space in SVMs

“…In this paper, we reinforce the argument of [25] within a more general framework. Our main purpose in adding a second SVM is not to improve the conventional SVM training methods but to support them.…”

“…From Tables 2 and 3, it seems that the skewness of X is not too severe and can be easily adjusted by all the homogeneous POLY kernels. Furthermore, the improvements by the various POLY kernels are fairly close to that of a simple low-order kernel, e.g., the first-order as in [25], homogeneous POLY kernel is adequate to perform the second SVM's task. The values given in the bold font for the second SVMs are larger than the ones for the corresponding first SVMs…”

“…Equation (43) shows that in K the distances from N to SVs The purpose of this section is to demonstrate that the separating hyperplane is well positioned in H, rather than to present an improved SVM training method, although a special form of our method is the model of the HSSVM [25]. We demonstrate that the separating hyperplane in H is slightly manipulable through operations in U.…”

“…which has, from the architectural viewpoint, degenerated into its special case, the HSSVM of [25]. Clearly, one can compute K 0 in advance before training an SVM, and the computation of K 0 needs only be done once.…”

“…hyperplanes in H. A special case of the framework with a second SVM was studied in [25], and [25] found that their classification accuracies are similar to those of conventional SVMs. In this paper, we reinforce the argument of [25] within a more general framework.…”