Feature space versus empirical kernel map and row kernel space in SVMs

Abstract

In machine-learning technologies, the support vector machine (SV machine, SVM) is a brilliant invention with many merits, such as freedom from local minima, the widest possible margins separating different clusters, and a solid theoretical foundation. In this paper, we first explore the linear separability relationships between the highdimensional feature space H and the empirical kernel map U as well as between H and the space of kernel outputs K. Second, we investigate the relations of the distances between separating hyperplanes and SVs in H and U, and derive an upper bound for the margin width in K. Third, as an application, we show experimentally that the separating hyperplane in H can be slightly adjusted through U. The experiments reveal that existing SVM training can linearly separate the data in H with considerable success. The results in this paper allow us to visualize the geometry of H by studying U and K.

Structural multiple empirical kernel learning

“…However, EKM can avoid the above issues and be more flexible. Liang et al [33] demonstrate that the separating hyperplane based on EKM is manipulable in the feature space. Further, Xiong et al [56] derive an effective kernel optimization algorithm through employing a datadependent kernel.…”

Electronic circuit fault diagnosis methods based on improved Support Vector Machines

“…Generally speaking, the dimension of original feature space is very high, sometimes infinite, but the dimension of empirical feature space is number of training instance at most [11] . So it is reasonable to process data in empirical feature space from point of theory and practice.…”

Design and implementation of support vector machines and information fusion methods for bio-medical decision support systems

“…The first layer through the primary kernel attempts to efficiently cluster the data, whereas the second one derives linear boundaries for class-separation of the clustered data. Furthermore, in (Liang 2010) the linear separability of samples in the SVM hidden space is examined and multiple parameterizations of polynomial second stage kernels are evaluated. The findings indicate that the linear decision plane in the hidden space might not be adequately optimal and thus more complex decision mapping may be needed.…”

“…Such mappings are used in DNA analysis for feature reduction (Sammon maps), and lately in social-network analysis to measure cluster compactness and affinity of single points to neighboring clusters. In view of its utility as a distance mapping, the two-layer kernel approach presents specific peculiarities that limit the predictive power of HS-SVMs compared to SVMs, as reported in (Liang 2010) (Liang 2010). More specifically, the first layer aims to cluster the input data in the augmented space (often denoted by φ(x)), by means of the distances among the samples defined by corresponding kernel.…”