A GP-based kernel construction and optimization method for RVM

Abstract

Selecting a suitable kernel for relevance vector machine is one of most challenging aspects of successfully using this learning tool. Efficiently automating the search for such a kernel is therefore desirable. This paper proposes a datadriven kernel function construction and optimization method, which combines genetic programming(GP) and relevance vector regression to evolve an optimal or near-optimal kernel function, named GP-Kernel. The evolved kernel is compared to several widely used kernels on several regression benchmark datasets. Empirical results demonstrate that RVM using such GP-Kernel can outperform or match the best performance of standard kernels.

In-depth analysis of SVM kernel learning and its components

“…Another undesirable downside of being able to compose more compact and flexible kernels is that these methods may generate non-Mercer kernels during the search. The simplest method to deal with this problem is simply not to guarantee that kernels meet Mercer's condition [3,24,19,49]. As a result, the optimization algorithm used to find the optimal hyperplane for SVMs may not converge.…”

“…On the other hand, the Periodic kernel is based on the RBF kernel but adds a spectral transformation to the space [23] in order to model periodic patterns in the data. Periodic elements, such as the spectral transformation, have been overlooked in the kernel search literature, and only some approaches [3] include them in their grammars.…”

“…All these algorithms were coded in Python, based on the EA software DEAP 1 [17], and made publicly available in the Python Package Index 2 . The code used to carry out the SVM classification with evolved kernels has been also published 3 .…”

Evolving Gaussian Process Kernels for Translation Editing Effort Estimation

“…In the GP literature this has been done by combining known kernels [20,11,22]. Kernels have also been evolved for Support Vector Machines (SVMs) [17,13,8,38,9,18] and Relevance Vector Machines (RVMs) [3]. Some of the SVM approaches are also based on combining the well-known kernels [38,9], although in some other works the kernels are learned from simple mathematical expressions [17,13,8,18].…”

Sentiment analysis with genetically evolved gaussian kernels

“…In the GaussProc literature this has been done by combining known kernels [30,31,32]. Kernels have also been evolved for Support Vector Machines (SVMs) [33,34,35,36,37,38] and Relevance Vector Machines (RVMs) [39]. Some of the SVM approaches are also based in combining the well-known kernels [36,37], although in some other works the kernels are learned from simple mathematical expressions [33,34,35,38].…”