Kernelized vector quantization in gradient-descent learning

Villmann, Thomas; Haase, Sven; Kaden, Marika

doi:10.1016/j.neucom.2013.11.048

Cited by 33 publications

(15 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A more elegant way is to replace the Euclidean distance directly by the kernel distance d κ generated from the kernel κ. If the kernel distance is d κ (v, w k ) is differentiable like the RBF kernel (23), we can immediately plugin this into GLVQ obtaining a kernel variant, which works exactly in the same Hilbert space [57]. Obviously, this trick could also be applied to RSLVQ.…”

Section: Beyond the Euclidean World -Glvq With Non-standard Dissimilamentioning

confidence: 99%

Can Learning Vector Quantization be an Alternative to SVM and Deep Learning? - Recent Trends and Advanced Variants of Learning Vector Quantization for Classification Learning

Villmann

Bohnsack

Kaden

2016

Journal of Artificial Intelligence and Soft Computing Research

Self Cite

View full text Add to dashboard Cite

Learning vector quantization (LVQ) is one of the most powerful approaches for prototype based classification of vector data, intuitively introduced by Kohonen. The prototype adaptation scheme relies on its attraction and repulsion during the learning providing an easy geometric interpretability of the learning as well as of the classification decision scheme. Although deep learning architectures and support vector classifiers frequently achieve comparable or even better results, LVQ models are smart alternatives with low complexity and computational costs making them attractive for many industrial applications like intelligent sensor systems or advanced driver assistance systems.Nowadays, the mathematical theory developed for LVQ delivers sufficient justification of the algorithm making it an appealing alternative to other approaches like support vector machines and deep learning techniques.This review article reports current developments and extensions of LVQ starting from the generalized LVQ (GLVQ), which is known as the most powerful cost function based realization of the original LVQ. The cost function minimized in GLVQ is an soft-approximation of the standard classification error allowing gradient descent learning techniques. The GLVQ variants considered in this contribution, cover many aspects like bordersensitive learning, application of non-Euclidean metrics like kernel distances or divergences, relevance learning as well as optimization of advanced statistical classification quality measures beyond the accuracy including sensitivity and specificity or area under the ROC-curve.According to these topics, the paper highlights the basic motivation for these variants and extensions together with the mathematical prerequisites and treatments for integration into the standard GLVQ scheme and compares them to other machine learning approaches. For detailed description and mathematical theory behind all, the reader is referred to the respective original articles.Thus, the intention of the paper is to provide a comprehensive overview of the stateof-the-art serving as a starting point to search for an appropriate LVQ variant in case of a given specific classification problem as well as a reference to recently developed variants and improvements of the basic GLVQ scheme.

show abstract

Section: Beyond the Euclidean World -Glvq With Non-standard Dissimilamentioning

confidence: 99%

Can Learning Vector Quantization be an Alternative to SVM and Deep Learning? - Recent Trends and Advanced Variants of Learning Vector Quantization for Classification Learning

Villmann

Bohnsack

Kaden

2016

Journal of Artificial Intelligence and Soft Computing Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…First integration attempts of kernel distances into GLVQ were suggested in [72] and [80] using various approximation techniques to determine the gradient learning in the kernel associated Hilbert space H. An elementary alternative is the utilization of differentiable universal kernels [103] based on the theory of universal kernels [61,88,94]. This approach allows the adaptation of the prototypes in the original data space but equipped with the kernel distance generated by the differentiable kernel, i.e.…”

Section: Appropriate Metrics and Metric Adaptation For Vector Datamentioning

confidence: 99%

“…This approach allows the adaptation of the prototypes in the original data space but equipped with the kernel distance generated by the differentiable kernel, i.e. the metric space (V, d κΦ ) [104,103]. Hence, such a distance is also differentiable according to (36), see For example, exponential kernels are universal, which can be used together with the above mentioned Minkowski-p-norms and the linear data mapping (35), revealing The natural extension of vector quantization is matrix quantization.…”

Section: Appropriate Metrics and Metric Adaptation For Vector Datamentioning

confidence: 99%

Aspects in Classification Learning - Review of Recent Developments in Learning Vector Quantization

Kaden¹,

Lange²,

Nebel³

et al. 2014

Foundations of Computing and Decision Sciences

View full text Add to dashboard Cite

Abstract. Classification is one of the most frequent tasks in machine learning. However, the variety of classification tasks as well as classifier methods is huge. Thus the question is coming up: which classifier is suitable for a given problem or how can we utilize a certain classifier model for different tasks in classification learning. This paper focuses on learning vector quantization classifiers as one of the most intuitive prototype based classification models. Recent extensions and modifications of the basic learning vector quantization algorithm, which are proposed in the last years, are highlighted and also discussed in relation to particular classification task scenarios like imbalanced and/or incomplete data, prior data knowledge, classification guarantees or adaptive data metrics for optimal classification.

show abstract

“…There are several methods that have been used to optimize the kernel and select the regression parameters, such as cross validation learning [5,6], gradient descent learning [7,8], evolutionary learning [9,10], and positive semidefinite programming learning [11,12]. The model of support vector regression and the selection of kernel parameters are relatively few, and it primarily uses the grid search cross validation method and evolutionary method.…”

Section: Introductionmentioning

confidence: 99%

Parameter Selection Method for Support Vector Regression Based on Adaptive Fusion of the Mixed Kernel Function

Wang

2017

Journal of Control Science and Engineering

View full text Add to dashboard Cite

Support vector regression algorithm is widely used in fault diagnosis of rolling bearing. A new model parameter selection method for support vector regression based on adaptive fusion of the mixed kernel function is proposed in this paper. We choose the mixed kernel function as the kernel function of support vector regression. The mixed kernel function of the fusion coefficients, kernel function parameters, and regression parameters are combined together as the parameters of the state vector. Thus, the model selection problem is transformed into a nonlinear system state estimation problem. We use a 5th-degree cubature Kalman filter to estimate the parameters. In this way, we realize the adaptive selection of mixed kernel function weighted coefficients and the kernel parameters, the regression parameters. Compared with a single kernel function, unscented Kalman filter (UKF) support vector regression algorithms, and genetic algorithms, the decision regression function obtained by the proposed method has better generalization ability and higher prediction accuracy.

show abstract

Kernelized vector quantization in gradient-descent learning

Cited by 33 publications

References 66 publications

Can Learning Vector Quantization be an Alternative to SVM and Deep Learning? - Recent Trends and Advanced Variants of Learning Vector Quantization for Classification Learning

Can Learning Vector Quantization be an Alternative to SVM and Deep Learning? - Recent Trends and Advanced Variants of Learning Vector Quantization for Classification Learning

Aspects in Classification Learning - Review of Recent Developments in Learning Vector Quantization

Parameter Selection Method for Support Vector Regression Based on Adaptive Fusion of the Mixed Kernel Function

Contact Info

Product

Resources

About