Divergence-based classification in learning vector quantization

Mwebaze, Ernest; Schneider, Petra; Schleif, Frank-Michael; Aduwo, Jennifer R.; Quinn, John A.; Haase, Sven; Villmann, Thomas; Biehl, Michael

doi:10.1016/j.neucom.2010.10.016

Cited by 49 publications

(33 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Obviously, the Euclidean distance is not based on a functional norm [2,3,23]. Yet, the transfer to real functional norms and distances like Sobolev norms [24,25], the Lee-norm [23,1], kernel based LVQ-approaches [26] or divergence based similarity measures [27,28], which carry the functional aspect inherently, is straightforward and topic of future investigations.…”

Section: Resultsmentioning

confidence: 99%

Functional relevance learning in generalized learning vector quantization

et al. 2012

View full text Add to dashboard Cite

Citation for published version (APA): Kästner, M., Hammer, B., Biehl, M., & Villmann, T. (2012). Functional relevance learning in generalized learning vector quantization. Neurocomputing, 90, 85-95. DOI: 10.1016/j.neucom.2011 Copyright Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).Take-down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum. b s t r a c tRelevance learning in learning vector quantization is a central paradigm for classification task depending feature weighting and selection. We propose a functional approach to relevance learning for high-dimensional functional data. For this purpose we compose the relevance profile by a superposition of only a few parametrized basis functions taking into account the functional character of the data. The number of these parameters is usually significantly smaller than the number of relevance weights in standard relevance learning, which is the number of data dimensions. Thus, instabilities in learning are avoided and an inherent regularization takes place. In addition, we discuss strategies to obtain sparse relevance models for further model optimization.

show abstract

Section: Resultsmentioning

confidence: 99%

Functional relevance learning in generalized learning vector quantization

et al. 2012

View full text Add to dashboard Cite

show abstract

“…Adequate processing requires a precise interpretation of the differentiability, the most convenient in context of GLVQ seems to be the so-called Wirtinger calculus [44]. Divergences for density and histogram data are considered in [45,46], metrics for functional data were reported in [47,48,49]. Correlation based GLVQ as preferred frequently in biological application was introduced in [50] using the differentiability of the Pearson correlation.…”

Section: Beyond the Euclidean World -Glvq With Non-standard Dissimilamentioning

confidence: 99%

Can Learning Vector Quantization be an Alternative to SVM and Deep Learning? - Recent Trends and Advanced Variants of Learning Vector Quantization for Classification Learning

Villmann

Bohnsack

Kaden

2016

Journal of Artificial Intelligence and Soft Computing Research

View full text Add to dashboard Cite

Learning vector quantization (LVQ) is one of the most powerful approaches for prototype based classification of vector data, intuitively introduced by Kohonen. The prototype adaptation scheme relies on its attraction and repulsion during the learning providing an easy geometric interpretability of the learning as well as of the classification decision scheme. Although deep learning architectures and support vector classifiers frequently achieve comparable or even better results, LVQ models are smart alternatives with low complexity and computational costs making them attractive for many industrial applications like intelligent sensor systems or advanced driver assistance systems.Nowadays, the mathematical theory developed for LVQ delivers sufficient justification of the algorithm making it an appealing alternative to other approaches like support vector machines and deep learning techniques.This review article reports current developments and extensions of LVQ starting from the generalized LVQ (GLVQ), which is known as the most powerful cost function based realization of the original LVQ. The cost function minimized in GLVQ is an soft-approximation of the standard classification error allowing gradient descent learning techniques. The GLVQ variants considered in this contribution, cover many aspects like bordersensitive learning, application of non-Euclidean metrics like kernel distances or divergences, relevance learning as well as optimization of advanced statistical classification quality measures beyond the accuracy including sensitivity and specificity or area under the ROC-curve.According to these topics, the paper highlights the basic motivation for these variants and extensions together with the mathematical prerequisites and treatments for integration into the standard GLVQ scheme and compares them to other machine learning approaches. For detailed description and mathematical theory behind all, the reader is referred to the respective original articles.Thus, the intention of the paper is to provide a comprehensive overview of the stateof-the-art serving as a starting point to search for an appropriate LVQ variant in case of a given specific classification problem as well as a reference to recently developed variants and improvements of the basic GLVQ scheme.

show abstract

“…The incorporation of symmetric and non-symmetric, differentiable divergences into GLVQ training and classification is introduced in [37]. As an application example, the detection of Mosaic Disease in Cassava plants based on various image histograms is discussed there.…”

Section: Divergencesmentioning

confidence: 99%

Distance Measures for Prototype Based Classification

Biehl¹,

Hammer

Villmann

2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. The basic concepts of distance based classification are introduced in terms of clear-cut example systems. The classical k-NearestNeigbhor (kNN) classifier serves as the starting point of the discussion. Learning Vector Quantization (LVQ) is introduced, which represents the reference data by a few prototypes. This requires a data driven training process; examples of heuristic and cost function based prescriptions are presented. While the most popular measure of dissimilarity in this context is the Euclidean distance, this choice is frequently made without justification. Alternative distances can yield better performance in practical problems. Several examples are discussed, including more general Minkowski metrics and statistical divergences for the comparison of, e.g., histogram data. Furthermore, the framework of relevance learning in LVQ is presented. There, parameters of adaptive distance measures are optimized in the training phase. A practical application of Matrix Relevance LVQ in the context of tumor classification illustrates the approach.

show abstract

Divergence-based classification in learning vector quantization

Cited by 49 publications

References 16 publications

Functional relevance learning in generalized learning vector quantization

Functional relevance learning in generalized learning vector quantization

Can Learning Vector Quantization be an Alternative to SVM and Deep Learning? - Recent Trends and Advanced Variants of Learning Vector Quantization for Classification Learning

Distance Measures for Prototype Based Classification

Contact Info

Product

Resources

About