Probabilistic Learning Vector Quantization with Cross-Entropy for Probabilistic Class Assignments in Classification Learning

Villmann, Andrea; Kaden, Marika; Saralajew, Sascha; Villmann, Thomas

doi:10.1007/978-3-319-91253-0_67

Cited by 14 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interpretability increases the trustworthiness and hence the acceptance of models for the potential users [27]. Further extensions improving transparency of the decision and already known for GLVQ approaches are the incorporation of reject options for ambiguous decisions or outliers as well as the use of interpretable probabilistic classifiers [133][134][135].…”

Section: Conclusion Remarks and Future Workmentioning

confidence: 99%

The Resolved Mutual Information Function as a Structural Fingerprint of Biomolecular Sequences for Interpretable Machine Learning Classifiers

Bohnsack

Kaden

Abel

et al. 2021

Entropy

Self Cite

View full text Add to dashboard Cite

In the present article we propose the application of variants of the mutual information function as characteristic fingerprints of biomolecular sequences for classification analysis. In particular, we consider the resolved mutual information functions based on Shannon-, Rényi-, and Tsallis-entropy. In combination with interpretable machine learning classifier models based on generalized learning vector quantization, a powerful methodology for sequence classification is achieved which allows substantial knowledge extraction in addition to the high classification ability due to the model-inherent robustness. Any potential (slightly) inferior performance of the used classifier is compensated by the additional knowledge provided by interpretable models. This knowledge may assist the user in the analysis and understanding of the used data and considered task. After theoretical justification of the concepts, we demonstrate the approach for various example data sets covering different areas in biomolecular sequence analysis.

show abstract

Section: Conclusion Remarks and Future Workmentioning

confidence: 99%

The Resolved Mutual Information Function as a Structural Fingerprint of Biomolecular Sequences for Interpretable Machine Learning Classifiers

Bohnsack

Kaden

Abel

et al. 2021

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…We note that the log-likelihood ratio used in RSLVQ can be extended to a more natural cross-entropy cost function employed in Probabilistic LVQ (PLVQ) (Villmann et al, 2018). In fact, PLVQ coincides with RSLVQ if the only stochastic component in the joint distribution p(x, y) over R n × {1, 2, ..., C} is the marginal over the inputs p(x) and the input-conditional class distributions p(y|x) are delta-functions (see (Villmann et al, 2018)). The framework of PLVQ is preferable in cases of genuine class uncertainty in (at least some regions of) the input space, leading to more representative class prototypes.…”

Section: Robust Soft Leaning Vector Quantizationmentioning

confidence: 99%

Probabilistic Learning Vector Quantization on Manifold of Symmetric Positive Definite Matrices

Tang

Feng

Tiňo

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we develop a new classification method for manifold-valued data in the framework of probabilistic learning vector quantization. In many classification scenarios, the data can be naturally represented by symmetric positive definite matrices, which are inherently points that live on a curved Riemannian manifold. Due to the non-Euclidean geometry of Riemannian manifolds, traditional Euclidean machine learning algorithms yield poor results on such data. In this paper, we generalize the probabilistic learning vector quantization algorithm for data points living on the manifold of symmetric positive definite matrices equipped with Riemannian natural metric (affine-invariant metric). By exploiting the induced Riemannian distance, we derive the probabilistic learning Riemannian space quantization algorithm, obtaining the learning rule through Riemannian gradient descent. Empirical investigations on synthetic data, image data , and motor imagery EEG data demonstrate the superior performance of the proposed method.

show abstract

“…Originally, the learning process of the prototypes was heuristically motivated, later it was refined to optimize a cost function which approximates the overall classification accuracy [32,33]. Probabilistic variants of LVQ are the Robust Soft LVQ (RSLVQ [34]), the soft nearest prototype classifier [35] or the probabilistic LVQ [36] to name just a few. See Fig.…”

Section: Vector Quantization and Clusteringmentioning

confidence: 99%

“…in the RSLVQ network, which obeys a Gaussian probability in presence of the squared Euclidean distance. It turns out that the maximum log-likelihood loss function of RSLVQ is equivalent to the cross entropy loss between p (x) and p (x) [36].…”

Section: Lvq Layers As Final Classification Layersmentioning

confidence: 99%

“…In contrast, most LVQ variants are optimized according to the underlying GLVQ-loss, which is a smooth approximation of the classification accuracy but, as already mentioned above, corresponds to a hypothesis margin maximizer. Probabilistic variants like RSLVQ rely on log-likelihood ratios, which turn out to be equivalent to the cross entropy loss under mild assumptions [36]. We wish that the PB community is inspired by the rapidly developing NN-world to think towards adaptation for vector quantization models.…”

Section: Shared Losses and Regularizations Between Pbs And Nnsmentioning

confidence: 99%

See 1 more Smart Citation

Prototype-based Neural Network Layers: Incorporating Vector Quantization

Saralajew,

Holdijk,

Rees

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

Neural networks currently dominate the machine learning community and they do so for good reasons. Their accuracy on complex tasks such as image classification is unrivaled at the moment and with recent improvements they are reasonably easy to train. Nevertheless, neural networks are lacking robustness and interpretability. Prototype-based vector quantization methods on the other hand are known for being robust and interpretable. For this reason, we propose techniques and strategies to merge both approaches. This contribution will particularly highlight the similarities between them and outline how to construct a prototype-based classification layer for multilayer networks. Additionally, we provide an alternative, prototypebased, approach to the classical convolution operation. Numerical results are not part of this report, instead the focus lays on establishing a strong theoretical framework. By publishing our framework and the respective theoretical considerations and justifications before finalizing our numerical experiments we hope to jumpstart the incorporation of prototype-based learning in neural networks and vice versa.

show abstract

Probabilistic Learning Vector Quantization with Cross-Entropy for Probabilistic Class Assignments in Classification Learning

Cited by 14 publications

References 13 publications

The Resolved Mutual Information Function as a Structural Fingerprint of Biomolecular Sequences for Interpretable Machine Learning Classifiers

The Resolved Mutual Information Function as a Structural Fingerprint of Biomolecular Sequences for Interpretable Machine Learning Classifiers

Probabilistic Learning Vector Quantization on Manifold of Symmetric Positive Definite Matrices

Prototype-based Neural Network Layers: Incorporating Vector Quantization

Contact Info

Product

Resources

About