Fusion of deep learning architectures, multilayer feedforward networks and learning vector quantizers for deep classification learning

Villmann, Thomas; Biehl, Michael; Villmann, Andrea; Saralajew, Sascha

doi:10.1109/wsom.2017.8020009

Cited by 12 publications

(4 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiple studies explored the integration of the hierarchical feature-extracting capability of deep feature extractors with VQ [12,50,4] which were also later adapted to Continual Learning. TPCIL [46] proposed to retain the topology of the feature space to preserve old knowledge over the increments.…”

Section: Vector Quantizationmentioning

confidence: 99%

Dimensionality reduction for visualizing high-dimensional biological data

Malepathirana¹,

Senanayake²,

Vidanaarachchi³

et al. 2022

Biosystems

View full text Add to dashboard Cite

Section: Vector Quantizationmentioning

confidence: 99%

Dimensionality reduction for visualizing high-dimensional biological data

Malepathirana¹,

Senanayake²,

Vidanaarachchi³

et al. 2022

Biosystems

View full text Add to dashboard Cite

“…This allows to rewrite both of them using equivalent operations. This strategy is already described for different levels of abstraction in [53,54,55,56,57].…”

Section: A Different View On Fully-connected Layersmentioning

confidence: 99%

“…One of the first contributions reporting an approach for the fusion of LVQ with NNs is [53]. Later on, the idea was formulated more precisely [54]. In [57], a fused network was applied to train a network on MNIST and Cifar10.…”

Section: Related Workmentioning

confidence: 99%

Prototype-based Neural Network Layers: Incorporating Vector Quantization

Saralajew,

Holdijk,

Rees

et al. 2018

Preprint

View full text Add to dashboard Cite

Neural networks currently dominate the machine learning community and they do so for good reasons. Their accuracy on complex tasks such as image classification is unrivaled at the moment and with recent improvements they are reasonably easy to train. Nevertheless, neural networks are lacking robustness and interpretability. Prototype-based vector quantization methods on the other hand are known for being robust and interpretable. For this reason, we propose techniques and strategies to merge both approaches. This contribution will particularly highlight the similarities between them and outline how to construct a prototype-based classification layer for multilayer networks. Additionally, we provide an alternative, prototypebased, approach to the classical convolution operation. Numerical results are not part of this report, instead the focus lays on establishing a strong theoretical framework. By publishing our framework and the respective theoretical considerations and justifications before finalizing our numerical experiments we hope to jumpstart the incorporation of prototype-based learning in neural networks and vice versa.

show abstract

“…deep neural networks in applications that involve sophisticated feature extraction, vector quantization methods can be found incorporated in current deep learning architectures [15], [22], [23], and have recently shown impressive robustness against adversarial attacks, suggesting suitability in security critical applications [24].…”

Section: Introductionmentioning

confidence: 99%

Online Deterministic Annealing for Classification and Clustering

Mavridis,

Baras

2021

Preprint

View full text Add to dashboard Cite

We introduce an online prototype-based learning algorithm for clustering and classification, based on the principles of deterministic annealing. We show that the proposed algorithm constitutes a competitive-learning neural network, the learning rule of which is formulated as an online stochastic approximation algorithm. The annealing nature of the algorithm prevents poor local minima, offers robustness with respect to the initial conditions, and provides a means to progressively increase the complexity of the learning model as needed, through an intuitive bifurcation phenomenon. As a result, the proposed approach is interpretable, requires minimal hyper-parameter tuning, and offers online control over the complexity-accuracy trade-off. Finally, Bregman divergences are used as a family of dissimilarity measures that are shown to play an important role in both the performance of the algorithm, and its computational complexity. We illustrate the properties and evaluate the performance of the proposed learning algorithm in artificial and real datasets.

show abstract

Fusion of deep learning architectures, multilayer feedforward networks and learning vector quantizers for deep classification learning

Cited by 12 publications

References 44 publications

Dimensionality reduction for visualizing high-dimensional biological data

Dimensionality reduction for visualizing high-dimensional biological data

Prototype-based Neural Network Layers: Incorporating Vector Quantization

Online Deterministic Annealing for Classification and Clustering

Contact Info

Product

Resources

About