Style Memory: Making a Classifier Network Generative

Abstract

Deep networks have shown great performance in classification tasks. However, the parameters learned by the classifier networks usually discard stylistic information of the input, in favour of information strictly relevant to classification. We introduce a network that has the capacity to do both classification and reconstruction by adding a "style memory" to the output layer of the network. We also show how to train such a neural network as a deep multi-layer autoencoder, jointly minimizing both classification and reconstruction losses. The generative capacity of our network demonstrates that the combination of style-memory neurons with the classifier neurons yield good reconstructions of the inputs when the classification is correct. We further investigate the nature of the style memory, and how it relates to composing digits and letters. Finally, we propose that this architecture enables the bidirectional flow of information used in predictive coding, and that such bidirectional networks can help mitigate against being fooled by ambiguous or adversarial input.

TextCaps: Handwritten Character Recognition With Very Small Datasets

“…First, we describe the results we obtained with full training sets and compare with the state-of-the-art. On EMNIST- [22] 89.7% -Bhatnagar et al [25] 92.54% -Zhong et al [26] 96.35% -TextCaps 93.71 ± 0.64% 85.36 ± 0.79% letters, we significantly outperform the state-of-the-art Wiyatno et al [14] by 4.09%. An average accuracy of 90.46% was achieved by our system for the EMNIST-balanced dataset, which outperforms the state-of-the-art Dufourq et al [13] by 2.16%.…”

“…With full train set With 200 samp/class Cohen et al [12] 85.15% -Wiyatnoet al [14] 91.27% -TextCaps 95.36 ± 0.30% 92.79 ± 0.30%…”

“…Apart from digit recognition, several attempts [12,13] have been made in handwritten character recognition with EMNIST datasets [12]. A bidirectional neural network is introduced in [14] which is capable of performing both image classification and image reconstruction by adding a style memory to the output layer of the network.…”