Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning

Ng, Hongwei; Nguyễn, Việt Dũng; Vonikakis, Vassilios; Winkler, Stefan

doi:10.1145/2818346.2830593

Cited by 503 publications

(266 citation statements)

References 21 publications

Supporting

Mentioning

239

Contrasting

Unclassified

Order By: Relevance

“…A model of the well-known convolutional network VGG-16 [7] is given, the configuration of ten layers of which is shown in Fig. 1.…”

Section: Methodsmentioning

confidence: 99%

“…It is proposed to use the principles of neural gas and sparse coding for the training of the hierarchical extractor of visual features using the example of a multi-layered neural network VGG-16 [7,8]. At the same time, the efficiency evaluation of the extractor is supposed to be carried out based on the results of learning the information-extreme classifier with binary coding of observations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Development of the Method of Unsupervised Training of Convolutional Neural Networks Based on Neural Gas Modification

Москаленко

2017

Technology transfer: fundamental principles and innovative tech

View full text Add to dashboard Cite

Technologies for computer analysis of visual information based on convolutional neural networks have been widely used, but there is still a shortage of working algorithms for continuous unsupervised training and re-training of neural networks in real time, limiting the effectiveness of their functioning under conditions of nonstationarity and a priori uncertainty. In addition, the back propagation method for learning multi-layer neural networks requires significant computational resources and the amount of marked learning data, which makes it difficult to implement them in autonomous systems with limited resources. One approach to reducing the computational complexity of deep machine learning and overfitting is use of the neural gas principles to implement learning in the process of direct information propagation and sparse coding to increase the compactness and informativeness of feature representation. The paper considers the use of sparse coding neural gas for learning ten layers of the VGG-16 neural network on selective data from the ImageNet database. At the same time, it is suggested that the evaluation of the effectiveness of the feature extractor learning be carried out according to the results of so-called information-extreme machine learning with the teacher of the output classifier. Information-extreme learning is based on the principles of population optimization methods for binary coding of observations and the construction of radial-basic decision rules optimal in the information criterion in the binary Hamming space. According to the results of physical modeling, it is shown that learning without a teacher ensures the accuracy of decision rules to 96.4 %, which is inferior to the accuracy of learning with the teacher, which is equal to 98.7 %. However, the absence of an error in the training algorithm for the backward propagation of the error causes the prospect of further research towards the development of meta-optimization algorithms to refine the feature extractor's filters and parameters of the unsupervised training algorithm

show abstract

“…A model of the well-known convolutional network VGG-16 [7] is given, the configuration of ten layers of which is shown in Fig. 1.…”

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Development of the Method of Unsupervised Training of Convolutional Neural Networks Based on Neural Gas Modification

Москаленко

2017

Technology transfer: fundamental principles and innovative tech

View full text Add to dashboard Cite

show abstract

“…Regarding initialization, in our experiments we trained the proposed deep architectures by either (i) randomly initializing the weight values, or (ii) using pre-trained weights from networks having been pre-trained on large databases, such as the ImageNet [6]. For the second approach we used transfer learning [17], especially of the convolutional and pooling part of the pre-trained networks. In more detail, we utilized the ResNet L50 and VGG-16 networks, which have been pre-trained for object detection tasks, along with VGG-Face, which has been pre-trained for face recognition tasks.…”

Section: The End-to-end Deep Neural Architecturesmentioning

confidence: 99%

Recognition of Affect in the Wild Using Deep Neural Networks

Kollias

Nicolaou

Kotsia

et al. 2017

2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

120

104

View full text Add to dashboard Cite

In this paper we utilize the first large-scale "in-the-wild" (Aff-Wild) database, which is annotated in terms of the valence-arousal dimensions, to train and test an end-to-end deep neural architecture for the estimation of continuous emotion dimensions based on visual cues. The proposed architecture is based on jointly training convolutional (CNN) and recurrent neural network (RNN) layers, thus exploiting both the invariant properties of convolutional features, while also modelling temporal dynamics that arise in human behaviour via the recurrent layers. Various pre-trained networks are used as starting structures which are subsequently appropriately fine-tuned to the Aff-Wild database. Obtained results show premise for the utilization of deep architectures for the visual analysis of human behaviour in terms of continuous emotion dimensions and analysis of different types of affect.

show abstract

“…-Transfer Learning: Transfer learning [6] is the main approach to avoid learning failure due to overfitting, when training complex CNNs with small amounts of (image) data. In transfer learning, we use networks previously trained with large image datasets (even of generic objects) and fine-tune the whole, or parts of them, using the small training datasets.…”

Section: System Implementation and Operational Phasementioning

confidence: 99%

“…Recent advances in machine learning and deep neural networks provided state-of-the-art performance in all significant signal processing tasks, being used in a large number of applications, ranging from healthcare and question answering systems, to human computer interaction, surveillance and defense [4][5][6]. Deep neural networks are also applied as end-to-endarchitectures which include different network types in their structure and are trained to analyse signals, images, text and other inputs [3,4].…”

Section: Introductionmentioning

confidence: 99%

Adaptation and contextualization of deep neural network models

Kollias

Tagaris

et al. 2017

2017 IEEE Symposium Series on Computational Intelligence (SSCI)

View full text Add to dashboard Cite

Abstract-The ability of Deep Neural Networks (DNNs) to provide very high accuracy in classification and recognition problems makes them the major tool for developments in such problems. It is, however, known that DNNs are currently used in a 'black box' manner, lacking transparency and interpretability of their decision-making process. Moreover, DNNs should use prior information on data classes, or object categories, so as to provide efficient classification of new data, or objects, without forgetting their previous knowledge. In this paper, we propose a novel class of systems that are able to adapt and contextualize the structure of trained DNNs, providing ways for handling the abovementioned problems. A hierarchical and distributed system memory is generated and used for this purpose. The main memory is composed of the trained DNN architecture for classification/prediction, i.e., its structure and weights, as well as of an extracted -equivalent -Clustered Representation Set (CRS) generated by the DNN during training at its final -before the output -hidden layer. The latter includes centroids -'points of attraction' -which link the extracted representation to a specific area in the existing system memory. Drift detection, occurring, for example, in personalized data analysis, can be accomplished by comparing the distances of new data from the centroids, taking into account the intra-cluster distances. Moreover, using the generated CRS, the system is able to contextualize its decisionmaking process, when new data become available. A new public medical database on Parkinson's disease is used as testbed to illustrate the capabilities of the proposed architecture.

show abstract

Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning

Cited by 503 publications

References 21 publications

Development of the Method of Unsupervised Training of Convolutional Neural Networks Based on Neural Gas Modification

Development of the Method of Unsupervised Training of Convolutional Neural Networks Based on Neural Gas Modification

Recognition of Affect in the Wild Using Deep Neural Networks

Adaptation and contextualization of deep neural network models

Contact Info

Product

Resources

About