Training Conditional Random Fields Using Transfer Learning for Gesture Recognition

Abstract

Recently, combining Conditional Random Fields (CRF) with Neural Network has shown the success of learning high-level features in sequence labeling tasks. However, such models are difficult to train because of the increase of the parameters to tune which needs enormous of labeled data to avoid overfitting. In this paper, we propose a transfer learning framework for the sequence labeling task of gesture recognition. Taking advantage of the frame correlation, we design an unsupervised sequence model as a pseudo auxiliary task to capture the underlying information from both the labeled and unlabeled data. The knowledge learnt by the auxiliary task can be transferred to the main task of CRF with a deep architecture by sharing the hidden layers, which is very helpful for learning meaningful representation and reducing the need of labeled data. We evaluate our model under 3 gesture recognition datasets. The experimental results of both supervised learning and semi-supervised learning show that the proposed model improves the performance of the CRF with Neural Network and other baseline models.

Thumbs up, thumbs down: non-verbal human-robot interaction through real-time EMG classification via inductive and supervised transductive transfer learning

“…STTL is the process in which the source domain labels are available but the target is not, this is the validation stage in this study, when a calibrated model is benchmarked on further unknown data during application of a calibrated model. Transfer learning is the process of knowledge transfer from one learned task to another (Zhuang et al 2019), in this study, it is shown to be difficult to generalise a model to new subjects and thus application of a model to new data is considered a task to be solved by transfer learning; transfer learning often shows strong results in the application of gesture classification in related state-of-the-art works (Liu et al 2010;Goussies et al 2014;Costante et al 2014;Yang et al 2018;Demir et al 2019).…”

Transfer Learning Decision Forests for Gesture Recognition

“…A set of labeled words in the source and target data is shared so as to build a word classifier for a new signer on a set of unlabeled target words. A transfer learning method for conditional random fields is implemented to exploit information in both labeled and unlabeled data to learn high-level features for gesture recognition by Liu et al (2010). More recently, the ChaLearn Gesture Competition (Guyon et al, 2013) provided a benchmark of methods that apply transfer learning to gesture recognition.…”

Aprendizado por Transferência para Aplicações Orientadas a Usuário: Uma Experiência em Língua de Sinais

“…Em [14],é proposto um novo método baseado na estratégia de Aprendizado por Transferência, em Redes Neurais Artificiais e em Campos Condicionais Aleatórios, para reconhecimento de padrões no contexto de gestos feitos com os olhos e gestos feitos com a cabeça. A ideia apresentada pelos autoresé criar um modelo que analisa sequências de frames de vídeo de forma não supervisionada, capturando informações referentes as dependências entre os frames; essa tarefaé considerada como uma tarefa auxiliar.…”