From Traditional to Modern: Domain Adaptation for Action Classification in Short Social Video Clips

Abstract

Abstract. Short internet video clips like vines present a significantly wild distribution compared to traditional video datasets. In this paper, we focus on the problem of unsupervised action classification in wild vines using traditional labeled datasets. To this end, we use a data augmentation based simple domain adaptation strategy. We utilize semantic word2vec space as a common subspace to embed video features from both, labeled source domain and unlabled target domain. Our method incrementally augments the labeled source with target samples and iteratively modifies the embedding function to bring the source and target distributions together. Additionally, we utilize a multi-modal representation that incorporates noisy semantic information available in form of hash-tags. We show the effectiveness of this simple adaptation technique on a test set of vines and achieve notable improvements in performance.

Learning to hash-tag videos with Tag2Vec

“…Similar to all these approaches we also used a neural network for learning embedding function but we focus only on hash tags. [16] maps the video representations to semantic space for improving action classification. Recent image captioning methods [23,6] have shown remarkable progress in generating rich descriptive sentences for natural images.…”

“…Vector representation of words is used by many recent approaches for joint visual semantic learning [17,16,23,27,6]. Socher et al [17] learns an embedding function which performs a mapping of image features to semantic word space.…”