GradMix: Multi-source Transfer across Domains and Tasks

Li, Junnan; Xu, Ziwei; Wang, Yongkang; Zhao, Qi; Kankanhalli, Mohan

doi:10.1109/wacv45572.2020.9093343

Cited by 8 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The selection of the specific feature is dependent on the connection between the attribute-object pair provided in an external knowledge base. It can also help attribute-based domain and task transfer [83], [84]. Another direction is to extend our model to recognize more complex composite concepts whose cardinalities are larger than two (e.g.…”

Section: Discussionmentioning

confidence: 99%

Relation-aware Compositional Zero-shot Learning for Attribute-Object Pair Recognition

Xu,

Wang,

Wong

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper proposes a novel model for recognizing images with composite attribute-object concepts, notably for composite concepts that are unseen during model training. We aim to explore the three key properties required by the taskrelation-aware, consistent, and decoupled -to learn rich and robust features for primitive concepts that compose attributeobject pairs. To this end, we propose the Blocked Message Passing Network (BMP-Net). The model consists of two modules. The concept module generates semantically meaningful features for primitive concepts, whereas the visual module extracts visual features for attributes and objects from input images. A message passing mechanism is used in the concept module to capture the relations between primitive concepts. Furthermore, to prevent the model from being biased towards seen composite concepts and reduce the entanglement between attributes and objects, we propose a blocking mechanism that equalizes the information available to the model for both seen and unseen concepts. Extensive experiments and ablation studies on two benchmarks show the efficacy of the proposed model.

show abstract

Section: Discussionmentioning

confidence: 99%

Relation-aware Compositional Zero-shot Learning for Attribute-Object Pair Recognition

Xu,

Wang,

Wong

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, most target classes of ImageNet, such as "flowers", "animals", and "foods," are irrelevant to the human activity classes, like "push-ups", "dribbling", or "pick-up" (see Figure 2). On the contrary, BU101 is a set of data from the web containing static images that depict a human action, as shown in Figure 3, and has been widely used to enhance the performance of activity identification in video sequences [64,131,134,135]. For the aforementioned reasons, BU101 is selected for training our networks before these are transferred and trained on the target datasets.…”

Section: Previously Trained On Imagenet and Bu101mentioning

confidence: 99%

Evaluating the Performance of Mobile-Convolutional Neural Networks for Spatial and Temporal Human Action Recognition Analysis

Moutsis,

Tsintotas,

Kansizoglou

et al. 2023

Robotics

View full text Add to dashboard Cite

Human action recognition is a computer vision task that identifies how a person or a group acts on a video sequence. Various methods that rely on deep-learning techniques, such as two- or three-dimensional convolutional neural networks (2D-CNNs, 3D-CNNs), recurrent neural networks (RNNs), and vision transformers (ViT), have been proposed to address this problem over the years. Motivated by the fact that most of the used CNNs in human action recognition present high complexity, and the necessity of implementations on mobile platforms that are characterized by restricted computational resources, in this article, we conduct an extensive evaluation protocol over the performance metrics of five lightweight architectures. In particular, we examine how these mobile-oriented CNNs (viz., ShuffleNet-v2, EfficientNet-b0, MobileNet-v3, and GhostNet) execute in spatial analysis compared to a recent tiny ViT, namely EVA-02-Ti, and a higher computational model, ResNet-50. Our models, previously trained on ImageNet and BU101, are measured for their classification accuracy on HMDB51, UCF101, and six classes of the NTU dataset. The average and max scores, as well as the voting approaches, are generated through three and fifteen RGB frames of each video, while two different rates for the dropout layers were assessed during the training. Last, a temporal analysis via multiple types of RNNs that employ features extracted by the trained networks is examined. Our results reveal that EfficientNet-b0 and EVA-02-Ti surpass the other mobile-CNNs, achieving comparable or superior performance to ResNet-50.

show abstract

“…Luo et al (2020) continued that line and suggested a method in the context of few-shot transfer learning, showing that using even a few samples from the target task can significantly improve the transferability of the trained models. Li et al (2020) presented a similar idea but suggested adjusting learning rates for each layer as a mean to improve the cosine similarity of different tasks.…”

Section: Few-shot Transfer Learningmentioning

confidence: 99%

Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents

Kachuee¹,

Yuan²,

Kim³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Turn-level user satisfaction is one of the most important performance metrics for conversational agents. It can be used to monitor the agent's performance and provide insights about defective user experiences. While endto-end deep learning has shown promising results, having access to a large number of reliable annotated samples required by these methods remains challenging. In a large-scale conversational system, there is a growing number of newly developed skills, making the traditional data collection, annotation, and modeling process impractical due to the required annotation costs and the turnaround times. In this paper, we suggest a self-supervised contrastive learning approach that leverages the pool of unlabeled data to learn user-agent interactions. We show that the pre-trained models using the self-supervised objective are transferable to the user satisfaction prediction. In addition, we propose a novel few-shot transfer learning approach that ensures better transferability for very small sample sizes. The suggested few-shot method does not require any inner loop optimization process and is scalable to very large datasets and complex models. Based on our experiments using real data from a large-scale commercial system, the suggested approach is able to significantly reduce the required number of annotations, while improving the generalization on unseen skills.

show abstract

GradMix: Multi-source Transfer across Domains and Tasks

Cited by 8 publications

References 23 publications

Relation-aware Compositional Zero-shot Learning for Attribute-Object Pair Recognition

Relation-aware Compositional Zero-shot Learning for Attribute-Object Pair Recognition

Evaluating the Performance of Mobile-Convolutional Neural Networks for Spatial and Temporal Human Action Recognition Analysis

Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents

Contact Info

Product

Resources

About