Semantic embeddings of generic objects for zero-shot learning

Hascoet, Tristan; Ariki, Yasuo; Takiguchi, Tetsuya

doi:10.1186/s13640-018-0371-x

Cited by 7 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, we focus on generic object recognition and make close to no assumption regarding the nature of these descriptions. The closest work to ours is probably Hascoet et al [9], in which different methods to obtain prototypes from WordNet definitions are evaluated, but reported performance is significantly below that of usual word embeddings.…”

Section: Introduction and Related Workmentioning

confidence: 91%

Using Sentences as Semantic Representations in Large Scale Zero-Shot Learning

Cacheux,

Borgne,

Crucianu

2020

Preprint

View full text Add to dashboard Cite

Zero-shot learning aims to recognize instances of unseen classes, for which no visual instance is available during training, by learning multimodal relations between samples from seen classes and corresponding class semantic representations. These class representations usually consist of either attributes, which do not scale well to large datasets, or word embeddings, which lead to poorer performance. A good trade-off could be to employ short sentences in natural language as class descriptions. We explore different solutions to use such short descriptions in a ZSL setting and show that while simple methods cannot achieve very good results with sentences alone, a combination of usual word embeddings and sentences can significantly outperform current state-of-the-art 3 .

show abstract

Section: Introduction and Related Workmentioning

confidence: 91%

Using Sentences as Semantic Representations in Large Scale Zero-Shot Learning

Cacheux,

Borgne,

Crucianu

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Veeranna et al ( 2016), adopted pre-trained word embedding for measuring semantic similarity between a label and documents. Further endeavor has been spent on zero-shot learning using semantic embedding by (Hascoet et al, 2019;Zhang et al, 2019;Xie and Virtanen, 2021;Rios and Kavuluru, 2018;Yin et al, 2019;Xia et al, 2018;Zhang et al, 2019;Pushp and Srivastava, 2017;Puri and Catanzaro, 2019;Yogatama et al, 2017;Pushp and Srivastava, 2017;Chen et al, 2021;Gong and Eldardiry, 2021).…”

Section: Topic Modeling and Inferencementioning

confidence: 99%

Zero-Shot Multi-Label Topic Inference with Sentence Encoders and LLMs

Sarkar,

Feng,

Karmaker Santu

2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

In this paper, we conducted a comprehensive study with the latest Sentence Encoders and Large Language Models (LLMs) on the challenging task of "definition-wild zero-shot topic inference", where users define or provide the topics of interest in real-time. Through extensive experimentation on seven diverse data sets, we observed that LLMs, such as ChatGPT-3.5 and PaLM, demonstrated superior generality compared to other LLMs, e.g., BLOOM and GPT-NeoX. Furthermore, Sentence-BERT, a BERT-based classical sentence encoder, outperformed PaLM and achieved performance comparable to ChatGPT-3.5.

show abstract

“…In other words, it addresses multi-class learning problems when some classes do not have sufficient training data. However, during the learning process, additional visual and semantic features such as word embeddings [132], visual attributes [133], or descriptions [134] can be assigned to both seen and unseen classes. In the context of multimodality, a multimodal mapping scheme typically combines visual and semantic attributes using only data related to the seen classes.…”

Section: Zero-shot Learningmentioning

confidence: 99%

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

et al. 2021

View full text Add to dashboard Cite

The research progress in multimodal learning has grown rapidly over the last decade in several areas, especially in computer vision. The growing potential of multimodal data streams and deep learning algorithms has contributed to the increasing universality of deep multimodal learning. This involves the development of models capable of processing and analyzing the multimodal information uniformly. Unstructured real-world data can inherently take many forms, also known as modalities, often including visual and textual content. Extracting relevant patterns from this kind of data is still a motivating goal for researchers in deep learning. In this paper, we seek to improve the understanding of key concepts and algorithms of deep multimodal learning for the computer vision community by exploring how to generate deep models that consider the integration and combination of heterogeneous visual cues across sensory modalities. In particular, we summarize six perspectives from the current literature on deep multimodal learning, namely: multimodal data representation, multimodal fusion (i.e., both traditional and deep learning-based schemes), multitask learning, multimodal alignment, multimodal transfer learning, and zero-shot learning. We also survey current multimodal applications and present a collection of benchmark datasets for solving problems in various vision domains. Finally, we highlight the limitations and challenges of deep multimodal learning and provide insights and directions for future research.

show abstract

Semantic embeddings of generic objects for zero-shot learning

Cited by 7 publications

References 23 publications

Using Sentences as Semantic Representations in Large Scale Zero-Shot Learning

Using Sentences as Semantic Representations in Large Scale Zero-Shot Learning

Zero-Shot Multi-Label Topic Inference with Sentence Encoders and LLMs

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

Contact Info

Product

Resources

About