Phraseformer: Multimodal Key-phrase Extraction using Transformer and Graph Embedding

Nikzad-Khasmakhi, Narjes; Feizi-Derakhshi, Mohammad-Reza; Asgari-Chenaghlu, Meysam; Balafar, Mohammad-Ali; Feizi-Derakhshi, Ali-Reza; Rahkar-Farshi, Taymaz; Ramezani, Majid; Jahanbakhsh-Nagadeh, Zoleikha; Zafarani-Moattar, Elnaz; Ranjbar-Khadivi, Mehrdad

doi:10.48550/arxiv.2106.04939

Cited by 3 publications

(5 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To achieve the same, we again query and use Chat-GPT to extract key phrases, as shown in Figure 3. The choice of ChatGPT over task-specific models such as Phraseformer [92], PromptRank [93], KeyBART [94] is mainly due to its better ability [72], [73] to capture the right key phrases.…”

Section: ) Extraction Of Entities From the Meme Imagementioning

confidence: 99%

Capturing the Concept Projection in Metaphorical Memes for Downstream Learning Tasks

Acharya,

Das,

Sudarshan

2024

IEEE Access

View full text Add to dashboard Cite

Metaphorical memes, where a source concept is projected into a target concept, are an essential construct in figurative language. In this article, we present a novel approach for downstream learning tasks on metaphorical multimodal memes. Our proposed framework replaces traditional methods using metaphor annotations with a metaphor-capturing mechanism. Besides using the significant zero-shot learning capability of state-of-the-art pretrained encoders, this work introduces an alternative external knowledge enhancement strategy based on ChatGPT (chatbot generative pretrained transformer), demonstrating its effectiveness in bridging the intermodal semantic gap. We propose a new concept projection process consisting of three distinct components to capture the intramodal knowledge and intermodal concept gap in the forms of text modality embedding, visual modality embedding, and concept projection embedding. This approach leverages the attention mechanism of the Graph Attention Network for fusing the common aspects of external knowledge related to the knowledge in the text and image modality to implement the concept projection process. Our experimental results demonstrate the superiority of our proposed approach compared to existing methods.

show abstract

Section: ) Extraction Of Entities From the Meme Imagementioning

confidence: 99%

Capturing the Concept Projection in Metaphorical Memes for Downstream Learning Tasks

Acharya,

Das,

Sudarshan

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…To identify the paragraphs expressing the document, we started from the assumption that these paragraphs are semantically similar to most of the paragraphs of the document, in particular title, abstract and conclusion. For this, we first calculate the score of each paragraph using (1).…”

Section: Score Paragraphmentioning

confidence: 99%

“…Before the process of extracting the present keyphrases, the paragraphs that express the content of the document are selected, by calculating the score of each paragraph by (1). The paragraphs with the highest score are chosen as paragraphs expressing the document.…”

Section: Paragraphs Of the Documentmentioning

confidence: 99%

See 1 more Smart Citation

Present and absent keyphrases extraction: an approach based on sentence embedding

Ajallouda

Zellou

2022

IJEECS

View full text Add to dashboard Cite

The automatic keyphrases extraction (AKE) of a document is any expression by which we can learn its content without having to read it. Keyphrases are exploited in natural language processing (NLP) applications. These phrases are often mentioned in the document but there may be some keyphrases that are not mentioned. In the field of AKE, researchers have exploited many techniques, such as statistical calculation, deep learning algorithms, graph representation, and sentence embedding techniques. Approaches that exploit embedding techniques calculate the similarity between a document and a candidate keyphrase, where similar phrases to the document are considered as keyphrases. Representing the document by a single vector makes its performance poor, especially in long documents. This is in addition to the inability of these methods to generate absent keyphrases. In order to overcome these problems, our paper proposes an unsupervised approach to AKE, based on the universal sentence encoder (USE) to represent candidate keyphrases and parts of the document probably containing keyphrases. Our method also generates keyphrases not mentioned in the text. We compared the performance of the proposed approach with other methods based on embedding techniques, where the results showed the superiority of our approach especially in long documents.

show abstract

“…Some off-the-shelf methods can parse a sentence to multiple phrases, such as Topicrank [2] and Sentence Transformers [27] and BERT [6]. However, these methods suffer from two main problem: 1) task-specific design and not for image editing; 2) large-scare network with massive parameters.…”

Section: Sentence Parsing and Attribute Combinationmentioning

confidence: 99%

Each Attribute Matters: Contrastive Attention for Sentence-based Image Editing

Zhao¹,

Lyu²,

Hu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Sentence-based Image Editing (SIE) aims to deploy natural language to edit an image. Offering potentials to reduce expensive manual editing, SIE has attracted much interest recently. However, existing methods can hardly produce accurate editing and even lead to failures in attribute editing when the query sentence is with multiple editable attributes. To cope with this problem, by focusing on enhancing the difference between attributes, this paper proposes a novel model called Contrastive Attention Generative Adversarial Network (CA-GAN), which is inspired from contrastive training. Specifically, we first design a novel contrastive attention module to enlarge the editing difference between random combinations of attributes which are formed during training. We then construct an attribute discriminator to ensure effective editing on each attribute. A series of experiments show that our method can generate very encouraging results in sentencebased image editing with multiple attributes on CUB and COCO dataset. Our code is available at https://github.com/Zlq2021/CA-GAN IntroductionAs billions of images are uploaded and shared every day [16,32], image editing has become one of the most demanding tasks in social media. However, to edit an image as desired, one may have to master professional software such as Adobe PhotoShop. In contrast with manual editing, automatic image editing, has recently attracted much interest in computer vision. This paper studies the problem of Sentence-based Image Editing (SIE) [7,19,25] that intends to deploy natural language to assist image editing automatically. One main

show abstract

Phraseformer: Multimodal Key-phrase Extraction using Transformer and Graph Embedding

Cited by 3 publications

References 37 publications

Capturing the Concept Projection in Metaphorical Memes for Downstream Learning Tasks

Capturing the Concept Projection in Metaphorical Memes for Downstream Learning Tasks

Present and absent keyphrases extraction: an approach based on sentence embedding

Each Attribute Matters: Contrastive Attention for Sentence-based Image Editing

Contact Info

Product

Resources

About