Attention Mechanism for Fashion Image Captioning

Nguyen, Bao T.; Prakash, Om; Vo, Anh H.

doi:10.1007/978-3-030-62324-1_9

Cited by 4 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extensive efforts in fashion-related research direction have been made in the past few years [1] [2]. However, to the best of our knowledge, there are limited research works focused on fashion image captioning [3] [4]. The main goal of fashion image captioning is to generate accurate and attractive sentences that describe input fashion images and specify the item attributes such as style, shape, and fabric.…”

Section: Introductionmentioning

confidence: 99%

“…This work also builds the FACAD dataset and proposed a semantic rewards-based reinforcement learning framework to increase the evaluation accuracy of fashion image captioning. Nguyen et al [4] propose to use channel and spatial attention mecha- The conventional controllable image captioning is to generate image descriptions following selected control signals [7][8]. In this paper, we propose using semantic attributes as a control signal together with visual information to generate fashion image caption.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Attribute Conditioned Fashion Image Captioning

Cai

Yap

Wang

2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Fashion image captioning aims to automatically generate product descriptions for fashion items. Existing fashion image captioning models predict a fixed caption for a particular fashion item once deployed. Differently, we explore a controllable way of fashion image captioning by taking the semantic attributes as the control signal. We propose a new multi-modal fashion image captioning method that allows the users to specify a few semantic attributes to guide the caption generation to suit unique preferences. To study the problem, we clean, filter, and assemble a new fashion image caption dataset called FACAD170K from the current FACAD dataset to facilitate learning. We investigate the effectiveness of the proposed approach on our assembled FACAD170K dataset. The results demonstrate that our method can outperform existing fashion image captioning models as well as conventional captioning methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Attribute Conditioned Fashion Image Captioning

Cai

Yap

Wang

2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…It is goal is to determine whether a particular outfit is well-coordinated or not. It can be fashion item-based recommendation [26], [27], or outfit-based recommendations using criteria such as versatility or compatibility [28], [29].…”

mentioning

confidence: 99%

Channel and spatial attention mechanism for fashion image captioning

Nguyen

2023

IJECE

View full text Add to dashboard Cite

<div class="page" title="Page 1"><div class="layoutArea"><div class="column"><p><span lang="EN-US">Image captioning aims to automatically generate one or more description sentences for a given input image. Most of the existing captioning methods use encoder-decoder model which mainly focus on recognizing and capturing the relationship between objects appearing in the input image. However, when generating captions for fashion images, it is important to not only describe the items and their relationships, but also mention attribute features of clothes (shape, texture, style, fabric, and more). In this study, one novel model is proposed for fashion image captioning task which can capture not only the items and their relationship, but also their attribute features. Two different attention mechanisms (spatial-attention and channel-wise attention) is incorporated to the traditional encoder-decoder model, which dynamically interprets the caption sentence in multi-layer feature map in addition to the depth dimension of the feature map. We evaluate our proposed architecture on Fashion-Gen using three different metrics (CIDEr, ROUGE-L, and BLEU-1), and achieve the scores of 89.7, 50.6 and 45.6, respectively. Based on experiments, our proposed method shows significant performance improvement for the task of fashion-image captioning, and outperforms other state-of-the-art image captioning methods.</span></p></div></div></div>

show abstract