Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback

Wu, Hui; Gao, Yupeng; Guo, Xiaoxiao; Al-Halah, Ziad; Rennie, Steven J.; Grauman, Kristen; Feris, Rogério

doi:10.1109/cvpr46437.2021.01115

Cited by 114 publications

(138 citation statements)

References 53 publications

Supporting

Mentioning

116

Contrasting

Order By: Relevance

“…Additional ranking results. We provide qualitative results for the FashionIQ (Wu et al, 2021), the Shoes (Guo et al, 2018;Berg et al, 2010) and the CIRR (Liu et al, 2021) datasets in Figures 5, 6, 7, 8 and 9. They illustrate the model's ability to properly encode and reason on general concepts.…”

Section: Qualitative Resultsmentioning

confidence: 99%

“…It was even observed that, in some cases, users tend to directly describe characteristics of the target image, without explicitly stating the discrepancy between the reference and the target image 1 (c.f. the analysis presented for FashionIQ in (Wu et al, 2021) and for CIRR in (Liu et al, 2021)) Therefore the text, which describes mandatory details for the target, can be directly used to query for the target image, as is common in text-to-image retrieval. Yet, due to the specificities of the task, we go beyond the usual text-to-image retrieval.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…In this case, the compatibility score becomes The performance of these baselines changes according to the nature of each dataset. For FashionIQ, where 68% of the modifiers make direct reference to the target image (Wu et al, 2021), the textonly baseline outperforms the image-only one. For Shoes, based on visual inspection, we conjecture that the reference-target image pairs are, on average, more similar than for FashionIQ, making the reference image more crucial for many queries.…”

Section: Ablation Studymentioning

confidence: 98%

“…We consider two datasets focusing on the fashion domain and one on open-domain images, all three using human-written textual modifiers in natural language. The Fashion IQ dataset (Wu et al, 2021) is composed of 46.6k training images and around 15.5k images for both the validation and test sets. There are 18k training queries, and 12k queries per evaluation split, covering three fashion categories: women's tops (toptee), women's dresses (dress) and men's shirts (shirt).…”

Section: Experimental Settingmentioning

confidence: 99%

“…Either a BiGRU (Cho et al, 2014), or a LSTM (Hochreiter & Schmidhuber, 1997) (followed by an average pooling and a fully-connected layer, as in Chen et al ( 2020)) are learned from scratch on top of the word embeddings to produce a sentence-level feature. For Fashion IQ (Wu et al, 2021), the only dataset for which each reference/target image pair is provided with two relative captions (used as text modifiers) from two different annotators, we follow previous works (Lee et al, 2021) and use the concatenation of the two captions (in both orders) for training and evaluation. This means that we use about 12k queries fr the validation split.…”

Section: A Implementation Detailsmentioning

confidence: 99%

See 4 more Smart Citations

ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity

Delmas¹,

Rezende²,

Larlus³

2022

Preprint

View full text Add to dashboard Cite

An intuitive way to search for images is to use queries composed of an example image and a complementary text. While the first provides rich and implicit context for the search, the latter explicitly calls for new traits, or specifies how some elements of the example image should be changed to retrieve the desired target image. Current approaches typically combine the features of each of the two elements of the query into a single representation, which can then be compared to the ones of the potential target images. Our work aims at shedding new light on the task by looking at it through the prism of two familiar and related frameworks: text-to-image and image-to-image retrieval. Taking inspiration from them, we exploit the specific relation of each query element with the targeted image and derive light-weight attention mechanisms which enable to mediate between the two complementary modalities. We validate our approach on several retrieval benchmarks, querying with images and their associated free-form text modifiers. Our method obtains state-of-the-art results without resorting to side information, multi-level features, heavy pre-training nor large architectures as in previous works.

show abstract

Section: Qualitative Resultsmentioning

confidence: 99%

Section: Proposed Methodsmentioning

confidence: 99%

Section: Ablation Studymentioning

confidence: 98%

Section: Experimental Settingmentioning

confidence: 99%

Section: A Implementation Detailsmentioning

confidence: 99%

See 3 more Smart Citations

ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity

Delmas¹,

Rezende²,

Larlus³

2022

Preprint

View full text Add to dashboard Cite

show abstract

Well googled is half done: Multimodal forecasting of new fashion product sales with image‐based google trends

Skenderi,

Joppi,

Denitto

et al. 2024

Journal of Forecasting

View full text Add to dashboard Cite

New fashion product sales forecasting is a challenging problem that involves many business dynamics and cannot be solved by classical forecasting approaches. In this paper, we investigate the effectiveness of systematically probing exogenous knowledge in the form of Google Trends time series and combining it with multi‐modal information related to a brand‐new fashion item, in order to effectively forecast its sales despite the lack of past data. In particular, we propose a neural network‐based approach, where an encoder learns a representation of the exogenous time series, while the decoder forecasts the sales based on the Google Trends encoding and the available visual and metadata information. Our model works in a non‐autoregressive manner, avoiding the compounding effect of large first‐step errors. As a second contribution, we present VISUELLE, a publicly available dataset for the task of new fashion product sales forecasting, containing multimodal information for 5,577 real, new products sold between 2016 and 2019 from Nunalie, an Italian fast‐fashion company. The dataset is equipped with images of products, metadata, related sales, and associated Google Trends. We use VISUELLE to compare our approach against state‐of‐the‐art alternatives and several baselines, showing that our neural network‐based approach is the most accurate in terms of both percentage and absolute error. It is worth noting that the addition of exogenous knowledge boosts the forecasting accuracy by 1.5% in terms of Weighted Absolute Percentage Error (WAPE), revealing the importance of exploiting informative external information. The code and dataset are both available online (at https://github.com/HumaticsLAB/GTM-Transformer).

show abstract

FashionViL: Fashion-Focused Vision-and-Language Representation Learning

Han

Zhu

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback

Cited by 114 publications

References 53 publications

ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity

ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity

Well googled is half done: Multimodal forecasting of new fashion product sales with image‐based google trends

FashionViL: Fashion-Focused Vision-and-Language Representation Learning

Contact Info

Product

Resources

About