ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity

Delmas, Ginger; Rezende, Rafael Sampaio de; Larlus, Diane

doi:10.48550/arxiv.2203.08101

Cited by 3 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Metrics. Following previous works (Baldrati et al 2022;Delmas et al 2022;Zhao, Song, and Jin 2022), we employ Recall within top-K as the retrieval performance, which indicates the ratio of the ground-truth target image in the top-K ranking list that is correctly retrieved.…”

Section: Methodsmentioning

confidence: 99%

Decomposing Semantic Shifts for Composed Image Retrieval

Yang,

Liu,

Zhang

et al. 2024

AAAI

View full text Add to dashboard Cite

Composed image retrieval is a type of image retrieval task where the user provides a reference image as a starting point and specifies a text on how to shift from the starting point to the desired target image. However, most existing methods focus on the composition learning of text and reference images and oversimplify the text as a description, neglecting the inherent structure and the user's shifting intention of the texts. As a result, these methods typically take shortcuts that disregard the visual cue of the reference images. To address this issue, we reconsider the text as instructions and propose a Semantic Shift Network (SSN) that explicitly decomposes the semantic shifts into two steps: from the reference image to the visual prototype and from the visual prototype to the target image. Specifically, SSN explicitly decomposes the instructions into two components: degradation and upgradation, where the degradation is used to picture the visual prototype from the reference image, while the upgradation is used to enrich the visual prototype into the final representations to retrieve the desired target image. The experimental results show that the proposed SSN demonstrates a significant improvement of 5.42% and 1.37% on the CIRR and FashionIQ datasets, respectively, and establishes a new state-of-the-art performance. The code is available at https://github.com/starxing-yuu/SSN.

show abstract

Section: Methodsmentioning

confidence: 99%

Decomposing Semantic Shifts for Composed Image Retrieval

Yang,

Liu,

Zhang

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…However, interactive recommendation in design scenarios depend on the user's intuitive visual experience. Therefore, visually-grounded dialog system researches have also emerged in e-commerce platforms in recent years [2,10,39,40]. For example, Yuan et al [39] proposes a conversational fashion image retrieval method that predicts the desired image based on text and image information in the conversation history.…”

Section: Interactive Recommendationmentioning

confidence: 99%

“…The interior design data used in this paper were gathered from a popular Chinese interior design website 2 . The data comprises (The open design of entire dining room weakens the TV-centered layout and focuses on comfort to create a comfortable living atmosphere.…”

Section: Data Descriptionmentioning

confidence: 99%

Interactive Interior Design Recommendation via Coarse-to-fine Multimodal Reinforcement Learning

Zhang,

Sun,

Guo

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

Personalized interior decoration design often incurs high labor costs. Recent efforts in developing intelligent interior design systems have focused on generating textual requirement-based decoration designs while neglecting the problem of how to mine homeowner's hidden preferences and choose the proper initial design. To fill this gap, we propose an Interactive Interior Design Recommendation System (IIDRS) based on reinforcement learning (RL). IIDRS aims to find an ideal plan by interacting with the user, who provides feedback on the gap between the recommended plan and their ideal one. To improve decision-making efficiency and effectiveness in large decoration spaces, we propose a Decoration Recommendation Coarse-to-Fine Policy Network (DecorRCFN). Additionally, to enhance generalization in online scenarios, we propose an objectaware feedback generation method that augments model training with diversified and dynamic textual feedback. Extensive experiments on a real-world dataset demonstrate our method outperforms traditional methods by a large margin in terms of recommendation accuracy. Further user studies demonstrate that our method reaches higher real-world user satisfaction than baseline methods. CCS CONCEPTS• Information systems → Recommender systems.

show abstract