Learnable Pillar-based Re-ranking for Image-Text Retrieval

Qu, Leigang; Li, Meng; Wang, Wenjie; Zheng, Zhedong; Nie, Liqiang; Chua, Tat-Seng

doi:10.1145/3539618.3591712

Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval 2023

DOI: 10.1145/3539618.3591712

|View full text |Cite

Learnable Pillar-based Re-ranking for Image-Text Retrieval

Leigang Qu

Meng Li

Wenjie Wang

et al.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2023

2024

Publication Types

Select...

Other4

Relationship

Self Cite1

Independent3

Authors

Journals

Cited by 4 publications

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Target-Guided Composed Image Retrieval

Wen,

Zhang,

Song

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

Composed image retrieval (CIR) is a new and flexible image retrieval paradigm, which can retrieve the target image for a multimodal query, including a reference image and its corresponding modification text. Although existing efforts have achieved compelling success, they overlook the conflict relationship modeling between the reference image and the modification text for improving the multimodal query composition and the adaptive matching degree modeling for promoting the ranking of the candidate images that could present different levels of matching degrees with the given query. To address these two limitations, in this work, we propose a Target-Guided Composed Image Retrieval network (TG-CIR). In particular, TG-CIR first extracts the unified global and local attribute features for the reference/target image and the modification text with the contrastive language-image pre-training model (CLIP) as the backbone, where an orthogonal regularization is introduced to promote the independence among the attribute features. Then TG-CIR designs a target-query relationship-guided multimodal query composition module, comprising a target-free student composition branch and a target-based teacher composition branch, where the target-query relationship is injected into the teacher branch for guiding the conflict relationship modeling of the student branch. Last, apart from the conventional batch-based classification loss, TG-CIR additionally introduces a batch-based target similarity-guided matching degree regularization to promote the metric learning process. Extensive experiments on three benchmark datasets demonstrate the superiority of our proposed method. CCS CONCEPTS• Information systems → Image search.

show abstract

Target-Guided Composed Image Retrieval

Wen,

Zhang,

Song

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

Hierarchical Semantics Alignment for 3D Human Motion Retrieval

Yang,

Shi,

Zhang

2024

Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

CaLa: Complementary Association Learning for Augmenting Comoposed Image Retrieval

Jiang,

Wang,

et al. 2024

Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

hi@scite.ai

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Learnable Pillar-based Re-ranking for Image-Text Retrieval

Cited by 4 publications

References 14 publications

Target-Guided Composed Image Retrieval

Target-Guided Composed Image Retrieval

Hierarchical Semantics Alignment for 3D Human Motion Retrieval

CaLa: Complementary Association Learning for Augmenting Comoposed Image Retrieval

Contact Info

Product

Resources

About