"Does it come in black?" CLIP-like models are zero-shot recommenders

Abstract

Product discovery is a crucial component for online shopping. However, item-to-item recommendations today do not allow users to explore changes along selected dimensions: given a query item, can a model suggest something similar but in a different color? We consider item recommendations of the comparative nature (e.g. "something darker") and show how CLIP-based models can support this use case in a zero-shot manner. Leveraging a large model built for fashion, we introduce GradREC and its industry potential, and offer a first rounded assessment of its strength and weaknesses. * * GradRECS started as a (failed) experiment by JT; PC actually made it work, and he is the lead researcher on the project. FB, CG and DC all contributed to the paper, providing support for modelling, industry context and domain knowledge. PC and JT are the corresponding authors.