StyleGuide: Zero-Shot Sketch-Based Image Retrieval Using Style-Guided Image Generation

Dutta, Titir; Singh, Anurag; Biswas, Soma

doi:10.1109/tmm.2020.3017918

Cited by 44 publications

(38 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The paired cyclic consistency loss proposed in SEM-PCYC Akata, 2019, 2020) helps in aligning the sketches and images in an encoded semantic space using adversarial training. Inspired by style transfer, (Dutta and Biswas, 2019; develops a styleguided image to image translation model for ZS-SBIR, while (Dey et al, 2019) uses a triplet-based network to solve the task at hand. highlights the implications of data and class imbalance in ZS-SBIR and introduces an adaptive margin diversity regularizer (AMD-reg) to combat the same.…”

Section: State-of-the-art Zs-sbir Algorithmsmentioning

confidence: 99%

“…As opposed to the real-valued feature embedding, hash-code based representations are also considered in this regard which offers a trade-off between performance and storage (Liu et al, 2017). The generative models for cross-modal style transfer are also explored (Dutta and Biswas, 2019) in this regard. While all the techniques showcase their performance on ZS-SBIR, a few works Akata, 2019, 2020;Pandey et al, 2020) also demonstrate their experiments for the GZS-SBIR setting.…”

Section: State-of-the-art Zs-sbir Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

BDA-SketRet: Bi-Level Domain Adaptation for Zero-Shot SBIR

Chaudhuri¹,

Ruchika²,

Banerjee³

et al. 2022

Preprint

View full text Add to dashboard Cite

The efficacy of zero-shot sketch-based image retrieval (ZS-SBIR) models is governed by two challenges. The immense distributions-gap between the sketches and the images requires a proper domain alignment. Moreover, the fine-grained nature of the task and the high intra-class variance of many categories necessitates a class-wise discriminative mapping among the sketch, image, and the semantic spaces. Under this premise, we propose BDA-SketRet, a novel ZS-SBIR framework performing a bi-level domain adaptation for aligning the spatial and semantic features of the visual data pairs progressively. In order to highlight the shared features and reduce the effects of any sketch or image-specific artifacts, we propose a novel symmetric loss function based on the notion of information bottleneck for aligning the semantic features while a cross-entropy-based adversarial loss is introduced to align the spatial feature maps. Finally, our CNN-based model confirms the discriminativeness of the shared latent space through a novel topology-preserving semantic projection network. Experimental results on the extended Sketchy, TU-Berlin, and QuickDraw datasets exhibit sharp improvements over the literature.

show abstract

Section: State-of-the-art Zs-sbir Algorithmsmentioning

confidence: 99%

Section: State-of-the-art Zs-sbir Algorithmsmentioning

confidence: 99%

BDA-SketRet: Bi-Level Domain Adaptation for Zero-Shot SBIR

Chaudhuri¹,

Ruchika²,

Banerjee³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…IV. RESULTS AND DISCUSSIONS We compare the performance of the proposed model with some of the existing state-of-the-art frameworks of [9]- [12], [14], [17], [18], [21], [34]. We also lay down the performances of some of the notable works in SBIR that uses the same datasets to show how the proposed model which solves a more challenging ZS-SBIR problem achieves comparable performance.…”

Section: Sketchy-ext Tu Berlin-extmentioning

confidence: 99%

“…It comprises of both generative and discriminative deep-learning based models. The generative models such as [9], [11]- [15] tries to align the visual sketch and the photo images onto a latent semantic space using adversarial training. In ZSIH [11], the authors propose a generative hashing mechanism to reconstruct the semantic embeddings of each class names.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Zero-Shot Sketch Based Image Retrieval using Graph Transformer

Sumrit¹,

Chaudhuri²,

Banerjee³

2022

Preprint

View full text Add to dashboard Cite

The performance of a zero-shot sketch-based image retrieval (ZS-SBIR) task is primarily affected by two challenges. The substantial domain gap between image and sketch features needs to be bridged, while at the same time the side information has to be chosen tactfully. Existing literature has shown that varying the semantic side information greatly affects the performance of ZS-SBIR. To this end, we propose a novel graph transformer based zero-shot sketch-based image retrieval (GTZSR) framework for solving ZS-SBIR tasks which uses a novel graph transformer to preserve the topology of the classes in the semantic space and propagates the context-graph of the classes within the embedding features of the visual space. To bridge the domain gap between the visual features, we propose minimizing the Wasserstein distance between images and sketches in a learned domain-shared space. We also propose a novel compatibility loss that further aligns the two visual domains by bridging the domain gap of one class with respect to the domain gap of all other classes in the training set. Experimental results obtained on the extended Sketchy, TU-Berlin, and QuickDraw datasets exhibit sharp improvements over the existing state-ofthe-art methods in both ZS-SBIR and generalized ZS-SBIR.

show abstract