Partially Does It: Towards Scene-Level FG-SBIR with Partial Input

Multimodal query-guided object localization

“…In [70], researchers have leveraged an attention model to solve fine-grained SBIR and have also introduced higher-order learnable energy function-based loss to alleviate the domain gap between the images and the sketches. In [4,10], researchers have tackled the task of noise-tolerant image retrieval. To improve the efficiency for large-scale image retrieval, hashing models have been explored [41,66,80,91].…”

“…The above loss function consists of two parts: (i) In (9), the first part of the loss function assures that the object proposals that are overlapping with the ground truth object locations are predicted as foreground with high probability. (ii) The second part of the loss function, i.e., (10), is a margin-ranking loss that takes pairs of the proposals as input. It aids in reinforcing a greater division between prediction probabilities of foreground and background object proposals, and therefore, it improves the ranking of all the foreground proposals overlapping with the true location(s) of the object of interest.…”

“…The learning rate was initially set at 0.01, but it decays with a rate of 0.1 after every four epochs, and it is trained for 30 epochs. The constant K in ( 4) is fixed at 256 and m + = 0.3 and m − = 0.7 in ( 9) and (10) for all experiments. To obtain the sentence embeddings average of the features of the final layer of BERT is utilized.…”