One-Shot Action Detection via Attention Zooming In

Multimodal Query-Guided Object Localization

“…However, they do not generalize well in an open set setting where an object category may or may not be seen during the training phase. Recently, Hsieh et al [25] have proposed one-shot object detection. In their work, an object image is used as a query, and all the instances of the query object in the target image are detected.…”

“…Li et al [35] proposed Attention to context Convolution Neural Networks (AC-CNN) in object detection to integrate local and global context. Leveraging the self-attention mechanism [78], Heish et al [25] presented Co-attention and co-excitation network (CoAtEx) for one-shot object localization. In their work, the response at each feature map location of an image is computed as a weighted combination of the feature vectors at each feature map location of the query.…”

“…Instead of using a neural network, cosine similarity can also be used to compute the score. Motivated from [25], the loss function used in training is defined as:…”

“…This baseline is referred to as modified Faster R-CNN in this paper. [25]: It is a recent one-shot object localization method using image queries. The query information is integrated into the image feature maps by utilizing non-local neural networks [78] and channel co-excitation [28].…”

“…In this work, we study a more challenging task of open-set query-guided object localization with the following goal -given an image of a natural scene and an object query, localize all the instances of the queried object in the image, even if no sample for this queried object is assumed available during the training phase. In the literature, query-guided object localization has been attempted using either object category name [79] or an image of the object as a query [25,67]. However, it is possible to encounter scenarios where neither an image of the object nor the class label is available as a query.…”