Tasks Integrated Networks: Joint Detection and Retrieval for Image Search

Abstract

The traditional object (person) retrieval (re-identification) task aims to learn a discriminative feature representation with intra-similarity and inter-dissimilarity, which supposes that the objects in an image are manually or automatically pre-cropped exactly. However, in many real-world searching scenarios (e.g., video surveillance), the objects (e.g., persons, vehicles, etc.) are seldom accurately detected or annotated. Therefore, object-level retrieval becomes intractable without bounding-box annotation, which leads to a new but challenging topic, i.e. image-level search with multi-task integration of joint detection and retrieval. In this paper, to address the image search issue, we first introduce an end-to-end Integrated Net (I-Net), which has three merits: 1) A Siamese architecture and an on-line pairing strategy for similar and dissimilar objects in the given images are designed. Benefited by the Siamese structure, I-Net learns the shared feature representation, because, on which, both object detection and classification tasks are handled. 2) A novel on-line pairing (OLP) loss is introduced with a dynamic feature dictionary, which alleviates the multi-task training stagnation problem, by automatically generating a number of negative pairs to restrict the positives. 3) A hard example priority (HEP) based softmax loss is proposed to improve the robustness of classification task by selecting hard categories. The shared feature representation of I-Net may restrict the task-specific flexibility and learning capability between detection and retrieval tasks. Therefore, with the philosophy of divide and conquer, we further propose an improved I-Net, called DC-I-Net, which makes two new contributions: 1) two modules are tailored to handle different tasks separately in the integrated framework, such that the task specification is guaranteed. 2) A class-center guided HEP loss (C 2 HEP) by exploiting the stored class centers is proposed, such that the intra-similarity and inter-dissimilarity can be captured for ultimate retrieval. Extensive experiments on famous image-level search oriented benchmark datasets, such as CUHK-SYSU dataset and PRW dataset for person search and the large-scale WebTattoo dataset for tattoo search, demonstrate that the proposed DC-I-Net outperforms the state-of-the-art tasks-integrated and tasks-separated image search models.

Retrieval and Characteristic Analysis of Multimedia Tester Based on Bragg Equation

“…Second, the difference between different media semantics lies not only in their corresponding feature values, but also in the characteristics used to define them; The methods of using these features in media content analysis may also be different. This makes it difficult to classify various media semantics accurately and reliably [9]. There are two basic ideas to solve this problem.…”

Decoupled and Memory-Reinforced Networks: Towards Effective Feature Learning for One-Step Person Search

“…4. The results of two-step methods (Chang et al 2018;Chen et al 2018;Lan, Zhu, and Gong 2018;Han et al 2019;Wang et al 2020) are shown in the upper block while the one-step methods (Xiao et al 2017(Xiao et al , 2019Liu et al 2017;Yan et al 2019;Zhang et al 2020;Munjal et al 2019;Chen et al 2020) To evaluate the performance consistency, we also compare with other competitive methods under varying gallery sizes of [50,100,500,1000,2000,4000]. Fig.…”

Decoupled and Memory-Reinforced Networks: Towards Effective Feature Learning for One-Step Person Search

“…4. The results of two-step methods (Chang et al 2018;Chen et al 2018;Lan, Zhu, and Gong 2018;Han et al 2019;Wang et al 2020) are shown in the upper block while the one-step methods (Xiao et al 2017(Xiao et al , 2019Liu et al 2017;Yan et al 2019;Zhang et al 2020;Munjal et al 2019;Chen et al 2020) in the lower block. To evaluate the performance consistency, we also compare with other competitive methods under varying gallery sizes of [50,100,500,1000,2000,4000].…”