PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data

Abstract

In comparison with person re-identification (ReID), which has been widely studied in the research community, vehicle ReID has received less attention. Vehicle ReID is challenging due to 1) high intra-class variability (caused by the dependency of shape and appearance on viewpoint), and 2) small inter-class variability (caused by the similarity in shape and appearance between vehicles produced by different manufacturers). To address these challenges, we propose a Pose-Aware Multi-Task Re-Identification (PAMTRI) framework. This approach includes two innovations compared with previous methods. First, it overcomes viewpointdependency by explicitly reasoning about vehicle pose and shape via keypoints, heatmaps and segments from pose estimation. Second, it jointly classifies semantic vehicle attributes (colors and types) while performing ReID, through multi-task learning with the embedded pose representations. Since manually labeling images with detailed pose and attribute information is prohibitive, we create a largescale highly randomized synthetic dataset with automatically annotated vehicle attributes for training. Extensive experiments validate the effectiveness of each proposed component, showing that PAMTRI achieves significant improvement over state-of-the-art on two mainstream vehicle ReID benchmarks: VeRi and CityFlow-ReID. Code and models are available at https://github.com/NVlabs/PAMTRI. * Work done as an intern at NVIDIA. Zheng is now with Amazon.

A Strong and Efficient Baseline for Vehicle Re-Identification Using Deep Triplet Embedding

“…Ours 128 ID GSTE [2] 1024 ID VAMI [62] 2048 ID + A OIFE [53] 256 ID + K MGR [58] 1024 ID + A ATT [58] 1024 ID + A C2F [9] 1024 ID + A CLVR [19] 1024 A PAMTRI (All)* [49] 1024 ID + K + A MSVR [20] 2048 ID FDA-Net [31] 1024 ID AAVER [21] 2048 ID + K Referring to the best results in Table 2, in the subsequent Sections we consider only triplet loss without embedding-normalization.…”

“…In order to deal with lack of fine grained data to include vehicle attributes for re-identification, [49] proposed PAMTRI (Pose-Aware Multi-Task Re-Identification), which explicitly reason about vehicle pose and shape via keypoints, heatmaps and segments from pose estimation. Training is performed in multi-task learning fashion.…”

“…Table 12 shows results of various sampling approaches and comparisons with state-of-the-art approaches. Our straightforward approach to training outperforms PAMTRI (RS) [49] uses synthetic data in conjunction with real data. Figure 4 shows qualitative assessment on our best performing sampling variant on this dataset.…”

Multi-Proxy Constraint Loss for Vehicle Re-Identification

“…Besides generating multi-view features, Chu et al [ 11 ] learned two metrics for similar viewpoints and different viewpoints, then used the corresponding matric to evaluate the similarity of two images based on whether the viewpoints are similar. Tang et al [ 9 ] reasoned the vehicle pose and shape with synthetic datasets and passed this information to the attributes and feature learning network. Khorramshahi et al [ 10 ] increased a path to detect vehicle key points using the orientation as a conditional factor and extract the local features to distinguish similar vehicles.…”

“…Some work [ 6 , 7 , 8 , 9 , 10 , 11 ] were devoted to addressing the intra-class variance problem of vehicle re-identification by predicting key points or viewpoints. The key points can be passed as input to feature extract network [ 9 ] or directly used as the discriminant regions to aggregate the orientation-invariant features [ 6 ] and trained supervised by IDs to distinguish similar vehicles [ 10 ]. Despite obtaining local discriminative features, key points require extra labels and are only partially visible in different viewpoints.…”