Coarse-to-Fine: A RNN-Based Hierarchical Attention Model for Vehicle Re-identification

Abstract

Xiu-Shen Wei 1[0000−0002−8200−1845] , Chen-Lin Zhang 2[0000−0002−3168−1852] , Lingqiao Liu 3[0000−0003−3584−795X] , Chunhua Shen 3[0000−0002−8648−8718] , and Jianxin Wu 2[0000−0002−2085−7568]Abstract Vehicle re-identification is an important problem and becomes desirable with the rapid expansion of applications in video surveillance and intelligent transportation. By recalling the identification process of human vision, we are aware that there exists a native hierarchical dependency when humans identify different vehicles. Specifically, humans always firstly determine one vehicle's coarse-grained category, i.e., the car model/type. Then, under the branch of the predicted car model/type, they are going to identify specific vehicles by relying on subtle visual cues, e.g., customized paintings and windshield stickers, at the fine-grained level. Inspired by the coarse-to-fine hierarchical process, we propose an end-to-end RNN-based Hierarchical Attention (RNN-HA) classification model for vehicle re-identification. RNN-HA consists of three mutually coupled modules: the first module generates image representations for vehicle images, the second hierarchical module models the aforementioned hierarchical dependent relationship, and the last attention module focuses on capturing the subtle visual information distinguishing specific vehicles from each other. By conducting comprehensive experiments on two vehicle re-identification benchmark datasets VeRi and VehicleID, we demonstrate that the proposed model achieves superior performance over state-of-theart methods. Coarse-to-fineModel level Vehicle level Query … … … … Figure 1: Illustration of coarse-to-fine hierarchical information as a latent but crucial cue for vehicle re-identification. (Best viewed in color and zoomed in.) has diverse applications in video surveillance [30], intelligent transportation [37]and urban computing [40]. Moreover, vehicle re-identification has recently drawn increasing attentions in the computer vision community [19,20,23]. Compared with the classic person re-identification problem, vehicle re-identification could be more challenging as different specific vehicles can only be distinguished by slight and subtle differences, such as some customized paintings, windshield stickers, favorite decorations, etc. Nevertheless, there still conceals some latent but crucial information for handling this problem. As shown in Fig. 1, when humans identify different vehicles, they always follow a coarse-to-fine identification process. Specifically, we tend to firstly determine this specific vehicle belongs to which car model/type. The first step can eliminate many distractors, i.e., vehicles with similar subtle visual appearances but belonging to the other different car models/types. In the following, within the candidate vehicle set of the same car model/type, humans will carefully distinguish different vehicles from each other by using these subtle visual cues. Apparently, there is a hierarchical dependency in this coarse-to-fine process, which is yet neg...

Multi-Proxy Constraint Loss for Vehicle Re-Identification

“…Recent works on vehicle Re-ID can be divided as viewpoint-independent and viewpoint-dependent. The viewpoint-independent methods such as PROVID [ 13 ], RHH-HA [ 16 ], RAM [ 3 ], Multi-Region Model (MRM) [ 31 ] mainly focus on learning of robust global and local features or learning of distance metric (Batch Sample (BS) [ 32 ] and Feature Distance Adversarial Network (FDA-Net) [ 5 ]). The viewpoint-dependent methods are dedicated to learning orientation-invariance features (Orientation Invariant Feature Embedding (OIFE) and OIFE + ST) [ 6 ] or multi-view features (VAMI and VAMI + ST) [ 8 ].…”

“…Cui et al [ 15 ] fused the classification features of color, vehicle model, and pasted marks on windshield as the final features to describe the vehicle. Some studies used a variety of attributes to identify vehicles from coarse to fine, such as Progress Vehicle Re-identification (PROVID) [ 13 ] and RNN-based Hierarchical Attention (RNN-HA) [ 16 ]. These coarse-to-fine approaches require multiple recognition processes and cannot be implemented end to end.…”

Local Feature-Aware Siamese Matching Model for Vehicle Re-Identification

“…Wang et al [19] utilized 20 key-point locations of vehicles to extract orientation information and proposed an orientation invariant feature embedding module. De et al [23] proposed a two-stream Siamese classification model for vehicle re-ID, and Wei et al [24] proposed an recurrent neural network-based hierarchical attention (RNN-HA) network, which combines a large number of attributes for vehicle re-ID. Bai et al [14] proposed a group sensitive triplet embedding approach that can model the interclass differences.…”

A Structured Graph Attention Network for Vehicle Re-Identification

“…To address these issues, various approaches have been proposed including deep metric learning models [1,4,6,16,43,46,48] and informative region based approaches [24,40,52]. Deep metric learning approaches focus on learning an appropriate distance metric mainly based on global representation towards reducing the distances between identical vehicle images and enlarging those between different vehicle images.…”

“…As shown in the third and forth rows of Figure 1, the regions within blue boxes are crucial for identifying the two similar vehicles. To utilize such local visual cues, some works [24,40,52] exploited attention models to localize discriminative regions and learn detailed local representations. However, not all regions present discriminative content and are spatially aligned within the parts.…”