Enhancing the association in multi‐object tracking via neighbor graph

Abstract

Most modern multi‐object tracking (MOT) systems for videos follow the tracking‐by‐detection paradigm, where objects of interest are first located in each frame then associated correspondingly to form their intact trajectories. In this setting, the appearance features of objects usually provide the most important cues for data association, but it is very susceptible to occlusions, illumination variations, and inaccurate detections, thus easily resulting in incorrect trajectories. To address this issue, in this study we propose to make full use of the neighboring information. Our motivations derive from the observations that people tend to move in a group. As such, when an individual target's appearance is remarkably changed, the observer can still identify it with its neighbor context. To model the contextual information from neighbors, we first utilize the spatiotemporal relations among trajectories to efficiently select suitable neighbors for targets. Subsequently, we construct neighbor graph for each target and corresponding neighbors then employ the graph convolutional networks (GCNs) to model their relations and learn the graph features. To the best of our knowledge, it is the first time to explicitly leverage neighbor cues via GCN in MOT. Finally, standardized evaluations on the MOT16 and MOT17 data sets demonstrate that our approach can remarkably reduce the identity switches whilst achieve state‐of‐the‐art overall performance.

Self-supervised Re-identification for Online Joint Multi-object Tracking

“…Tracking-by-detection (TBD) methods [26,27] usually use powerful detection networks such as Faster R-CNN [7] and YOLO families [8,9] or dedicated pedestrian detection networks [28,29] to detect objects in each frame, and next extract the features of these objects such as appearance features [30][31][32] through a feature extraction network. Then, matching algorithms such as the Hungarian algorithm [33], network flow [34], multi-hypothesis tracking [35], and graph-based learning [36][37][38] are used to associate all objects in the frame, thereby generating tracking trajectories. The advantage of TBD method is that it can track new targets that appear at any time throughout the entire video, but this method requires a "good" object detection algorithm.…”

LMOT: Efficient Light-Weight Detection and Tracking in Crowds

“…Tracking-by-detection approaches [13,14,15,16] separate detection and tracking steps. These approaches deal only with the current frame and usually apply popular off-shelf detector networks to generate detection bounding boxes for objects of interest, such as DPM [17], Faster-RCNN [18] and SDP [19].…”

“…Moreover, Bae [15] proposed a tracking framework that models objects' visual and radar features and their affinity using a confidence-based data association model and a visual learning object model. Alternatively, Liang et al [16] use graph neighbor networks to model full contextual relations for each tracklet with its surrounding neighbor tracklets for effective data association. Accordingly, the tracking-bydetection paradigm's powerful aspect is that for each task, we build the most convenient model for its goal.…”

Learning to rank: An intelligent system for person reidentification

“…The main purpose of P‐Reid is to match and retrieve all images of a person having similar patterns, captured through visual sensors. Recently, this area has attracted numerous researchers in the field of information retrieval, due to its potential applications to surveillance for industrial security 2,3 . Although a huge number of P‐Reid techniques have been developed in recent years, however, there is still room for improvement due to numerous challenges, such as similarities in background information, differences in illumination and viewpoint, the diverse range of body positions, full/partial occlusion, among others 4 …”

“…Recently, this area has attracted numerous researchers in the field of information retrieval, due to its potential applications to surveillance for industrial security. 2,3 Although a huge number of P-Reid techniques have been developed in recent years, however, there is still room for improvement due to numerous challenges, such as similarities in background information, differences in illumination and viewpoint, the diverse range of body positions, full/partial occlusion, among others. 4 In recent years, substantial research efforts have been made to address the above-mentioned challenges, where deep learning models have attracted more attention than traditional approaches.…”