Deep learning in video multi-object tracking: A survey

Ciaparrone, Gioele; Sánchez, Francisco Luque; Tabik, Siham; Troiano, Luigi; Tagliaferri, Roberto; Herrera, Francisco

doi:10.1016/j.neucom.2019.11.023

Cited by 507 publications

(266 citation statements)

References 161 publications

(258 reference statements)

Supporting

Mentioning

236

Contrasting

Order By: Relevance

“…Object Tracking. In the last few years, MOT (Multi-Object Tracking) based on deep-learning has reached state-of-the-art performance in terms of quality of tracking [15]. For example, an object detector like Faster-RCNN [10] associated with a linear Kalman filter allows a good compromise between processing time and quality of tracking, as shown by SORT (Simple Online and Realtime racking) [16].…”

Section: Related Workmentioning

confidence: 99%

Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility

Mauri

Khemmar

Decoux

et al. 2020

Sensors

View full text Add to dashboard Cite

In core computer vision tasks, we have witnessed significant advances in object detection, localisation and tracking. However, there are currently no methods to detect, localize and track objects in road environments, and taking into account real-time constraints. In this paper, our objective is to develop a deep learning multi object detection and tracking technique applied to road smart mobility. Firstly, we propose an effective detector-based on YOLOv3 which we adapt to our context. Subsequently, to localize successfully the detected objects, we put forward an adaptive method aiming to extract 3D information, i.e., depth maps. To do so, a comparative study is carried out taking into account two approaches: Monodepth2 for monocular vision and MADNEt for stereoscopic vision. These approaches are then evaluated over datasets containing depth information in order to discern the best solution that performs better in real-time conditions. Object tracking is necessary in order to mitigate the risks of collisions. Unlike traditional tracking approaches which require target initialization beforehand, our approach consists of using information from object detection and distance estimation to initialize targets and to track them later. Expressly, we propose here to improve SORT approach for 3D object tracking. We introduce an extended Kalman filter to better estimate the position of objects. Extensive experiments carried out on KITTI dataset prove that our proposal outperforms state-of-the-art approches.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility

Mauri

Khemmar

Decoux

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…One standard approach in multi-object tracking algorithms is tracking-by-detection [34]. Detecting is performed separately.…”

Section: The Proposed Methodsmentioning

confidence: 99%

A Multilayer Motion Direction Based Model for Tracking Vehicles at Intersections

Delavarian

Marouzi

Hassanpour

2020

IJE

View full text Add to dashboard Cite

Visual vehicle tracking is an important topic in intelligent transportation systems. Intersections are challenging locations for visual systems to track vehicles which are simultaneously moving in different directions. In addition, normal traffic flow may change at intersections due to accidents. Congestion, occlusion and undetermined motion flows are the nominated challenging issues of vehicle tracking at intersections. In this paper, a method for tracking multiple vehicles is proposed considering the vehicle motion directions to overcome undetermined motion flows. For this purpose, a multilayer model is presented, which assigns each motion flows to distinct layers. Moreover, we introduce different neighborhoods for various layers considering the regular motion flows in a layer. Hence, vehicles entering from the same side of intersection with the same motion direction are assigned to the same layer. Then the tracking is performed on different layers separately. In special cases such as vehicles crossing each other, misdetections or occlusion, the proposed tracking method can predict the vehicles tracks by using the stored tracking history and considering neighborhoods in that layer. Experimental results show consistency between proposed tracking method results and ground truth, also outperformance of other tracking methods in tracking vehicles crossing the intersection.

show abstract

“…Frequently, the dominant flows of movement in the crowd are also determined [20] . Many existing works have successfully tackled this research sub-area [21] . Feature extraction stage.…”

Section: A Taxonomy For Crowd Behaviour Analysismentioning

confidence: 99%

Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects

Sánchez

Hupont²,

Tabik

et al. 2020

Information Fusion

Self Cite

View full text Add to dashboard Cite

Crowd behaviour analysis is an emerging research area. Due to its novelty, a proper taxonomy to organise its different sub-tasks is still missing. This paper proposes a taxonomic organisation of existing works following a pipeline, where sub-problems in last stages benefit from the results in previous ones. Models that employ Deep Learning to solve crowd anomaly detection, one of the proposed stages, are reviewed in depth, and the few works that address emotional aspects of crowds are outlined. The importance of bringing emotional aspects into the study of crowd behaviour is remarked, together with the necessity of producing real-world, challenging datasets in order to improve the current solutions. Opportunities for fusing these models into already functioning video analytics systems are proposed.

show abstract

Deep learning in video multi-object tracking: A survey

Cited by 507 publications

References 161 publications

Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility

Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility

A Multilayer Motion Direction Based Model for Tracking Vehicles at Intersections

Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects

Contact Info

Product

Resources

About