Fooling Detection Alone is Not Enough: First Adversarial Attack against Multiple Object Tracking

Abstract

Recent work in adversarial machine learning started to focus on the visual perception in autonomous driving and studied Adversarial Examples (AEs) for object detection models. However, in such visual perception pipeline the detected objects must also be tracked, in a process called Multiple Object Tracking (MOT), to build the moving trajectories of surrounding obstacles. Since MOT is designed to be robust against errors in object detection, it poses a general challenge to existing attack techniques that blindly target objection detection: we find that a success rate of over 98% is needed for them to actually affect the tracking results, a requirement that no existing attack technique can satisfy. In this paper, we are the first to study adversarial machine learning attacks against the complete visual perception pipeline in autonomous driving, and discover a novel attack technique, tracker hijacking, that can effectively fool MOT using AEs on object detection. Using our technique, successful AEs on as few as one single frame can move an existing object in to or out of the headway of an autonomous vehicle to cause potential safety hazards. We perform evaluation using the Berkeley Deep Drive dataset and find that on average when 3 frames are attacked, our attack can have a nearly 100% success rate while attacks that blindly target object detection only have up to 25%. * Equal contribution Preprint. Under review.

DIMBA: discretely masked black-box attack in single object tracking

“…Taking the realm of physical world attacks into account, Eykholt et al (2018) analyzed adversarial stickers on stop signs in the context of autonomous driving to fool YOLO (Redmon et al, 2016). Jia et al (2019) proposed a 'tracking hijacking' technique to fool multiple object trackers with imperceptible perturbations computed for object detectors in the perceptual pipeline of autonomous driving. Meanwhile, Yan et al (2020a) developed an attacking technique to deceive single object trackers based on SiamRPN++ (Li et al, 2018).…”

A Unified Multi-Scenario Attacking Network for Visual Object Tracking

“…For example, (Szegedy et al 2013) first shows that adversarial examples, generated by adding visually imperceptible perturbations to the original images, could make classification models predict a wrong label with high confidence. Further, (Thys, Van Ranst, and Goedemé 2019) successfully generates adversarial patches that can hide a person from a person detector while (Jia et al 2019) studies adversarial attacks against the visual perception pipeline in autonomous Figure 1: The framework of our UEN, which consists of a generator G and a attacked victim model F . Three different losses are developed for the multi-scenario attacks.…”

“…Using typical attack methods (Jia et al 2019;Szegedy et al 2013;Thys, Van Ranst, and Goedemé 2019) to generate adversarial perturbations against SOT is difficult and we analyze and summarize several reasons for it. Firstly, SOT algorithms could handle information across frames in realtime to locate the trajectory of the target in videos.…”