Indirect Local Attacks for Context-Aware Semantic Segmentation Networks

Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey

“…Even for the tasks beyond classification, multiple other attacks exist that do not fall under the categories described above. For example, Nakka et al [257] devised an attack to demonstrate the vulnerability of semantic segmentation networks against holistic perturbations and localized ones. Similarly, for the problem of segmentation, a data membership attack is devised in [258].…”

On the Real-World Adversarial Robustness of Real-Time Semantic Segmentation Models for Autonomous Driving

“…First, they evaluated the adversarial effect against non-real-time networks only. From our point of view, this leads to unrealistic assessments, since real-time architectures usually do not rely on the rich sequence of modules used by the evaluated networks, which could be practically prone to adversarial attacks [28]. Second, and most importantly, they did not consider realworld adversarial objects, which might represent a real threat for driving scenarios.…”

“…b) Adversarial studies on SS: Previous works [19], [20], [21], [22], [23], [17], [24], [25] proved that both targeted and untargeted pixel-based perturbations easily fool SS models by extending well-known adversarial strategies (e.g., [26], [27]) from image classification. Consequently, Nakka et al [28] presented an interesting study on the robustness of SS models on autonomous driving datasets by showing that it is possible to perturb a precise area of pixels to change the SS prediction corresponding to specific objects placed in the whole image.…”

“…There are many ways to define a target for this problem (for instance, providing the label of a completely different scene), but we focus to a case that is more interesting for real-world applications: forcing the network to make a specific class "disappear" from its prediction. This can be done by uniformly changing the pixels belonging to class c attacked into the ones of another class c target , or by applying the nearest neighbor algorithm [28] to provide a more realistic target label, as illustrated in Figure 3. The nearest neighbor algorithm associates to each pixel belonging to c attacked the class of the closest pixel of a different class.…”

“…The pixel-wise cross-entropy (CE) loss, denoted by L CE , has been shown to work well for untargeted digital attacks by adding a perturbation r to pixels values [24] [19]. In this case, the loss is…”