Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving

Sobh, Ibrahim; Hamed, Ahmed Abdeen; Kumar, Varun; Yogamani, Senthil

doi:10.2352/j.imagingsci.technol.2021.65.6.060408

Cited by 13 publications

(7 citation statements)

References 25 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rear-view fisheye cameras have become a standard feature for dashboard viewing and reverse parking, even in lower-cost vehicles. Fisheye cameras are used in for autonomous driving tasks such as perception which involves object detection [6], [7], soiling detection [8], [9], semantic segmentation [10], [11], weather classification [12], depth prediction [13], [14], [15], [16], [5], moving object detection [17] and SLAM [18], [19], [20] are challenging due to the highly dynamic and interactive nature of surrounding objects in the automotive scenarios [21]. Fisheye cameras are also used commonly in other domains like video surveillance [22] and augmented reality [23].…”

Section: A Surround-view Camera Systemmentioning

confidence: 99%

SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving

Sekkat,

Dupuis,

Kumar

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Section: A Surround-view Camera Systemmentioning

confidence: 99%

SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving

Sekkat,

Dupuis,

Kumar

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Camera calibration was converted to a pixel-wise tensor and fed into the model to have adaptation to various camera intrinsics. Sobh et al [151] studied the effect of adversarial attacks in a multi-task setup using OmniDet, which is important for safety-critical applications. The tests addressed both white and black box attacks for targeted and untargeted cases and the effect of using a simple defense strategy while attacking a task and analyzing the effect on the others.…”

Section: Mulitask Modelsmentioning

confidence: 99%

Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges

Kumar¹,

Eising²,

Witt³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Surround-view fisheye cameras are commonly used for near-field sensing in automated driving. Four fisheye cameras on four sides of the vehicle are sufficient to cover 360°around the vehicle capturing the entire near-field region. Some primary use cases are automated parking, traffic jam assist, and urban driving. There are limited datasets and very little work on near-field perception tasks as the main focus in automotive perception is on far-field perception. In contrast to far-field, surround-view perception poses additional challenges due to high precision object detection requirements of 10cm and partial visibility of objects. Due to the large radial distortion of fisheye cameras, standard algorithms can not be extended easily to the surround-view use case. Thus we are motivated to provide a selfcontained reference for automotive fisheye camera perception for researchers and practitioners. Firstly, we provide a unified and taxonomic treatment of commonly used fisheye camera models. Secondly, we discuss various perception tasks and existing literature. Finally, we discuss the challenges and future direction. I. INTRODUCTIONSurround-view systems use four sensors to form a network with overlap regions, sufficient to cover the near-field area around the car. Figure 1 shows the four views of a typical surround-view system, along with a representation of the typical parking use-case. Wide-angle views exceeding 180 • are used for this near-field sensing. Any perception algorithm must consider the significant fisheye distortion inherent with such camera systems. This is a significant challenge, as most work in computer vision focuses on narrow field-of-view cameras with mild radial distortion. However, as such camera systems are more widely deployed, work has been completed in this area. It is the aim of this paper to give the reader an overview of surround view cameras (e.g., image formation, configuration and rectification), to survey the existing state of the art, and to provide insights into the current challenges in the area.In theory, the field-of-view of a pinhole camera is 180 • . However, in practice, due to the practical limitations of the size of the aperture and imager, it is not easy to get over 80 • , as illustrated in Figure 2 (top). Fisheye lenses are commonly used to effectively increase the field-of-view to 180 • or more. It is interesting to note that the term fisheye is a bit of a misnomer, as illustrated in Figure 2 (bottom). Due to the bending of light rays due to refraction at the junction of water and air surface, a large field-of-view of nearly 180 • is compressed to a smaller V. Ravi Kumar, C. Witt, and S. Yogamani are with Valeo. C. Eising is with the Dept. of Electronic and Computer Engineering at the University of Limerick. V. Ravi Kumar and C. Eising are co-first authors.

show abstract

“…Therefore, it has become popular in near field sensing at low speed [13,14]. Several experiments have been conducted to enhance the performance of CNN on fisheye dataset by investigating the impact of adversarial attacks [15] on a multi-task visual perception network [16]. The domain of autonomous driving in- volves object detection [17,18,19], soiling detection [20,21,22], semantic segmentation [23], weather classification [24,25], dynamic object detection [26], depth prediction [27,28,29,30,31], fusion [32], key-point detection and description [33] and multitask learning [34,35,36].…”

Section: Introductionmentioning

confidence: 99%

FisheyePixPro: Self-supervised pretraining using Fisheye images for semantic segmentation

Ramchandra¹,

Sistu²,

Eising³

et al. 2022

Self Cite

View full text Add to dashboard Cite

Self-supervised learning has been an active area of research in the past few years. Contrastive learning is a type of selfsupervised learning method that has achieved a significant performance improvement on image classification task. However, there has been no work done in its application to fisheye images for autonomous driving. In this paper, we propose FisheyePixPro, which is an adaption of pixel level contrastive learning method PixPro [1] for fisheye images. This is the first attempt to pretrain a contrastive learning based model, directly on fisheye images in a self-supervised approach. We evaluate the performance of learned representations on the WoodScape dataset using segmentation task. Our FisheyePixPro model achieves a 65.78 mIoU score, a significant improvement over the PixPro model. This indicates that pre-training a model on fisheye images have a better performance on a downstream task.

show abstract

Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving

Cited by 13 publications

References 25 publications

SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving

SynWoodScape: Synthetic Surround-view Fisheye Camera Dataset for Autonomous Driving

Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges

FisheyePixPro: Self-supervised pretraining using Fisheye images for semantic segmentation

Contact Info

Product

Resources

About