Safe Traffic Sign Recognition through Data Augmentation for Autonomous Vehicles Software

Jöckel, Lisa; Kläs, Michael; Martínez‐Fernández, Silverio

doi:10.1109/qrs-c.2019.00114

Cited by 17 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are DL models whose dataset is increased when there is not enough data. This can be done through techniques such as obtaining new entries from those that already exist, carrying out transformations to these including cuts and rotations, among others [8,14,38]. Another way is using adversary examples, which C. Szegedy discovered in 2013, when he noticed that several machine learning and DL models are vulnerable to slightly different inputs from those that are correctly classified, i.e.…”

Section: Adversarial Attack: Fast Gradient Sign Methods (Fgsm)mentioning

confidence: 99%

Guiding the retraining of convolutional neural networks againstadversarial inputs: best guidance metrics and configurations

2022

View full text Add to dashboard Cite

Background: When using deep learning models, there are many possible vulnerabilities and some of the most worrying are the adversarial inputs, which can cause wrong decisions (e.g., wrongly classifying an image) with minor perturbations. Therefore, it becomes necessary to retrain these models against adversarial inputs, as part of the software testing process addressing the vulnerability to adversarial inputs. Furthermore, for an energy efficient testing and retraining process, data scientists need support on which are the best guidance metrics for reducing the adversarial inputs to use as well as optimal dataset configurations.Aims: We examined four guidance metrics for retraining deep learning models, specifically with convolutional neural network architecture, and three retraining configurations. Our goal is to improve the convolutional neural networks against adversarial inputs with regard to accuracy, resource utilization and time from the point of view of a data scientist in the context of image classification.Method: We conduced an empirical study in two datasets for image classification. We explore: (a) the accuracy, resource utilization and time of retraining convolutional neural networks by ordering new training set by four different guidance metrics (neuron coverage, likelihood-based surprise adequacy, distance-based surprise adequacy and random), (b) the accuracy and resource utilization of retraining convolutional neural networks with three different configurations (from scratch and augmented dataset, using weights and augmented dataset, and using weights and only adversarial inputs).Results: We reveal that retraining with adversarial inputs from original model weights and by ordering with surprise adequacy metrics gives the best model w.r.t. accuracy, resource utilization and time.Conclusions: Although more studies are necessary, we recommend data scientists to use the above configuration and metrics to deal with the vulnerability to adversarial inputs of deep learning models, as they can improve their models against adversarial inputs without using many inputs. We also show that dataset size has an important impact on the results. CCS Concepts• Computing methodologies → Machine learning; Artificial intelligence; • Software and its engineering → Software creation and management.

show abstract

Section: Adversarial Attack: Fast Gradient Sign Methods (Fgsm)mentioning

confidence: 99%

Guiding the retraining of convolutional neural networks againstadversarial inputs: best guidance metrics and configurations

2022

View full text Add to dashboard Cite

show abstract

“…According to the analyses and studies performed together with the existing models, the algorithms were improved and subsequently configured in order to integrate them into the simulated scenarios, presenting themselves in the first simulations with satisfactory results and in accordance with the previously proposed hardware architecture. The erosion and analysis of input blocks were based on the analysis of an architecture based in MobileNetV2 and YOLOv3, improved by increasing the analysis capacity of several blocks, and using Neural Intel Coral increased processing time [57,58].…”

Section: Description Pedestrian Setup and Practical Scenariosmentioning

confidence: 99%

Inter-Urban Analysis of Pedestrian and Drivers through a Vehicular Network Based on Hybrid Communications Embedded in a Portable Car System and Advanced Image Processing Technologies

Zadobrischi

Dimian

2021

Remote Sensing

View full text Add to dashboard Cite

Vehicle density and technological development increase the need for road and pedestrian safety systems. Identifying problems and addressing them through the development of systems to reduce the number of accidents and loss of life is imperative. This paper proposes the analysis and management of dangerous situations, with the help of systems and modules designed in this direction. The approach and classification of situations that can cause accidents is another feature analyzed in this paper, including detecting elements of a psychosomatic nature: analysis and detection of the conditions a driver goes through, pedestrian analysis, and maintaining a preventive approach, all of which are embedded in a modular architecture. The versatility and usefulness of such a system come through its ability to adapt to context and the ability to communicate with traffic safety systems such as V2V (vehicle-to-vehicle), V2I (vehicle-to-infrastructure), V2X (vehicle-to-everything), and VLC (visible light communication). All these elements are found in the operation of the system and its ability to become a portable device dedicated to road safety based on (radio frequency) RF-VLC (visible light communication).

show abstract

“…There are DL models whose dataset is augmented when there is not enough data. This can be done through techniques such as obtaining new entries from those that already exist, carrying out transformations to these including cuts and rotations, and synthetic data generation, among others ( Tian et al, 2018 ; Feinman et al, 2017 ; Jöckel, Kläs & Martínez-Fernández, 2019 ). Another way is using adversary inputs, which C. Szegedy discovered in 2013 when he noticed that several DL models are vulnerable to slightly different inputs from those that are correctly classified, i.e ., the adversarial inputs ( Szegedy et al, 2013 ).…”

Section: Introductionmentioning

confidence: 99%

Guiding the retraining of convolutional neural networks against adversarial inputs

Durán

Martínez‐Fernández

Felderer

et al. 2023

PeerJ Computer Science

View full text Add to dashboard Cite

Background When using deep learning models, one of the most critical vulnerabilities is their exposure to adversarial inputs, which can cause wrong decisions (e.g., incorrect classification of an image) with minor perturbations. To address this vulnerability, it becomes necessary to retrain the affected model against adversarial inputs as part of the software testing process. In order to make this process energy efficient, data scientists need support on which are the best guidance metrics for reducing the adversarial inputs to create and use during testing, as well as optimal dataset configurations. Aim We examined six guidance metrics for retraining deep learning models, specifically with convolutional neural network architecture, and three retraining configurations. Our goal is to improve the convolutional neural networks against the attack of adversarial inputs with regard to the accuracy, resource utilization and execution time from the point of view of a data scientist in the context of image classification. Method We conducted an empirical study using five datasets for image classification. We explore: (a) the accuracy, resource utilization, and execution time of retraining convolutional neural networks with the guidance of six different guidance metrics (neuron coverage, likelihood-based surprise adequacy, distance-based surprise adequacy, DeepGini, softmax entropy and random), (b) the accuracy and resource utilization of retraining convolutional neural networks with three different configurations (one-step adversarial retraining, adversarial retraining and adversarial fine-tuning). Results We reveal that adversarial retraining from original model weights, and by ordering with uncertainty metrics, gives the best model w.r.t. accuracy, resource utilization, and execution time. Conclusions Although more studies are necessary, we recommend data scientists use the above configuration and metrics to deal with the vulnerability to adversarial inputs of deep learning models, as they can improve their models against adversarial inputs without using many inputs and without creating numerous adversarial inputs. We also show that dataset size has an important impact on the results.

show abstract

Safe Traffic Sign Recognition through Data Augmentation for Autonomous Vehicles Software

Cited by 17 publications

References 9 publications

Guiding the retraining of convolutional neural networks againstadversarial inputs: best guidance metrics and configurations

Guiding the retraining of convolutional neural networks againstadversarial inputs: best guidance metrics and configurations

Inter-Urban Analysis of Pedestrian and Drivers through a Vehicular Network Based on Hybrid Communications Embedded in a Portable Car System and Advanced Image Processing Technologies

Guiding the retraining of convolutional neural networks against adversarial inputs

Contact Info

Product

Resources

About