Adversarial Appearance Learning in Augmented Cityscapes for Pedestrian Recognition in Autonomous Driving

Savkin, Artem; Lapotre, Thomas; Strauss, Kevin A.; Akbar, Uzair; Tombari, Federico

doi:10.1109/icra40945.2020.9197024

Cited by 6 publications

(6 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our setup is based on [24], which augments the Cityscapes dataset [8] with 3D pedestrian models and adversarially learns to cast realistic style on those models by means of a multi-discriminator. Due to the inpainting of virtual model instances, the pedestrian class becomes the most imbalanced class between the original and augmented dataset (see 1).…”

Section: Attention With Adversarial Lossmentioning

confidence: 99%

“…To counteract such undesired behavior, several works proposed to split discriminator of the adversarial network into multiple ones to overcome distribution discrepancies [16,24]. The intuition behind it is to restrict the decisive context of the discriminator and let it consider only specific aspects (e.g.…”

Section: Multi-discriminator Architecturementioning

confidence: 99%

“…For (a) and (b) M c (s) is down-sampled to match the dimensions of the feature maps and output layer. This functionality is enabled by MaskLayer which is shown in figure 3 [24].…”

Section: Multi-discriminator Architecturementioning

confidence: 99%

“…In the final step of our experimental setup, we applied the augmentation strategy from [24] to enrich our custom dataset with pedestrian instances. This strategy picks randomly a number of objects from the pool of CAD pedestrians and puts them onto a so-called spawn map -set of collision-free spots on the ground.…”

Section: Datasetsmentioning

confidence: 99%

“…Predicted discrepancy areas are then utilized in the second step of the multi-discriminator adaptation setup. We employ a data augmentation pipeline based on [24] that blends virtual pedestrians into existing real image scenes to enable a controlled generation of yet unseen urban traffic scenarios involving pedestrians.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Attention-based Adversarial Appearance Learning of Augmented Pedestrians

Strauss,

Savkin,

Tombari

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Synthetic data became already an essential component of machine learning-based perception in the field of autonomous driving. Yet it still cannot replace real data completely due to the sim2real domain shift. In this work, we propose a method that leverages the advantages of the augmentation process and adversarial training to synthesize realistic data for the pedestrian recognition task. Our approach utilizes an attention mechanism driven by an adversarial loss to learn domain discrepancies and improve sim2real adaptation. Our experiments confirm that the proposed adaptation method is robust to such discrepancies and reveals both visual realism and semantic consistency. Furthermore, we evaluate our data generation pipeline on the task of pedestrian recognition and demonstrate that generated data resemble properties of the real domain.

show abstract

Section: Attention With Adversarial Lossmentioning

confidence: 99%

Section: Multi-discriminator Architecturementioning

confidence: 99%

“…For (a) and (b) M c (s) is down-sampled to match the dimensions of the feature maps and output layer. This functionality is enabled by MaskLayer which is shown in figure 3 [24].…”

Section: Multi-discriminator Architecturementioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations