Learning Pedestrian Detection from Virtual Worlds

Amato, Giuseppe; Ciampi, Luca; Falchi, Fabrizio; Gennaro, Claudio; Messina, Nicola

doi:10.1007/978-3-030-30642-7_27

Cited by 21 publications

(24 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Introductionmentioning

confidence: 63%

“…Specifically, in this work, we extend our previous paper [ 13 ]. Compared to [ 13 ], we obtain better results, employing a new state-of-the-art detector that exhibits higher performance and introducing a new domain adaptation strategy. Furthermore, we carry out extensive experimentation over additional publicly available datasets, demonstrating the robustness of our approach over different real-world scenarios.…”

Section: Introductionmentioning

confidence: 63%

“…We exploit Faster-RCNN [ 36 ] as object detector architecture. In our previous work [ 13 ], we employed YOLOv3 [ 38 ], another state-of-the-art object detector. Here, our choice fell on Faster R-CNN since it provides better performance.…”

Section: Domain Adaptation For Synthetic2real Pedestrian Detectionmentioning

confidence: 99%

See 2 more Smart Citations

Virtual to Real Adaptation of Pedestrian Detectors

Ciampi

Messina

Falchi

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

Pedestrian detection through Computer Vision is a building block for a multitude of applications. Recently, there has been an increasing interest in convolutional neural network-based architectures to execute such a task. One of these supervised networks’ critical goals is to generalize the knowledge learned during the training phase to new scenarios with different characteristics. A suitably labeled dataset is essential to achieve this purpose. The main problem is that manually annotating a dataset usually requires a lot of human effort, and it is costly. To this end, we introduce ViPeD (Virtual Pedestrian Dataset), a new synthetically generated set of images collected with the highly photo-realistic graphical engine of the video game GTA V (Grand Theft Auto V), where annotations are automatically acquired. However, when training solely on the synthetic dataset, the model experiences a Synthetic2Real domain shift leading to a performance drop when applied to real-world images. To mitigate this gap, we propose two different domain adaptation techniques suitable for the pedestrian detection task, but possibly applicable to general object detection. Experiments show that the network trained with ViPeD can generalize over unseen real-world scenarios better than the detector trained over real-world data, exploiting the variety of our synthetic dataset. Furthermore, we demonstrate that with our domain adaptation techniques, we can reduce the Synthetic2Real domain shift, making the two domains closer and obtaining a performance improvement when testing the network over the real-world images.

show abstract

Section: Introductionmentioning

confidence: 63%

Section: Introductionmentioning

confidence: 63%

See 1 more Smart Citation

Virtual to Real Adaptation of Pedestrian Detectors

Ciampi

Messina

Falchi

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…[MVGL10, VLM*13] was probably the first to explore synthetic images generation, using a computer game (Half‐Life 2 [HL2]) that depicts urban environments, by developing appropriate game mods. In such manner, several data generation methods followed based on capturing video sequences from the GTA‐V game and provided large‐scale synthetic training sets with vehicles, pedestrians and various objects detection annotations [JRBM*16, RHK17, HCW19, ACF*19]. More real‐time rendering generation pipelines have been developed in 3D development platforms utilizing non‐procedural physically based modelling [GWCV16, QY16, QZZ*17], non‐procedural non‐physically based modelling with domain and rendering randomization and object infusion [TPA*18] (Figure 12a) and procedural, physically based modelling in a structured domain and rendering randomization manner [PBB*18] (Figure 12b).…”

Section: Image Synthesis Methods Overviewmentioning

confidence: 99%

“…[TPA*18], [WU18]

^{false(13 false)}

, [PBB*18], [KPL*19]

^{false(10 false)}

, [HCW19]

^{false(11 false)}

, [HVG*19], [ACF*19]

^{false(16 false)}

…”

Section: Introductionmentioning

confidence: 99%

A Survey of Image Synthesis Methods for Visual Machine Learning

Tsirikoglou

Eilertsen

Unger

2020

Computer Graphics Forum

View full text Add to dashboard Cite

Image synthesis designed for machine learning applications provides the means to efficiently generate large quantities of training data while controlling the generation process to provide the best distribution and content variety. With the demands of deep learning applications, synthetic data have the potential of becoming a vital component in the training pipeline. Over the last decade, a wide variety of training data generation methods has been demonstrated. The potential of future development calls to bring these together for comparison and categorization. This survey provides a comprehensive list of the existing image synthesis methods for visual machine learning. These are categorized in the context of image generation, using a taxonomy based on modelling and rendering, while a classification is also made concerning the computer vision applications they are used. We focus on the computer graphics aspects of the methods, to promote future image generation for machine learning. Finally, each method is assessed in terms of quality and reported performance, providing a hint on its expected learning potential. The report serves as a comprehensive reference, targeting both groups of the applications and data development sides. A list of all methods and papers reviewed herein can be found at https://computergraphics.on.liu.se/image_synthesis_methods_for_visual_machine_learning/.

show abstract