The effect of simulation variety on a deep learning-based military vehicle detector

Eker, Thijs A.; Heslinga, Friso G.; Ballan, Luca; den Hollander, Richard J. M.; Schutte, Klamer

doi:10.1117/12.2679808

Cited by 2 publications

(8 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Its use has been shown to improve the performance of object detection models in a number of different studies. [1][2][3][4] In the context of computer vision, synthetic data can be defined as images or videos (i) generated from scratch, for example through computer graphics or rendering engines, or generative models like GANs or diffusion models, 41 (ii) modified versions of existing real data, for example employing DL-based augmentations 42 or style transfer, 43 or (iii) a combination of the two, for example inpainting. 44 Besides saving time on data gathering and annotation, synthetic data can also offer the advantage of a large degree of precision and control over the environment through which synthetic datasets are built.…”

Section: Synthetic Datamentioning

confidence: 99%

“…44 Besides saving time on data gathering and annotation, synthetic data can also offer the advantage of a large degree of precision and control over the environment through which synthetic datasets are built. 2 However, synthetic data also comes with its challenges. Bridging the reality gap, as well as building datasets with enough variability to represent all possible target scenarios and characteristics sufficiently, can be a challenge in creating and using synthetic data.…”

Section: Synthetic Datamentioning

confidence: 99%

“…We perform experiments on the challenging VisDrone dataset 62 with DGTA-VisDrone 14 as a synthetic training counterpart, the synthetic-to-real RarePlanes dataset, 13 and a synthetic and real in-house vehicle detection dataset from concurrent work. 2 The details of each dataset are listed below.…”

Section: Datamentioning

confidence: 99%

“…The last dataset we evaluated models on is one developed in-house for concurrent work. 2 The dataset was designed through iterative variation of relevant axes, such as the number of images per object class, objectcamera distance, and object pitch, to reach optimal mAP performance when training a vehicle object detector on synthetic data. 2 The resulting dataset contains 1,600 synthetic and 333 real-world images of 4 types of vehicles.…”

Section: Vehicle Detectionmentioning

confidence: 99%

“…The proliferation of synthetic datasets, driven by advancements in graphics technology and simulation, has opened new avenues for training object detection models. [1][2][3][4] Synthetic data offers the advantage of scalability, cost-effectiveness, and control over various environmental factors. However, bridging the reality gap, the difference between synthetic or simulated and real(-world) data, remains a significant hurdle.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Improving object detector training on synthetic data by starting with a strong baseline methodology

Ruis,

Liezenga,

Heslinga

et al. 2024

Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications II

View full text Add to dashboard Cite

Collecting and annotating real-world data for the development of object detection models is a time-consuming and expensive process. In the military domain in particular, data collection can also be dangerous or infeasible. Training models on synthetic data may provide a solution for cases where access to real-world training data is restricted. However, bridging the reality gap between synthetic and real data remains a challenge. Existing methods usually build on top of baseline Convolutional Neural Network (CNN) models that have been shown to perform well when trained on real data, but have limited ability to perform well when trained on synthetic data. For example, some architectures allow for fine-tuning with the expectation of large quantities of training data and are prone to overfitting on synthetic data. Related work usually ignores various best practices from object detection on real data, e.g. by training on synthetic data from a single environment with relatively little variation. In this paper we propose a methodology for improving the performance of a pre-trained object detector when training on synthetic data. Our approach focuses on extracting the salient information from synthetic data without forgetting useful features learned from pre-training on real images. Based on the state of the art, we incorporate data augmentation methods and a Transformer backbone. Besides reaching relatively strong performance without any specialized synthetic data transfer methods, we show that our methods improve the state of the art on synthetic data trained object detection for the RarePlanes and DGTA-VisDrone datasets, and reach near-perfect performance on an in-house vehicle detection dataset.

show abstract

Section: Synthetic Datamentioning

confidence: 99%

Section: Synthetic Datamentioning

confidence: 99%

Section: Datamentioning

confidence: 99%

Section: Vehicle Detectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Improving object detector training on synthetic data by starting with a strong baseline methodology

Ruis,

Liezenga,

Heslinga

et al. 2024

Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications II

View full text Add to dashboard Cite

show abstract

Automatic satellite characterization using simulated ISAR data

Heslinga,

Caro Cuenca,

Knight

et al. 2024

Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications II

View full text Add to dashboard Cite

Space domain awareness has gained traction in recent years, encompassing the charting and cataloging of space objects, anticipating orbital paths, and keeping track of re-entering objects. Radar techniques can be used to monitor the fast-growing population of satellites, but so far this is mainly used for detection and tracking. For the characterization of a satellite’s capabilities, more detailed information, such as inverse synthetic-aperture radar (ISAR) imaging, is needed. Deep learning has become the preferred method for automated image analysis in various applications. Development of deep learning models typically requires large amounts of training data, but recent studies have shown that synthetic data can be used as an alternative in combination with domain adaption techniques to overcome the domain gap between synthetic and real data. In this study, we present a deep learning-based methodology for automated segmentation of the satellite’s bus and solar panels in ISAR images. We first train a segmentation model using thousands of fast simulated ISAR images and then we finetune the model using a domain adaptation technique that only requires a few samples of the target domain. As a proof of concept, we use a small set of high fidelity simulated ISAR images closely resembling real ISAR images as the target domain. Our proof of concept demonstrates that this domain adaptation technique effectively bridges the domain gap between the training and target radar image domains. Consequently, fast simulated (low fidelity) synthetic datasets are proven to be invaluable for training segmentation models for ISAR images, especially when combined with domain adaptation techniques.

show abstract

The effect of simulation variety on a deep learning-based military vehicle detector

Cited by 2 publications

References 28 publications

Improving object detector training on synthetic data by starting with a strong baseline methodology

Improving object detector training on synthetic data by starting with a strong baseline methodology

Automatic satellite characterization using simulated ISAR data

Contact Info

Product

Resources

About