Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles

Kiefer, Benjamin; Ott, David J.; Zell, Andreas

doi:10.48550/arxiv.2112.12252

Cited by 6 publications

(10 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This result is in line with the conclusions of previous works addressing the problem of a so-called “reality” gap, existing in the case of using both real (experimental) and “synthetic” data for deep neural network training. Namely, the results obtained during the application of convolutional neural networks in the field of computer vision − allow one to conclude that the optimal ratio of real data to “synthetic” data in any deep learning task could be about 5–20% to 80–95% (for a more comprehensive review we refer the reader to Section S6 in the Supporting Information). Since our current experimental data set contains 214 PIs with 607 values of their T g values, one may assume that starting with about 214 real samples at the lowest possible real-to-synthetic data ratio 5% to 95% we will obtain just about 5000 samples in total effective real-and-synthetic training set.…”

Section: Resultsmentioning

confidence: 99%

“…In addition, we address the “reality gap” problem that arises from the differences in the size of the experimental and “synthetic” data sets influencing the GCNN training quality. This problem has been first explored in the area of deep learning in computer vision tasks: − “synthetic datasets” may contain rather huge values with fail-proof ground-truth labeling; a large amount of data could significantly decrease the model performance on real data. Regarding this issue, we derive important estimates about the necessary amount of data on pretraining stage for the effective development of GCNN to predict polymer properties.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning with Enormous “Synthetic” Data Sets: Predicting Glass Transition Temperature of Polyimides Using Graph Convolutional Neural Networks

et al. 2022

View full text Add to dashboard Cite

In the present work, we address the problem of utilizing machine learning (ML) methods to predict the thermal properties of polymers by establishing "structure−property" relationships. Having focused on a particular class of heterocyclic polymers, namely polyimides (PIs), we developed a graph convolutional neural network (GCNN), being one of the most promising tools for working with big data, to predict the PI glass transition temperature T g as an example of the fundamental property of polymers. To train the GCNN, we propose an original methodology based on using a "transfer learning" approach with an enormous "synthetic" data set for pretraining and a small experimental data set for its fine-tuning. The "synthetic" data set contains more than 6 million combinatorically generated repeating units of PIs and theoretical values of their T g values calculated using the well-established Askadskii's quantitative structure−property relationship (QSPR) computational scheme. Additionally, an experimental data set for 214 PIs was also collected from the literature for training, fine-tuning, and validation of the GCNN. Both "synthetic" and experimental data sets are included into a PolyAskInG database (Polymer Askadskii's Intelligent Gateway). By using the PolyAskInG database, we developed GCNN which allows estimation of T g of PI with a mean absolute error (MAE) of about 20 K, which is 1.5 times lower than in the case of Askadskii QSPR analysis (33 K). To prove the efficiency and usability of the proposed GCNN architecture and training methodology for predicting polymer properties, we also employed "transfer learning" to develop alternative GCNN pretrained on proxy-characteristics taken from the popular quantumchemical QM9 database for small compounds and fine-tuned on an experimental T g values data set from PolyAskInG database. The obtained results indicate that pretraining of GCNN on the "synthetic" polymer data set provides MAE which is almost twice as low as that in the case of using the QM9 data set in the pretraining stage (∼41 K). Furthermore, we address the questions associated with the influence of the differences in the size of the experimental and "synthetic" data sets (so-called "reality gap" problem), as well as their chemical composition on the training quality. Our results state the overall priority of using polymer data sets for developing deep neural networks, and GCNN in particular, for efficient prediction of polymer properties. Moreover, our work opens up a challenge for the theoretically supported generation of large "synthetic" data sets of polymer properties for the training of the complex ML models. The proposed methodology is rather versatile and may be generalized for predicting other properties of different polymers and copolymers synthesized through the polycondensation reaction.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Machine Learning with Enormous “Synthetic” Data Sets: Predicting Glass Transition Temperature of Polyimides Using Graph Convolutional Neural Networks

et al. 2022

View full text Add to dashboard Cite

show abstract

“…For Video Anomaly Detection, we fall back to the Open-Water data set of the Maritime Anomaly Detection Benchmark [17] since it is the only one featuring metadata.…”

Section: Results and Analysismentioning

confidence: 99%

“…We apply the same memory map from before on both UAVs' object detectors' output, but this time, we average their memory maps in the overlapping region. For that, we take an EfficientDet-D0 trained on POG [17] and test it on the following data. We capture four minutes of footage in a similar environment as POG from the viewpoint of two UAVs, a DJI Matrice 210 and a DJI Mavic 2 Pro, denoted 2AVs.…”

Section: Cooperative Detection Via Multiple Uavsmentioning

confidence: 99%

Diminishing Domain Bias by Leveraging Domain Labels in Object Detection on UAVs

Kiefer

Messmer

Zell

2021

2021 20th International Conference on Advanced Robotics (ICAR)

View full text Add to dashboard Cite

This paper introduces a novel approach to video object detection detection and tracking on Unmanned Aerial Vehicles (UAVs). By incorporating metadata, the proposed approach creates a memory map of object locations in actual world coordinates, providing a more robust and interpretable representation of object locations in both, image space and the real world. We use this representation to boost confidences, resulting in improved performance for several temporal computer vision tasks, such as video object detection, short and long-term single and multi-object tracking, and video anomaly detection. These findings confirm the benefits of metadata in enhancing the capabilities of UAVs in the field of temporal computer vision and pave the way for further advancements in this area.

show abstract

“…Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles [6] RetinaNet feature pyramid network (FPN) backbone…”

mentioning

confidence: 99%

Computer Vision Application Analysis based on Object Detection

Chavan¹,

Hembade²,

Jadhav³

et al. 2023

IJSREM

View full text Add to dashboard Cite

Object detection is a computer vision method that allows for the identification and localization of specific classes of objects in images or videos. It goes beyond simple object classification and helps provide a better understanding of the object in question. Object detection has numerous applications, including automating business processes like inventory management in retail [1]. It can detect objects that occupy between 2% and 60% of an image's area and clusters of objects as a single entity. Additionally, it can localize objects at high speeds, typically greater than 15 frames per second. Vehicle detection is a crucial component in the development of autonomous vehicles, enabling them to identify and perceive objects in their environment. It involves identifying and locating vehicles in image or video frames and has various applications in surveillance and security systems. There are different techniques and models for object detection, including traditional image processing methods and modern deep learning networks. Traditional methods like Viola-Jones, SIFT, and histogram of oriented gradients do not require historical data for training and are unsupervised. Popular image processing tools like OpenCV can be used for these techniques. On the other hand, modern deep learning networks like CNN, RCNN, YOLO, ResNet, RetinaNet, and MANet are supervised and efficient for object detection. Key Words: Deep learning, OpenCV, object detection

show abstract

Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles

Cited by 6 publications

References 31 publications

Machine Learning with Enormous “Synthetic” Data Sets: Predicting Glass Transition Temperature of Polyimides Using Graph Convolutional Neural Networks

Machine Learning with Enormous “Synthetic” Data Sets: Predicting Glass Transition Temperature of Polyimides Using Graph Convolutional Neural Networks

Diminishing Domain Bias by Leveraging Domain Labels in Object Detection on UAVs

Computer Vision Application Analysis based on Object Detection

Contact Info

Product

Resources

About