Efficient generation of image chips for training deep learning algorithms

Han, Sanghui; Fafard, Alexander; Kerekes, John P.; Gartley, Michael; Ientilucci, Emmett J.; Savakis, Andreas; Parhan, Jason; Turek, Matt; Fieldhouse, Keith; Rovito, Todd V.

doi:10.1117/12.2261702

Cited by 17 publications

(15 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge no rigorous work has been conducted on utilizing synthetic overhead imagery (e.g., satellite or high-altitude aerial photography) for training machine-learning models. The only existing work exploring this idea was recently (2018 presented in [34], however this work suffers from numerous limitations. Most crucially, the authors employed a military-grade rendering software that is inaccessible to the public, and their raw synthetic imagery was not made publicly available.…”

Section: Related Workmentioning

confidence: 99%

The Synthinel-1 dataset: a collection of high resolution synthetic overhead imagery for building segmentation

Kong

Huang

Bradbury

et al. 2020

2020 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Recently deep learningnamely convolutional neural networks (CNNs)have yielded impressive performance for the task of building segmentation on large overhead (e.g., satellite) imagery benchmarks. However, these benchmark datasets only capture a small fraction of the variability present in real-world overhead imagery, limiting the ability to properly train, or evaluate, models for real-world application. Unfortunately, developing a dataset that captures even a small fraction of real-world variability is typically infeasible due to the cost of imagery, and manual pixel-wise labeling of the imagery. In this work we develop an approach to rapidly and cheaply generate large and diverse virtual environments from which we can capture synthetic overhead imagery for training segmentation CNNs. Using this approach, generate and publicly-release a collection of synthetic overhead imagery termed Synthinel-1 with full pixel-wise building labels. We use several benchmark dataset to demonstrate that Synthinel-1 is consistently beneficial when used to augment real-world training imagery, especially when CNNs are tested on novel geographic locations or conditions.

show abstract

Section: Related Workmentioning

confidence: 99%

The Synthinel-1 dataset: a collection of high resolution synthetic overhead imagery for building segmentation

Kong

Huang

Bradbury

et al. 2020

2020 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

show abstract

“…Four types of vehicles were placed within the road network using the simulation of urban mobility (SUMO) [36], [37] where the location of the vehicles changed at time increments similar to real vehicles moving in traffic. When simulating multiple captures at a given rate, the location of the vehicles in each image was different from one image to another [38], [39]. This allowed a single simulation run to generate multiple images with changing target locations that were time correlated.…”

Section: Assessment: Dirsigmentioning

confidence: 99%

Optimizing Requirements for a Compact Spaceborne Adaptive Spectral Imaging System in Subpixel Target Detection Applications

Han

Kerekes

Higbee

et al. 2020

IEEE J. Miniat. Air Space Syst.

Self Cite

View full text Add to dashboard Cite

We developed a process to provide design recommendations for compact spaceborne spectral imaging systems with adaptive band selection capabilities. Our focus application was subpixel target detection, and we analyzed a set of mission scenarios to find relationships in detection performance between selected parameters of interest. We used an analytic model to predict performance and generate trade curves, then simulated a scene to analyze potential operational effects on performance for the selected target and background combinations. Using these models, we predicted and assessed each scenario to provide recommendations for mission feasibility and system design. The parameters we selected for analysis were target fill fraction, noise, number of bands, and scene complexity to find critical points in the trade space and reach a set of recommendations. We examined the operational effects by simulating a realistic scenario and ensuring key real-world phenomena were captured within the spectral images. Our results produced recommendations for each mission and provided a proof of concept for a process to analyze designs of miniature spaceborne imaging systems.

show abstract

“…2. In addition, we generate a synthetic single-channel aerial dataset for training a CNN and use it to perform vehicle classification on the real WAMI platform, similar to [14] (Sect. IV).…”

Section: Synthetic Imagery Conceptmentioning

confidence: 99%

“…The Digital Imaging and Remote Sensing (DIRSIG) software has been used before to generate spectral scenarios for varied applications that use conventional computer vision techniques and deep learning based models [13], [6], [14], [15]. Since flying spectral sensors on an aerial platform is still an ongoing arXiv:1711.07235v3 [cs.CV] 6 May 2018 area of development due to the high costs involved, we evaluate our tracker on synthetic scenarios generated using DIRSIG by [16], [17], [5].…”

Section: Synthetic Imagery Conceptmentioning

confidence: 99%

Tracking in Aerial Hyperspectral Videos Using Deep Kernelized Correlation Filters

Uzkent

Rangnekar

Hoffman

2019

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

Hyperspectral imaging holds enormous potential to improve the state-of-the-art in aerial vehicle tracking with low spatial and temporal resolutions. Recently, adaptive multi-modal hyperspectral sensors have attracted growing interest due to their ability to record extended data quickly from aerial platforms. In this study, we apply popular concepts from traditional object tracking, namely (1) Kernelized Correlation Filters (KCF) and (2) Deep Convolutional Neural Network (CNN) features to aerial tracking in hyperspectral domain. We propose the Deep Hyperspectral Kernelized Correlation Filter based tracker (DeepHKCF) to efficiently track aerial vehicles using an adaptive multi-modal hyperspectral sensor. We address low temporal resolution by designing a single KCF-in-multiple Regions-of-Interest (ROIs) approach to cover a reasonably large area. To increase the speed of deep convolutional features extraction from multiple ROIs, we design an effective ROI mapping strategy. The proposed tracker also provides flexibility to couple with the more advanced correlation filter trackers. The DeepHKCF tracker performs exceptionally well with deep features set up in a synthetic hyperspectral video generated by the Digital Imaging and Remote Sensing Image Generation (DIRSIG) software. Additionally, we generate a large, synthetic, single-channel dataset using DIRSIG to perform vehicle classification in the Wide Area Motion Imagery (WAMI) platform. This way, the high-fidelity of the DIRSIG software is proved and a large scale aerial vehicle classification dataset is released to support studies on vehicle detection and tracking in the WAMI platform.

show abstract

Efficient generation of image chips for training deep learning algorithms

Cited by 17 publications

References 1 publication

The Synthinel-1 dataset: a collection of high resolution synthetic overhead imagery for building segmentation

The Synthinel-1 dataset: a collection of high resolution synthetic overhead imagery for building segmentation

Optimizing Requirements for a Compact Spaceborne Adaptive Spectral Imaging System in Subpixel Target Detection Applications

Tracking in Aerial Hyperspectral Videos Using Deep Kernelized Correlation Filters

Contact Info

Product

Resources

About