Bridging a Gap in SAR-ATR: Training on Fully Synthetic and Testing on Measured Data

Abstract

Obtaining measured Synthetic Aperture Radar (SAR) data for training Automatic Target Recognition (ATR) models can be too expensive (in terms of time and money) and complex of a process in many situations. In response, researchers have developed methods for creating synthetic SAR data for targets using electromagnetic prediction software, which is then used to enrich an existing measured training dataset. However, this approach relies on the availability of some amount of measured data. In this work, we focus on the case of having 100% synthetic training data, while testing on only measured data. We use the SAMPLE dataset public released by AFRL, and find significant challenges to learning generalizable representations from the synthetic data due to distributional differences between the two modalities and extremely limited training sample quantities. Using deep learning-based ATR models, we propose data augmentation, model construction, loss function choices, and ensembling techniques to enhance the representation learned from the synthetic data, and ultimately achieved over 95% accuracy on the SAMPLE dataset. We then analyze the functionality of our ATR models using saliency and feature-space investigations and find them to learn a more cohesive representation of the measured and synthetic data. Finally, we evaluate the out-oflibrary detection performance of our synthetic-only models and find that they are nearly 10% more effective than baseline methods at identifying measured test samples that do not belong to the training class set. Overall, our techniques and their compositions significantly enhance the feasibility of using ATR models trained exclusively on synthetic data.

Improving Out-of-Distribution Detection by Learning From the Deployment Environment

“…As observed in [52], [6], [28], using an adversarial maximizer in the DNN's training objective can significantly improve the robustness and quality of learned representations in SAR-ATR models. Inkawhich et al [28] directly found that this modification also leads to sizable gains in OOD detection performance.…”

“…Previous work has shown that when the majority of the training distribution is measured, the detection problem for the SAMPLE dataset is nearly solved [28]. However, it is also shown that at K = 0, OOD detection of the SAMPLE-Holdout classes is particularly difficult because there is a distribution gap between the training and test sets that is the result of inaccuracies in the complex SAR simulation process [6]. For this reason, we focus on the K = 0 setting in our SAR-ATR experiments.…”

“…the clean data sample, and directly controls the size of the perturbation δ that is applied to x. We follow [6] and [28] by defining S adv as an ∞ ball centered at x with a radius of = 8/255; and approximate the inner maximization using 7iteration Projected Gradient Descent (PGD) adversarial attack [53]. The intuition for adding the maximizer to the first term of AdvOE and AdvMixupOE is to learn more robust features of the ID classes.…”

“…Importantly, regardless of the exact data-format/modality/task such RS systems are most commonly built using Deep Neural Networks (DNNs) to perform the underlying recognition function [1], [2], [3], [4]. While DNNs have shown to be exceptionally accurate in many settings [5], [6], [7], they have also shown to yield poor confidence calibration on anomalous/out-of-distribution (OOD) data, which leads to erroneous high-confidence predictions on input samples that do not belong to one of the known in-distribution (ID) classes [8], [9], [10]. This is a particularly dangerous failure-mode of a system that is expected to be deployed in an "open-world" operating environment.…”

“…In this domain, we focus exclusively on the scenario where the offline training distribution is composed of 100% syntheticallygenerated SAR targets, while the test distribution contains 100% measured/real SAR data. This presents a considerable challenge for existing algorithms [13], [6], however, is a problem we believe our learn-online workflow is positioned to excel at.…”