Improving Out-of-Distribution Detection by Learning From the Deployment Environment

Abstract

Recognition systems in the remote sensing domain often operate in "open-world" environments, where they must be capable of accurately classifying data from the indistribution categories while simultaneously detecting and rejecting anomalous/out-of-distribution (OOD) inputs. However, most modern designs use Deep Neural Networks (DNNs) to perform this recognition function that are trained under "closedworld" assumptions in offline-only environments. As a result, by construction these systems are ill-posed to handle anomalous inputs and have no mechanism for improving OOD detection abilities during deployment. In this work, we address these weaknesses from two aspects. First, we introduce advanced DNN training methods to co-design for accuracy and OOD detection in the offline training phase. We then propose a novel "learn-online" workflow for updating the DNNs during deployment using a small library of carefully collected samples from the operating environment. To show the efficacy of our methods we consider experimenting with two popular recognition tasks in remote sensing: scene classification in electro-optical satellite images and automatic target recognition in synthetic aperture radar imagery. In both, we find that our two primary design contributions can individually improve detection performance, while also being complementary. Additionally, we find that detection performance on difficult and highly-granular OOD samples can be drastically improved using only tens or hundreds of samples collected from the environment. Finally, through analysis we determine that the logic for adding/removing samples from the collection library is of key importance and using a proper learning rate during the model update step is critical.

Target Recognition in SAR Images by Deep Learning with Training Data Augmentation

“…Regarding this research line, we have submitted a paper to the 2023 International Radar Symposium (IRS), which is titled as “Super-class labeling for out-of-library targets with deep learning and multiview information fusion”. Second, we could resort to active learning, cross-domain transfer learning, and transductive learning to compensate for the lack of annotated SAR training samples, and organize the scene recognition and the OOD sample detection problem into a single framework [ 30 ]. Regarding this research line, we have submitted a paper to the 2023 International Geoscience and Remote Sensing Symposium (IGARSS), which is titled as “SAR image scene classification and out-of-library target detection with cross-domain transfer learning”.…”

Open Set Recognition and Category Discovery Framework for SAR Target Classification Based on K-Contrast Loss and Deep Clustering

“…At present, most studies focus on incremental learning. By using limited labeled new category samples to fine-tune the classifier or feature extractor, the model can identify new categories at the lowest possible cost [20]- [22]. Such methods of handling detected unknown class data are indeed appealing, but they overlook some essential steps: determining the number of categories of unknown classes in a batch, followed by classification and labeling.…”

A Global Model Approach to Robust Few-Shot SAR Automatic Target Recognition

“…Intuitively, the classifier is taught to be accurate and confident on labeled task data while being minimally confident on the OOD data. This objective creates a confidence calibration effect which can be exploited to detect novel OOD samples during deployment [9].…”