BFAct: Out-of-Distribution Detection with Butterworth Filter Rectified Activations

Kong, Haojia; Li, Haoan

doi:10.1007/978-981-99-0617-8_9

Cited by 17 publications

(24 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The feature distributions on ModelNet10 and ShapeNet are not exactly the rectified Gaussian distribution ( 16) used to derive the control algorithms. However, features distributions on realworld datasets are well-known to be in the shape of sparse activation, i.e., being (near) zero with high probability, and being high-magnitude with low probability [29]. In that sense, the rectified Gaussian distribution is similar to real-world distributions (see Fig.…”

Section: Performance Of Max-airpoolingmentioning

confidence: 85%

See 1 more Smart Citation

Over-the-Air Multi-View Pooling for Distributed Sensing

Liu¹,

Li²,

Kalør³

et al. 2023

Preprint

View full text Add to dashboard Cite

Sensing is envisioned as a key network function of the sixth-generation (6G) mobile networks.Artificial intelligence (AI)-empowered sensing fuses features of multiple sensing views from devices distributed in edge networks for the edge server to perform accurate inference. This process, known as multi-view pooling, creates a communication bottleneck due to multi-access by many devices. To alleviate this issue, we propose a task-oriented simultaneous access scheme for distributed sensing called Overthe-Air Pooling (AirPooling). The existing Over-the-Air Computing (AirComp) technique can be directly applied to enable Average-AirPooling, which exploits the waveform superposition property of a multiaccess channel to implement fast over-the-air averaging of pooled features. However, despite being most popular in practice, the over-the-air maximization, called Max-AirPooling, is not AirComp realizable given the fact that AirComp addresses only a limited subset of functions. We tackle the challenge by proposing the novel generalized AirPooling framework that can be configured to support both Maxand Average-AirPooling by controlling a configuration parameter and extended to even other pooling functions. The former is realized by adding to AirComp the designed pre-processing at devices and postprocessing at the server. To characterize the End-to-End (E2E) sensing performance in object recognition, the theory of classification margin is applied to relate the classification accuracy and the AirPooling error, which allows the latter to be a tractable surrogate of the former. Furthermore, the analysis reveals an inherent tradeoff of Max-AirPooling between the accuracy of the pooling-function approximation and the effectiveness of noise suppression. Using the tradeoff, we make an attempt to optimize the configuration parameter of Max-AirPooling, yielding a sub-optimal closed-form method of adaptive parametric control. Experimental results obtained on real-world datasets show that AirPooling provides sensing accuracies close to those achievable by the traditional digital air interface but dramatically reduces the communication latency, by up to an order of magnitude.

show abstract

Section: Performance Of Max-airpoolingmentioning

confidence: 85%

“…To solve Problem (P2), we adopt an assumption on the feature distribution following the learning literature (see, e.g., [28], [29]), that results from Gaussian distributed raw features traversing through a ReLU activation unit.…”

Section: B Optimization Of Max-airpoolingmentioning

confidence: 99%

Over-the-Air Multi-View Pooling for Distributed Sensing

Liu¹,

Li²,

Kalør³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Past approaches trained a classification model without knowledge about outliers and conducted OD detection during inference. For example, [19] used simple softmax probability scores during inference to detect OD samples, while other works used Mahalanobis distance [32], rectified activations [48], KL divergence [24] and Gram Matrices [46] instead of softmax scores to detect such samples. On the other hand, ODIN [33] and Generalized ODIN [23] performed perturbations to the test examples to enhance the performance of softmax function for OD detection.…”

Section: Related Workmentioning

confidence: 99%

Normalizing Flow based Feature Synthesis for Outlier-Aware Object Detection

Kumar¹,

Šegvić²,

Eslami³

et al. 2023

Preprint

View full text Add to dashboard Cite

Real-world deployment of reliable object detectors is crucial for applications such as autonomous driving. However, general-purpose object detectors like Faster R-CNN are prone to providing overconfident predictions for outlier objects. Recent outlier-aware object detection approaches estimate the density of instance-wide features with classconditional Gaussians and train on synthesized outlier features from their low-likelihood regions. However, this strategy does not guarantee that the synthesized outlier features will have a low likelihood according to the other classconditional Gaussians. We propose a novel outlier-aware object detection framework that learns to distinguish outliers from inlier objects by learning the joint data distribution of all inlier classes with an invertible normalizing flow. The flow model ensures that the synthesized outliers have a lower likelihood than inliers of all object classes, thereby modeling a better decision boundary between inlier and outlier objects. Our approach significantly outperforms the state-of-the-art for outlier-aware object detection on both image and video datasets.

show abstract

“…8. Sun et al [17] propose ReAct, which works by clipping the activation value that exceeds a certain threshold, and then keeping the activations within a certain range based on the observation that abnormally high activations on OOD data can harm their detection. 9.…”

Section: Background and Related Workmentioning

confidence: 99%

Timing Performance Benchmarking of Out-of-Distribution Detection Algorithms

Luan

Saremi

et al. 2023

J Sign Process Syst

View full text Add to dashboard Cite

In an open world with a long-tail distribution of input samples, Deep Neural Networks (DNNs) may make unpredictable mistakes for Out-of-Distribution (OOD) inputs at test time, despite high levels of accuracy obtained during model training. OOD detection can be an effective runtime assurance mechanism for safe deployment of machine learning algorithms in safety–critical applications such as medical imaging and autonomous driving. A large number of OOD detection algorithms have been proposed in recent years, with a wide range of performance metrics in terms of accuracy and execution time. For real-time safety–critical applications, e.g., autonomous driving, timing performance is of great importance in addition to accuracy. We perform a comprehensive and systematic benchmark study of multiple OOD detection algorithms in terms of both accuracy and execution time on different hardware platforms, including a powerful workstation and a resource-constrained embedded device, equipped with both CPU and GPU. We also profile and analyze the internal details of each algorithm to identify the performance bottlenecks and potential for GPU acceleration. This paper aims to provide a useful reference for the practical deployment of OOD detection algorithms for real-time safety–critical applications.

show abstract

BFAct: Out-of-Distribution Detection with Butterworth Filter Rectified Activations

Cited by 17 publications

References 21 publications

Over-the-Air Multi-View Pooling for Distributed Sensing

Over-the-Air Multi-View Pooling for Distributed Sensing

Normalizing Flow based Feature Synthesis for Outlier-Aware Object Detection

Timing Performance Benchmarking of Out-of-Distribution Detection Algorithms

Contact Info

Product

Resources

About