OutlierNets: Highly Compact Deep Autoencoder Network Architectures for On-Device Acoustic Anomaly Detection

Abstract

Human operators often diagnose industrial machinery via anomalous sounds. Given the new advances in the field of machine learning, automated acoustic anomaly detection can lead to reliable maintenance of machinery. However, deep learning-driven anomaly detection methods often require an extensive amount of computational resources prohibiting their deployment in factories. Here we explore a machine-driven design exploration strategy to create OutlierNets, a family of highly compact deep convolutional autoencoder network architectures featuring as few as 686 parameters, model sizes as small as 2.7 KB, and as low as 2.8 million FLOPs, with a detection accuracy matching or exceeding published architectures with as many as 4 million parameters. The architectures are deployed on an Intel Core i5 as well as a ARM Cortex A72 to assess performance on hardware that is likely to be used in industry. Experimental results on the model’s latency show that the OutlierNet architectures can achieve as much as 30x lower latency than published networks.

Machine Learning for Microcontroller-Class Hardware: A Review

“…Customized data layout and loop reordering of each attention kernel, coupled with quantization, have allowed porting transformers onto microcontrollers [119] by minimizing computationally intensive data marshaling operations. The use of depthwise and pointwise convolution has been shown to yield autoencoder architectures as small as 2.7 kB for anomaly detection [120].…”

“…Anomaly detection or one-class classification detects outliers or in the input data stream to indicate malfunctions [120] in an unsupervised fashion. Included in MLPerf Tiny v0.5 benchmark, applications of anomaly detection include diagnosis of industrial machinery [8], [9], [120], physiological disorders (e.g., heart attacks and seizures) [171], and climate conditions [211]. The two most common network architectures for microcontroller-based anomaly detection are fully connected autoencoders (FC-AEs) and depthwise CNN.…”

“…DROOC assumes that normal points lie on a low-dimensional linear manifold, while points surrounding the normal points outside a threshold radius are outliers, which can be augmented in a generative adversarial manner into the training set. Other audio-based anomaly detectors generally operate on mel-spectrograms [8], [9], [120].…”

An Intelligent Model Based on Deep Transfer Learning for Detecting Anomalies in Cyber-Physical Systems

“…The difficulty here is to obtain information from several sensors that differ in their specific acoustic properties [6]. Researchers propose new methods and expand existing algorithms for detecting industrial equipment faults [6][7][8][9][10][11][12][13].…”

“…OutlierNets, a family of very compact deep convolutional autoencoder architectures adapted for real-time acoustic anomaly detection, were proposed [12]. It has extremely low complexity and matches or exceeds large convolutional autoencoder architecture by AUC (area under the receiver operating characteristic curve) exhibiting microsecond scale latency on embedded hardware.…”