Cyber-Physical Analytics: Environmental Sound Classification at the Edge

Abstract

With the growth of the Internet of Things and the rise of Big Data, data processing and machine learning applications are being moved to cheap and low size, weight, and power (SWaP) devices at the edge, often in the form of mobile phones, embedded systems, or microcontrollers. The field of Cyber-Physical Measurements and Signature Intelligence (MASINT) makes use of these devices to analyze and exploit data in ways not otherwise possible, which results in increased data quality, increased security, and decreased bandwidth. However, methods to train and deploy models at the edge are limited, and models with sufficient accuracy are often too large for the edge device. Therefore, there is a clear need for techniques to create efficient AI/ML at the edge. This work presents training techniques for audio models in the field of environmental sound classification at the edge. Specifically, we design and train Transformers to classify office sounds in audio clips. Results show that a BERTbased Transformer, trained on Mel spectrograms, can outperform a CNN using 99.85% fewer parameters. To achieve this result, we first tested several audio feature extraction techniques designed for Transformers, using ESC-50 for evaluation, along with various augmentations. Our final model outperforms the state-ofthe-art MFCC-based CNN on the office sounds dataset, using just over 6,000 parameters -small enough to run on a microcontroller.

Forest Sound Classification Dataset: FSC22

“…Figure 1 shows a section of the ESC-50 dataset taxonomy emphasizing forest-specific sounds. Moreover, U8K [ 35 ] is another popular dataset used in many types of research on audio-based monitoring systems [ 18 , 36 ]. U8K is a subset of the main Urban Sound dataset, which contains 8732 labelled sound clips of urban sounds from 10 classes.…”

“…Although several studies have been carried out in the forest acoustic monitoring context, still, a standard benchmark dataset specific to forest sounds is unavailable. Therefore, most of the existing studies have utilized publicly available environmental sound datasets such as ESC-50 [ 4 , 13 , 14 , 15 , 16 , 17 ], UrbanSound8K (U8k) [ 14 , 18 , 19 , 20 , 21 ], FSD50K [ 22 , 23 ], and SONYC-UST [ 24 , 25 ]. These datasets contain a large quantity of audio data categorized into several groups covering a broad area of sound events.…”

ESC-NAS: Environment Sound Classification Using Hardware-Aware Neural Architecture Search for the Edge

“…However, the employment of NAS is limited within the domain of environmental sound classification. Moreover, in the existing audio classification studies employing NAS, feature extraction techniques such as Mel and MFCC were commonly utilized [ 16 , 17 ], akin to those in image classification tasks. Addressing the aforementioned research gaps, this study endeavored to pursue the following research questions: RQ1: With the general nature of DL models being highly resource intensive, how can one design a DL model for environmental sound classification in resource-constrained environments?…”

“…Another key application of ESC is urban sound classification, which delves into the acoustic ecology within urban landscapes, notably cities. With advanced approaches such as DL-based audio classifiers on edge devices, urban sound classification holds immense potential for designing, managing, and monitoring sustainable urban environments that promote human well-being, and security, and can minimize noise pollution alongside ecological diversity [ 16 , 26 , 27 ].…”

“…There have been several studies conducted in the domain of ESC. Studies such as [ 16 , 23 , 26 , 31 ] used DL for ESC, while focusing on deploying DL-based audio classifiers on IoT devices. Out of all the studies related to DL-based ESC on resource-constrained edge devices, by Mohaimenuzzaman et al [ 32 ], stands out as a state-of-the-art DL model that performed well under the aforementioned conditions.…”