A 0.8 mW TinyML-Based PDM-to-PCM Conversion for In-Sensor KWS Applications

Vitolo, Paola; Liguori, Rosalba; Benedetto, Luigi Di; Rubino, Alfredo; Pau, Danilo; Licciardo, Gian Domenico

doi:10.1007/978-3-031-26066-7_23

Cited by 4 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the autoencoder, two loss functions were employed: the Mean Absolute Error (MAE) and the Fast-Fourier-Transform Mean Absolute Error (FFT-MAE). The FFT-MAE is a custom loss function introduced in [25], [26]. As described by ( 4), it returns the mean absolute error between the FFT of the model outputs and the FFT of the corresponding labels.…”

Section: Trainingmentioning

confidence: 99%

Automatic Audio Feature Extraction for Keyword Spotting

Vitolo,

Liguori,

Benedetto

et al. 2024

IEEE Signal Process. Lett.

Self Cite

View full text Add to dashboard Cite

The accuracy and computational complexity of keyword spotting (KWS) systems are heavily influenced by the choice of audio features in speech signals. This paper introduces a novel approach for audio feature extraction in KWS by leveraging a convolutional autoencoder, which has not been explored in the existing literature. Strengths of the proposed approach are in the ability to automate the extraction of the audio features, keep its computational complexity low, and allow accuracy values of the overall KWS systems comparable with the state of the art. To evaluate the effectiveness of our proposal, we compared it with the widely-used Mel Frequency Cepstrum (MFC) method in terms of classification metrics in noisy conditions and the number of required operators, using the public Google speech command dataset. Results demonstrate that the proposed audio feature extractor achieves an average classification accuracy on 12 classes ranging from 81.84% to 90.36% when the signal-tonoise ratio spans from 0 to 40 dB, outperforming the MFC up to 5.2%. Furthermore, the required number of operations is one order of magnitude lower than that of the MFC, resulting in a reduction in computational complexity and processing time, which makes it well-suited for integration with KWS systems in resource-constrained edge devices.

show abstract

Section: Trainingmentioning

confidence: 99%

Automatic Audio Feature Extraction for Keyword Spotting

Vitolo,

Liguori,

Benedetto

et al. 2024

IEEE Signal Process. Lett.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The proposed system is based on a tiny CNN, chosen for its highly proven effectiveness in image classification and fewer parameters compared to other architecture. [22][23][24][25][26][27] The model is designed to classify images into 3 classes, representing different percentage intervals of Cloud Coverage (CCov) as reported in: 28 clear (CCov < 35%), mid-cloudy (35% ≤ CCov ≤ 65%) and cloudy (CCov > 65%).…”

Section: The Proposed Systemmentioning

confidence: 99%

“…[8][9][10] In particular, Convolutional Neural Networks (CNNs) have demonstrated highly effective in image classification tasks, due to their ability to automatically capture local patterns and spatial features by leveraging convolutional filters. [11][12][13][14][15][16][17] Furthermore, techniques such as pooling and normalization contribute to the network's robustness and ability to generalize. [18][19][20] However, the use of NN-based cloud detection methods onboard satellites is constrained due to their typically high computational and memory demands.…”

Section: Introductionmentioning

confidence: 99%

Real-time on-board satellite cloud cover detection hardware architecture using spaceborne remote sensing imagery

Vitolo,

Fasolino,

Liguori

et al. 2024

Real-Time Processing of Image, Depth, and Video Information 2024

View full text Add to dashboard Cite

The growing demand for high-resolution hyperspectral satellite imagery, particularly from nano and microsatellites, is driven by a wide range of critical applications, ranging from disaster management and agriculture to military intelligence. The increasing deployment of thousands of satellites into orbit and the resulting massive data production introduce challenges related to stringent edge-device constraints, especially for nanosatellites. These constraints include limited processing capabilities, power consumption, memory usage, and transmission bandwidth. Clouds, which cover over 66% of the Earth's surface, introduce noise and inaccuracies into satellite imagery, compromising the quality and reliability of image-based systems and potentially leading to invalid data, while unnecessarily consuming transmission bandwidth. To address these issues, this paper introduces a real-time on-board satellite cloud cover detection system based on a lightweight neural network. By discarding excessively cloudy images, the proposed approach can lead to an improvement in the efficiency and accuracy of satellite image-based systems. At the same time, it allows to minimize the data to be transmitted to the ground, consequently mitigating bandwidth problems and reducing transmission power. The proposed CNN shows a compact architecture, requiring fewer than 9 thousand parameters, while maintaining a detection accuracy of 89% when evaluated using the Landsat 8 dataset. An optimized hardware accelerator is designed to meet the on-board nanosatellites constraints. Post-implementation simulations on a Xilinx Artix 7 FPGA demonstrate state-of-the-art results with a utilization of about 12 thousand and 7 thousand of mapped LUTs and FFs, respectively, with a power consumption of 116 mW.

show abstract

“…3 These aspects are even more advantageous in the context of In-Sensor Computing (ISC), where the processing circuits are moved close to the audio sensor, integrated into its auxiliary circuitry, or in the same package, realizing a compact smart sensor. [4][5][6][7] A KWS pipeline involves three main stages: a conditioning signal module to adapt the microphone output to audio processing systems, a feature extractor, and a classifier. The quality of the extracted audio features is of primary importance because it directly affects the complexity and accuracy of the classifier.…”

Section: Introductionmentioning

confidence: 99%

In-sensor neural network for real-time KWS by image processing

Vitolo

Esposito

Pau

et al. 2023

Real-Time Processing of Image, Depth and Video Information 2023

View full text Add to dashboard Cite

KeyWord Spotting (KWS), i.e. the capability to identify vocal commands as they are pronounced, is becoming one of the most important features of Human-Machine Interface (HMI), also thanks to the pervasive diffusion of high-performance MEMS audio sensors with very reduced dimensions. In-Sensor Computing (ISC) appears the most viable solution to get the maximum advantage of KWS, since the dimensions of MEMS microphones remain reduced and minimally invasive. ISC, indeed, represents the extreme evolution of the edge computing paradigm, where the processing circuits are moved close to the audio sensor, integrated into its auxiliary circuitry or in the same package. However, ISC introduces severe area and power constraints and must trade off with processing speed to meet real-time operations naturally required by KWS. In this work, we want to show a neural networkbased KWS suitable for ISC contexts, when audio sensor data are converted into MEL spectrogram images and a Depthwise Separable Convolutional Neural Network (DSCNN) with feature extraction capabilities is designed. To show the advantages of the above approach, the DSCNN is compared with an alternative Fully Connected Neural Network (FCNN), operating on audio signals not converted into images. The considered models have been profiled on a microcontroller and implemented on an FPGA. Their performances are compared in terms of classification accuracy and HW resources. Comparisons show that the FCNN is very far from meeting the ISC real-time processing requirements, showing a number of parameters and a frame latency respectively of 3 and 1 orders of magnitude higher than required by the DSCNN alternative when mapped to a Xilinx Zynq Ultrascale+ MPSoC.

show abstract

A 0.8 mW TinyML-Based PDM-to-PCM Conversion for In-Sensor KWS Applications

Cited by 4 publications

References 14 publications

Automatic Audio Feature Extraction for Keyword Spotting

Automatic Audio Feature Extraction for Keyword Spotting

Real-time on-board satellite cloud cover detection hardware architecture using spaceborne remote sensing imagery

In-sensor neural network for real-time KWS by image processing

Contact Info

Product

Resources

About