Environmental Sound Recognition on Embedded Systems: From FPGAs to TPUs

Abstract

In recent years, Environmental Sound Recognition (ESR) has become a relevant capability for urban monitoring applications. The techniques for automated sound recognition often rely on machine learning approaches, which have increased in complexity in order to achieve higher accuracy. Nonetheless, such machine learning techniques often have to be deployed on resource and power-constrained embedded devices, which has become a challenge with the adoption of deep learning approaches based on Convolutional Neural Networks (CNNs). Field-Programmable Gate Arrays (FPGAs) are power efficient and highly suitable for computationally intensive algorithms like CNNs. By fully exploiting their parallel nature, they have the potential to accelerate the inference time as compared to other embedded devices. Similarly, dedicated architectures to accelerate Artificial Intelligence (AI) such as Tensor Processing Units (TPUs) promise to deliver high accuracy while achieving high performance. In this work, we evaluate existing tool flows to deploy CNN models on FPGAs as well as on TPU platforms. We propose and adjust several CNN-based sound classifiers to be embedded on such hardware accelerators. The results demonstrate the maturity of the existing tools and how FPGAs can be exploited to outperform TPUs.

A Review of AIoT-based Edge Devices and Lightweight Deployment

“…Usually, we first apply knowledge distillation to improve the performance, then apply network pruning [132], [135] or quantization. Or first, apply network pruning, then quantization [88], [90], [95], [96], [100]. In summary, knowledge distillation should be applied first to ensure performance; then network pruning for model compression; quantization should be placed in the last step because the training of the quantized network is challenging.…”

The Internet of Sounds: Convergent Trends, Insights, and Future Directions

“…Various embedded systems have been utilized for this purpose, using both real-time and non-real-time architectures. Relevant examples concern environmental monitoring [159], ecoacoustics (e.g., birds monitoring) [160]- [162] and urban sounds [31]. Smartphones are also increasingly used for similar purposes [163] along with drones [164], [165].…”

Embedded Sensing System for Recognizing Citrus Flowers Using Cascaded Fusion YOLOv4-CF + FPGA

“…Figure 6 shows the main steps of the YOLOv4-CF + FPGA object recognition model migration and deployment process, which comprise three parts: model format conversion in a TensorFlow environment on the server, the compilation and quantification of a model in a Vitis AI environment, and programming processing of the ZCU104 platform on the embedded FPGA side. In the first part, the Xilinx Vitis AI tool is affected by the technical update iteration speed and version limitation [ 30 ] because it does not support the direct quantification of Keras weight files created in the TensorFlow environment. Thus, this study transforms the weight of the Keras model and network architecture file into a binary protobuf file.…”