Edge Deployment Framework of GuardBot for Optimized Face Mask Recognition With Real-Time Inference Using Deep Learning

Manzoor, Sumaira; Kim, Eun-Jin; Joo, Sung-Hyeon; Bae, Sung-Hun; In, Gun-Gyo; Joo, Kyeong-Jin; Choi, Jun-Hyeon; Kuc, Tae-Yong

doi:10.1109/access.2022.3190538

Cited by 14 publications

(12 citation statements)

References 87 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TensorRT contains a deep learning inference optimizer for trained deep learning models and runtime for execution. It speeds up the inference by quantization and other techniques for deployment in the production environment 85 . Therefore, we optimized the proposed custom Keras‐TensorFlow trained DCNNs with Nvidia TensorRT at FP32 (TF‐TRT FP32), FP16 (TF‐TRT FP16), and INT8 (TF‐TRT INT8) precision modes to improve the inference speed.…”

Section: Optimization and Inference Resultsmentioning

confidence: 99%

“…For the optimization steps of the proposed Keras‐TensorFlow model using Nvidia TensorRT SDK, we refer the readers to recent works 14,85 . At first, we loaded the Keras‐TensorFlow DCNN models and saved them in native TensorFlow format.…”

Section: Optimization and Inference Resultsmentioning

confidence: 99%

“…85 Therefore, we optimized the proposed custom Keras-TensorFlow trained DCNNs with Nvidia TensorRT at FP32 (TF-TRT FP32), FP16 (TF-TRT FP16), and INT8 (TF-TRT INT8) precision modes to improve the inference speed. The experiments use TensorFlow (2.7.0), Keras (2.7.0), TensorRT (8.2.1), and JetPack 4.6.3.For the optimization steps of the proposed Keras-TensorFlow model using Nvidia TensorRT SDK, we refer the readers to recent works 14,85. At first, we loaded the Keras-TensorFlow DCNN models and saved them in native TensorFlow format.…”

mentioning

confidence: 99%

See 2 more Smart Citations

A dual‐channel ensembled deep convolutional neural network for facial expression recognition in the wild

Saurav

Saini

Singh

2023

Computational Intelligence

View full text Add to dashboard Cite

Facial expression recognition (FER) in the wild is an active and challenging field of research. A system for automatic FER finds use in a wide range of applications related to advanced human–computer interaction (HCI), human–robot interaction (HRI), human behavioral analysis, gaming and entertainment, etc. Since their inception, convolutional neural networks (CNNs) have attained state‐of‐the‐art accuracy in the facial analysis task. However, recognizing facial expressions in the wild with high confidence running on a low‐cost embedded device remains challenging. To this end, this study presents an efficient dual‐channel ensembled deep CNN (DCE‐DCNN) for FER in the wild. Initially, two DCNNs, namely the and , are trained separately on the grayscale and Scharr‐convolved vertical gradient facial images, respectively. The proposed network later integrates the two pre‐trained DCNNs to obtain the dual‐channel integrated DCNN (DCI‐DCNN). Finally, all three neural networks, namely the , , and DCI‐DCNN, are jointly fine‐tuned to get a single dual‐channel‐multi‐output model. The multi‐output model produces three prediction scores for the given input facial image. The prediction scores are thus fused using the max‐voting ensemble scheme to obtain the DCE‐DCNN with the final classification label. On the FER2013, RAF‐DB, NCAER‐S, AffectNet, and CKPlus benchmark FER datasets, the proposed DCE‐DCNN consistently outperforms the two individual DCNNs and numerous state‐of‐the‐art CNNs. Moreover, the network achieves competitive recognition accuracy on all four FER in the wild datasets with reduced memory storage size and parameters. The proposed DCE‐DCNN model with high throughput on resource‐limited embedded devices is suitable for applications that seek real‐time classification of facial expressions in the wild with high confidence.

show abstract

Section: Optimization and Inference Resultsmentioning

confidence: 99%

Section: Optimization and Inference Resultsmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

A dual‐channel ensembled deep convolutional neural network for facial expression recognition in the wild

Saurav

Saini

Singh

2023

Computational Intelligence

View full text Add to dashboard Cite

show abstract

“…Hardware: The experiments for all the trackers were conducted on the Nvidia Jetson Xavier NX board. However, the limited memory of the Xavier NX posed a challenge as it does not provide enough space to load and run heavy deep learning models efficiently [ 92 ]. To overcome this problem, we created a swap file that allows us to use more memory than the physically installed memory.…”

Section: Resultsmentioning

confidence: 99%

SPT: Single Pedestrian Tracking Framework with Re-Identification-Based Learning Using the Siamese Model

Manzoor¹,

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

Pedestrian tracking is a challenging task in the area of visual object tracking research and it is a vital component of various vision-based applications such as surveillance systems, human-following robots, and autonomous vehicles. In this paper, we proposed a single pedestrian tracking (SPT) framework for identifying each instance of a person across all video frames through a tracking-by-detection paradigm that combines deep learning and metric learning-based approaches. The SPT framework comprises three main modules: detection, re-identification, and tracking. Our contribution is a significant improvement in the results by designing two compact metric learning-based models using Siamese architecture in the pedestrian re-identification module and combining one of the most robust re-identification models for data associated with the pedestrian detector in the tracking module. We carried out several analyses to evaluate the performance of our SPT framework for single pedestrian tracking in the videos. The results of the re-identification module validate that our two proposed re-identification models surpass existing state-of-the-art models with increased accuracies of 79.2% and 83.9% on the large dataset and 92% and 96% on the small dataset. Moreover, the proposed SPT tracker, along with six state-of-the-art (SOTA) tracking models, has been tested on various indoor and outdoor video sequences. A qualitative analysis considering six major environmental factors verifies the effectiveness of our SPT tracker under illumination changes, appearance variations due to pose changes, changes in target position, and partial occlusions. In addition, quantitative analysis based on experimental results also demonstrates that our proposed SPT tracker outperforms the GOTURN, CSRT, KCF, and SiamFC trackers with a success rate of 79.7% while beating the DiamSiamRPN, SiamFC, CSRT, GOTURN, and SiamMask trackers with an average of 18 tracking frames per second.

show abstract

“…Our work diverges from previous studies that assess the optimization of transfer learning-trained models, including [16] which explores multiple models and [17] whose optimization focuses exclusively on reducing computational load. We examine both energy efficiency and performance across two devices, in contrast to prior research that only concentrates on inference tasks.…”

Section: Related Workmentioning

confidence: 90%

Performance and energy efficiency: quantization of models for IoT devices

Hernández,

Almeida,

Blanco

2023

Preprint

View full text Add to dashboard Cite

This document addresses some inherent problems in Machine Learning (ML), such as the high computational and energy costs associated with their implementation on IoT devices. It aims to study and analyze the performance and efficiency of quantization as an optimization method, as well as the possibility of training ML models directly on an IoT device. Quantization involves reducing the precision of model weights and activations while still maintaining acceptable levels of accuracy. Using a facial recognition system developed with TensorFlow and TensorRT as a use case, Post-Training Quantization and Quantization-Aware Training are employed to reduce computational load and improve energy efficiency. The computational experience was conducted on a general-purpose computer featuring an Intel i7-1260P processor and an NVIDIA RTX 3080 graphics card used as an accelerator. Additionally, an NVIDIA Jetson AGX Orin was used as an example of an IoT device. We analyze the feasibility of training on an IoT device, the impact of quantization optimization on knowledge transfer-trained models, and evaluate the differences between Post-Training Quantization and Training-Aware Quantization in such networks on different devices. Furthermore, the performance and efficiency of NVIDIA's inference accelerator (Deep Learning Accelerator - DLA, in its 2.0 version) available at the Jetson Orin architecture are studied. We concluded that the Jetson device is capable of performing training on its own. The IoT device can achieve inference performance similar to that of the more powerful processor, thanks to the optimization process, with better energy efficiency. Post-Training Quantization has shown better performance, while Quantization-Aware Training has demonstrated higher energy efficiency. Nevertheless, since the accelerator cannot execute certain layers of the models, the use of DLA worsens both performance and efficiency results.

show abstract

Edge Deployment Framework of GuardBot for Optimized Face Mask Recognition With Real-Time Inference Using Deep Learning

Cited by 14 publications

References 87 publications

A dual‐channel ensembled deep convolutional neural network for facial expression recognition in the wild

A dual‐channel ensembled deep convolutional neural network for facial expression recognition in the wild

SPT: Single Pedestrian Tracking Framework with Re-Identification-Based Learning Using the Siamese Model

Performance and energy efficiency: quantization of models for IoT devices

Contact Info

Product

Resources

About