Emotion Recognition on Edge Devices: Training and Deployment

Pandelea, Vlad; Ragusa, Edoardo; Apicella, Tommaso; Gastaldo, Paolo; Wang, Zhaoxia

doi:10.3390/s21134496

Cited by 11 publications

(6 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is not within the scope of this paper to perform a complete analysis of edge device performance. However, using suggestions on how to adapt the text model architecture or using even more lightweight versions of BERT in combination with previous work that uses similar text models on edge devices [97], we claim that our approach is also feasible to adapt and use on edge devices.…”

Section: Discussion and Recommendationsmentioning

confidence: 87%

Multimodal Fine-Grained Grocery Product Recognition using Image and OCR Text

Pettersson,

Riveiro,

Löfström

2023

Preprint

View full text Add to dashboard Cite

Automatic recognition of grocery products can be used to improve customer flow at checkouts and reduce labor costs and store losses. Product recognition is, however, a challenging task for machine learning-based solutions due to the large number of products and their variations in appearance. In this work, we tackle the challenge of fine-grained product recognition by first extracting a large dataset from a grocery store containing products that are only differentiable by subtle details. Then, we propose a multimodal product recognition approach that uses product images with extracted OCR text from packages to improve fine-grained recognition of grocery products. We evaluate several image and text models separately and then combine them using different multimodal models of varying complexities. The results show that image and textual information complement each other in multimodal models and enable a classifier with greater recognition performance than unimodal models, especially when the number of training samples is limited. Therefore, this approach is suitable for many different scenarios in which product recognition is used to further improve recognition performance.

show abstract

Section: Discussion and Recommendationsmentioning

confidence: 87%

Multimodal Fine-Grained Grocery Product Recognition using Image and OCR Text

Pettersson,

Riveiro,

Löfström

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…ML algorithms, when employed in the context of IoT devices and wearable systems, should be adapted to consider the resource constraints in such systems [66], [67], [68], [69], [70]. In particular, in cross-device FL, where we have a large number of mobile or IoT devices, communication is often the main bottleneck [50].…”

Section: State Of the Artmentioning

confidence: 99%

Extremely Randomized Trees With Privacy Preservation for Distributed Structured Health Data

et al. 2022

View full text Add to dashboard Cite

Machine learning has recently attracted a lot of attention in the healthcare domain. The data used by machine learning algorithms in healthcare applications is often distributed over multiple sources, e.g., hospitals. One main difficulty lies in analyzing such data without compromising personal information, which is a primary concern in healthcare applications. Therefore, in these applications, we are interested in running machine learning algorithms over distributed data without disclosing sensitive information about data subjects. In this paper, we propose a distributed extremely randomized tree algorithm for learning with privacy preservation. We present the implementation of our technique on a cloud platform and demonstrate its performance based on medical data, including the mental health datasets associated with the Norwegian INTROducing Mental health through Adaptive Technology (INTROMAT) project.

show abstract

“…Therefore, many approaches have arisen to address this computational gap [22]. Pandelea et al [3] combined a large transformer as a feature extractor with a simple classifier, deployed it on Jetson Nano and two smartphones, and optimized latency and performance using dimensionality reduction and pre-training. Xu et al [2] proposed an edge-based caching framework for voice assistant systems called CHA on three edge devices, Raspberry Pi, Intel Fog Reference Design, and Jetson AGX Xavier.…”

Section: Edge Intelligencementioning

confidence: 99%

“…They store personal data at the edge to get void of privacy leakage, as opposed to the cloud-based commercial services, i.e., Google Assistant and Amazon Alexia. Dialogue systems such as task-oriented dialogue systems and intelligent personal assistants have been deployed near the user at different edge platforms, i.e., Raspberry Pi [2], Jetson Nano [3], and smartphones. The major challenge is guaranteeing real-time user experience on hardware-constrained devices with limited computation, memory storage, and energy resources.…”

Section: Introductionmentioning

confidence: 99%

“…From our perspective, although large pre-trained language models such as BERT bring significant performance gains, these models also have increasing demands on computational resources and memory [21]. Due to high latency or memory overflow, these models are challenging to deploy directly in certain computing resource-constrained conditions, such as on edge devices [3]. Edge computing is a real need due to the response latency and privacy issues associated with computing from the cloud [22].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Fast Attention Network for Joint Intent Detection and Slot Filling on Edge Devices

Huang¹,

Liang²,

Ye³

et al. 2022

Preprint

View full text Add to dashboard Cite

Intent detection and slot filling are two main tasks in natural language understanding and play an essential role in task-oriented dialogue systems. The joint learning of both tasks can improve inference accuracy and is popular in recent works. However, most joint models ignore the inference latency and cannot meet the need to deploy dialogue systems at the edge. In this paper, we propose a Fast Attention Network (FAN) for joint intent detection and slot filling tasks, guaranteeing both accuracy and latency. Specifically, we introduce a clean and parameter-refined attention module to enhance the information exchange between intent and slot, improving semantic accuracy by more than 2%. FAN can be implemented on different encoders and delivers more accurate models at every speed level. Our experiments on the Jetson Nano platform show that FAN inferences fifteen utterances per second with a small accuracy drop, showing its effectiveness and efficiency on edge devices.Impact Statement-Dialogue systems at the edge are an emerging technology in real-time interactive applications. They improve the user experience with low latency and secure privacy without transferring personal data to the cloud servers. However, it is challenging to guarantee inference accuracy and low latency on hardware-constrained devices with limited computation, memory storage, and energy resources. The neural network models we introduce in this paper overcome these limitations. With a significant increase in semantic accuracy by more than 2% after adopting our algorithms, the technology reduces the inference latency to less than 100ms. From this viewpoint, our approaches accelerate the boosting of secure personal assistants to end-users.

show abstract

Emotion Recognition on Edge Devices: Training and Deployment

Cited by 11 publications

References 48 publications

Multimodal Fine-Grained Grocery Product Recognition using Image and OCR Text

Multimodal Fine-Grained Grocery Product Recognition using Image and OCR Text

Extremely Randomized Trees With Privacy Preservation for Distributed Structured Health Data

A Fast Attention Network for Joint Intent Detection and Slot Filling on Edge Devices

Contact Info

Product

Resources

About