Multistep Deep System for Multimodal Emotion Detection With Invalid Data in the Internet of Things

Li, Minjia; Xie, Liang-Liang; Lv, Zhen; Li, Juan; Wang, Zhiliang

doi:10.1109/access.2020.3029288

Cited by 12 publications

(9 citation statements)

References 50 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RecNet [58] MARN [59] GRU-MP/EVE [60] SibNet [61] SymSynAN [61] HRL [66] PickNet [69] E2E [70] RecNet(RL) [72] DPRN [73] Wei [74] B_4 RecNet [58] MARN [59] GRU-MP/EVE [60] SibNet [61] SymSynAN [61] HRL [66] PickNet [69] E2E [70] RecNet(RL) [72] DPRN [73] Wei [74] M 0 10 20 30 40 50 60 70…”

Section: Multimodal Speech Synthesis Resultsmentioning

confidence: 99%

“…RecNet [58] MARN [59] GRU-MP/EVE [60] SibNet [61] SymSynAN [61] HRL [66] PickNet [69] E2E [70] RecNet(RL) [72] DPRN [73] Wei [74] R 0 10 20 30 40 50 60…”

Section: Multimodal Speech Synthesis Resultsmentioning

confidence: 99%

“…RecNet [58] MARN [59] GRU-MP/EVE [60] SibNet [61] SymSynAN [61] PickNet [69] E2E [70] RecNet(RL) [72] DPRN [73] Wei [74] C 4.40 ± 0.04 models. Parallel TACOTRON [28] is also the closest model in terms of experimental results, but its relevancy is much more than TACOTRON2.…”

Section: Multimodal Speech Synthesis Resultsmentioning

confidence: 99%

“…During the chatting user can know each other's emotions in this network. M. Li et al [59] proposed a multi-step deep system for reliable detection of emotions by using collected data that contains invalid data as well. Neural networks are used to filter this invalid data from videos and physiological signals by using continuity and semantic compatibility.…”

Section: Multimodal Emotion Recognition (Mmer)mentioning

confidence: 99%

See 3 more Smart Citations

A Review on Methods and Applications in Multimodal Deep Learning

Summaira¹,

Li²,

Shoib³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep Learning has implemented a wide range of applications and has become increasingly popular in recent years. The goal of multimodal deep learning (MMDL) is to create models that can process and link information using various modalities. Despite the extensive development made for unimodal learning, it still cannot cover all the aspects of human learning. Multimodal learning helps to understand and analyze better when various senses are engaged in the processing of information. This paper focuses on multiple types of modalities, i.e., image, video, text, audio, body gestures, facial expressions, and physiological signals. Detailed analysis of the baseline approaches and an in-depth study of recent advancements during the last five years (2017 to 2021) in multimodal deep learning applications has been provided. A fine-grained taxonomy of various multimodal deep learning methods is proposed, elaborating on different applications in more depth. Lastly, main issues are highlighted separately for each domain, along with their possible future research directions.CCS Concepts: • Computing methodologies → Machine learning; • Information systems → Multimedia and multimodal retrieval.

show abstract

Section: Multimodal Speech Synthesis Resultsmentioning

confidence: 99%

“…RecNet [58] MARN [59] GRU-MP/EVE [60] SibNet [61] SymSynAN [61] HRL [66] PickNet [69] E2E [70] RecNet(RL) [72] DPRN [73] Wei [74] R 0 10 20 30 40 50 60…”

Section: Multimodal Speech Synthesis Resultsmentioning

confidence: 99%

Section: Multimodal Speech Synthesis Resultsmentioning

confidence: 99%

Section: Multimodal Emotion Recognition (Mmer)mentioning

confidence: 99%

See 2 more Smart Citations

A Review on Methods and Applications in Multimodal Deep Learning

Summaira¹,

Li²,

Shoib³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Several studies generated machine learning models from 1039 noisy IoT data, and the models could achieve high accuracy 1040 [73], [74]. In contrast, studies on differential privacy showed 1041 that if the value of the privacy budget is small, i.e., if the 1042 noise added using differential privacy techniques is large, 1043 the accuracy of machine learning models will deteriorate 1044 significantly [53]- [55].…”

Section: B Machine Learning With Noisy Iot Data 1038mentioning

confidence: 99%

Private True Data Mining: Differential Privacy Featuring Errors to Manage Internet-of-Things Data

Sei

Ohsuga

2022

IEEE Access

View full text Add to dashboard Cite

Available data may differ from true data in many cases due to sensing errors, especially for the Internet of Things (IoT). Although privacy-preserving data mining has been widely studied during the last decade, little attention has been paid to data values containing errors. Differential privacy, which is the de facto standard privacy metric, can be achieved by adding noise to a target value that must be protected. However, if the target value already contains errors, there is no reason to add extra noise. In this paper, a novel privacy model called true-value-based differential privacy (TDP) is proposed. This model applies traditional differential privacy to the "true value" unknown by the data owner or anonymizer but not to the "measured value" containing errors. Based on TDP, the amount of noise added by differential privacy techniques can be reduced by approximately 20% by our solution. As a result, the error of generated histograms can be reduced by 40.4% and 29.6% on average according to mean square error and Jensen-Shannon divergence, respectively.We validate this result on synthetic and five real data sets. Moreover, we proved that the privacy protection level does not decrease as long as the measurement error is not overestimated. INDEX TERMSData mining, Data privacy, Differentnial privacy, Internet of Things I. INTRODUCTION 1 Significant amounts of IoT data are generated every day by 2 many different sensors, such as thermal cameras, home appli-3 ance sensors, automotive sensors, and smartphone-equipped 4 sensors. These IoT data can be used for health monitoring [1], 5 context-aware recommendation (or recommender) systems 6 [2], navigation [3], and other applications.7 However, sensing people or their surrounding environment 8 might involve information that identifies an individual [4]. 9 Thus private information is at risk of leakage. By anonymiz-10 ing data based on ϵ-differential privacy [5], [6], which is the 11 de facto standard privacy metric (ϵ represents the privacy 12 budget), privacy leakage can be controlled. Differential pri-13 vacy has been used in many studies, such as [7]-[9], as it is 14 one of the most critical privacy metrics [10]. It is considered 15 an important concept for data analysis [11], [12]. 16 Local differential privacy is a specialized concept of dif-17 ferential privacy especially for data collection from each 18 person. In this paper, "differential privacy," refers to "lo-19

show abstract

A review on multimodal zero‐shot learning

Cao

Sun

et al. 2023

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Multimodal learning provides a path to fully utilize all types of information related to the modeling target to provide the model with a global vision. Zero-shot learning (ZSL) is a general solution for incorporating prior knowledge into data-driven models and achieving accurate class identification. The combination of the two, known as multimodal ZSL (MZSL), can fully exploit the advantages of both technologies and is expected to produce models with greater generalization ability. However, the MZSL algorithms and applicationshave not yet been thoroughly investigated and summarized. This study fills this gap by providing an objective overview of MZSL's definition, typical algorithms, representative applications, and critical issues. This article will not only provide researchers in this field with a comprehensive perspective, but it will also highlight several promising research directions.

show abstract

Multistep Deep System for Multimodal Emotion Detection With Invalid Data in the Internet of Things

Cited by 12 publications

References 50 publications

A Review on Methods and Applications in Multimodal Deep Learning

A Review on Methods and Applications in Multimodal Deep Learning

Private True Data Mining: Differential Privacy Featuring Errors to Manage Internet-of-Things Data

A review on multimodal zero‐shot learning

Contact Info

Product

Resources

About