Evaluating fusion of RGB-D and inertial sensors for multimodal human action recognition

Imran, Javed; Raman, Balasubramanian

doi:10.1007/s12652-019-01239-9

Cited by 65 publications

(45 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, it will be the preferred strategy in this work. Moreover, contrary to Imran and Raman [9], who reported that the performances of their CNN based method decayed with the augmentation of the amount of zero-padding, our method proved itself to be robust against it.…”

Section: Temporal Normalizationcontrasting

confidence: 92%

“…It can be deduced from Equation (5) that the width of the max pooling operator is 2; hence, the length of the time series is halved after each convolution block (i.e., T l = 1 2 T l−1 ). Rather then connecting the output of the last convolution block to a classifier, as was done in previous 1D-CNN based works [9][10][11][12], the proposed method instead achieves high-level reasoning (inference and learning) by connecting to a classifier the maximum and minimum values of each dimension of each time series generated throughout the network. Equivalently, this concept can be regarded as generating a feature vector, V, by sampling elements of the different time series as such:…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Table 2. Mean accuracy of the k-fold cross-validation analysis produced by Imran et al [9] with respect to zero-padding on their 5 layer 1D-CNN.…”

Section: Robustness Against Zero-padding and Flexibility Towards The mentioning

confidence: 99%

“…Moreover, in the conclusion of their work, Imran and Raman argued that it was not only the complementarity between the different modalities that helped them achieve these results, but also the complementarity between the architecture treating the different data streams, thus further stressing the importance of 1D-CNN architectures. Chen et al [21] Depth + inertial 79.10% Wang et al [26] Depth + skeleton 89.04% El Madany et al [27] Depth + Inertial + Skeleton 93.26% Imran et al [9] RGB + inertial + skeleton 97.91%…”

Section: Performancesmentioning

confidence: 99%

“…Similarly, Weiss et al [ 8 ] observed that MLP outperformed decision trees, k-NN, naive Bayes and logistic regression when the training and evaluation was done on data gathered from the same user and also noted that these user-specific models performed dramatically better than impersonal models where training and evaluation users are different. Since then, models based on convolution neural networks (CNN) have been proposed for inertial sensor based HAR [ 9 , 10 , 11 , 12 ] and have also been proven superior to k-NN and SVM [ 11 ]. With these methods, the input signals usually go through multiple convolution and pooling layers before being fed to a classifier.…”

Section: Related Workmentioning

confidence: 99%

See 4 more Smart Citations

A Hierarchical Learning Approach for Human Action Recognition

Lemieux

Noumeir

2020

Sensors

View full text Add to dashboard Cite

In the domain of human action recognition, existing works mainly focus on using RGB, depth, skeleton and infrared data for analysis. While these methods have the benefit of being non-invasive, they can only be used within limited setups, are prone to issues such as occlusion and often need substantial computational resources. In this work, we address human action recognition through inertial sensor signals, which have a vast quantity of practical applications in fields such as sports analysis and human-machine interfaces. For that purpose, we propose a new learning framework built around a 1D-CNN architecture, which we validated by achieving very competitive results on the publicly available UTD-MHAD dataset. Moreover, the proposed method provides some answers to two of the greatest challenges currently faced by action recognition algorithms, which are (1) the recognition of high-level activities and (2) the reduction of their computational cost in order to make them accessible to embedded devices. Finally, this paper also investigates the tractability of the features throughout the proposed framework, both in time and duration, as we believe it could play an important role in future works in order to make the solution more intelligible, hardware-friendly and accurate.

show abstract

Section: Temporal Normalizationcontrasting

confidence: 92%

Section: Proposed Methodsmentioning

confidence: 99%

“…Table 2. Mean accuracy of the k-fold cross-validation analysis produced by Imran et al [9] with respect to zero-padding on their 5 layer 1D-CNN.…”

Section: Robustness Against Zero-padding and Flexibility Towards The mentioning

confidence: 99%

Section: Performancesmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

A Hierarchical Learning Approach for Human Action Recognition

Lemieux

Noumeir

2020

Sensors

View full text Add to dashboard Cite

show abstract

Intelligent Material Data Preparation Mechanism Based on Ensemble Learning for AI‐Based Ceramic Material Analysis

Imran

Iqbal

Kim

2022

Advcd Theory and Sims

View full text Add to dashboard Cite

Research interest in ceramic materials increased due to their extensive environmental, biomedical, and electronic applications. Increased demand for ceramics with specialized experimental conditions and limited resources has resulted in a higher cost for scientific practices and applications. Enormous material data is accumulated from traditional and high-tech experimentation, but the manual recording process has shown inconsistencies in the analysis results. Recently, application based on artificial intelligence (AI) and machine learning has been able to address the issues of traditional scientific experiments in material science. However, no machine learning mechanisms are proposed for sophisticated data preparation and AI-based discovery application of ceramics. This paper proposed an intelligent material data preparation mechanism based on ensemble learning for AI-assisted material screening and discovery. The current method can potentially resolve the problems of missing and inconsistent material data. As a case study, a material data preparation platform for ceramic material data pre-processing is developed. For performance evaluation of the proposed mechanism, machine learning regression models are trained before and after the imputation techniques applied to the data. Performance analysis shows that the ensemble model of deep learning network (DNN) and automated machine learning (autoML) performed better as compared to previously reported imputation approaches.

show abstract