Deep ConvLSTM With Self-Attention for Human Activity Decoding Using Wearable Sensors

Singh, Satya P.; Sharma, Madan Kumar; Lay-Ekuakille, A.; Gangwar, Deepak; Gupta, Sukrit

doi:10.1109/jsen.2020.3045135

Cited by 109 publications

(47 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is followed by [7], an approach based on a Temporal Convolutional Network (TCN), an architecture that is formed by a hierarchy of convolutions to abstract the temporal relations. The two approaches that we found with the best performance in the literature (i.e., DeepConvLSTM [9] and METIER [2]) consider an architecture that combines convolutional models with LSTMs to account for the temporal information. In all cases, our approach outperforms the results of these existing approaches.…”

Section: Discussionmentioning

confidence: 99%

VIDENS: Vision-based User Identification from Inertial Sensors

Guinea

Heinrich

Mühlhäuser

2021

2021 International Symposium on Wearable Computers

View full text Add to dashboard Cite

In this paper we propose the VIDENS (vision-based user identification from inertial sensors) approach, which transforms inertial sensors time-series data into images that represent in pixel form patterns found over time, allowing even a simple CNN to outperform complex ad-hoc deep learning models that combine RNNs and CNNs for user identification. Our evaluation shows promising results when comparing our approach to some relevant existing methods. CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile computing systems and tools; Empirical studies in ubiquitous and mobile computing.

show abstract

Section: Discussionmentioning

confidence: 99%

VIDENS: Vision-based User Identification from Inertial Sensors

Guinea

Heinrich

Mühlhäuser

2021

2021 International Symposium on Wearable Computers

View full text Add to dashboard Cite

show abstract

“…The first part, as per Figure 3 a, is based on the acquisition of images captured by the video-sensors in order to retrieve the following data from the waves: temperature, vertical speed, and their magnitude. The algorithm is essentially based on convolution [ 20 ] along with a pooling of frames [ 21 ], and the final processing is provided by a multilayer perceptron (MLP) [ 22 ]. This first part must classify and extract features connected to temperature, vertical speed and wave detection.…”

Section: Methodsmentioning

confidence: 99%

Image Sensors for Wave Monitoring in Shore Protection: Characterization through a Machine Learning Algorithm

Lay-Ekuakille

Okitadiowo

Luccio

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

Waves propagating on the water surface can be considered as propagating in a dispersive medium, where gravity and surface tension at the air–water interface act as restoring forces. The velocity at which energy is transported in water waves is defined by the group velocity. The paper reports the use of video-camera observations to study the impact of water waves on an urban shore. The video-monitoring system consists of two separate cameras equipped with progressive RGB CMOS sensors that allow 1080p HDTV video recording. The sensing system delivers video signals that are processed by a machine learning technique. The scope of the research is to identify features of water waves that cannot be normally observed. First, a conventional modelling was performed using data delivered by image sensors together with additional data such as temperature, and wind speed, measured with dedicated sensors. Stealth waves are detected, as are the inverting phenomena encompassed in waves. This latter phenomenon can be detected only through machine learning. This double approach allows us to prevent extreme events that can take place in offshore and onshore areas.

show abstract

“…The self-attention (Vaswani et al, 2017;Singh et al, 2021) mechanism allows the inputs to interact with each other and find out who they should pay more attention to or which features are more important. The attention layer aims to learn the important time points from the sensor time-series data that aid in determining the state label.…”

Section: Self-attentionmentioning

confidence: 99%

Sabotage Detection Using DL Models on EEG Data From a Cognitive-Motor Integration Task

Chaudhary

Adams

Mukhopadhyay

et al. 2021

Front. Hum. Neurosci.

View full text Add to dashboard Cite

Objective clinical tools, including cognitive-motor integration (CMI) tasks, have the potential to improve concussion rehabilitation by helping to determine whether or not a concussion has occurred. In order to be useful, however, an individual must put forth their best effort. In this study, we have proposed a novel method to detect the difference in cortical activity between best effort (no-sabotage) and willful under-performance (sabotage) using a deep learning (DL) approach on the electroencephalogram (EEG) signals. The EEG signals from a wearable four-channel headband were acquired during a CMI task. Each participant completed sabotage and no-sabotage conditions in random order. A multi-channel convolutional neural network with long short term memory (CNN-LSTM) model with self-attention has been used to perform the time-series classification into sabotage and no-sabotage, by transforming the time-series into two-dimensional (2D) image-based scalogram representations. This approach allows the inspection of frequency-based, and temporal features of EEG, and the use of a multi-channel model facilitates in capturing correlation and causality between different EEG channels. By treating the 2D scalogram as an image, we show that the trained CNN-LSTM classifier based on automated visual analysis can achieve high levels of discrimination and an overall accuracy of 98.71% in case of intra-subject classification, as well as low false-positive rates. The average intra-subject accuracy obtained was 92.8%, and the average inter-subject accuracy was 86.15%. These results indicate that our proposed model performed well on the data of all subjects. We also compare the scalogram-based results with the results that we obtained by using raw time-series, showing that scalogram-based gave better performance. Our method can be applied in clinical applications such as baseline testing, assessing the current state of injury and recovery tracking and industrial applications like monitoring performance deterioration in workplaces.

show abstract

Deep ConvLSTM With Self-Attention for Human Activity Decoding Using Wearable Sensors

Cited by 109 publications

References 36 publications

VIDENS: Vision-based User Identification from Inertial Sensors

VIDENS: Vision-based User Identification from Inertial Sensors

Image Sensors for Wave Monitoring in Shore Protection: Characterization through a Machine Learning Algorithm

Sabotage Detection Using DL Models on EEG Data From a Cognitive-Motor Integration Task

Contact Info

Product

Resources

About