Masked reconstruction based self-supervision for human activity recognition

Haresamudram, Harish; Beedu, Apoorva; Agrawal, Varun; Grady, Patrick; Essa, Irfan; Hoffman, Judy; Plötz, Thomas

doi:10.1145/3410531.3414306

Cited by 89 publications

(79 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar technique is proposed in [47] for representation learning from raw sensory data to mitigate data limitations and shows that method is effective even for small sized data. Likewise, masked reconstruction is proposed as a viable self-supervised pre-training objective for time series data of HAR [17]. Similarly, an SSL method is proposed for enhancing performance by pre-training the network by predicting the values of sensor signals in future time-steps [53] and their findings show that the SSL technique is effective in boosting the performance.…”

Section: Self-supervised Learningmentioning

confidence: 99%

“…In the field of human activity recognition (HAR), some researchers have successfully attempted to employ SSL to solve the issue of the lack of labeled data by employing simple signal transformations to produce the pretext task for sensor data [17,46]. Their goal is to recognize physical human activities by using accelerometers (ACC) and gyroscopes (GYRO) as sensors to distinguish body movements that characterize activities.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Self-supervised Learning for Reading Activity Classification

Islam

Sakamoto

Yamada

et al. 2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

Reading analysis can relay information about user's confidence and habits and can be used to construct useful feedback. A lack of labeled data inhibits the effective application of fully-supervised Deep Learning (DL) for automatic reading analysis. We propose a Self-supervised Learning (SSL) method for reading analysis. Previously, SSL has been effective in physical human activity recognition (HAR) tasks, but it has not been applied to cognitive HAR tasks like reading. We first evaluate the proposed method on a four-class classification task on reading detection using electrooculography datasets, followed by an evaluation of a two-class classification task of confidence estimation on multiple-choice questions using eye-tracking datasets. Fully-supervised DL and support vector machines (SVMs) are used as comparisons for the proposed SSL method. The results show that the proposed SSL method is superior to the fully-supervised DL and SVM for both tasks, especially when training data is scarce. This result indicates the proposed method is the superior choice for reading analysis tasks. These results are important for informing the design of automatic reading analysis platforms.

show abstract

Section: Self-supervised Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Self-supervised Learning for Reading Activity Classification

Islam

Sakamoto

Yamada

et al. 2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

show abstract

“…Haresamudram et al 6 presented an approach that utilized masked reconstruction based self-supervision to derive recognition systems. The key idea is to make economic use of existing (small) sets of training data and to use unlabeled data, in specifically modified versions, for pretraining feature extractors.…”

Section: Activity Recognitionmentioning

confidence: 99%

Iswc 2020

Plötz

Healey

Laerhoven

et al. 2021

IEEE Pervasive Comput.

View full text Add to dashboard Cite

“…Mobiact dataset [2] contains data for 11 activities of daily living and 4 types of falls recorded from a smartphone's inertial sensors. Following [7,16], we utilize data for the daily living activities -sitting, walking, jogging, jumping, stairs up, stairs down, stand to sit, sitting on a chair, sit to stand, car step-in, and car step-out, from 61 participants.…”

Section: Daphnet Gait Datasetmentioning

confidence: 99%

Personalization Models for Human Activity Recognition with Distribution Matching-Based Metrics

Nguyen

Kwon

Haresamudram

et al. 2021

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

Building activity recognition systems conventionally involves training a common model from all data of training users and utilizing this model to recognize activities of unseen subjects. However, participants come from diverse demographics, so that different users can perform the same actions in diverse ways. Each subject might exhibit user-specific signal patterns, yet a group of users may perform activities in similar manners and share analogous patterns. Leveraging this intuition, we explore Frechet Inception Distance (FID) as a distribution matchingbased metric to measure the similarity between users. From that, we propose the nearest-FID-neighbors and the FID-graph clustering techniques to develop user-specific models that are trained with data from the community the testing user likely belongs to. Verified on a series of benchmark wearable datasets, the proposed techniques significantly outperform the model trained with all users. H. T. Nguyen and H. Kwon-Both authors contributed equally to this research.

show abstract

Masked reconstruction based self-supervision for human activity recognition

Cited by 89 publications

References 18 publications

Self-supervised Learning for Reading Activity Classification

Self-supervised Learning for Reading Activity Classification

Iswc 2020

Personalization Models for Human Activity Recognition with Distribution Matching-Based Metrics

Contact Info

Product

Resources

About