Sensor-based and vision-based human activity recognition: A comprehensive survey

Dang, L. Minh; Min, Kyungbok; Wang, Hanxiang; Piran, Md. Jalil; Lee, Cheol Hee; Moon, Hyeonjoon

doi:10.1016/j.patcog.2020.107561

Cited by 353 publications

(169 citation statements)

References 185 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, drinking activity can be detected by installing an accelerometer on a cup. A GPS or RFID module is attached to a worker's helmet to monitor the user's location to infer the worker's activity status [36][37][38].…”

Section: Literature Reviewmentioning

confidence: 99%

“…RGB-D data is produced by RGB-D cameras which can not only capture the original RGB data but also collect depth information. Dang et al [37] pointed out that RGB data has the advantage that it is extensively available as well as affordable and contains rich content on the subjects. While compared to RGB data, RGB-D data provides depth information, which can enhance the performance of HAR.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Scaffolding Workface Assessment for Activity Analysis through Machine Learning

et al. 2021

View full text Add to dashboard Cite

Scaffolding serves as one construction trade with high importance. However, scaffolding suffers from low productivity and high cost in Australia. Activity Analysis is a continuous procedure of assessing and improving the amount of time that craft workers spend on one single construction trade, which is a functional method for monitoring onsite operation and analyzing conditions causing delays or productivity decline. Workface assessment is an initial step for activity analysis to manually record the time that workers spend on each activity category. This paper proposes a method of automatic scaffolding workface assessment using a 2D video camera to capture scaffolding activities and the model of key joints and skeleton extraction, as well as machine learning classifiers, were used for activity classification. Additionally, a case study was conducted and showed that the proposed method is a feasible and practical way for automatic scaffolding workface assessment.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Automatic Scaffolding Workface Assessment for Activity Analysis through Machine Learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Recent research, involving both dynamic and static HAR, uses sensor data collected from wearable devices to better understand the relationship between health and behavioral biometric information [ 10 , 11 ]. The HAR methods can be categorized into two categories according to data sources: visual-based and sensor-based [ 12 ]. With visual-based HAR, video or image data are recorded and processed using computer vision techniques [ 13 ].…”

Section: Introductionmentioning

confidence: 99%

LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes

Mekruksavanich

Jitpattanakul

2021

Sensors

198

View full text Add to dashboard Cite

Human Activity Recognition (HAR) employing inertial motion data has gained considerable momentum in recent years, both in research and industrial applications. From the abstract perspective, this has been driven by an acceleration in the building of intelligent and smart environments and systems that cover all aspects of human life including healthcare, sports, manufacturing, commerce, etc. Such environments and systems necessitate and subsume activity recognition, aimed at recognizing the actions, characteristics, and goals of one or more individuals from a temporal series of observations streamed from one or more sensors. Due to the reliance of conventional Machine Learning (ML) techniques on handcrafted features in the extraction process, current research suggests that deep-learning approaches are more applicable to automated feature extraction from raw sensor data. In this work, the generic HAR framework for smartphone sensor data is proposed, based on Long Short-Term Memory (LSTM) networks for time-series domains. Four baseline LSTM networks are comparatively studied to analyze the impact of using different kinds of smartphone sensor data. In addition, a hybrid LSTM network called 4-layer CNN-LSTM is proposed to improve recognition performance. The HAR method is evaluated on a public smartphone-based dataset of UCI-HAR through various combinations of sample generation processes (OW and NOW) and validation protocols (10-fold and LOSO cross validation). Moreover, Bayesian optimization techniques are used in this study since they are advantageous for tuning the hyperparameters of each LSTM network. The experimental results indicate that the proposed 4-layer CNN-LSTM network performs well in activity recognition, enhancing the average accuracy by up to 2.24% compared to prior state-of-the-art approaches.

show abstract

“…Recent studies showed that deep learning models outperform conventional machine learning methods as Support Vector Machines (SVMs) in sensor-based human activity recognition (HAR) in sports [8][9][10]. In contrast to conventional methods which require hand-crafted, domain-specific features, deep learning models automatically extract abstract features from sensor signals [11]. However, the training of neural networks generally requires large labelled datasets to achieve satisfactory performance explaining its rare use for sensor-based performance analysis in field-sports [6,12].…”

Section: Introductionmentioning

confidence: 99%

From the Laboratory to the Field: IMU-Based Shot and Pass Detection in Football Training and Game Scenarios Using Deep Learning

Stoeve

Schuldhaus

Gamp

et al. 2021

Sensors

View full text Add to dashboard Cite

The applicability of sensor-based human activity recognition in sports has been repeatedly shown for laboratory settings. However, the transferability to real-world scenarios cannot be granted due to limitations on data and evaluation methods. On the example of football shot and pass detection against a null class we explore the influence of those factors for real-world event classification in field sports. For this purpose we compare the performance of an established Support Vector Machine (SVM) for laboratory settings from literature to the performance in three evaluation scenarios gradually evolving from laboratory settings to real-world scenarios. In addition, three different types of neural networks, namely a convolutional neural net (CNN), a long short term memory net (LSTM) and a convolutional LSTM (convLSTM) are compared. Results indicate that the SVM is not able to reliably solve the investigated three-class problem. In contrast, all deep learning models reach high classification scores showing the general feasibility of event detection in real-world sports scenarios using deep learning. The maximum performance with a weighted f1-score of 0.93 was reported by the CNN. The study provides valuable insights for sports assessment under practically relevant conditions. In particular, it shows that (1) the discriminative power of established features needs to be reevaluated when real-world conditions are assessed, (2) the selection of an appropriate dataset and evaluation method are both required to evaluate real-world applicability and (3) deep learning-based methods yield promising results for real-world HAR in sports despite high variations in the execution of activities.

show abstract

Sensor-based and vision-based human activity recognition: A comprehensive survey

Cited by 353 publications

References 185 publications

Automatic Scaffolding Workface Assessment for Activity Analysis through Machine Learning

Automatic Scaffolding Workface Assessment for Activity Analysis through Machine Learning

LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes

From the Laboratory to the Field: IMU-Based Shot and Pass Detection in Football Training and Game Scenarios Using Deep Learning

Contact Info

Product

Resources

About