Context-Aware Cyber-Physical Assistance Systems in Industrial Systems: A Human Activity Recognition Approach

Abstract

The increasing demand for product customisation is leading to higher complexities within industrial systems. This imposes new challenges for the workforce. One way to support operators' productivity may be context-aware, human-centred cyber-physical assistance systems. Human Activity Recognition (HAR) is a promising approach to enable context-awareness. However, standardised approaches to integrate HAR into existing industrial environments are rare. Particularly, there is a lack of available datasets of industrial activities. Moreover, comparative studies of inertial and visual HAR approaches are still rare. This work therefore proposes Methods-Time Measurement (MTM) as a standardised foundation for creating an industrial activity dataset. Subsequently, five different machine learning algorithms are tested for their recognition performance based on the dataset captured with an inertial sensor suit and an RGB-D sensor. A proof-of-concept is delivered for both sensor categories applied to the scope of 18 MTM-1 activities, whereas inertial data outperformed depth data. K-Nearest Neighbour and Bagged Tree algorithms revealed the best classification accuracy results in this context.

Expert–Novice Level Classification Using Graph Convolutional Network Introducing Confidence-Aware Node-Level Attention Mechanism

“…Specifically, the expert-novice soccer dataset contains motion data of eight participants for nine types of soccer plays (penalty kick (PK), free kick (FK), direct shot (DS), cross shot (CS), volley, long dribble, straight dribble, short dribble, and juggling), four times each, for 288 samples. These motion data were obtained using the PERCEPTION NEURON PRO (https://neuronmocap.com), which was used to capture whole-body motion [49,50]. The nine types of soccer plays in this dataset are illustrated in Figure 5.…”

Classification of Expert-Novice Level Using Eye Tracking And Motion Data via Conditional Multimodal Variational Autoencoder

“…Since the number of histogram bins of eye tracking data is set as 100, the dimension of vector x is 2,500 (25 features × 100 bins). Also, motion data were acquired by PERCEPTION NEU-RON PRO 4 , which is utilized to record whole-body motions [26,27]. This device obtains motion data from 21 small sensors attached to several body parts (e.g., leg, hip, arm, shoulder, and head).…”

Feature representation learning for human activity recognition

“…Thus, modern factories have an opportunity to monitor and analyze human performance, behavior, and activities [247]. Examples of studies that focus on monitoring humans within the manufacturing processes typically include recognizing assembly tasks based on body and hand tracking [174,120], and motion sensors data using neural networks [203], and assistance systems for workers [177]. Typically, both cameras and wearable devices can be used to monitor people in smart factory settings.…”