Acceleration-based Activity Recognition of Repetitive Works with Lightweight Ordered-work Segmentation Network

Abstract

This study presents a new neural network model for recognizing manual works using body-worn accelerometers in industrial settings, named Lightweight Ordered-work Segmentation Network (LOS-Net). In industrial domains, a human worker typically repetitively performs a set of predefined processes, with each process consisting of a sequence of activities in a predefined order. State-of-the-art activity recognition models, such as encoder-decoder models, have numerous trainable parameters, making their training difficult in industrial domains because of the consequent substantial cost for preparing a large amount of labeled data. In contrast, the LOS-Net is designed to be trained on a limited amount of training data. Specifically, the decoder in the LOS-Net has few trainable parameters and is designed to capture only the necessary information for precise recognition of ordered works. These are (i) the boundary information between consecutive activities, because a transition in the performed activities is generally associated with the trend change of the sensor data collected during the manual works and (ii) long-term context regarding the ordered works, e.g., information about the previous and next activity, which is useful for recognizing the current activity. This information is obtained by introducing a module that can collect it at distant time steps using few trainable parameters. Moreover, the LOS-Net can refine the activity estimation by the decoder by incorporating prior knowledge regarding the order of activities. We demonstrate the effectiveness of the LOS-Net using sensor data collected from workers in actual factories and a logistics center, and show that it can achieve state-of-the-art performance.

Exploring deep learning techniques for wild animal behaviour classification using animal‐borne accelerometers

“…Domain adaptation techniques such as domain adversarial neural networks (Ganin et al., 2016) can be explored to further reduce F1‐score variations between individuals. The development of a new model architecture for more specific tasks (Xia et al., 2022; Yoshimura, Maekawa, et al., 2022), the use of a specific loss function to deal with class imbalance (e.g. Park et al., 2021) and the use of multimodal sensor data (e.g.…”

“…Yoshimura, Maekawa, et al. (2022) proposed a model for recognising complex, ordered and repetitive activities during line production systems and packaging tasks in the logistics domain. As such, the application of deep learning techniques in HAR is more varied and advanced than that in wild animals.…”

Contrastive Predictive Coding for Human Activity Recognition

“…We perform parameter tuning over (2,4,6) blocks to derive the number of causal convolution layers to utilize (and thus to determine the receptive field of the aggregator). The kernel sizes of layers are (2,3,4,5,6,7), and setting the number of causal convolution blocks to 2 results in two blocks with the causal convolution filter sizes being (2, 3) respectively. This is extended for situations where the number of blocks is 4 or 6.…”