Unsafe Mining Behavior Identification Method Based on an Improved ST-GCN

Abstract

Aiming to solve the problems of large environmental interference and complex types of personnel behavior that are difficult to identify in the current identification of unsafe behavior in mining areas, an improved spatial temporal graph convolutional network (ST-GCN) for miners’ unsafe behavior identification network in a transportation roadway (NP-AGCN) was proposed. First, the skeleton spatial-temporal map constructed using multi-frame human key points was used for behavior recognition to reduce the interference caused by the complex environment of the coal mine. Second, aiming to solve the problem that the original graph structure cannot learn the association relationship between the non-naturally connected nodes, which leads to the low recognition rate of climbing belts, fighting and other behaviors, the graph structure was reconstructed and the original partitioning strategy was changed to improve the recognition ability of the model for multi-joint interaction behaviors. Finally, in order to alleviate the problem that the graph convolution network has difficulty learning global information due to the small receptive field, multiple self-attention mechanisms were introduced into the graph convolution to improve the recognition ability of the model for unsafe behaviors. In order to verify the detection ability of the model regarding identifying unsafe behaviors of personnel in a coal mine belt area, our model was tested on the public datasets NTU-RGB + D and the self-built datasets of unsafe behaviors in a coal mine belt area. The recognition accuracies of the proposed model in the above datasets were 94.7% and 94.1%, respectively, which were 6.4% and 7.4% higher than the original model, which verified that the proposed model had excellent recognition accuracies.

Study on the Interaction Behaviors Identification of Construction Workers Based on ST-GCN and YOLO

“…Many previous studies utilized ST-GCN for non-similar motion recognition and found it performed well. Cao et al [ 21 ] identified miners’ unsafe behavior (10 different types of behaviors) based on ST-GCN in their self-built dataset, with an overall identification accuracy of 86.7%. Lee et al [ 65 ] used ST-GCN to identify 5 different unsafe behaviors of workers, with an overall identification accuracy of 87.20%.…”

“…It takes advantage of the fact that skeletons are represented by graphs rather than 2D or 3D grids, and it has achieved great success in the field of action identification. Cao et al [ 21 ] proposed an improved ST-GCN method for recognizing unsafe mining behaviors, and achieved good performances on both public datasets and their own constructed datasets. In addition, some researchers have also made improvements based on the ST-GCN model [ 22 , 23 ].…”

Recognition of Miner Action and Violation Behavior Based on the ANODE-GCN Model

“…They demonstrated that by compensating for missing key points through preprocessing, there is a significant enhancement in recognition performance. Cao et al [20] proposed an improved network for recognizing unsafe behaviors of miners in transportation roadways based on ST-GCN, named NP-AGCN. This network is designed to overcome environmental solid interference and the high complexity of personnel behavior patterns in mining areas.…”

Hybrid Convolutional Neural Network Approaches for Recognizing Collaborative Actions in Human–Robot Assembly Tasks

“…Liu et al [21] improved the model's ability to extract temporal features by incorporating residual networks into the time information processing of the node. Cao et al [22] proposed a new partition self-attention spatial-temporal graph convolutional network (NP-AGCN).…”