Human Activity Recognition by Sequences of Skeleton Features

Ramirez, Heilym; Velastín, Sergio A.; Aguayo, Paulo; Fábregas, Ernesto; Farías, Gonzalo

doi:10.3390/s22113991

Cited by 16 publications

(44 citation statements)

References 51 publications

(68 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the report of Gomez et al [ 25 ], they analyzed the evoked facial gestures in patients with 'Parkinson's Disease from the video of patients and indicated that the detection rate significantly improved (from 75.00% to 88.46%) by using the 17 facial features derived from the landmark detection algorithm. In addition, similar attempts have also been applied in the assessment of cerebral palsy [ 26 ], pose evaluation in sports [ 27 , 28 ], and human activity recognition [ 29 ]. The combination of pose estimation methods and machine learning classifiers presented superior performances in these works.…”

Section: Discussionmentioning

confidence: 99%

Automatic Detection of Horner Syndrome by Using Facial Images

Fan¹,

Qin

Gu³

et al. 2022

Journal of Healthcare Engineering

View full text Add to dashboard Cite

Horner syndrome is a clinical constellation that presents with miosis, ptosis, and facial anhidrosis. It is important as a warning sign of the damaged oculosympathetic chain, potentially with serious causes. However, the diagnosis of Horner syndrome is operator dependent and subjective. This study aims to present an objective method that can recognize Horner sign from facial photos and verify its accuracy. A total of 173 images were collected, annotated, and divided into training and testing groups. Two types of classifiers were trained (two-stage classifier and one-stage classifier). The two-stage method utilized the MediaPipe face mesh to estimate the coordinates of landmarks and generate facial geometric features accordingly. Then, ten machine learning classifiers were trained based on this. The one-stage classifier was trained based on one of the latest algorithms, YOLO v5. The performance of the classifier was evaluated by the diagnosis accuracy, sensitivity, and specificity. For the two-stage model, the MediaPipe successfully detected 92.2% of images in the testing group, and the Decision Tree Classifier presented the highest accuracy (0.790). The sensitivity and specificity of this classifier were 0.432 and 0.970, respectively. As for the one-stage classifier, the accuracy, sensitivity, and specificity were 0.65, 0.51, and 0.84, respectively. The results of this study proved the possibility of automatic detection of Horner syndrome from images. This tool could work as a second advisor for neurologists by reducing subjectivity and increasing accuracy in diagnosing Horner syndrome.

show abstract

Section: Discussionmentioning

confidence: 99%

Automatic Detection of Horner Syndrome by Using Facial Images

Fan¹,

Qin

Gu³

et al. 2022

Journal of Healthcare Engineering

View full text Add to dashboard Cite

show abstract

“…The current work is focused on detecting falls based on the overall movement of images in the scene. Pose estimation can be a potential future direction for this research [39]; however it is out of scope for the focus of this work. In future work, methods developed on this dataset can be applied on other public domain videos for further validation.…”

Section: Discussionmentioning

confidence: 99%

Multi Visual Modality Fall Detection Dataset

et al. 2022

View full text Add to dashboard Cite

Falls are one of the leading causes of injury-related deaths among the elderly worldwide. Effective detection of falls can reduce the risk of complications and injuries. Fall detection can be performed using wearable devices or ambient sensors; these methods may struggle with user compliance issues or false alarms. Video cameras provide a passive alternative; however, regular red, green, and blue (RGB) cameras are impacted by changing lighting conditions and privacy concerns. From a machine learning perspective, developing an effective fall detection system is challenging because of the rarity and variability of falls. Many existing fall detection datasets lack important real-world considerations, such as varied lighting, continuous activities of daily living (ADLs), and camera placement. The lack of these considerations makes it difficult to develop predictive models that can operate effectively in the real world. To address these limitations, we introduce a novel multi-modality dataset (MUVIM) that contains four visual modalities: infra-red, depth, RGB and thermal cameras. These modalities offer benefits such as obfuscated facial features and improved performance in low-light conditions. We formulated fall detection as an anomaly detection problem, in which a customized spatio-temporal convolutional autoencoder was trained only on ADLs so that a fall would increase the reconstruction error. Our results showed that infra-red cameras provided the highest level of performance (AUC ROC=0.94), followed by thermal (AUC ROC=0.87), depth (AUC ROC=0.86) and RGB (AUC ROC=0.83). This research provides a unique opportunity to analyze the utility of camera modalities in detecting falls in a home setting while balancing performance, passiveness, and privacy.

show abstract

“…For example, H. Abdo et al [ 15 ] used RetinaNet to detect people in videos and obtain motion features and human shape features (including the aspect ratio of the human bounding box and motion history images), and then input the improved mobile nets to determine whether they fell or not; Y. Chen et al [ 16 ] used an OpenPose–SVM combination algorithm to detect people falling, which can accurately determine whether people tend to fall. The experiments proved that they achieved recognition rates of 92.5% and 95.8% on the two public data sets of MCFD and URFD, respectively; Ramirez et al [ 17 ] used the AlphaPose–kNN combination algorithm to detect people falling. First, AlphaPose was used to obtain the skeleton information of the human body; the information was then input into the kNN network for classification.…”

Section: Related Workmentioning

confidence: 99%

Fall Detection for Shipboard Seafarers Based on Optimized BlazePose and LSTM

Liu

et al. 2022

Sensors

View full text Add to dashboard Cite

Aiming to avoid personal injury caused by the failure of timely medical assistance following a fall by seafarer members working on ships, research on the detection of seafarer’s falls and timely warnings to safety officers can reduce the loss and severe consequences of falls to seafarers. To improve the detection accuracy and real-time performance of the seafarer fall detection algorithm, a seafarer fall detection algorithm based on BlazePose–LSTM is proposed. This algorithm can automatically extract the human body key point information from the video image obtained by the vision sensor, analyze its internal data correlation characteristics, and realize the process from RGB camera image processing to seafarer fall detection. This fall detection algorithm extracts the human body key point information through the optimized BlazePose human body key point information extraction network. In this section, a new method for human bounding-box acquisition is proposed. In this study, a head detector based on the Vitruvian theory was used to replace the pre-trained SSD body detector in the BlazePose preheating module. Simultaneously, an offset vector is proposed to update the bounding box obtained. This method can reduce the frequency of repeated use of the head detection module. The algorithm then uses the long short-term memory neural network to detect seafarer falls. After extracting fall and related behavior data from the URFall public data set and FDD public data set to enrich the self-made data set, the experimental results show that the algorithm can achieve 100% accuracy and 98.5% specificity for the seafarer’s falling behavior, indicating that the algorithm has reasonable practicability and strong generalization ability. The detection frame rate can reach 29 fps on a CPU, which can meet the effect of real-time detection. The proposed method can be deployed on common vision sensors.

show abstract

Human Activity Recognition by Sequences of Skeleton Features

Cited by 16 publications

References 51 publications

Automatic Detection of Horner Syndrome by Using Facial Images

Automatic Detection of Horner Syndrome by Using Facial Images

Multi Visual Modality Fall Detection Dataset

Fall Detection for Shipboard Seafarers Based on Optimized BlazePose and LSTM

Contact Info

Product

Resources

About