Learning spatiotemporal representations for human fall detection in surveillance video

Kong, Yongqiang; Huang, Jianhui; Huang, Shanshan; Wei, Zhengang; Wang, Shengke

doi:10.1016/j.jvcir.2019.01.024

Cited by 59 publications

(31 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Though such a limitation is imposed, literature shows that traditional machine learning, based on support vector machines, hidden Markov models, and decision trees are still very active in the field of fall detection that uses individual wearable non-visual or ambient sensors (e.g., accelerometer) (Wang et al, 2017a , b ; Chen et al, 2018 ; Saleh and Jeannès, 2019 ; Wu et al, 2019 ). For visual sensors the trend has been moving toward deep learning for convolutional neural networks (CNN) (Adhikari et al, 2017 ; Kong et al, 2019 ; Han et al, 2020 ), or LSTM (Shojaei-Hashemi et al, 2018 ). Deep learning is a sophisticated learning framework that besides the mapping function (as mentioned above and used in traditional machine learning), it also learns the features (in a hierarchy fashion) that characterize the concerned classes (e.g., falls and no falls).…”

Section: Fall Detection Using Individual Sensorsmentioning

confidence: 99%

Elderly Fall Detection Systems: A Literature Survey

2020

View full text Add to dashboard Cite

Falling is among the most damaging event elderly people may experience. With the ever-growing aging population, there is an urgent need for the development of fall detection systems. Thanks to the rapid development of sensor networks and the Internet of Things (IoT), human-computer interaction using sensor fusion has been regarded as an effective method to address the problem of fall detection. In this paper, we provide a literature survey of work conducted on elderly fall detection using sensor networks and IoT. Although there are various existing studies which focus on the fall detection with individual sensors, such as wearable ones and depth cameras, the performance of these systems are still not satisfying as they suffer mostly from high false alarms. Literature shows that fusing the signals of different sensors could result in higher accuracy and lower false alarms, while improving the robustness of such systems. We approach this survey from different perspectives, including data collection, data transmission, sensor fusion, data analysis, security, and privacy. We also review the benchmark data sets available that have been used to quantify the performance of the proposed methods. The survey is meant to provide researchers in the field of elderly fall detection using sensor networks with a summary of progress achieved up to date and to identify areas where further effort would be beneficial.

show abstract

Section: Fall Detection Using Individual Sensorsmentioning

confidence: 99%

Elderly Fall Detection Systems: A Literature Survey

2020

View full text Add to dashboard Cite

show abstract

“…Then, the VGG-16-based Con-vNet takes dynamic image as input and outputs the scores of four phases: standing, falling, fallen, and not moving. In reference [34], a three-stream Convolutional Neural Network is used to model the spatio-temporal representations in videos. The inputs to the three-stream Convolutional Neural Network are silhouettes, motion history images, and dynamic images.…”

Section: The Overview Of Proposed Methodsmentioning

confidence: 99%

“…Then, the extracted C3D features are fed into an LSTM-based attention network. In references [21,[33][34][35], the proposed models detect fall at frame level, therefore, can only detect fall only in the temporal dimension. In essence, the four methods refs.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Movement Tube Detection Network Integrating 3D CNN and Object Detection Framework to Detect Fall

et al. 2021

View full text Add to dashboard Cite

Unlike most of the existing neural network-based fall detection methods, which only detect fall at the time range, the algorithm proposed in this paper detect fall in both spatial and temporal dimension. A movement tube detection network integrating 3D CNN and object detection framework such as SSD is proposed to detect human fall with constrained movement tubes. The constrained movement tube, which encapsulates the person with a sequence of bounding boxes, has the merits of encapsulating the person closely and avoiding peripheral interference. A 3D convolutional neural network is used to encode the motion and appearance features of a video clip, which are fed into the tube anchors generation layer, softmax classification, and movement tube regression layer. The movement tube regression layer fine tunes the tube anchors to the constrained movement tubes. A large-scale spatio-temporal (LSST) fall dataset is constructed using self-collected data to evaluate the fall detection in both spatial and temporal dimensions. LSST has three characteristics of large scale, annotation, and posture and viewpoint diversities. Furthermore, the comparative experiments on a public dataset demonstrate that the proposed algorithm achieved sensitivity, specificity an accuracy of 100%, 97.04%, and 97.23%, respectively, outperforms the existing methods.

show abstract

“…It increases the user's complexity and reduces the user's experience in repeated wearing. Kong et al [27] propose an effective fall detection method based on computer vision-based framework, which learned to take full advantage of the appearance and motion information. Leila et al [28] propose a machine vision-based system combined with Support Vector Machine (SVM) classifier, which has high sensitivity and specificity of 100% and 97.5% respectively.…”

Section: Introductionmentioning

confidence: 99%

Walking Assist Robot: A Novel Non-Contact Abnormal Gait Recognition Approach Based on Extended Set Membership Filter

et al. 2019

View full text Add to dashboard Cite

This work develops intelligent walking assist robot and gait rehabilitation robot for people with walking disabilities, where distinguishing of abnormal gait is crucial during safe human-robot interaction. Although vision sensors and wearable inertial devices are widely used in fall detection system, shielding of robot mechanical structure brings instability for visual detection method, and utilizing a large number of wearable sensors brings discomfort and complexity for users. Meanwhile, the frequent drag-to-drop gait has never been considered, which is a kind of intermittent state between normal state and fall. It's a major challenge to achieve an abnormal gait recognition system with high stability and good comfort. In this paper, a novel non-contact gait recognition method is proposed, which can identify fall and drag-to-drop gait in a stable and comfortable way. Leg movement data is obtained by a built-in-robot camera, which accurately describes gait law through contrast test and error analysis. New-type ESMF algorithm is proposed to evaluate knee angle precisely and resolve the issue of marker point loss effectively. The comprehensive experiments have shown that this innovative approach calculates accurately knee angle in the walking process so as to recognize the phenomenon of drag-to-drop gait and fall. Additionally, this method having a sound theoretical basis and higher stability solves the issue of marker point loss, which could be applied to a walker with a similar structure for elderly and disabled in a comfortable way. INDEX TERMS Walking assist robot, fall recognition, drag-to-drop gait, marker point loss, non-contact detection, extended set membership filter.

show abstract

Learning spatiotemporal representations for human fall detection in surveillance video

Cited by 59 publications

References 48 publications

Elderly Fall Detection Systems: A Literature Survey

Elderly Fall Detection Systems: A Literature Survey

Movement Tube Detection Network Integrating 3D CNN and Object Detection Framework to Detect Fall

Walking Assist Robot: A Novel Non-Contact Abnormal Gait Recognition Approach Based on Extended Set Membership Filter

Contact Info

Product

Resources

About