Fall Detection: a Camera and LiDAR Based Fall Detection System using Pose Estimation model (Preprint)

Abstract

BACKGROUND Fall is dangerous for human, sometimes may cause body malfunction. Moreover, fall is dangerous especial for elder & newborn babies. According to statistics of World Health Organization, falls are the second leading cause of unintentional injury deaths worldwide. Each year an estimated 684,000 individuals die from falls globally of which over 80% are in low and middle income countries. Adults older than 60 years of age suffer the greatest number of fatal falls. 37.3 million of falls that are severe enough that require medical attention each year. It is possible to minimize the damage if we can make medical treatment as soon as possible after fall. We propose a new mechanism to detect fall using camera and LiDAR technology. OBJECTIVE Our goal is to make fall detection as precise as possible. We take advantage of camera and LiDAR to get the world 3D coordinate of the scene with human, and through pose estimation model to get human skeleton. Then we can do fall analysis using the 3D coordinate of human body. We can save more time for rescuers to rescue someone after fall. The scenario application of this system can be home, hospital, road, construction site, even factory, any place with human being. METHODS We take a photo of human using a camera and LiDAR. Then we use the 2D camera photo as the input of pose estimation machine learning model to get the 2D coordinate of human body. Then we transfer these 2D coordinates of human body into 3D coordinates by the 2D coordinates data and the depth information of LiDAR. Then we can detect if a man/woman is falling or not according to the 3D coordinate of the human body. RESULTS We analyze 4 poses of 5 rounds of 3 testers. The accuracy of this system is good (92.5%) even when parts of the body go out of camera sight. Both male and female and child testers with various height could be correctly detected (male of 182 cm, female of 160 cm and child of 98 cm). A fall can be detected and alerted in real time (more than 20 frames per second). Although there are some false negative when testing a fall pose. But many frames are analyzed during the single fall. Most of fall correctly alerted is enough for a rescue. CONCLUSIONS 3D coordinate is more precise and easier to do judgement than 2D coordinate. We can get 3D coordinate by combining the 2D coordinate of camera and the depth information of LiDAR. And Apple already launched some iOS and iPadOS device with camera and LiDAR. In this paper we use iPad Pro 11” 3rd edition to do this work, which means this mechanism is of low cost and easy to implement.