Automated Detection of Movements During Sleep Using a 3D Time-of-Flight Camera: Design and Experimental Evaluation

Abstract

Analyses of sleep-related movement disorders have gained importance due to an increase in life expectancy. The present approaches for measuring movements are based on electromyography or accelerometry and provide only local or specific results from muscles/limbs to which sensors have been attached. The motivation of this work was to investigate the detection of a more complete spectrum of sleeprelated movements using a three-dimensional (3D) camera instead of the current conventional methods. In contrast to most of the previously published literature, this method allows for the detection of movements even when patients are covered with a blanket. This is the first work to evaluate movement detection with a clinical dataset and replicate the clinical environment in a laboratory setup. The laboratory setup allowed for the characterization of detectable movements through the determination of speed and amplitude limits. We used the Kinect One time-of-flight sensor to record 3D videos. Movements were quantified based on the temporal depth change in these 3D videos. A computer-controlled lifting table allowed for the controlled simulation of movements. Our algorithm detected movements with amplitude values >3.0 mm and velocity values >3.5 mm/s with an F1 score ≥95%. The shortest reliably detected duration of movement was 350 ms. In an ethically approved clinical study including 44 patients, 93.1% of electromyography-detected leg movements were also found in 3D. A significant correlation (ρ = 0.86) was found between movements detected by the 3D system and polysomnography. The 3D system detected 31.2% more movements than electromyography. In addition to obtaining a broader spectrum of movements not limited to local and muscle/limb-specific movements, the usage of a contactless 3D camera simplifies the recording setup and preserves natural sleeping behavior. The presented 3D system may become useful for diagnostic purposes during sleep studies.

Automatic 3D Video Analysis of Upper and Lower Body Movements to Identify Isolated REM Sleep Behavior Disorder: A Pilot Study

“…For each (𝑖, 𝑗) and for each frame 𝑡, the motion signal 𝑚 𝑖,𝑗 (𝑡) was obtained via convolution over time. A detailed description can be found elsewhere [22]. By combining the motion signal of each pixel, it was possible to generate motion maps for each frame (Fig.…”

“…For each ROI and for each frame 𝑡, the movement strength 𝑆(𝑡) was calculated as the sum of all the values 𝑚 𝑖,𝑗 (𝑡) included in the ROI. Based on the signal 𝑆(𝑡), movements were identified by using two thresholds, as previously described [22].…”

Improved automatic identification of isolated rapid eye movement sleep behavior disorder with a 3D time‐of‐flight camera

“…To identify movements from depth images, a pipeline previously described was applied [12,21]. Compared with our previous work where only lower limb movements were analyzed [12], further regions of interest (ROIs) (Figure 1) were added.…”

Convolutional Neural Network and LSTM Applied To Abnormal Behaviour Detection From Highway Footage

“…Despite this, there are papers in recent literature detailing the application of computer vision for the detection and identification of behaviours, and it is important to note that these are ad hoc applications, thus distancing them from generalism by definition. For example: 1) [4] presented an example where a method was proposed to autonomously extract vehicles' trajectories from aerial images, thus also allowing the analysis of behavioural deviations; 2) [7] presented a paper where long-range cameras were applied to detect suspicious activity on the Texas-Mexico border; 3) the paper of [8], where they presented a system for detecting irregular behaviour in courts; 4) [9] presented an approach to analyzing and detecting sleep-related movement disorders based on nocturnal behaviours captured using sensors on the Kinect One device.…”