Objective: Video and sound acquisition and processing technologies have seen great improvements in recent decades, with many applications in the biomedical area. The aim of this paper is to review the overall state of the art of advances within these topics in paediatrics and to evaluate their potential application for monitoring in the neonatal intensive care unit (NICU). Approach: For this purpose, more than 150 papers dealing with video and audio processing were reviewed. For both topics, clinical applications are described according to the considered cohorts—full-term newborns, infants and toddlers or preterm newborns. Then, processing methods are presented, in terms of data acquisition, feature extraction and characterization. Main results: The paper first focuses on the exploitation of video recordings; these began to be automatically processed in the 2000s and we show that they have mainly been used to characterize infant motion. Other applications, including respiration and heart rate estimation and facial analysis, are also presented. Audio processing is then reviewed, with a focus on the analysis of crying. The first studies in this field focused on induced-pain cries and the newest ones deal with spontaneous cries; the analyses are mainly based on frequency features. Then, some papers dealing with non-cry signals are also discussed. Significance: Finally, we show that even if recent improvements in digital video and signal processing allow for increased automation of processing, the context of the NICU makes a fully automated analysis of long recordings problematic. A few proposals for overcoming some of the limitations are given.

International audienceObjectives: In the context of neonatal non invasive monitoring, this paper proposes the estimation and characterization of the motion of premature newborns from long duration video recordings.Material and Methods: A set of 13 videos from 9 different patients, corresponding to 190 hours of recordings, have been studied. An algorithm based on the analysis of changes in the image border has been used to remove intervals artifacted by adults' presence. Then, some features were computed to characterize the baby's motion. The approach was applied to compare two groups of premature newborns, with different severities of prematurity, recorded at the same postmenstrual age.Results: Detection of adults' presence was achieved with 96.8% of sensitivity. All features were found statistically significant to differentiate the two groups.Conclusion: This study shows that the automated video monitoring on long periods is achievable and provides relevant information about the premature newborns motion activity

No abstract

Premature babies have several immature functions and begin their life under high medical supervision. Since the sleep organization diers across postmenstrual age, its analysis may give a good indication of the degree of brain maturation. However, sleep analysis (polysomnography or behavioral observation) is dicult to install, time consuming and cannot systematically be used. In this context, development of new ways to automatically monitor the neonates, using contactless modalities, is necessary. Therefore, this study presents an innovative non-invasive approach to semi-automatize the classication of infant behavioral sleep states. Methods First, three descriptors were extracted from audio and video recordings: vocalizations, motion and eye state of the baby. For this purpose, an original semi-automatic algorithm for the estimation of the eye state was proposed. Secondly, the three descriptors were used in order to obtain an estimation of the behavioral sleep states. Five classiers (K-Nearest Neighbors, Linear Discriminant Analysis, Support Vector Machine, Random Forest and Multi-Layer Perceptron) were compared to an expert annotation. Results Firstly, the comparison of the semi-automatic eye state estimation to manual annotations of 10 videos led to a mean accuracy of 99.4%. Secondly, sleep stage classication was performed. Best results were obtained with Random Forest, for Quiet Alert and Active Alert stages, with 93.5% and 99.0% of accuracy respectively. Conclusion The proposed method provides a high capacity to identify alert sleep stages but the dierentiation between Quiet Sleep and Active Sleep only by behavioral observations still remains a dicult task to achieve. Signicance Results presented in this paper are new since no similar approach was proposed in the literature in the context of neonatal intensive care unit. They augur well for the automatic sleep organization assessment to improve newborn care.