Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences

Abstract

Statistical models of the human body surface are generally learned from thousands of high-quality 3D scans in predefined poses to cover the wide variety of human body shapes and articulations. Acquisition of such data requires expensive equipment, calibration procedures, and is limited to cooperative subjects who can understand and follow instructions, such as adults. We present a method for learning a statistical 3D Skinned Multi-Infant Linear body model (SMIL) from incomplete, low-quality RGB-D sequences of freely moving infants. Quantitative experiments show that SMIL faithfully represents the RGB-D data and properly factorizes the shape and pose of the infants. To demonstrate the applicability of SMIL, we fit the model to RGB-D sequences of freely moving infants and show, with a case study, that our method captures enough motion detail for General Movements Assessment (GMA), a method used in clinical practice for early detection of neurodevelopmental disorders in infants. SMIL provides a new tool for analyzing infant shape and movement and is a step towards an automated system for GMA.

AI Approaches towards Prechtl’s Assessment of General Movements: A Systematic Literature Review

“…In comparison, Schroeder et al [ 104 ] recently evaluated a Skinned Multi-Infant Linear Model (SMIL) including 3D body surface additionally to the skeleton of the infant. SMIL model creation consists out of several steps, including background and clothing segmentation, landmark (body, face and hand) estimation and a personalization step, where an initial base template is transferred to the “infant specific shape space by performing PCA on all personalized shapes” [ 105 ]. The base template represents an infant-based model instead of just downsampling already existing adult models.…”

Analyse der Spontanmotorik im 1. Lebensjahr: Markerlose 3-D-Bewegungserfassung zur Früherkennung von Entwicklungsstörungen

“…Neben dem geringen Kostenaufwand (herkömmliche RGB-D-Videokamera, Notebook, KineMAT-Software) zeichnet sich der KineMAT durch seine den bisher entwickelten Methoden gegenüber überlegene Präzision in der korrekten Erfassung der Spontanbewegung über die Aufnahmezeit aus. Bereits 2019 konnten wir zeigen, dass in 98,8 % der Zeit die Bewegungen des SMIL-Motion-Videos mit dem konventionellen Farbvideo übereinstimmen [14].…”

An Open-Source Articulated Multi-Person Shape Model Training and Inference Pipeline

“…Many state-of-the-art methods for the construction and inference of statistical human shape models are readily available. One of the most important is SMPL [6], which was extended to faces [7], hands [8], and infants [9]. SMPL is compatible with 3D modelling software but relies heavily on high-quality registrations and is thus limited to the application on costly preprocessed data.…”