Interaction and behaviour imaging: a novel method to measure mother–infant interaction using video 3D reconstruction

Abstract

Studying early interaction is essential for understanding development and psychopathology. Automatic computational methods offer the possibility to analyse social signals and behaviours of several partners simultaneously and dynamically. Here, 20 dyads of mothers and their 13–36-month-old infants were videotaped during mother–infant interaction including 10 extremely high-risk and 10 low-risk dyads using two-dimensional (2D) and three-dimensional (3D) sensors. From 2D+3D data and 3D space reconstruction, we extracted individual parameters (quantity of movement and motion activity ratio for each partner) and dyadic parameters related to the dynamics of partners heads distance (contribution to heads distance), to the focus of mutual engagement (percentage of time spent face to face or oriented to the task) and to the dynamics of motion activity (synchrony ratio, overlap ratio, pause ratio). Features are compared with blind global rating of the interaction using the coding interactive behavior (CIB). We found that individual and dyadic parameters of 2D+3D motion features perfectly correlates with rated CIB maternal and dyadic composite scores. Support Vector Machine classification using all 2D–3D motion features classified 100% of the dyads in their group meaning that motion behaviours are sufficient to distinguish high-risk from low-risk dyads. The proposed method may present a promising, low-cost methodology that can uniquely use artificial technology to detect meaningful features of human interactions and may have several implications for studying dyadic behaviours in psychiatry. Combining both global rating scales and computerized methods may enable a continuum of time scale from a summary of entire interactions to second-by-second dynamics.

Relations between Automatically Extracted Motion Features and the Quality of Mother-Infant Interactions at 4 and 13 Months

“…In fact, Hammal et al ( 2015a ) suggest a continuum with increasing levels of motion activity going from depressed, to neutral, to positive and finally negative affect. The previous studies, excepting the results from Leclère et al ( 2016 ), also suggest that higher levels of movement coordination, i.e., where the partners adjust their level of motion activity to each other, are associated with non-conflictual periods and a better rating of the relationship. In mother-infant interaction, when the mother coordinates her movements to her infant's, she shows the infant that she is aware of him and that she would like to participate in his experience (Stern, 2004 ).…”

“…For example, Væver et al ( 2013 ) reported that the variability in head distance between mothers and infants was lower for mothers with postpartum depression than for non-clinical mothers. In another study, Leclère et al ( 2016 ) showed that it was possible to discriminate between non-clinical mothers and mothers who were emotionally neglecting, based on motion features such as motion activity, pause, and overlapping motion. Finally, Delaherche et al ( 2013 ) demonstrated that it was possible to discriminate between autistic and non-clinical children interacting with therapists using motion features such as gestural and pause durations.…”

“…It could be that the oxytocin-induced alterations in fathers' motion characteristics increased the level of motionese, i.e., moving in an exaggerated way, which enhanced the infant's attention. Leclère et al ( 2016 ) found that higher levels of motion activity, overlapping motion, and contingent motion responses in mother-infant interactions were associated with higher levels of aversive interaction qualities, such as maternal intrusiveness and child avoidance, and with lower levels of adaptive interaction qualities, such as maternal sensitivity. In line with these results, other studies have found that an increase in the velocity of infant head movements is related to an increase in infant negative affect and distress within mother-infant interactions at 4 and 13 months of infant age (Hammal et al, 2015a , b ).…”

“…The study is based on the results from the study by Leclère et al ( 2016 ) who examined the role of upper-body motion in a pre-determined play situation between 12- and 36-month-old children and emotionally neglecting or non-clinical mothers. The present study extends the results from the study by Leclère et al ( 2016 ) by investigating whether the same relationships between motion features and interaction qualities are present for different body parts, i.e., arm, head, and upper-body motion, at different infant ages, specifically, 4 and 13 months, and in a different interaction arrangement, namely a free face-to-face interaction.…”

Dyssynchrony and perinatal psychopathology impact of child disease on parents-child interactions, the paradigm of Prader Willi syndrom

“…This synchrony phenomena remains true accross modalities and behaviors whether focusing on speech turns (e.g. Weisman et al, 2015;Jaffe et al, 2001), motion (Leclère et al, 2016), head movements (e.g. Hammal et al, 2015) and smile (Messinger et al, 2008).…”