Using automatic face analysis to score infant behaviour from video collected online

Chouinard, Brea; Scott, Kimberly; Cusack, Rhodri

doi:10.1016/j.infbeh.2018.11.004

Cited by 34 publications

(51 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These broad concerns are certainly not limitations of all in-person studies in the field of cognitive development, and there are many well-established and exciting new ways to mitigate them, including having blind experimenters, developing multi-lab collaborations, adopting new practices for data and protocol sharing, and making efforts to diversify samples for in-person research (see Chouinard, Scott, & Cusack, 2019;Gilmore et al, 2018;Rubio-Fernández, 2019;Scott, Chu, & Schultz, 2017;The Many Babies Consortium, 2020). We note these limitations simply to acknowledge the challenges inherent to doing work in cognitive development -a science that, perhaps more than most other subdisciplines of psychology, has relied heavily on interpersonal interaction between the researcher and participant.…”

mentioning

confidence: 99%

Advancing Developmental Science via Unmoderated Remote Research with Children

Rhodes

Rizzo

Foster‐Hanson

et al. 2020

Journal of Cognition and Development

126

View full text Add to dashboard Cite

This article introduces an accessible approach to implementing unmoderated remote research in developmental science-research in which children and families participate in studies remotely and independently, without directly interacting with researchers. Unmoderated remote research has the potential to strengthen developmental science by: (1) facilitating the implementation of studies that are easily replicable, (2) allowing for new approaches to longitudinal studies and studies of parent-child interaction, and (3) including families from more diverse backgrounds and children growing up in more diverse environments in research. We describe an approach we have used to design and implement unmoderated remote research that is accessible to researchers with limited programming expertise, and we describe the resources we have made available on a new website (discoveriesonline.org) to help researchers get started with implementing this approach. We discuss the potential of this method for developmental science and highlight some challenges still to be overcome to harness the power of unmoderated remote research for advancing the field. The field of cognitive development was founded upon remarkable insights gleaned from everyday interactions with children. Piaget's theory of cognitive development (1954) began with his observations of his own children playing on their mats, dropping things from their highchairs, and playing with marbles. Carey (1985) revolutionized our understanding of how concepts originate and change by analyzing conversations with her own child about birth, the nature of life, and death. The field is full of stories of great theoretical insights made by researchers closely watching children as they crawl around near the sides of high beds (Adolph, Kretch, & LoBue, 2014), negotiate the rules of games among themselves on a playground (Borman, 1981), and try to sit down on way-too-tiny toy tractors (DeLoache, 1987). Of course, the field has never relied on the observations of individual researchers alone. We use these observations to design experiments that recreate the situations in which the behaviors were first observed, which can then be replicated by labs around the world. But still, individual interactions between researchers and their participants have always been central to the field of cognitive development. The notion that we can learn from watching and interacting with children, along with the intriguing challenges of thinking about how to recreate the conditions of everyday life in CONTACT Marjorie Rhodes

show abstract

mentioning

confidence: 99%

Advancing Developmental Science via Unmoderated Remote Research with Children

Rhodes

Rizzo

Foster‐Hanson

et al. 2020

Journal of Cognition and Development

126

View full text Add to dashboard Cite

show abstract

“…Rekognition's face detection and face recognition accuracy were independently evaluated in [26], while Transcribe's ASR performance in conversational settings was assessed in [27], both with satisfactory results for our application. Rekognition face detection output was employed to study infant behavior in [28].…”

Section: Methodsmentioning

confidence: 99%

Automatized analysis of children’s exposure to child-directed speech in reschool settings: Validation and application

Villasanti¹,

Justice²,

Chaparro-Moreno³

et al. 2020

PLoS ONE

View full text Add to dashboard Cite

The present study explored whether a tool for automatic detection and recognition of interactions and child-directed speech (CDS) in preschool classrooms could be developed, validated, and applied to non-coded video recordings representing children’s classroom experiences. Using first-person video recordings collected by 13 preschool children during a morning in their classrooms, we extracted high-level audiovisual features from recordings using automatic speech recognition and computer vision services from a cloud computing provider. Using manual coding for interactions and transcriptions of CDS as reference, we trained and tested supervised classifiers and linear mappings to measure five variables of interest. We show that the supervised classifiers trained with speech activity, proximity, and high-level facial features achieve adequate accuracy in detecting interactions. Furthermore, in combination with an automatic speech recognition service, the supervised classifier achieved error rates for CDS measures that are in line with other open-source automatic decoding tools in early childhood settings. Finally, we demonstrate our tool’s applicability by using it to automatically code and transcribe children’s interactions and CDS exposure vertically within a classroom day (morning to afternoon) and horizontally over time (fall to winter). Developing and scaling tools for automatized capture of children’s interactions with others in the preschool classroom, as well as exposure to CDS, may revolutionize scientific efforts to identify precise mechanisms that foster young children’s language development.

show abstract

“…The video for each trial was cropped with a time window from when the question ended to 2,000 ms thereafter since, at a later time, looking behavior is no longer considered to be related to the stimulus (Delle Luche et al, 2015). We set the starting point of the potential time window not at the onset or offset of the target word but at the end of the question sentence because, in Japanese, a listener cannot determine if a sentence is a question unless they hear the sentence through to the end due to the grammatical difference in word order from English.…”

Section: Video Recordingsmentioning

confidence: 99%

“…In line with recent vigorous studies on automatic gaze estimation using webcam-based data (Chouinard et al, 2019;Papoutsaki et al, 2016), we used an open-source library, OpenFace 2.2.0 (Baltrušaitis et al, 2018) for automatic face and gaze estimation. We adopted this toolkit because it provides rich information such as facial landmarks, gaze directions, and three-dimensional head positions and angles, because it is freely available for research purposes, and works in the local environments, which reduces ethical concerns.…”

Section: Video Recordingsmentioning

confidence: 99%

Looking represents choosing in toddlers: Exploring the equivalence between multimodal measures in forced‐choice tasks

Hagihara

Ienaga

Terayama

et al. 2020

Infancy

View full text Add to dashboard Cite

In the two‐alternative forced‐choice (2AFC) paradigm, manual responses such as pointing have been widely used as measures to estimate cognitive abilities. While pointing measurements can be easily collected, coded, analyzed, and interpreted, absent responses are often observed particularly when adopting these measures for toddler studies, which leads to an increase of missing data. Although looking responses such as preferential looking can be available as alternative measures in such cases, it is unknown how well looking measurements can be interpreted as equivalent to manual ones. This study aimed to answer this question by investigating how accurately pointing responses (i.e., left or right) could be predicted from concurrent preferential looking. Using pre‐existing videos of toddlers aged 18–23 months engaged in an intermodal word comprehension task, we developed models predicting manual from looking responses. Results showed substantial prediction accuracy for both the Simple Majority Vote and Machine Learning‐Based classifiers, which indicates that looking responses would be reasonable alternative measures of manual ones. However, the further exploratory analysis revealed that when applying the created models for data of toddlers who did not produce clear pointing responses, the estimation agreement of missing pointing between the models and the human coders slightly dropped. This indicates that looking responses without pointing were qualitatively different from those with pointing. Bridging two measurements in forced‐choice tasks would help researchers avoid wasting collected data due to the absence of manual responses and interpret results from different modalities comprehensively.

show abstract

Using automatic face analysis to score infant behaviour from video collected online

Cited by 34 publications

References 27 publications

Advancing Developmental Science via Unmoderated Remote Research with Children

Advancing Developmental Science via Unmoderated Remote Research with Children

Automatized analysis of children’s exposure to child-directed speech in reschool settings: Validation and application

Looking represents choosing in toddlers: Exploring the equivalence between multimodal measures in forced‐choice tasks

Contact Info

Product

Resources

About