iCatcher: A Neural Network Approach for Automated Coding of Young Children’s Eye Movements

Abstract

Infants’ looking behaviors are often used for measuring attention, real-time processing, and learning – often using low-resolution videos. Despite the ubiquity of gaze-related methods in developmental science, current techniques usually involve laborious post hoc coding, imprecise real-time coding, or expensive eye trackers that may increase data loss and require a calibration phase. As a solution, we used computer-vision methods to perform automatic gaze estimation from low-resolution videos. At the core of this approach is an artificial neural network that classifies gaze directions in real time. We tested our method, called iCatcher, on data collected using the looking-while-listening procedure, where infants look at one of two locations on a screen. Using a large dataset of manually-annotated videos from prior research, we demonstrate that the accuracy of iCatcher approximates that of human annotators and replicates the results. Our method is publicly available as an open-source repository at https://github.com/yoterel/iCatcher.

Conducting developmental research online vs. in-person: A meta-analysis

“…Thus, our meta-analysis might underestimate the effectiveness of online studies due to researcher and experimenter inexperience. Over the next several years, as developmental researchers develop expertise and experience with online studies, effect sizes might increase for any number of reasons, including better experimenter-participant interactions, better stimulus design, and more accurate methods of measurements (i.e., automatic looking time measures, see Erel et al, 2022). Relatedly, as new methods are developed and adapted for online experiments, researchers should not take the current findings as a blanket declaration that all online studies produce comparable results to their in-person counterparts; some might underperform, while others might outperform.…”

“…Accurate looking measures require precise camera positioning and coding schemes, and are thus more likely to deviate from their in-person counterparts compared to studies that measure children's verbal responses. To that end, automated gaze annotation is currently being developed and represents an exciting future direction in online methodology (see Erel, Potter, Jaffe-Dax, Lew-Williams, & Bermano, 2022). We examine how the kind of dependent measure employed (looking vs. verbal) might moderate the difference between online and in-person results.…”

Using video calls to study children’s conversational development: The case of backchannel signaling

“…Thus, we settled on a manageable sample size. That said, progress in automatic annotation of children's data (Sagae et al, 2007;Nikolaus et al, 2021;Long et al, 2022;Erel et al, 2022) should alleviate the constraint on large-scale data collection in future research. The current work is also supposed to contribute to this effort by providing substantial hand-annotated data that can be used for automatic model training and validation.…”

Understanding Children's Multimodal Conversational Development: Challenges and Opportunities

“…Existing programs need to be benchmarked in terms of their ability to provide reliable performance on children's hand-annotated datasets [e.g., 2], and new tools should be created to correct for potential limitations of existing programs. We are starting to witness interest in such endeavors among researchers in the developmental community [23,12,28]. Nevertheless, much more effort should be devoted to this line of work before we can adequately account for the complexity of children's multimodal signaling.…”