Digital Behavioral Phenotyping Detects Atypical Pattern of Facial Expression in Toddlers with Autism

Abstract

Commonly used screening tools for autism spectrum disorder (ASD) generally rely on subjective caregiver questionnaires. While behavioral observation is more objective, it is also expensive, time‐consuming, and requires significant expertise to perform. As such, there remains a critical need to develop feasible, scalable, and reliable tools that can characterize ASD risk behaviors. This study assessed the utility of a tablet‐based behavioral assessment for eliciting and detecting one type of risk behavior, namely, patterns of facial expression, in 104 toddlers (ASD N = 22) and evaluated whether such patterns differentiated toddlers with and without ASD. The assessment consisted of the child sitting on his/her caregiver's lap and watching brief movies shown on a smart tablet while the embedded camera recorded the child's facial expressions. Computer vision analysis (CVA) automatically detected and tracked facial landmarks, which were used to estimate head position and facial expressions (Positive, Neutral, All Other). Using CVA, specific points throughout the movies were identified that reliably differentiate between children with and without ASD based on their patterns of facial movement and expressions (area under the curves for individual movies ranging from 0.62 to 0.73). During these instances, children with ASD more frequently displayed Neutral expressions compared to children without ASD, who had more All Other expressions. The frequency of All Other expressions was driven by non‐ASD children more often displaying raised eyebrows and an open mouth, characteristic of engagement/interest. Preliminary results suggest computational coding of facial movements and expressions via a tablet‐based assessment can detect differences in affective expression, one of the early, core features of ASD. Lay Summary This study tested the use of a tablet in the behavioral assessment of young children with autism. Children watched a series of developmentally appropriate movies and their facial expressions were recorded using the camera embedded in the tablet. Results suggest that computational assessments of facial expressions may be useful in early detection of symptoms of autism.

Information and Communication Technologies to Support Early Screening of Autism Spectrum Disorder: A Systematic Review

“…We identified 28 studies that used mainstream or adapted information technologies to screen children up to 6 years for ASD (see Table 1 ). Seven of the included studies [ 22 , 31 , 32 , 33 , 34 , 35 , 36 ] involved children recruited from primary care or pediatric services, while five studies involved children referred to tertiary care or specialized ASD centers [ 37 , 38 , 39 , 40 , 41 ]. A total of 7308 children participated in the studies.…”

“…Two studies involved children from 6 to 18 months [ 42 , 43 ]. Six studies involved children within the 10 to 48 months range [ 36 , 40 , 41 , 44 , 45 , 46 ], seven studies involved participants aged between 16 and 30 months [ 22 , 25 , 31 , 32 , 34 , 35 , 47 ], and four studies involved children within the 18- and 72-months range [ 37 , 39 , 48 , 49 ]. Three studies involved a sample of children aged between 48 and 72 months [ 33 , 50 , 51 ].…”

“…The former category (i.e., NUI) included 11 papers. Of these, five papers involved the use of eye trackers [ 38 , 40 , 41 , 49 , 51 ], two studies used voice-based recording systems [ 44 , 53 ], two studies employed face-recognition to detect facial expressions [ 25 , 36 ], one paper involved motion recognition using touch screen sensor technologies [ 48 ], and one paper tracked pupil diameter [ 54 ].…”

“…The studies reported by Abbas et al [ 37 ] and Kanne et al [ 39 ] were included in both categories (i.e., PC and Mobile) as they combined the two strategies within the same application. In a similar vein, the studies reported by Egger et al [ 25 ] and Carpenter et al [ 36 ] were included both in the NUI and PC/Mobile category. Accordingly, 11 papers reported on the use of computerized solutions (PC or mobile platforms) to administer parent-reported questionnaires [ 22 , 31 , 32 , 33 , 34 , 35 , 37 , 39 , 46 , 47 , 55 ], and seven papers employed screening tools in which videos were collected from [ 37 , 39 , 43 , 50 ] or showed via [ 25 , 36 , 42 ] parents’ mobile/PC devices.…”

“…Nine papers were identified that involved children in the 16–30 months age range [ 22 , 25 , 31 , 32 , 34 , 35 , 36 , 46 , 47 ]. Of these, two papers reported on studies aimed at adapting the M-CHAT for its administration via tablet [ 3 , 31 ].…”