Using 2D video-based pose estimation for automated prediction of autism spectrum disorders in young children

Abstract

Clinical research in autism has recently witnessed promising digital phenotyping results, mainly focused on single feature extraction, such as gaze, head turn on name-calling or visual tracking of the moving object. The main drawback of these studies is the focus on relatively isolated behaviors elicited by largely controlled prompts. We recognize that while the diagnosis process understands the indexing of the specific behaviors, ASD also comes with broad impairments that often transcend single behavioral acts. For instance, the atypical nonverbal behaviors manifest through global patterns of atypical postures and movements, fewer gestures used and often decoupled from visual contact, facial affect, speech. Here, we tested the hypothesis that a deep neural network trained on the non-verbal aspects of social interaction can effectively differentiate between children with ASD and their typically developing peers. Our model achieves an accuracy of 80.9% (F1 score: 0.818; precision: 0.784; recall: 0.854) with the prediction probability positively correlated to the overall level of symptoms of autism in social affect and repetitive and restricted behaviors domain. Provided the non-invasive and affordable nature of computer vision, our approach carries reasonable promises that a reliable machine-learning-based ASD screening may become a reality not too far in the future.

Deep Learning Approach for Screening Autism Spectrum Disorder in Children with Facial Images and Analysis of Ethnoracial Factors in Model Development and Application

“…The standard approaches to diagnosing autism spectrum disorder (ASD) evaluate between 20 and 100 behaviors and take several hours to complete [ 39 ]. To make this approach easier and faster, several researchers reported using videos with machine learning to accelerate and automate the process [ 39 , 40 , 41 , 42 ]. These proposed video-based approaches use tablets or other devices that can capture the child’s behaviors, for example, eye gaze, or responses to stimuli, while the child is watching the specially designed movie clips or engaging in activities.…”

The Role of Intelligent Technologies in Early Detection of Autism Spectrum Disorder (ASD): A Scoping Review

“…However, the children with ASD displayed higher comfort and engagement with robots and a high IJA towards the therapist during the transition. In addition, [67] developed and validated a deep neural network (CNN-LSTM architecture) trained on the non-verbal aspects of social interaction from video recordings captured during ADOS-2 assessments that distinguished ASD and TD peers with an accuracy of 80.9%.…”

BabyPose: Real-Time Decoding of Baby’s Non-Verbal Communication Using 2D Video-Based Pose Estimation

“…It was based on a conceptual framework developed by the World Health Organization (WHO) and employed a real-world dataset of self-care behaviors. Kojovic et al [26] described a machine learning method that could distinguish between those with Autism spectrum disorders (ASD) and those who were typically developing (TD). They trained a deep neural network over the gold standard diagnostic examination using recordings collected as part of a wider study on early development in autism, which includes social interactions between a kid (with autism or TD) and an adult.…”