Engagement detection for children with Autism Spectrum Disorder

Chorianopoulou, Arodami; Tzinis, Efthymios; Iosif, Elias; Papoulidi, Asimenia; Papailiou, Christina; Potamianos, Alexandros

doi:10.1109/icassp.2017.7953119

Cited by 20 publications

(26 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This may particularly be challenging due to the large individual and cultural heterogeneity in image data of this population. Also, most of existing works on analysis of facial cues in autism focus on eye-gaze, blinking, and head-pose [12], [29], which are shown to be a good proxy of joint attention and engagement -the lack of which is pertinent to ASC. Extracting these cues from face images is usually done using detectors specifically built for each facial cue.…”

Section: Introductionmentioning

confidence: 99%

CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism

Rudovic

Utsumi

Lee

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Many children on autism spectrum have atypical behavioral expressions of engagement compared to their neurotypical peers. In this paper, we investigate the performance of deep learning models in the task of automated engagement estimation from face images of children with autism. Specifically, we use the video data of 30 children with different cultural backgrounds (Asia vs. Europe) recorded during a single session of a robot-assisted autism therapy. We perform a thorough evaluation of the proposed deep architectures for the target task, including within-and across-culture evaluations, as well as when using the child-independent and child-dependent settings. We also introduce a novel deep learning model, named Cul-tureNet, which efficiently leverages the multi-cultural data when performing the adaptation of the proposed deep architecture to the target culture and child. We show that due to the highly heterogeneous nature of the image data of children with autism, the child-independent models lead to overall poor estimation of target engagement levels. On the other hand, when a small amount of data of target children is used to enhance the model learning, the estimation performance on the held-out data from those children increases significantly. This is the first time that the effects of individual and cultural differences in children with autism have empirically been studied in the context of deep learning performed directly from face images.

show abstract

Section: Introductionmentioning

confidence: 99%

CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism

Rudovic

Utsumi

Lee

et al. 2018

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

show abstract

“…[14][15][16][17][18][19] Chorianopoulou et al collected structured home videos from participants and had expert annotators label the dataset with the actions, emotions, gaze fixations, utterances, and overall level of engagement in each video; this information was then used to train a classifier to identify specific engagement features that could be correlated with ASD. 20 Rudovic et al trained a large and generalizable neural network to estimate engagement in children with ASD from different cultural backgrounds. 21 Engagement labels were manually annotated by trained individuals.…”

Section: Manual Annotation Methodsmentioning

confidence: 99%

Identification of social engagement indicators associated with autism spectrum disorder using a game-based mobile application

Varma

Washington

Chrisman

et al. 2021

Preprint

View full text Add to dashboard Cite

ObjectiveAutism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Children with ASD exhibit behavioral and social impairments, giving rise to the possibility of utilizing computational techniques to evaluate a child’s social phenotype from home videos.MethodsHere, we use a mobile health application to collect over 11 hours of video footage depicting 95 children engaged in gameplay in a natural home environment. We utilize automated dataset annotations to analyze two social indicators that have previously been shown to differ between children with ASD and their neurotypical (NT) peers: (1) gaze fixation patterns and (2) visual scanning methods. We compare the gaze fixation and visual scanning methods utilized by children during a 90-second gameplay video in order to identify statistically-significant differences between the two cohorts; we then train an LSTM neural network in order to determine if gaze indicators could be predictive of ASD.ResultsOur work identifies one statistically significant region of fixation and one significant gaze transition pattern that differ between our two cohorts during gameplay. In addition, our deep learning model demonstrates mild predictive power in identifying ASD based on coarse annotations of gaze fixations.DiscussionUltimately, our results demonstrate the utility of game-based mobile health platforms in quantifying visual patterns and providing insights into ASD. We also show the importance of automated labeling techniques in generating large-scale datasets while simultaneously preserving the privacy of participants. Our approaches can generalize to other healthcare needs.

show abstract

“…In [26], wearable sensors were used to measure the electrodermal activity of the children and a Support Vector Machine (SVM) classifier was applied to classify the children being engaged or not. In [27], acoustic and linguistic data were utilized to detect the social engagement in conversational interactions of children with ASD and their parents, using an SVM classifier. e first indepth study of measuring the engagement of children when interacting with social robots was proposed by Anzalone et al [28].…”

Section: Related Workmentioning

confidence: 99%

Estimating Children Engagement Interacting with Robots in Special Education Using Machine Learning

Papakostas

Sidiropoulos

Lytridis

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

The task of child engagement estimation when interacting with a social robot during a special educational procedure is studied. A multimodal machine learning-based methodology for estimating the engagement of the children with learning difficulties, participating in appropriate designed educational scenarios, is proposed. For this purpose, visual and audio data are gathered during the child-robot interaction and processed towards deciding an engaged state of the child or not. Six single and three ensemble machine learning models are examined for their accuracy in providing confident decisions on in-house developed data. The conducted experiments revealed that, using multimodal data and the AdaBoost Decision Tree ensemble model, the children’s engagement can be estimated with 93.33% accuracy. Moreover, an important outcome of this study is the need for explicitly defining the different engagement meanings for each scenario. The results are very promising and put ahead of the research for closed-loop human centric special education activities using social robots.

show abstract

Engagement detection for children with Autism Spectrum Disorder

Cited by 20 publications

References 14 publications

CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism

CultureNet: A Deep Learning Approach for Engagement Intensity Estimation from Face Images of Children with Autism

Identification of social engagement indicators associated with autism spectrum disorder using a game-based mobile application

Estimating Children Engagement Interacting with Robots in Special Education Using Machine Learning

Contact Info

Product

Resources

About