Autism spectrum disorder (ASD) impacts communication and cognitivedevelopment of children and adults, has aworldwide prevalenceof 1% on children and affects not only the people with this disorder,but also their family and the surrounding community. In the familycircle, individuals on the spectrum require greater support and attentionrelative to its cognitive capacity, impacting the mental andemotional health and even the financial life of families. The lack ofinfrastructure, professionals, and public health policies to deal withASD is a known problem, specially in low income countries. Tomitigate this issue, computer-aided ASD diagnosis and treatmentrepresent a powerful ally, reducing the workload of professionalsand allowing a better overall therapeutic experience. This paperintends to investigate how machine learning techniques can helpspecialists by providing an automated analysis of ASD recordedtherapy sessions. The proposed solution is capable of handling largeamounts of video data, filtering out irrelevant frames and keepingonly relevant scenes for posterior analysis. Our results show thatthe proposed solution is capable of reducing manual checks by upto 51.4%, which represents a significant workload reduction forhealth experts. This solution will hopefully provide researchers,therapists and specialists with a tool that assists the automatedidentification of features and events of interest in video-recordedtherapy sessions, reducing the amount of time spent on this task.