Usability and Reliability of a Remotely Administered Adult Autism Assessment, the Autism Diagnostic Observation Schedule (ADOS) Module 4

Abstract

Introduction: The Autism Diagnostic Observation Schedule (ADOS) Module 4 is an autism assessment designed for verbally fluent adolescents and adults. Because of a shortage of available clinical expertise, it can be difficult for adults to receive a proper autism spectrum disorder (ASD) diagnostic assessment. A potential option to address this shortage is remote assessment. The objective of this study was to examine the feasibility, usability, and reliability of administering the ADOS Module 4 remotely using the Versatile and Integrated System for Telerehabilitation (VISYTER). Materials and Methods: VISYTER consists of computer stations at the client site and clinician site for video communication and a Web portal for managing and coordinating the assessment process. Twenty-three adults with an ASD diagnosis participated in a within-subject crossover design study in which both a remote ADOS and a face-to-face ADOS were administered. After completing the remote ADOS, participants completed a satisfaction survey. Results: Participant satisfaction with the remote ADOS delivery system was high. The kappa value was greater than 0.61 on 21 of 31 ADOS items. There was substantial agreement on ADOS classification (i.e., diagnosis) between assessments delivered face-to-face versus assessments delivered remotely (interclass coefficient = 0.92). Nonagreement may have been due to outside factors or practice effect despite a washout period. Conclusions: The results of this study demonstrate that an autism assessment designed to be delivered face to face can be administered remotely using an integrated Web-based system with high levels of usability and reliability.

Review of key telepsychiatry outcomes

“…Prominent positive themes include ease of use and decreased burden of transportation to and from appointments; whereas prominent negative themes include privacy concerns, challenging to establish patient-doctor relationship, and technical challenges. Finally, some studies (n = 7) used an experimental design to assess patient satisfaction [10,21,26,36,[41][42][43] . In a study that compared reactions of 48 outpatients randomized to telepsychiatry or face-to-face (FTF) psychiatric consultation, telepsychiatry patients reported comfort in disclosing the same information they would disclose in FTF consultation but reported slightly lower levels of satisfaction regarding feeling supported and encouraged than did FTF patients [36] .…”

“…Two studies compared the accuracy of diagnoses made using the SCID [59,60] , two studies evaluated reliability of child assessments [42,61] , four studies evaluated reliability of neuropsychological assessments [62][63][64][65] , and three studies evaluated reliability of measures of depressive symptom severity [66][67][68] . The remaining studies evaluated a range of targets such as alcohol use severity [69] , diagnostic accuracy [70][71][72] , competency to stand trial [73] , psychosis [26] , and adult autism [21] . The majority of studies report moderate to high level of agreement between raters using telepsychiatry and FTF regardless of instrument, provider, or setting type.…”

Telehealth and autism: Are telehealth language assessments reliable and feasible for children with autism?

“…Diagnostic assessments for autism have been completed via telehealth with adults (Schutte et al . ). Diagnostic assessments via telehealth for children have also been researched, with parents coached via telehealth to deliver assessment items to their children (Reese et al .…”

“…It has been used to provide parent training in early intervention (e.g., Ingersoll and Berger 2015, Meadan et al 2016, Wainer and Ingersoll 2015, Simacek et al 2017, with positive outcomes reported for communication and other skills, including behaviour and imitation, as well as to support older children with autism and co-occurring anxiety (Hepburn et al 2015) and to provide functional behavioural assessment and functional communication training for children with behaviour support needs (Lindgren et al 2016, Suess et al 2014) via parent training. Diagnostic assessments for autism have been completed via telehealth with adults (Schutte et al 2015). Diagnostic assessments via telehealth for children have also been researched, with parents coached via telehealth to deliver assessment items to their children (Reese et al 2013, Reese et al 2015.…”

Impact of Patient-centered eHealth Applications on Patient Outcomes: A Review on the Mediating Influence of Human Factor Issues

“…Previous research within the field of patient-centered eHealth services shows that HFE approaches are nearly always adopted in the design and development of these HIT interventions [21]. Furthermore, insights on human factors that influence the acceptance, usability, and implementation of patient-centered eHealth services have grown [26][27][28][29][30]. However, evidence on how human factors, such as patient characteristics, mediate the use and impact of these eHealth services on patient behavior changes, self-efficacy, and health-related outcomes is scarce [21,25].…”