Assessing the Implementation Fidelity of Early Interventions: Data Collection Methods

Abstract

There is a lack of standards regarding methods used to collect and assess data on implementation fidelity of early childhood intervention practices.Objectives: The purpose of this article is to document and discuss data collection methods that researchers could use for the complex process of evaluating implementation fidelity.Methods: We conducted a critical analysis of 27 empirical articles published between 1998 and 2019 on implementation of early childhood intervention practices, using the Matrix Method (Garrard, 2014).Results: Eight of the 26 studies used a log or implementation report. In 50 % of the studies (13/26), questionnaires collected information about quality of the intervention delivered, satisfaction and participant responsiveness. Direct or indirect observation (using video) was the most common collection method utilized. Most studies (16/26) used video observation. Conclusion:Using several methods rather than a single one is advantageous and allows for data triangulation. Different aspects of implementation fidelity must be considered, and information collected through a variety of methods, from different sources and on many occasions.

Evaluation of a Telehealth Parent-Training Program in Japan: Collaboration with Parents to Teach Novel Mand Skills to Children Diagnosed with Autism Spectrum Disorder

“…Third, procedural integrity was evaluated for every mand-training trial conducted by the parents. Procedural integrity is often assessed intermittently in early interventions, for example, once a month (Lemire et al, 2020 ). Evaluating procedural integrity for every trial was resource intensive and not feasible in many applied settings.…”