Taking Language Samples Home: Feasibility, Reliability, and Validity of Child Language Samples Conducted Remotely With Video Chat Versus In-Person

Abstract

Purpose There has been increased interest in using telepractice for involving more diverse children in research and clinical services, as well as when in-person assessment is challenging, such as during COVID-19. Little is known, however, about the feasibility, reliability, and validity of language samples when conducted via telepractice. Method Child language samples from parent–child play were recorded either in person in the laboratory or via video chat at home, using parents' preferred commercially available software on their own device. Samples were transcribed and analyzed using Systematic Analysis of Language Transcripts software. Analyses compared measures between-subjects for 46 dyads who completed video chat language samples versus 16 who completed in-person samples; within-subjects analyses were conducted for a subset of 13 dyads who completed both types. Groups did not differ significantly on child age, sex, or socioeconomic status. Results The number of usable samples and percent of utterances with intelligible audio signal did not differ significantly for in-person versus video chat language samples. Child speech and language characteristics (including mean length of utterance, type–token ratio, number of different words, grammatical errors/omissions, and child speech intelligibility) did not differ significantly between in-person and video chat methods. This was the case for between-group analyses and within-child comparisons. Furthermore, transcription reliability (conducted on a subset of samples) was high and did not differ between in-person and video chat methods. Conclusions This study demonstrates that child language samples collected via video chat are largely comparable to in-person samples in terms of key speech and language measures. Best practices for maximizing data quality for using video chat language samples are provided.

Genesis of a New Generation of Telepractitioners: The COVID-19 Pandemic and Pediatric Speech-Language Pathology Services

“…For example, Aggarwal et al (2020) reported not only an increase in therapy services delivered remotely in India immediate after the COVID-19 pandemic but also acceptance and predicted future use. Speech-language pathologists who previously used telehealth understood its potential to provide access to clients in a larger geographical area and not predominantly clients who lived in rural areas (Manning et al, 2020;Sutherland et al, 2016). The pandemic enabled a diverse group of pediatric speech-language pathologists to finally experience this utility.…”

“…Although previous studies have collected information about the use of and the attitudes toward telehealth in the provision of pediatric speech-language services, most of these data were collected prior to the COVID-19 pandemic (Manning et al, 2020;Orlando et al, 2019). Prior to March 2020, telehealth research was limited due to the prolific barriers and restrictions creating underutilization of this delivery method within the pediatric population.…”

The Bilingual Home Language Boost Through the Lens of the COVID-19 Pandemic

“…Before posing our questions, it is important to point out that the testing modes are different between the two samples by necessity: one group was tested in person, the other remotely via video chat. However, emerging evidence suggests that the two testing modes are largely comparable in terms of child engagement, speech intelligibility, amount and characteristics of language elicited, and reliability of transcription and scoring ( Waite et al, 2010 ; Ciccia et al, 2011 ; Sutherland et al, 2017 ; Manning et al, 2020 ). It is also important to point out that the current comparisons are cross-sectional rather than longitudinal.…”

Online Developmental Research: Observations from Moderated Studies

“…Various stages of research such as participant recruitment (Amon, Campbell, Hawke, & Steinbeck, 2014;Gilligan, Kypri, & Bourke, 2014;Thornton, Batterham, Fassnacht, Kay-Lambkin, Calear, & Hunt, a Boğaziçi University, Department of Psychology, gunes.oner@boun.edu.tr, ORCID: 0000-0002-0312-6712 b Boğaziçi University, Department of Psychology, neslihan.oguz@boun.edu.tr, ORCID: 0000-0003-4980-0108 c Boğaziçi University, Department of Psychology, ozdes.cetin@boun.edu.tr, ORCID: 0000-0001-5007-1411 d Boğaziçi University, Department of Psychology, simge.ersonmez@boun.edu.tr, ORCID: 0000-0001-9212-1333 e Boğaziçi University, Department of Psychology, pinar.karan@boun.edu.tr, ORCID: 0000-0001-9895-0175 f Boğaziçi University, Department of Psychology, mahmut.kurupinar@boun.edu.tr, ORCID: 0000-0001-5376-1303 g Boğaziçi University, Department of Psychology, oya.serbest@boun.edu.tr, ORCID: 0000-0001-7442-4699 h Corresponding author: Boğaziçi University, Department of Psychology, gaye.soley@boun.edu.tr, ORCID: 0000-0002-6333-6017 2016), testing (e.g., Discoveries Online [Rhodes et al, 2020]; Lookit ; TheChildLab.com [Sheskin & Keil, 2018]), coding (Chouinard, Scott, & Cusack, 2019) and sharing the materials and data (e.g., the OSF [Foster & Deardorff, 2017], The Child Language Data Exchange System -CHILDES [MacWhinney, 2000]; databrary.org [Adolph, Gilmore, & Kennedy, 2017]) have increasingly moved to online platforms. While remote data collection has long been in place for other fields of psychology, developmental scientists started adopting online methods for data collection only in recent years (e.g., Chouinard et al, 2019;Chuey, Lockhart, Sheskin, & Keil, 2020;Johnston, Sheskin, & Keil, 2019;Kominsky, Gerstenberg, Pelz, Sheskin, Singmann, Schulz, & Keil, 2021;Leshin, Leslie, & Rhodes, 2020;Manning, Harpole, Harriott, Postolowicz, & Norton, 2020;Nussenbaum, Scheuplein, Phaneuf, Evans, & Hartley, 2020;Richardson, Sheskin, & Keil, 2021;Scott, Chu, & Shulz, 2017;Smith-Flores, Perez, Zhang, & Feigenson, 2021;Soley & Köseler, 2021;Tran, Cabral, Patel, & Cusack, 2017). COVID-19 pandemic conditions have further spread the use of online research tools in many fields, including developmental science.…”

“…For instance, studies done with children include short tasks (10-to-20 minutes) such as showing children animations and asking them questions about these (e.g., Chuey et al, 2020;Leshin et al, 2020) as well as longer tasks (e.g., 30-to-60 minutes) that require children to exhibit sustained concentration in order to make several repeated decisions (Nussenbaum et al, 2020). Online platforms are also used to collect language samples (Manning et al, 2020) and looking time data from infants (e.g., Smith-Flores et al, 2021). Importantly, through both moderated and unmoderated online studies, researchers seem to be able to collect data that are comparable to data from lab-based studies (e.g., Manning et al, 2020;Nussenbaum et al, 2020;Rhodes et al, 2020;Sheskin & Keil, 2018;Smith-Flores et al, 2021;Soley & Köseler, 2021), providing preliminary evidence about the reliability of these methods.…”