A Comparison of Spoken and Written Language Use in Traditional andTechnology‐MediatedLearning Environments

Abstract

A key piece of a validity argument for a language assessment tool is clear overlap between assessment tasks and the target language use (TLU) domain (i.e., the domain description inference). The TOEFL 2000 Spoken and Written Academic Language (T2K‐SWAL) corpus, which represents a variety of academic registers and disciplines in traditional learning environments (e.g., lectures, office hours, textbooks, course packs), has served as an important foundation for the TOEFL iBT® test's domain description inference for more than 15 years. There are, however, signs that the characteristics of the registers that students encounter may be changing. Increasingly, typical university courses include technology‐mediated learning environments (TMLEs), such as those represented by course management software and other online educational tools. To ensure that the characteristics of TOEFL iBT test tasks continue to align with the TLU domain, it is important to analyze the registers that are typically encountered in TMLEs. In this study, we address this issue by collecting a relatively large (4.5 million words) corpus of spoken and written TMLE registers across the six primary disciplines represented in T2K‐SWAL. This corpus was subsequently tagged for a wide variety of linguistic features, and a multidimensional analysis was conducted to compare and contrast written and spoken language in TMLE and T2K‐SWAL. The results indicate that although some similarities exist across spoken and written texts in traditional learning environments and TMLEs, language use also differs across learning environments (and modes) with regard to key linguistic dimensions.

Register variation in spoken and written language use across technology-mediated and non-technology-mediated learning environments

“…In order to investigate the language used in TMLEs, Kyle et al (2021) collected a relatively large corpus (4.5 million words) of texts that represent various TMLEs from students and instructors at public universities and from publicly available online courses (i.e., massive open online course or MOOCs). They then used the TMLE corpus and the T2K-SWAL corpus (2.8 million words) to compare and contrast the lexicogrammatical features of language used in written and spoken university registers in technology-mediated and non-technology-mediated learning environments using MDA.…”

“…However, typical academic learning environments have been increasingly technology-mediated (e.g., through the affordances of course management software; Jacoby, 2014;Means et al, 2013), a trend that has increased dramatically due to the COVID-19 pandemic. Recently, Kyle et al (2021) collected a corpus of language encountered in technologymediated learning environments (TMLEs) and compared the linguistic features of spoken and written language across TMLEs and non-TMLEs using multidimensional analysis (MDA; Biber, 1988). Their analysis indicated that TMLEs and non-TMLEs demonstrated similar general patterns of language use across spoken and written modes (e.g., texts in spoken modes tended to include more clausal coordination, contractions, and emphatics while texts in written modes tended to include longer words and more phrasal coordination).…”

“…Within modes, however, substantial differences were found across learning environments, indicating that the language produced in non-TMLEs is not necessarily representative of the language produced in TMLEs. One limitation of Kyle et al (2021) is that spoken and written modes were analyzed without regard to more specific registers, and accordingly, the distribution of dimension scores varied widely. As language is known to vary based on register (e.g., Biber et al, 2004), it is important to determine the degree to which the variation identified in Kyle et al (2021) is due to particular registers.…”

“…One limitation of Kyle et al (2021) is that spoken and written modes were analyzed without regard to more specific registers, and accordingly, the distribution of dimension scores varied widely. As language is known to vary based on register (e.g., Biber et al, 2004), it is important to determine the degree to which the variation identified in Kyle et al (2021) is due to particular registers. This study builds on Kyle et al (2021) by exploring variation across registers in technology-mediated and non-technology-mediated learning environments.…”

“…As language is known to vary based on register (e.g., Biber et al, 2004), it is important to determine the degree to which the variation identified in Kyle et al (2021) is due to particular registers. This study builds on Kyle et al (2021) by exploring variation across registers in technology-mediated and non-technology-mediated learning environments.…”