Measurement Properties of a Standardized Elicited Imitation Test: An Integrative Data Analysis

Abstract

Elicited Imitation Tests (EITs) are commonly used in second language acquisition (SLA)/bilingualism research contexts to assess the general oral proficiency of study participants. While previous studies have provided valuable EIT construct-related validity evidence, some key gaps remain. This study uses an integrative data analysis to further probe the validity of the Korean EIT score interpretations by examining the performances of 318 Korean learners (198 second language, 79 foreign language, and 41 heritage) on the Korean EIT scored by five different raters. Expanding on previous EIT validation efforts, this study (a) examined both inter-rater reliability and differences in rater severity, (b) explored measurement bias across subpopulations of language learners, (c) identified relevant linguistic features which relate to item difficulty, and (d) provided a norm-referenced interpretation for Korean EIT scores. Overall, findings suggest that the Korean EIT can be used in diverse SLA/bilingualism research contexts, as it measures ability similarly across subgroups and raters.

Elicited Imitation Tasks as a Measure of L2 Proficiency

“…It is obvious that in the current landscape, there is a prevalence of studies that have centered around adult, college-level learners of English as a second language. To provide a more trustworthy and robust picture of the usefulness of EIT to assess L2 proficiency, future studies need to expand their research into other contexts and participant demographics (see Andringa & Godfroid, 2020; Plonsky, 2017; Isbell & Son, 2021). Additionally, studies that focus on target languages other than English will not only help us understand the appropriateness of elicited imitation for learners of these languages but also contribute to the research on EIT as a functional assessment instrument independent of the L2.…”

“…To bolster the validity argument for EIT as a trustworthy measure of L2 proficiency, L2 research has largely focused on examining the relationship between learners’ performance on EI tasks and more established measures of proficiency. Several EIT validation studies found significant positive correlations between learners’ EIT scores and scores on standardized proficiency measures with correlations ranging from 0.62 for IELTS (Erlam, 2006) to 0.76 for Test of Proficiency in Korean (TOPIK) (Kim et al, 2016; see also Isbell & Son, 2021). Similarly, strong and positive correlations from r = 0.87 to 0.91 have been found between EIT and Simulated Oral Proficiency Interviews (SOPI), particularly in L2 Spanish (Bowden, 2016; Ortega, 2000; Ortega et al, 2002).…”

(Why) Are Open Research Practices the Future for the Study of Language Learning?

“…In another kind of activity, researchers are beginning to conduct integrative data analyses-pooling related data collected by the same elicitation technique though not synchronously. Although not a new concept (Bauer & Hussong, 2009), it has rarely been used in the language sciences (cf., Isbell & Son, 2022). Bringing these endeavors together, we envisage freely available and accessible data collection tools (such as measures of individual differences) that output full datasets to facilitate the large-scale accumulation of ethically compliant open datasets that are tagged with metadata and analyzable by those who did not collect the data.…”

(Why) Are Open Research Practices the Future for the Study of Language Learning?

“…In another kind of activity, researchers are beginning to conduct integrative data analyses -pooling related data collected by the same elicitation technique though not synchronously. Although not a new concept (Bauer & Hussong, 2009), it has rarely been used in the language sciences (cf., Isbell & Son, 2022). Bringing these endeavours together, we envisage freely available and accessible data collection tools (such as measures of individual differences) that output full datasets to facilitate the large-scale accumulation of ethically-compliant open datasets that are tagged with metadata and analysable by those who did not collect the data.…”