Online assessment of narrative macrostructure in adult Irish-English multilinguals

Abstract

BackgroundOnline assessment of narrative production and comprehension became an important component of language assessment during the COVID-19 pandemic. This study aimed to establish quantitative measures of narrative macrostructure in the production and comprehension of adult Irish-English bilinguals in an online assessment.MethodsA total of 30 Irish-English bilingual adults participated in an online assessment of oral narrative production and comprehension. Narratives were elicited using LITMUS-MAIN for Irish and English. Story-tell elicitation method was used for all stories. Twenty participants produced Baby Birds and Baby Goats story pairs while 10 participants produced Cat and Dog story pairs. Quantitative measures of story structure, comprehension score, and the overall number of Internal State Terms (ISTs) in production and comprehension were compared across the story pairs, languages, and the output type (production vs. comprehension).ResultsA general linear model indicated no differences in either story structure or story comprehension scores across languages for both sets of stories. Combined analysis for all participants and stories indicated no difference in the story structure scores or comprehension scores across the languages or the story pairs. While the overall number of ISTs was the same across languages, a higher number of ISTs was observed in comprehension relative to production in both languages for Cat and Dog story pair only, but not for Baby birds and Baby goats' stories. The major benefit of using online assessment was the accessibility of participants. The major drawback was the inability to control the environment and the quality of the internet connection.Conclusion and implicationsWhile online assessment increased the availability of participants, which is a significant factor in rural Ireland characterized by low population density and the high percentage of Irish speakers, the availability of stable internet connection limited the applicability of online assessment. Measures of narrative macrostructure were stable across the languages and the story pairs. This is important because of high variability in exposure to Irish, frequent code-switching, and a high number of morphosyntactic errors due to rapid language change that characterizes Irish-English bilinguals. Identifying reliable measures of language performance for Irish-English adult speakers is an important step toward establishing developmental norms for Irish-English bilinguals.

Adapting MAIN to eliciting stories from adults and in a remote context: What do we have to consider, and what do we know?

“…Its pictorial representation was originally developed for children between the ages of 3 and 10 (cf. Gagarina, Klop et al, 2019, p. 1) but later was successfully used with teenagers and adults (Antonijević et al, 2022;Gagarina et al, 2019b;Jažić et al, 2023;Krasnoshchekova & Kashleva, 2019;Mattiuada et al, 2022;Vogels & Lindgren, 2022). However, it also requires a background view, which I want to provide here.…”

“…Thus, there is essential comparability. In addition, there are studies collecting data from adults and using them as a control group (e.g., Hržica & Kuvač Kraljević, 2022;Vogels & Lindgren, 2022) and two important studies focusing on adult's narratives elicited with the help of MAIN in a remote context (Antonijević et al, 2022;Jažić et al, 2023). Antonijević et al (2022) aimed to establish measures of macrostructure in narratives of Irish-English bilingual adults to create a baseline for comparison with narratives of respectively bilingual children.…”

“…In addition, there are studies collecting data from adults and using them as a control group (e.g., Hržica & Kuvač Kraljević, 2022;Vogels & Lindgren, 2022) and two important studies focusing on adult's narratives elicited with the help of MAIN in a remote context (Antonijević et al, 2022;Jažić et al, 2023). Antonijević et al (2022) aimed to establish measures of macrostructure in narratives of Irish-English bilingual adults to create a baseline for comparison with narratives of respectively bilingual children. Therefore, they tested 30 Irish-English speaking adults (aged 22-59 years) via Zoom with the help of an unpublished "custom-made PowerPoint presentation embedding the 6 pictures for each story [..] [to] share the pictures with participants."…”

Multilingual Assessment Instrument for Narratives: Recent developments and new language adaptations

“…Its pictorial representation was originally developed for children between the ages of 3 and 10 (cf. Gagarina, Klop et al, 2019, p. 1) but later was successfully used with teenagers and adults (Antonijević et al, 2022;Gagarina et al, 2019b;Jažić et al, 2023;Krasnoshchekova & Kashleva, 2019;Mattiuada et al, 2022;Vogels & Lindgren, 2022). However, it also requires a background view, which I want to provide here.…”

“…Thus, there is essential comparability. In addition, there are studies collecting data from adults and using them as a control group (e.g., Hržica & Kuvač Kraljević, 2022;Vogels & Lindgren, 2022) and two important studies focusing on adult's narratives elicited with the help of MAIN in a remote context (Antonijević et al, 2022;Jažić et al, 2023). Antonijević et al (2022) aimed to establish measures of macrostructure in narratives of Irish-English bilingual adults to create a baseline for comparison with narratives of respectively bilingual children.…”