This study partially replicates Paquot’s (2018, 2019) study of phraseological complexity in L2 English by investigating how phraseological complexity compares across proficiency levels as well as how phraseological complexity measures relate to lexical, syntactic and morphological complexity measures in a corpus of L2 French argumentative essays. Phraseological complexity is operationalized as the diversity (root type-token ratio; RTTR) and sophistication (pointwise mutual information; PMI) of three types of grammatical dependencies: adjectival modifiers, adverbial modifiers and direct objects. Results reveal a significant increase in the mean PMI of direct objects and the RTTR of adjectival modifiers across proficiency levels. In addition to phraseological sophistication, important predictors of proficiency include measures of lexical diversity, lexical sophistication, syntactic (phrasal) complexity and morphological complexity. The results provide cross-linguistic validation for the results of Paquot (2018, 2019) and further highlight the importance of including phraseological measures in the current repertoire of L2 complexity measures.
The main objective of this Methods Showcase Article is to show how the technique of adaptive comparative judgment, coupled with a crowdsourcing approach, can offer practical solutions to reliability issues as well as to address the time and cost difficulties associated with a text-based approach to proficiency assessment in L2 research. We showcased this method by reporting on the methodological framework implemented in the Crowdsourcing Language Assessment Project and by presenting the results of a first study that demonstrated that a crowd is able to assess learner texts with high reliability. We found no effect of language skills or language assessment experience on the assessment task, but judges who had received formal language assessment training seemed to differ in their decisions from judges who had not received such training. However, the scores generated by the crowdsourced task exhibited a strong positive correlation with the rubric-based scores provided with the learner corpus used. Keywords learner corpus; language assessment; proficiency; adaptive comparative judgment; crowdsourcingThe Crowdsourcing Language Assessment Project (CLAP) project was developed within the framework of the Lexicogrammatical Complexity Across Mode T.0086.18 FNRS project. We thank Alex König (then at EURAC, Bolzano, Italy) for his technical help at the start of the project. We are grateful to our colleagues at UCLouvain for taking part in the pilot study and to all national and international colleagues who contributed to CLAP and/or provided feedback on the project. We also thank the reviewers for their very constructive and insightful comments. The usual disclaimers apply. We have no known conflict of interest to disclose.
Ce document est protégé par la loi sur le droit d'auteur. L'utilisation des services d'Érudit (y compris la reproduction) est assujettie à sa politique d'utilisation que vous pouvez consulter en ligne. https://apropos.erudit.org/fr/usagers/politique-dutilisation/ Cet article est diffusé et préservé par Érudit.Érudit est un consortium interuniversitaire sans but lucratif composé de l
This study builds upon previous research investigating the construct validity of phraseological complexity as an index of L2 development and proficiency. Whereas previous studies have focused on cross-sectional comparisons of written productions across proficiency levels, the current study compares the longitudinal development of phraseological complexity in written and oral productions elicited over a 21-month period from learners of French. We also improve upon the state of the art by including L1 data to benchmark learner levels of phraseological complexity. Phraseological complexity, operationalized as the diversity (no. types) and sophistication (PMI) of adjectival modifiers (adjective + noun) and direct objects (verb + noun), was generally higher in learner writing as compared to speaking. Over the study period, the sophistication of phraseological units increased slightly but developmental patterns were found to differ between tasks, highlighting the importance of considering task characteristics when measuring phraseological complexity.
This article reports on an open-source R package for the extraction of syntactic units from dependency-parsed French texts. To evaluate the reliability of the package, syntactic units were extracted from a corpus of L2 French and were compared to units extracted manually from the same corpus. The f-score of the extracted units ranged from 0.53–0.97. Although units were not always identical between the two methods, manual and automatically-derived syntactic complexity measures were strongly and significantly correlated (ρ = 0.62–0.97, p < 0.001), suggesting that this package may be a suitable replacement for manual annotation in some cases where manual annotation is not possible but that care should be used in interpreting the measures based on these units.
This study investigates whether re-thinking the separation of lexis and grammar in language testing could lead to more valid inferences about proficiency across modes. As argued by Römer, typical scoring rubrics ignore important information about proficiency encoded at the lexis–grammar interface, in particular how the co-selection of lexical and grammatical features is mediated by communicative function. This is especially evident when assessing oral versus written exam tasks, where the modality of a task may intersect with register-induced variation in linguistic output. This article presents the results of an empirical study in which we measured the diversity and sophistication of four-word lexical bundles extracted from a corpus of French proficiency exams. Analysis revealed that the diversity of noun-based bundles was a significant predictor of written proficiency scores and the sophistication of verb-based bundles was a significant predictor of proficiency scores across both modes, suggesting that communicative function as well as the constraints of online planning mediated the effect of lexicogrammatical phenomena on proficiency scores. Importantly, lexicogrammatical measures were better predictors of proficiency than solely lexical-based measures, which speaks to the potential utility of considering lexicogrammatical competence on scoring rubrics.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.