Assessing Lexical Psychological Properties in Second Language Production: A Dynamic Semantic Similarity Approach

Abstract

Previous studies of the lexical psycholinguistic properties (LPPs) in second language (L2) production have assessed the degree of an LPP dimension of an L2 corpus by computing the mean ratings of unique content words in the corpus for that dimension, without considering the possibility that learners at different proficiency levels may perceive the degree of that dimension of the same words differently. This study extended a dynamic semantic similarity algorithm to estimate the degree of five different LPP dimensions of several sub-corpora of the Education First-Cambridge Open Language Database representing L2 English learners at different proficiency levels. Our findings provide initial evidence for the validity of the algorithm for assessing the LPPs in L2 production and contribute useful insights into between-proficiency relationships and cross-proficiency differences in the LPPs in L2 production as well as the relationships among different LPP dimensions.

Assessing receptive vocabulary using state‑of‑the‑art natural language processing techniques

“…While lexical annotations that explore breadth, depth, and core lexical knowledge features have become commonplace in many studies of L2 performance (Grant & Ginther, 2000;Graesser et al, 2004;Koizumi & In'nami, 2013;Sundqvist, 2019), modeling lexical knowledge based on semantic features is rare (cf. Monteiro et al, 2021;Sun & Lu, 2021;Lu & Hu, 2021;Zhang et al, 2021). Additionally, little research has investigated links between receptive and productive vocabulary as found in this study (Meara, 2010).…”

“…In a more recent study, Lu and Hu (2021) explored contextual embeddings from BERT as a means of sense disambiguation and found that augmenting existing measures of lexical sophistication with sense-aware frequency counts improved predictive power for L2 English writing quality. Sun and Lu (2021) utilized a vector space model (fastText, Bojanowski et al 2017) to extrapolate psycholinguistic dimensions of unseen words from smaller sets of labelled lexemes (i.e., psycholinguistic databases). They then measured variation within these psycholinguistic properties in a large, longitudinal corpus (EFCAMDAT, Huang et al, 2017) and found that the tested word properties can be inferred from their positions in a vector space model.…”

Evaluating Familiarity Ratings of Domain Concepts with Interpretable Machine Learning: A Comparative Study

“…Likewise, when evaluating word representation with lexical and semantic knowledge, regression models can be constructed from part of a psycholinguistic database based on human judgment ratings. The remaining database can then be tested with these models [16][17][18][19]. However, no approaches exist to reliably assess familiarity ratings for domain concepts that involve compound words or phrases and include rich semantic information.…”