Estimating second language productive vocabulary size

Abstract

This study provides validity evidence for the Capture-Recapture (CR) method, borrowed from ecology, as a measure of second language (L2) productive vocabulary size (PVS). Two separate “captures” of productive vocabulary were taken using written word association tasks (WAT). At Time 1, 47 bilinguals provided at least 4 associates to each of 30 high-frequency stimulus words in English, their first language (L1), and in French, their L2. A few days later (Time 2), this procedure was repeated with a different set of stimulus words in each language. Since the WAT was used, both Lex30 and CR PVS scores were calculated in each language. Participants also completed an animacy judgment task assessing the speed and efficiency of lexical access. Results indicated that, in both languages, CR and Lex30 scores were significantly positively correlated (evidence of convergent validity). CR scores were also significantly larger in the L1, and correlated significantly with the speed of lexical access in the L2 (evidence of construct validity). These results point to the validity of the technique for estimating relative L2 PVS. However, CR scores are not a direct indication of absolute vocabulary size. A discussion of the method’s underlying assumptions and their implications for interpretation are provided.

Issues with the capture-recapture measure of vocabulary size

“…Two methods have been used to generate data for the CR measure, essay writing (Meara & Olmos Alcoy, 2010) and word association (Williams, Segalowitz & Leclair, 2014). Word frequency in text is well known to show a power-law distribution (indeed, the power-law PDF of word frequencies in text has its own name, Zipf 's Law, after G. K. Zipf, who calculated its parameters from a copy of Joyce's Ulysses).…”

“…In terms of word generation, essay writing is prototypically a power-law process, and this fact is well enough established that it need not be defended here. Williams, et al (2014) mention the lack of fit between the actual distribution of the sampled data and that required by the measure in their critique of the method of the Meara and Olmos Alcoy, however their use of word association data may not be a sufficient corrective: research suggests that word association is likewise a power-law process. Indeed, prior to the discovery of Zipf 's law, Skinner (1937) and Cook and Skinner (1939) found a power-law relation between rank and frequency in the Kent-Rosanoff free association norms and the Schellenberg free association norms, respectively.…”

“…1. Deleting high frequency words: This intervention assumes that the better performance observed in Williams, Segalowitz, and Leclair (2014) occurs because word association exposes only the content words to sampling. In the simulated data, this was approximated by deleting all tokens of the top 5% of most frequently occurring types (labeled 'minus 5%' in Table 1).…”

“…Unfortunately, when attempting to apply this measure to learner texts, Meara and Olmos Alcoy (2010) noted that their results were "ridiculously low, and clearly (could not) be interpreted at face value" (p. 221). There was a general understanding in both Meara and Olmos Alcoy (2010) and in subsequent work by Williams, Segalowitz and Leclair (2014) that the issue was related to the presence of a few high frequency words and very many low frequency words in the text samples. In the latter paper, an attempt was made to address this issue by using word association as the sampling procedure.…”

Exploring the construct validity of tests used to assess L2 productive vocabulary knowledge

“…However, the potential for a clean mapping of assessment measures onto these four potential constructs is belied by the characterizations and actual uses of many tests. Two examples are illustrative: (i) although there is agreement that Lex30 (Meara & Fitzpatrick, 2000) elicits productive vocabulary knowledge, disagreements arise as to whether it provides a measure of depth of vocabulary knowledge (see Read, 2004, p. 220) or vocabulary size (i.e., Williams, Segalowitz & Leclair, 2014); and, (ii) although the Productive Vocabulary Levels Test (PVLT, Laufer & Nation, 1999) was designed to reveal the size of an individual's productive vocabulary, given that participants must complete words presented in carrier sentences, Webb (2005, p. 82) suggests that it might "actually test receptive knowledge. "…”