Evaluating Language Environment Analysis System Performance for Chinese: A Pilot Study in Shanghai

Abstract

LENA adult word count and conversational turn provided reasonably accurate estimates for SDM over the age range tested. Theoretical and practical considerations regarding LENA performance in non-English languages are discussed. Despite the pilot nature and other limitations of the study, results are promising for broader cross-linguistic applications.

The LENA System Applied to Swedish: Reliability of the Adult Word Count Estimate

“…The reliability of the AWC as estimated by the LENA system when applied to Swedish was found to be within the range of its reliability for Spanish, Mandarin and French [2,15,16], and most similar to its reliability for French [16]. Further evaluations of other estimates and different aspects of the LENA system are crucial for any language to which it is applied, in order for it to be used appropriately in research and clinical applications.…”

“…Further, it is also necessary to evaluate the automatic segmentation that is the basis of the estimates [18]. There have been reports of instances where the LENA system categorized large portions of the audio recording erroneously, for example speech in a TV program being segmented as male speech [19] or an elderly woman being categorized as a child [15]. Any automated segmentation based on acoustic characteristics of the signal is expected to make some erroneous predictions.…”

“…The present study differs from several of previous evaluations of the AWC measure in terms of how segments used for analysis were selected. In most previous evaluations, segments have been selected based on high amounts of adult and/or child speech (estimated either by the LENA system or by an unspecified estimation procedure) [11,15,16]. In the evaluation of LENA applied to Spanish [2], segment selection was based on the AWC estimate, but the selection procedure ensured segments with low AWC were included.…”

“…The reliability (r = .80) was found to be within the range of that for American English [2]. Similarly, in the Mandarin evaluation, a correlation of r=.73 between transcribed Shanghai dialect words and the estimated AWC was found [15]. When applying the LENA system to French however, the correlation between the system's AWC and human word count was somewhat lower (r = .64) [16].…”

“…The LENA system has also been evaluated when applied to languages other than American English. The reliability of the AWC estimate specifically has so far been evaluated for Spanish, Mandarin and French [2,15,16]. In the evaluation of the AWC estimate of the LENA system applied to Spanish, 60 minutes out of ten recordings were transcribed, and the words in the transcription counted.…”