Partial dictation is a measure of EFL listening proficiency that can be easily constructed, administered, and scored by EFL teachers. However, it is controversial whether this form of test measures lower-order abilities exclusively or involves both lower- and higher-order abilities. In order to answer this question, a study was designed to examine the difference between partial dictation and test forms believed to measure more higher-order abilities. In a series of confirmatory factor analyses, the simplex, second-order, and bi-factor models were fitted to the scores of 367 college-level EFL learners in China in a listening test composed of partial dictation, gap-filling and constructed response tasks. The bi-factor model was identified as the best-fitting and this supports the view that partial dictation measures the same construct as test forms believed to measure more higher-order abilities. Concomitant statistical analyses also showed that the partial dictation tasks were suited to the ability level of the test takers and had high internal consistency.

This study is an attempt to classify raters according to their weighting patterns and explore systematic differences between rater types in the rating process. In the context of an EFL speaking test, 126 raters were classified into three types-form-oriented, balanced, and content-oriented-through cluster analyses of their weighting patterns derived from a holistic judgment task. In subsequent verbal protocols, a smaller sample of raters from each type rated the performance of real test takers and justified their ratings. The procedure for classifying raters yielded reliable results that agreed broadly with self-perceived and actual weights of the raters in the rating process. Verbal protocol analyses found systematic differences across rater types in the distinction between criteria and conformity to rubric-implied criteria. The form-oriented raters were found to overweigh form-related criteria and underweigh content-related criteria, which suggested unbalanced construct representation and led to more halos. Preliminary evidence also suggested that the form-oriented raters might overrate digressive content due to lower sensitivity to digression.

No abstract

This study aims to resolve contradictory conclusions on the relative importance of lexical and syntactic knowledge in second language (L2) listening with evidence from academic English. It was hypothesized that when lexical and syntactic knowledge is measured in auditory receptive tasks contextualized in natural discourse, the measures will be more relevant to L2 listening, so that both lexical and syntactic knowledge will have unique contributions to L2 listening. To test this hypothesis, a quantitative study was designed, in which lexical and syntactic knowledge was measured via partial dictation, an auditory receptive task contextualized in a discourse context. Academic English listening was measured via a retired IELTS listening test. A group of 258 college-level native Chinese learners of English completed these tasks. Pearson correlations showed that both lexical and syntactic measures correlated strongly with English listening (r = 0.77 and r = 0.67 respectively). Hierarchical regression analyses showed that both measures jointly explained 62% of the variance in the listening score and that each measure had its unique contribution. These results demonstrated the importance of considering construct representation substantially and using measures that well reflect constructs in practical research.

No abstract