Assessing Speaking Proficiency: A Narrative Review of Speaking Assessment Research Within the Argument-Based Validation Framework

Abstract

This paper provides a narrative review of empirical research on the assessment of speaking proficiency published in selected journals in the field of language assessment. A total of 104 published articles on speaking assessment were collected and systematically analyzed within an argument-based validation framework (Chapelle et al., 2008). We examined how the published research is represented in the six inferences of this framework, the topics that were covered by each article, and the research methods that were employed in collecting the backings to support the assumptions underlying each inference. Our analysis results revealed that: (a) most of the collected articles could be categorized into the three inferences of evaluation, generalization, and explanation; (b) the topics most frequently explored by speaking assessment researchers included the constructs of speaking ability, rater effects, and factors that affect spoken performance, among others; (c) quantitative methods were more frequently employed to interrogate the inferences of evaluation and generalization whereas qualitative methods were more frequently utilized to investigate the explanation inference. The paper concludes with a discussion of the implications of this study in relation to gaining a more nuanced understanding of task-or domain-specific speaking abilities, understanding speaking assessment in classroom contexts, and strengthening the interfaces between speaking assessment, and teaching and learning practices.

An Extensive Knowledge Mapping Review of Measurement and Validity in Language Assessment and SLA Research

“…Bachman ( 2005 ) stressed that, before using an assessment for decision-making purposes, a validity argument should be fully fledged in terms of evidence supporting test developers' claims. On the other hand, empirical validation studies have demonstrated that collecting such evidence to establish an all-encompassing validity argument is an arduous and logistically complex task (Chapelle et al, 2008 ; Aryadoust, 2013 ; Fan and Yan, 2020 ). We are, hence, keen to determine the extent to which language assessment and SLA studies involving measurement and assessment have fulfilled the requirements of validation in the research clusters that are identified through DCA.…”

Perspectives of Test Examiners of the Localized Speaking Assessment Framework: A Case Study in Vietnam

“…To sum up, although numerous factors such as rating scales, task characteristics, and rater effects have been reported in previous studies as influential variables to speaking assessment quality (Fan & Yan, 2020;McNamara et al, 2019;O'Sullivan, 2002), very few studies provide insights into test examiners' perspectives about these factors. Besides, qualitative aspects of VSTEP have not been extensively reported in the literature (Nguyen, 2015).…”

“…Speaking assessment practices can be affected by diverse factors such as task and interlocutor characteristics, test validity and reliability, assessment criteria (Fan & Yan, 2020;Kang & Wang, 2014), rater effects ( McNamara et al, 2019), and rater training (Kang et al, 2019). Studies have shown that interaction tasks involving interactions with examiners can be unnatural compared to paired or oral group tests, in which the test taker interacts Pertanika J.…”

Language Assessment for Immigration: A Review of Validation Research Over the Last Two Decades

“…To collect papers, the method of preferred reporting items for systematic reviews and meta-analyses (PRISMA) was adopted ( Moher et al, 2009 ; Shintani and Wallace, 2014 ; Fan and Yan, 2020 ). The literature search was intended to identify published studies investigating the validity issues of language tests for immigration purposes.…”