New and not so new methods for assessing oral communication

Abstract

The assessment of oral communication has continued to evolve over the past few decades. The construct being assessed has broadened to include interactional competence, and technology has played a role in the types of tasks that are currently popular. In this paper, we discuss the factors that affect the process of oral communication assessment, current conceptualizations of the construct to be assessed, and five tasks that are used to assess this construct. These tasks include oral proficiency interviews, paired/group oral discussion tasks, simulated tasks, integrated oral communication tasks, and elicited imitation tasks. We evaluate these tasks based on current conceptualizations of the construct of oral communication, and conclude that they do not assess a broad construct of oral communication equally. Based on our evaluation, we advise test developers to consider the aspects of oral communication that they aim to include or exclude in their assessment when they select one of these task types. ABSTRACTThe assessment of oral communication has continued to evolve over the past few decades. The construct being assessed has broadened to include interactional competence, and technology has played a role in the types of tasks that are currently popular. In this paper, we discuss the factors that affect the process of oral communication assessment, current conceptualizations of the construct to be assessed, and five tasks that are used to assess this construct. These tasks include oral proficiency interviews, paired/group oral discussion tasks, simulated tasks, integrated oral communication tasks, and elicited imitation tasks. We evaluate these tasks based on current conceptualizations of the construct of oral communication, and conclude that they do not assess a broad construct of oral communication equally. Based on our evaluation, we advise test developers to consider the aspects of oral communication that they aim to include or exclude in their assessment when they select one of these task types.

Post hoc evaluation of analytic rating scales for improved functioning in the assessment of interactive L2 speaking ability

“…With more attention being paid to dialogic speaking tasks conducted in pairs or groups in recent years, researchers such as Nitta and Nakatsuhara (2014) have proposed adding measures of interactional competence or interaction, such as turn length, to their discourse analytic measures in an attempt to better represent the co-constructed nature of dialogic speech. Put another way, Ockey and Li (2015) characterized interactional competence as "an individual's underlying ability to actively structure appropriate speech in response to incoming stimuli, such as information from another speaker, in real time" (p. 5). As such, interactional competence encompasses more than merely the ability to take turns, it also involves the ability to appropriately engage with others and develop topics in a given context.…”

Approaches and Challenges to Assessing Oral Communication on Japanese Entrance Exams

“…The construct validity of both the face-to-face and the computer-delivered paired or group oral assessments are both potentially quite high. All aspects of Ockey and Li's (2015) construct could be assessed by this task type. These task types are designed to elicit aspects of interactional competence via the interaction among the test takers.…”

“…The construct validity of computer-delivered and computer-scored oral communication assessments is quite low compared to other direct assessments of oral communication. The task types that are typically used with this format have been criticized for assessing a narrow construct of oral communication (Chun, 2006;O'Sullivan, 2013;Ockey & Li, 2015), which does not include interactional competence. Computer-scored speaking assessments typically rely on measuring features such as speech rate, latency of response, length and position of pauses, word stress, segmental forms of words, and a comparison of the vocabulary with words used by highly proficient speakers (Ockey, 2009a), and, as a result, fail to assess interactional competence to a substantial extent.…”

“…Computer-scored speaking assessments typically rely on measuring features such as speech rate, latency of response, length and position of pauses, word stress, segmental forms of words, and a comparison of the vocabulary with words used by highly proficient speakers (Ockey, 2009a), and, as a result, fail to assess interactional competence to a substantial extent. In addition, tasks, such as sentence repeat have been criticized on the grounds that test takers may be able to imitate spoken sentences without being able to segment the speech into meaningful units, calling into question their efficacy in assessing pronunciation and fluency, two important sub-constructs of oral communication ability (Ockey & Li, 2015).…”

Evaluating interactional competence in interview and paired discussion tasks: A rater cognition study

“…However, this interview format is not likely to elicit authentic discourse as in a conversation (Van Lier, 1989). It is also unlikely to measure some aspects of IC, like taking turns, opening and closing gambits, and developing topics with appropriate pragmatic use (Ockey & Li, 2015).To that end, the development of more authentic task types, such as paired or group assessments, where two or more test-takers engage in a task together without an examiner's involvement, is necessary. This is important because "the oversimplified view on human interactions taken by the proficiency movement can impair and even prevent the attainment of true interactional competence within a cross-cultural framework and jeopardize our chances of contributing to interactional understanding" (Kramsch, 1986, p. 367).…”