Comparing Direct and Semi-Direct Modes for Speaking Assessment: Affective Effects on Test Takers

Abstract

In recent decades, with an increasing application of computer technology to the delivery of oral language proficiency assessment, there have been renewed debates over the appropriateness of two different testing modes, namely, (a) face-to-face, or direct, testing, and (b) person-to-machine, or semi-direct, testing. Previous research conducted in various geographical regions has produced mixed results as to which format is more acceptable to test takers. The present investigation, carried out in the context of a university setting in Hong Kong, compared the popularity of two testing modes by analyzing reactions and perceptions of a group of test takers who had just sat for a direct test as well as a semi-direct test. The results indicate that, although a large proportion of the participants had no particular preference in terms of the testing mode, the number of participants who strongly favored direct testing far exceeded the number strongly favoring semi-direct testing. The participants' main reason cited for disliking semi-direct testing was its inability for the examiner and examinee to interact during the test, which appears to have created a psychological barrier for the test taker. BACKGROUND

Analysing Test-Takers’ Views on a Computer-Based Speaking Test

“…Similar to previous research findings (Qian, 2009;Shohamy, 1994), face-to-face speaking tests drew the most positive results on this aspect. Thus, 75% of test-takers favoured face-to-face speaking tests over technology-mediated speaking tests since, as previously anticipated, oral communication is mainly regarded as a human phenomenon (Kenyon & Malabonga, 2001).…”

“…The main reasons provided by candidates in this respect were mostly related to the interactional nature of conversation (i.e., use of real or "authentic" language, importance of gestures and body language, and provision of feedback), which could not be appropriately captured in technologymediated assessment (Douglas & Hegelheimer, 2007;Kenyon & Malabonga, 2001). These findings point to the importance attached by candidates to interpersonal cues and to the negotiation of meaning between interlocutors to interact and reach communicative goals (Ockey, 2009;Qian, 2009). A minority of test-takers also commented that they felt disadvantaged by talking into a computer since computer-based tests could present some logistical or technical problems such as sound or audio problems.…”

“…In this regard, numerous researchers point out that the advantages of using computer-based technology can outweigh some of its limitations (García-Laborda, 2007;García-Laborda, Magal Royo, Litzler, & Giménez López, 2014) as well as the positive attitude of many test takers (Litzler & García-Laborda, 2016). Under this perspective, computer language testing is presented as a feasible alternative to other traditional methods of testing oral skills such as face-to-face assessment since it clearly facilitates test administration and delivery by reducing testing time and costs (Araújo, 2010;Bulut & Kan, 2012;García-Laborda, 2007;Qian, 2009). Furthermore, various research studies provide evidence of score equivalence between the two types of delivery modes (computer-based tests vs. face-to-face tests) on the testing of oral skills (Shohamy, 1994;Zhou, 2015).…”

Does a Speaking Task Affect Second Language Comprehensibility?

“…The IELTS task exemplified a dialogic, direct test which assessed speaking through face‐to‐face oral communication with an interviewer, although the interviewer's role was limited. In contrast, the TOEFL task was a monologic, semi‐direct test, which included a machine‐mediated assessment involving a test taker speaking into a recording device (Ellis, ; Qian, ). In sum, the tasks differed in the cognitive load they imposed on the speaker, with the IELTS task being a less demanding face‐to‐face speaking task, compared with the TOEFL task, which not only placed higher cognitive demands on speakers but also engaged them in monologic performance, without the need to interact with a human interlocutor.…”

The Effect of Mode of Response on a Semidirect Test of Oral Proficiency

“…Currently only a few studies have included an analysis of test takers' perceptions. Test takers were generally found to be in favor of human interaction in oral proficiency assessments (e.g., Dean, 2008;Qian, 2009).…”