Assessing Language Using Computer Technology

Abstract

In this article, we propose to follow up on the most recent ARAL survey article on trends in computer-based second language assessment (Jamieson, 2005) and review developments in the use of technology in the creation, delivery, and scoring of language tests. We will discuss the promise and threats associated with computer-based language testing, including the language construct in relation to computer-based delivery and response technologies; computer-based authoring options; current developments; scoring, feedback, and reporting systems; and validation issues.

Computer‐Assisted Language Testing

“…The use of multimedia, which may incorporate audio, images, videos, animation, and graphics, has gained much attention among researchers because it is believed to have the potential for enhancing the authenticity of language tasks. However, Douglas and Hegelheimer (2007) warn that this issue is not as straightforward as it might first seem because the implementation of multimedia in computer-assisted language tests results in a more complex construct to measure, which, in turn, poses a threat to test validity.…”

“…In particular, a noticeable amount of discussion in the literature has been dedicated to the issues plaguing computer-adaptive testing, which, according to some researchers, led to the decline of its popularity, especially in large scale assessment (e.g., Douglas & Hegelheimer, 2007;Ockey, 2009). Of primary concern for CATs is the security of test items (Wainer & Eignor, 2000).…”

“…Although the use of multimedia input is believed to result in a greater level of authenticity in test tasks by providing more realistic content and contextualization cues, it remains unclear how the inclusion of multimedia affects the L2 construct being measured by CBTs (Jamieson, 2005). Some researchers even question the extent to which multimedia enhances the authenticity of tests (e.g., Douglas & Hegelheimer, 2007) since comparative studies on the role of multimedia in language assessment have yielded mixed results (see Ginther, 2002;Wagner, 2007;Suvorov, 2009). With regards to integrated tasks, their implementation in CBTs is generally viewed favorably because such tasks seem to better reflect what test takers would be required to do in real-life situations.…”

Actitudes docentes hacia los ejercicios de la Prueba de Acceso a la Universidad informatizada

“…Las razones por las que el formato informatizado es cada vez más común son la economía y practicidad para las administraciones encargadas de los mismos y la capacidad, cada vez mayor, de realizar tareas de evaluación de manera rápida y eficiente (CHAPELLE; DOUGLAS, 2006;ENRIGHT;JAMIESON, 2007;WANG, 2007), difícilmente realizables de otra manera (CHOI; KIM; BOO, 2003;IN'NAMI;KOIZUMI, 2009) y sin que la seguridad del sistema empleado se vea comprometida. Uno de los aspectos más destacados es la versatilidad que el software de evaluación del conocimiento de una lengua extranjera ha alcanzado en los últimos cinco años (DOOEY, 2008), en parte gracias al hecho de que se pueda realizar cierto tipo pruebas de evaluación de manera eficiente y a costes reducidos (por ejemplo, ciertas tareas orales), factores que han dado lugar a la necesidad del uso de la informática en la evaluación MARTÍN MONJE, 2010;2012;.…”

Analysing Test-Takers’ Views on a Computer-Based Speaking Test

“…Indeed, the more individual view of competence highlighted in computer-delivered tests of oral proficiency seems to contradict the social oriented view of communicative performance as a jointly constructed event involving interaction between individuals (Bachman & Palmer, 1996;Chalhoub-Deville, 2003;Kramsch, 1986;McNamara, 1997). As Douglas and Hegelheimer (2007) explain, computer-based tests cannot currently capture the complexity of natural language use. Furthermore, this focus on individual competence rather than on interactional competence (Kramsch, 1986;May, 2009) may have a negative influence or washback effect on current communicative language teaching practices (Amengual-Pizarro, 2009;Green, 2013;May, 2009).…”