Construct implications of including still image or video in computer-based listening tests

Abstract

Over the past decade, listening comprehension tests have been converting to computer-based tests that include visual input. However, little research is available to suggest how test takers engage with different types of visuals on such tests. The present study compared a series of still images to video in academic computer-based tests to determine how test takers engage with these two test modes. The study, which employed observations, retrospective reports and interviews, used data from university-level non-native speakers of English. The findings suggest that test takers engage differently with these two modes of delivery. Specifically, while test takers engaged minimally and similarly with the still images, there was wide variation in the ways and degree to which they engaged with the video stimulus. Implications of the study are that computer-based tests of listening comprehension could include still images with only minimally altering the construct that is measured by audio-only listening tests, but the utilization of video in such computer-based tests may require a rethinking of the listening construct.

Iranian EFL Students’ Listening Comprehension Problems

“…For example, when learners fail to recognize the overall organization of the ideas in a text, they have discoursal problems. Gruba (1997) and Ockey (2007) maintain that focusing on rhetorical signaling cues can help listeners understand the discoursal features of the text they are listening to. For instance, a competent listener would recognize the cue, "Let me put it in another way!"…”

Testing audiovisual comprehension tasks with questions embedded in videos as subtitles: a pilot multimethod study

“…Second, it is in harmony with common practice in the classroom (Buck, 2001;Gruba, 1997;PardoBallester, 2016). Third, it increases the validity of the test (Bejar, Douglas, Jamieson, Nissan, & Turner, 2000;Wagner, 2007Wagner, , 2008Wagner, , 2010a, its authenticity (Alderson, 2005;Bejar et al 2000;Ockey, 2007;Wagner, 2007Wagner, , 2008 and its naturalness (Alderson, 2005). Fourth, "seeing the situation and the participants tends to call up relevant schemes" (Buck, 2001, p. 172).…”

Test-takers' Attitudes toward Taking Pictorial and Visual Modalities of Listening Comprehension Test in an EFL Context

“…Ockey (2007) found that students shown the minimum amount of engagement while taking the video listening test. On the other hand, investigated the test-takers' viewing rate and found that students showed 69% eye contact with the visual materials while taking the listening test.…”

“…Brett (1997) argued that students became busy reading the test books and neglected watching the visual materials. Coniam (2001) and Ockey (2007) believed that sidetracking and distraction are other factors that affect test-takers in this kind of test. According to Buck (2001), the ability to comprehend the language similar to those of the target language domain is different from the ability to understand "subtle visual information" (p. 172).…”

“…However, incorporating these devices with listening comprehension test needs more consideration as it is suspected to be the reason for test-takers' distraction (MacWilliam, 1986; Bejar et al, 2000; Alderson, Clapham, and Wall, 1995; Brett, 1997;Coniam, 2001;Ockey, 2007;Buck, 2001).…”