Test-taker perception of and test performance on computer-delivered speaking tests: the mediational role of test-taking motivation

Abstract

Background: Research on the test-taker perception of assessments has been conducted under the assumption that negative test-taker perception may influence test performance by decreasing test-taking motivation. This assumption, however, has not been verified in the field of language testing. Building on expectancy-value theory, this study explored the relationships between test-taker perception, test-taking motivation, and test performance in the context of a computer-delivered speaking test. Methods: Sixty-four Japanese university students took the TOEIC Speaking test and completed a questionnaire that included statements about their test perception, testtaking motivation, and self-perceived test performance. Five students participated in follow-up interviews. Results: Questionnaire results showed that students regarded the TOEIC Speaking test positively in terms of test validity but showed reservations about computer delivery, and that they felt sufficiently motivated during the test. Interview results revealed various reasons for their reservations about computer delivery and factors that distracted them during the test. According to correlation analysis, the effects of testtaker perception and test-taking motivation seemed to be minimal on test performance, and participants' perception of computer delivery was directly related to test-taking effort, but their perception of test validity seemed to be related to testtaking effort only indirectly through the mediation of perceived test importance. Conclusion: Our findings not only provide empirical evidence for the relationship between test-taker perception and test performance but also highlight the importance of considering test-taker reactions in developing tests.

The Recurring Issue of Aviation English Test Validity: Echoes from Test-takers and Assessors of the English for Aviation Language Testing System in Algeria

“…The test-takers' dissatisfaction might be rooted, according to the authors, in the teachers' newly acquired expertise, resulting in confidence in their skills to work more effectively in the target situation; thus, claiming that PEAT did not meet their needs. Zhou and Yoshitomi (2019) investigated test-takers' perceptions of 64 Japanese university students who took the Test of English for International Communication (TOEIC). The researchers assumed that negative test-taker perception may influence students' test performance by decreasing test-taking motivation.…”

Video-conferencing speaking tests: do they measure the same construct as face-to-face tests?

“…Given its advantage of allowing for quicker and more generalisable comparisons among a large number of performances, the functions checklist has been modified and used with various other speaking tests, such as the IELTS Speaking test, Trinity's GESE and ISE examinations, and British Council's Aptis Speaking test (e.g. Brooks, 2003;Inoue & Nakatsuhara, forthcoming;Zhou et al, 2018). For example, Brooks (2003) made some adjustments to the checklist to suit the specific examiner/test-taker discourse of the IELTS Speaking test and provided validation evidence that supported the 2001 revision of that test, as well as demonstrating the usefulness of the checklist for the analysis of the IELTS Speaking test.…”

Interrogating the Construct of PRETCO-Oral: Longitudinal Evidence From Raters and Test-Takers

“…Aside from the rater stability, test takers' perceptions or attitudes should be deemed as a vital source of evidence for construct validity (Messick, 1989 ; Fan and Ji, 2014 ). For instance, test takers' perceptions might impact their performance on test tasks (Cheng, 2005 ), and extant studies show that both test candidates for high-stakes tests, such as TOEIC (Zhou and Yoshitomi, 2019 ) or IELTS (Rasti, 2009 ) and those of low-stakes school-based English test (Fan and Ji, 2014 ) react positively toward the construct of those tests mentioned. Furthermore, test takers' perceptions of assessment affect the measurement of the intended construct (Xie, 2011 ).…”