High-stakes online assessments: A case study of National Benchmark Tests during COVID-19

Abstract

Owing to the COVID-19 pandemic, paper-based delivery of the National Benchmark Tests (NBTs) was not possible during the 2020 testing cycle. The NBTs, being a large-scale national assessment project, did not have alternative options, other than to offer the tests online. Moving these high-stakes tests online meant that certain considerations had to be considered to retain the credibility and security of the tests, without compromising the validity and reliability of the scores. Digitising the paper-based NBTs required an innovative, flexible and robust solution, which promotes fairness and ensures the quality of testing is maintained, while in many ways remains comparable to the paper-based implementation. To deliver the NBTs online, the following important considerations needed to be addressed: test security and integrity, test candidate identification processes, the prevention of dishonest behaviour, test scheduling and timing and technical support. The online testing solution chosen integrates the following aspects: it 1) enables all candidates to take the same test at the same time; 2) ensures the quality and similarity in experience of test delivery for all candidates as far as possible; 3) prevents candidates from accessing other applications and devices during the test; 4) enables proctoring before, during and after the tests to encourage appropriate behaviour similar to that expected during paper-based tests; 5) provides live support to assist candidates to deal with technical challenges and to guide them through the test sessions and 6) processes and presents data and scores in the same way as for the paper-based tests. In this article, we analyse the integration and complexity of the online NBTs solution, the opportunities and challenges associated with this form of delivery and reflect on test candidates' and the team's experiences. We discuss components of online assessment and wish to argue that this is also relevant to high-stakes course assessments. This case study should help to refine the scope of further research and development in the use of online high-stakes assessments.

The feasibility of computerized adaptive testing of the national benchmark test: A simulation study

“…Africa has seen an increase in CBT, but it is still not ubiquitous. In South Africa, candidates across provinces are required to take paper-pencil tests; the NBT continues to deploy both online and paper-pencil tests (Sango et al, 2022). In contrast, many other large-scale testing programs in developed countries (Kimura, 2017;Veldkamp & Verschoor, 2019) have adopted computer-based tests.…”

“…In response to these challenges, CETAP announced on 25th July 2020 a transition to a highly secure computer-based assessment (CETAP, 2020). Evidence showing the shift to computer-based tests came from a recent study by Sango et al (2022), who demonstrated that NBTs have many restrictions that prevent them from being administered in a paper-based format, as they were before the COVID-19 pandemic. The study also submits that researchers do not anticipate ever returning to exclusively paper-based delivery, even if circumstances return to making paper delivery easier (Sango et al, 2022).…”