Higher-Order e-Assessment for Physics in the Digital Age Using Sakai

Abstract

In any science field, including physics, it is important to remain abreast of new assessment methods to cater to the 21st-century student. The rationale of this paper is to argue for a move away from the use of lower-order thinking skills (LOTS) in e-assessment in favor of higher-order thinking skills (HOTS), in line with Bloom’s Revised Taxonomy. This builds on the work of Livingston, who argued that multiple-choice questions, while presenting some advantages, also entail disadvantages since there are forms of knowledge and skills that they cannot evaluate. This is especially true when certain forms of knowledge and skills cannot be accurately evaluated if the answer is presented within a list of options. Livingston argued that constructed-response questions are a fruitful avenue for addressing this shortcoming. Similarly, as Jones points out, while multiple-choice questions can—with careful design—be used to assess higher-order thinking, it is far more common to find them being used to test factual information recall, which requires LOTS. Schultz echoes this view and made use of randomized non-multiple-choice assignments, delivered via a Learning Management System (LMS) in a chemistry course. The benefits reported include automatic marking, reduction of copying among students due to randomization, and the targeting of higher-order learning outcomes since students were required to work out the answer rather than choosing it from a list. While Schultz did not use the term constructed-response question, Livingston defines it as “questions that require the test taker to produce the answer, rather than simply choosing it from a list.” In this paper, we aim to illustrate a range of new possibilities (within physics) for using constructed-response questions in an increasingly technologically advanced learning and teaching environment. While we use the Sakai LMS, the examples we offer are equally valid for other systems. The use of constructed-response questions to support physics learning and teaching within Sakai is, however, an underresearched area. This is especially the case for the calculated and numeric response questions, and their complementary tools, that we showcase.

E-assessment and Academic Integrity: A Literature Review

“…In a similar cheating practice, many students simultaneously log on to the system, using the same user account, in order to co-solve the quiz [4]. To prevent mutual assistance between students in multiple choice tests, various countermeasures are applied, such as asking students different questions, changing the order of the answers and producing multiple variations of questions that differ in their numerical values [8,10]. The implementation of such countermeasures has been simplified through the use of Learning Management Systems (LMS) that facilitate the construction and delivering of online quizzes and the use of question banks.…”

“…Many question variations could be automatically produced by the LMS, once free variables take the place of constant numerical values. Particularly, the question is typed into the LMS, as a "calculated question" [8,10] such as the following example: «How long would it take for a bicycle to travel a distance of X km at a constant speed of Y km/h?». In the example, free variables (X and Y) replace the constant numerical values.…”

Maximizing the utility of online assessment tools in the pandemic era- A narrative review

“…10 f. For deeper engagement of students, case studies, patient videos, simulations, or other resources can be provided. 11 g. After the completion of the assessment, constructive feedback has to be provided to the students. 12 h. Proper planning should be done to maintain academic integrity.…”

“…18 e. To assess the higher-order thinking skills, casebased/problem based /videos/simulations/reflections can be added at the end of the quiz. 11,21 f. To prevent any technical glitch, more than one attempt can be allowed and the highest score can be included in the final grade.…”