Investigation of Usage Frequency and Department on Usability Issues with the Online Examination System (OSSI)

Abstract

Usability is an important term for systems related to learning and teaching because usability issues can negatively affect student achievement. The aim of this study is to investigate the usability of a specific online examination system, OSSI, which is used by a public university for midterm and final exams. Researchers examined mean differences between System Usability Scale (SUS) scores of students from different departments with different usage frequencies. Usability issues were investigated by gathering students' ideas. Both quantitative and qualitative data procedures were employed. Quantitative data were analyzed through analysis of variance, while content analysis was applied to qualitative data. Scores showed no statistical mean difference between usage frequencies but did show a statistically significant mean difference between CEIT and other two departments. Qualitative analysis revealed the importance of screen size, placement of buttons, question formatting, and other screen elements. The results of this study contribute to the design and development of OSSI and other online examination systems.

Usability Evaluation of the Online Skill Assessment Tool

“…Moreover, although no matter how the system was designed in line with universal design principles, situations where features such as the prior knowledge and skill level of the target group come to the fore, and as in this study, they oblige the system to be designed specifically for the target users. As a matter of fact, Battal & Çağıltay (2015) emphasized that the user profile that will use the system according to the intended use of the system should also be taken into consideration while designing systems. Finally, for future research, it may be recommended to conduct studies on the usability evaluation of online assessment systems with larger samples, where different methods may be employed, and in-depth findings and indicators may be revealed.…”

“…In other words, it is aimed to reveal the progress made in line with the objectives of a course. Within this context, in the literature, it is possible to encounter studies on evaluation of the usability of various assessment systems developed for different purposes (Bayrak & Akcam, 2015;Battal & Çağıltay, 2015;Karahoca, Karaoca, & Günoğlu, 2009;Lai, Chen, & Chou, 2017;Majzub, 2009;Vairamuthu & Anouncia, 2016), even though the number of such studies is limited. However, when these studies were examined, it was seen that survey data mostly based on self-reported user opinions were used in the usability evaluation of assessment systems.…”

Assessment in online education during the COVID-19 pandemic: from the perspective of university instructors

“…Durak et al (2020) confirmed this issue by arguing that most universities in Turkey were found unprepared, reporting that instructors were "not ready for offering a course" in an online format (p. 789). The responsibility of institutions was not only to provide the necessary technical infrastructure and maintain a learning management system (Doğan et al, 2020) but also to provide usable online systems for the students and instructors (Battal & Çağıltay, 2015).…”

Milenyumdan koronavirüs dönemine Türkiye'de çevrimiçi sınavlar üzerinde yapılan çalışmalara ilişkin bir inceleme

“…In order to meet the expected needs, online exams should provide certain competencies such as being valid, reliable, fair and flexible (Al-Mashaqbeh ve Al Hamad, 2010). According to Battal and Çağıltay (2015), the exam system should be easy to access and use, [211] the questions should be free from distractions. In comparative studies on the exam types, the online exams were found to be close to the classical exams in terms of usefulness and reaching the purpose, and positive contributions were made to the learning process (Karakaya, 2001).…”