Evaluating security and usability of profile based challenge questions authentication in online examinations

Abstract

Student authentication in online learning environments is an increasingly challenging issue due to the inherent absence of physical interaction with online users and potential security threats to online examinations. This study is part of ongoing research on student authentication in online examinations evaluating the potential benefits of using challenge questions. The authors developed a Profile Based Authentication Framework (PBAF), which utilises challenge questions for students' authentication in online examinations. This paper examines the findings of an empirical study in which 23 participants used the PBAF including an abuse case security analysis of the PBAF approach. The overall usability analysis suggests that the PBAF is efficient, effective and usable. However, specific questions need replacement with suitable alternatives due to usability challenges. The results of the current research study suggest that memorability, clarity of questions, syntactic variation and question relevance can cause usability issues leading to authentication failure. A configurable traffic light system was designed and implemented to improve the usability of challenge questions. The security analysis indicates that the PBAF is resistant to informed guessing in general, however, specific questions were identified with security issues. The security analysis identifies challenge questions with potential risks of informed guessing by friends and colleagues. The study was performed with a small number of participants in a simulation online course and the results need to be verified in a real educational context on a larger sample size.

A study into the usability and security implications of text and image based challenge questions in the context of online examination

“…The usability analysis of text-based questions is not significantly different than other similar studies discussed in Table 1. While the effectiveness of text-based questions increased with improved question design from our previous study (Ullah et al 2014a), the issues contributing to incorrect answers such as syntactic variation, capitalization, incorrect spellings, memorability remained unchanged. In their study, Just and Aspinall (2009a) reported similar concerns for incorrect answers i.e.…”

“…We implemented an string-tostring comparison algorithm for authentication purposes (Schechter et al 2009;Ullah et al 2014a). Results in Table 4 show that, of the 890 text-based challenge questions randomly presented to students, 583 (66%) were answered correctly during the authentication process using an equality algorithm, which would increase to 74% if a more relaxed algorithm was implemented.…”

“…Our previous study revealed usability and security issues associated with the use of text-based challenge questions due to question design (Ullah et al 2014a). In response to the risks and usability issues identified, multiple-choice image questions were introduced for use in this study.…”

“…In their previous work, the authors conducted an online course to analyse usability, security and privacy factors in an online examination context (Ullah et al 2014a(Ullah et al , 2012b. The overall findings of the study reported positive outcomes.…”

“…& Questions Design: Questions reported with usability and security issues in a previous study (Ullah et al 2014a) were replaced with alternatives giving a careful design consideration to reduce ambiguity and clarity issues. Text-based and imagebased questions were classified into five themes: academic, favourite, personal, date and image.…”