The impact of an audience response system on a summative assessment, a controlled field study

Abstract

Background: Audience response systems allow to activate the audience and to receive a direct feedback of participants during lectures. Modern systems do not require any proprietary hardware anymore. Students can directly respond on their smartphone. Several studies reported about a high level of satisfaction of students when audience response systems are used, however their impact on learning success is still unclear. Methods: In order to evaluate the impact of an audience response system on the learning success we implemented the audience response system eduVote into a seminar series and performed a controlled crossover study on its impact on assessments. One hundred fifty-four students in nine groups were taught the same content. In four groups, eduVote was integrated for the first topic while five groups were taught this topic without the audience response systems. For a second topic, the groups were switched: Those groups who were taught before using eduVote were now taught without the audience response system and vice versa. We then analysed the impact of the audience response system on the students' performance in a summative assessment and specifically focused on questions dealing with the topic, for which the audience response system was used during teaching. We further assessed the students' perception on the use of eduVote using questionnaires. Results: In our controlled crossover study we could not confirm an impact of the audience response system eduVote on long-term persistence i.e. the students' performance in the summative assessment. Our evaluation revealed that students assessed the use of eduVote very positively, felt stronger engaged and better motivated to deal with the respective topics and would prefer their integration into additional courses as well. In particular we identified that students who feel uncomfortable with answering questions in front of others profit from the use of an audience response system during teaching. Conclusions: Audience response systems motivate and activate students and increase their engagement during classes. However, their impact on long-term persistence and summative assessments may be limited. Audience response systems, however, specifically allow activating students which cannot be reached by the traditional way of asking questions without such an anonymous tool.

Impact of Audience Response System in Enhancing Teaching of Anatomy and Physiology for Health Sciences Students at King Saud bin Abdulaziz University for Health Sciences

“…In a traditional lecture, it has been shown that the students’ attention substantially decreases after 20–25 minutes, 26 requiring an intervention such as posing lecture-based questions using ARS to get back the class’ attention. 10 Therefore, the current study, like others, found the ARS to be an efficient tool for getting enhance students’ attention. 13 , 18 , 19 This would help in providing the perceived benefit of learning and assisting students to retain their learning.…”

“…These results are in line with earlier studies that had introduced ARS in different teaching modalities like lectures, 19 case-based discussions 24 and seminars. 10 As has been seen in other studies, 8 , 19 , 24 the participation of the students also increased when ARS was introduced into anatomy and physiology lectures. This could be because the use of ARS enhanced students’ self-confidence and allowed them to gauge their abilities in comparison with their classmates.…”

“…This feature allowed students to overcome their fear of participation, as demonstrated by several researchers. 10 , 19 , 24 Moreover, the immediate discussion of the students’ answers in classroom provided the students with individualized and quick formative feedback during the learning process and about their learning progress compared to their classmates. The instant feedback, as shown in this study and in other studies, also allowed the instructors to clarify, repeat and rephrase the key points and/or concepts that students misunderstood the most during lectures.…”

“…The instant feedback, as shown in this study and in other studies, also allowed the instructors to clarify, repeat and rephrase the key points and/or concepts that students misunderstood the most during lectures. 10 , 11 , 24 Therefore, the instructors can employ the time of the class more effectively by spending more time reviewing the points and/or concepts that appear to be misunderstood by many students instead of responding to questions raised by individual students. Without ARS, the instructors might imprecisely estimate which lecture objectives require further and immediate review till the lecture is completed or the exam is conducted.…”

Unplugging Students: Utilizing Guided Technology Policies to Enhance Classroom Engagement

“…Most commonly these devices are used for formative assessment, which is related to improved performance on summative assessment, but they can also be utilized for summative assessment. (McKenzie & Ziemann, 2020;Schmidt et al, 2020). When ARS usage is linked to a portion of the student's final grade it results in increased attendance, and increased attendance is closely linked to the final course grade (Kay & LeSage, 2009).…”