A Tool for Comparing Mathematics Tasks from Paper-Based and Digital Environments

Abstract

Comparative studies on paper and pencil– and computer-based tests principally focus on statistical analysis of students’ performances. In educational assessment, comparing students’ performance (in terms of right or wrong results) does not imply a comparison of problem-solving processes followed by students. In this paper, we present a theoretical tool for task analysis that allows us to highlight how students’ problem-solving processes could change in switching from paper to computer format and how these changes could be affected by the use of one environment rather than another. In particular, the aim of our study lies in identifying a set of indexes to highlight possible consequences that specific changes in task formulation have, in terms of task comparability. Therefore, we propose an example of the use of the tool for comparing paper-based and computer-based tasks.

Computer-based assessment in mathematics

“…Even when simpler tools such as a ruler or protractor are integrated digitally, using them demands other skills than if the physical tools are used. If such tools are integrated in tests, students must be accustomed to using them to perform well (Lemmo, 2021). Harris et al (2021) showed that students' performance on interactive tasks was lower than on static tasks in the digital environment.…”

“…The vertical order in which text elements are usually arranged on ordinary sheets also works less well on a screen with its proportions, and tasks may therefore need to be structured differently, for example in columns. This restructuring may complicate reading because the student needs to coordinate information from different parts of the text in a way that they may not be used to (Lemmo, 2021).…”

The transition from school to university in mathematics education research: new trends and ideas from a systematic literature review

“…First, most data are collected through online surveys: this choice is 'not neutral'. Psychologist have discussed the impact of computer versus paper-pencil survey in collecting self-reports (Bates & Cox, 2008) and, in mathematics education, recent publications address the impact in students' performance in online versus pen and paper tasks (Lemmo, 2021).…”

A Teacher Training Project to Promote Mathematics Laboratory During the COVID-19 Health Crisis in Italy

“…In Italy, in many cases, the COVID-19 emergency has led to mere reproduction at a distance of what was previously done in the physical classroom (Bolondi, 2020), but this is not the same in all countries as reported by Drijvers et al (2021) considering Belgium, Germany, and Netherlands. The assessment practices in mathematics were also called for consideration: the same task, according to whether it is set in a classroom with paper and pencil or on a computer, can lead to different solution strategies and different difficulties for students (Lemmo, 2021;Puentedura, 2006). However, in many cases, the emergency has led to deep reflection and represented an opportunity for teachers to become involved and collaborate, identifying new approaches and new tools useful for teaching their subject.…”