On the Development of a Computer-Based Tool for Formative Student Assessment: Epistemological, Methodological, and Practical Issues

Abstract

Formative assessments in schools have the potential to improve students’ learning outcomes and self-regulation skills; they make learning visible and provide evidence-based guidelines for setting up and pursuing individual learning goals. With the recent introduction of the computer-based formative assessment systems for the educational contexts, there is much hope that such systems will provide teachers and students with valuable information to guide the learning process without taking much time from teaching and learning to spend on generating, evaluating and interpreting assessments. In this paper, we combine the theoretical and applied perspectives by addressing (a) the epistemological aspects of the formative assessment, with an emphasis on data collection, model building, and interpretation; (b) the methodological challenges of providing feedback in the context of instruction in the classroom; and (c) practical requirements for and related challenges of setting up and delivering the assessment system to a large number of students. In the epistemological section, we develop and explicate the interpretive argument of formative assessment and discuss the challenges of obtaining data with high validity. From the methodological perspective, we argue that computer-based formative assessment systems are generally superior to the traditional methods of providing feedback in the classroom, as they better allow supporting inferences of the interpretive argument. In the section on practical requirements, we first introduce an existing computer-based formative assessment system, as a case in point, for discussing related practical challenges. Topics covered in this section comprise the specifications of assessment content, the calibration and maintenance of the item bank, challenges concerning teachers’ and students’ assessment literacy, as well as ethical and data-protection requirements. We conclude with an outlook on possible future directions for computer-based formative assessment systems and the field in general.

Informatics literacy among first-year students of medicine, dentistry and pharmacy at the University of Nis in compliance with European Computer Driving Licence

“…Despite an upbringing in a digitized world, many students still lack some basic computing skills [29,30]. Recent surveys in different countries have also declared that biomedical sciences students' IL was unsatisfactory [31,32,33]. IL skills have become a necessity and an integral part of preparing tomorrow's doctors to be sufficiently competent to use informatics resources effectively and efficiently for the best practice of biomedical sciences [34,35].…”

Development and Validation of a Vertical Scale for Formative Assessment in Mathematics

“…In spring 2017 and 2018, we administered the 35 mathematics booklets to a sample of 2,733 students from schools in Northwestern Switzerland. The mathematics booklets were available for teachers besides other assessment templates in a first version of the computer-based assessment system MINDSTEPS (Tomasik et al, 2018). Teachers were invited to administer the booklets to their students during school lessons to support the calibration of the system's item pool.…”

“…Modern computer technology can be used as a tool for providing formative feedback in classrooms on a regular basis (e.g., Hattie and Brown, 2007;Brown, 2013). It allows for implementing complex measurement models and itemselection algorithms that support teachers in providing objective, reliable, and valid feedback (e.g., Glas and Geerlings, 2009;Wauters et al, 2010;Tomasik et al, 2018). In Northwestern Switzerland, four cantons-Aargau, Basel-Landschaft, Basel-Stadt, and Solothurn-joined forces to develop a computerbased, formative feedback system for classrooms (Tomasik et al, 2018; see also https://www.mindsteps.ch/) to serve teachers and their nearly 100,000 third-through ninth-grade students, as an instrument for data-based decision making (Schildkamp et al, 2013;van der Kleij et al, 2015).…”

“…It allows for implementing complex measurement models and itemselection algorithms that support teachers in providing objective, reliable, and valid feedback (e.g., Glas and Geerlings, 2009;Wauters et al, 2010;Tomasik et al, 2018). In Northwestern Switzerland, four cantons-Aargau, Basel-Landschaft, Basel-Stadt, and Solothurn-joined forces to develop a computerbased, formative feedback system for classrooms (Tomasik et al, 2018; see also https://www.mindsteps.ch/) to serve teachers and their nearly 100,000 third-through ninth-grade students, as an instrument for data-based decision making (Schildkamp et al, 2013;van der Kleij et al, 2015). The data-based decision making approach to formative assessments (van der Kleij et al, 2015) originates from the No Child Left Behind Act in the United States and places a strong emphasis on monitoring the attainment of specific learning targets through objective data.…”

“…The specific purpose of the computer-based, formative feedback system is to support teachers and students in collecting objective information about students' current abilities (i.e., strengths and weaknesses) as well as learning progress within the domains and competencies stated in the curriculum for four school subjects: German, the schools' language; English and French, the two foreign languages taught; and mathematics (Tomasik et al, 2018). To provide targeted feedback (i.e., objective data targeted on students and teachers' specific needs), the system is conceptualized as an item bank with several thousand assessment items that teachers and students can select based on curriculum-related, as well as empirical, criteria, such as curriculum-related competence levels or empirical item difficulty estimates.…”