Video‐enhanced or textual rubrics: Does the Viewbrics' formative assessment methodology support the mastery of complex (21st century) skills?

Abstract

Learners in the process of developing complex skills need a rich mental model of what such skills entail. Textual analytics rubrics (TR) are a widely used instrument to support formative assessment of complex skills, supporting feedback, reflection, and thus mental model development of complex skills. However, the textual nature of a rubric limits the possibility to deliver contextual and dynamic (time‐and behaviour‐oriented) information. In the Viewbrics online tool, we developed a version supporting the delivery of contextual and dynamic information by adding video‐modelling examples with embedded self‐explanation prompts to textual analytics rubrics. We called this combination as video‐enhanced rubrics (VERs). Our current study investigates whether the Viewbrics online tool supports complex skills development and whether either textual‐ or video‐enhanced rubrics best support complex skills' mastery. The study was a three‐group (VERS n = 49, TR n = 54, control n = 50) within‐subjects design. Learners' performance of complex skills was measured through expert, peer, and self‐assessment using the Viewbrics online tool. A multilevel regression analysis shows learners in the TR and VERS conditions consistently outperforming the control condition to varying degrees across skills. However, no differences have been found between the two experimental conditions. Positive results across different complex skills indicate the Viewbrics online tool can be used to support the development of a wide range of complex skills in secondary education.

A whole learning process-oriented formative assessment framework to cultivate complex skills

“…The concept of formative assessment emphasizes the integration of instruction and evaluation. It requires a dynamic, continuous framework for evaluating performance that promotes students' progress and the mastery of increasingly sophisticated skills, which is important for the cultivating of complex 21CS (Ackermans et al, 2021;Care et al, 2019;Nadolski et al, 2021). However, since complex 21CS involve complex learning of different but interconnected knowledge and competencies, formative evaluation practices in the school curricula are relatively scarce (Ackermans et al, 2021;Webb et al, 2018).…”

“…It requires a dynamic, continuous framework for evaluating performance that promotes students' progress and the mastery of increasingly sophisticated skills, which is important for the cultivating of complex 21CS (Ackermans et al, 2021;Care et al, 2019;Nadolski et al, 2021). However, since complex 21CS involve complex learning of different but interconnected knowledge and competencies, formative evaluation practices in the school curricula are relatively scarce (Ackermans et al, 2021;Webb et al, 2018). As a result, there are still some key issues that must be explored in how to evaluate and monitor students' learning progress under complex context, such as how and when formative assessment should be offered (Bhagat and Spector, 2017;Kim, 2021).…”

Pe(e)rfectly Skilled

“…Students are supported to form a mental model of what skills mastery level is expected of them through the availability of video-enhanced analytic rubrics and video-modeling examples, by using the rubrics on a standard video-modeling example and by comparing their feedback with the feedback of an expert. A video-enhanced rubric is an integration of a videomodeling example with example situations and role models, and an analytic rubric (Ackermans et al, 2021a(Ackermans et al, , 2021b. A video-modeling example is an (good or bad) example of task execution and associated behavior that can support observational learning and mental model formation to acquire skills.…”

Use of Educational Technology to Improve the Quality of Learning and Teaching: A Systematic Research Review and New Perspectives

“…Studies have shown that educational technology can be used very well for the process of formative assessment. Throughout the formative assessment, teachers can use educational technology to provide evidence regarding the achievement of learning outcomes and to provide feedback and improve the teaching process [38], [39], [40]. The use of technology for assessment has had a particularly great impact on students with special needs [41].…”