Assessing Collaborative Problem Solving Skills in Technology-Enhanced Learning Environments – The PISA Framework and Modes of Communication

Abstract

Abstract-As been highlighted by many, for instance by PISA, Collaborative Problem Solving (CPS) is a critical and necessary 21 st century skill across educational settings. While many initiatives have been launched to investigate the nature of these skills, fewer are the attempts to understand how they should be assessed. However, in 2015, the PISA organization presented a framework for assessing CPS skills. This paper reports on an exploratory study investigating the predictive validity of the PISA assessment framework and if and how modes of communication influence the assessment of 24 students' collaborative problem solving activities when using a computer-based assessment task system. The findings presented demonstrate that the PISA CPS assessment framework have a weak predictive validity, does not count for quality or productivity in communication, and that the mode of communication indeed influence CPS processes and in turn what is possible to assess.

Advancing the Science of Collaborative Problem Solving

“…Only two subskills showed significant correlations (surprisingly, both positive and negative) with the overall task performance, and only in the audio condition. Hence, the predictive validity of the PISA assessment framework was questioned by the authors of this study (Nouri et al, 2017).…”

“…A similar study also applied the PISA framework design and compared audio and text chats with regard to H-H interactions (Nouri, Åkerfeldt, Fors, & Selander, 2017). More utterances were found in the audio chat condition, and some subskills from the PISA system were more prevalent with audio chat.…”

ICT Self-Efficacy and ICT Interest Mediate the Gender Differences in Digital Reading: A Multilevel Serial Mediation Analysis

“…PISA 2018 is the seventh and the most recent cycle of PISA, and reading is the major domain of assessment for this cycle, in which seventy-nine countries/regions have participated. PISA allows the comparison across different participating countries/regions [31], and has strong educational implication for educators and policy makers worldwide [32][33]. 6,173 samples with a balanced gender ratio (51.09% for girls) from 192 schools in New Zealand were selected in this study.…”

Multimodal Texts and Literacy in a Digitized World

“…In an international developmental project, researchers and teachers from Australia, Ireland, and Sweden collaborated in developing a game for collaborative problem solving and on the assessment and teaching of "21st Century Skills" (Nouri et al 2017). In this game, students worked in pairs to solve a joint problem.…”