Lost in transition – Learning analytics on the transfer from knowledge acquisition to knowledge application in complex problem solving

Abstract

Since Complex Problem Solving (CPS) skills represent a key competence for educational success, they are of great relevance for learning analytics. More specifically, CPS serves as a pertinent showcase for addressing a crucial existing gap contemporary education is facing, the gap between students' ability to acquire and subsequently apply knowledge in uncertain situations, which are increasingly important in the 21st century. While the CPS process incorporates both the acquisition and application of knowledge, many earlier studies have focused on identifying the factors relevant for success in knowledge acquisition. Given the dearth of existing research on factors influencing a successful transition between both CPS phases, we investigated the rates of successful and unsuccessful knowledge transition over the course of nine CPS items in a sample of N = 1,151 students in 9th grade. Results showed that many participants were unable to transition their knowledge from the acquisition to the application phase, which was presumably due to an inefficient mental model transfer. Furthermore, the likelihood of students being 'lost in transition' was higher in difficult items. Implications are discussed in light of learning analytics, and particularly with regard to the factors to be taken into account by future CPS training programs.

Measuring Complex Problem-Solving in Jordan: Feasibility, Construct Validity and Behaviour Pattern Analyses

“…The processes of the KAC and KAP phases can also be empirically distinguished in the Jordanian context. The bivariate correlation (r=.45) between KAC and KAP was consistent with earlier research results, which varied between r=0.14 and r=0.94 (Nicolay, Krieger, Stadler, Gobert, & Greiff, 2021). The reason for this wide range of correlation coefficients is the use of different problem-solving approaches and CPS assessments to measure KAC and KAP.…”

How we explore, interpret, and solve complex problems: A cross-national study of problem-solving processes

“…To this end, technology-based assessment offers an opportunity to collect contextual information gathered beyond the final response data and analyze different behavioral indicators, such as strategy effectiveness, number of trials, and time-on-task. Logfile analysis has the potential to look at developmental differences from different perspectives (Nicolay et al, 2021) and to provide more sophisticated feedback instead of using single indicators, such as an overall test score.…”

“…The CPS approach and the CPS tasks have been employed extensively at both the national and international levels (see Csap o and Eichmann et al, 2020;Greiff et al, 2013b;Greiff et al, 2015b;Mustafic et al, 2019;Nicolay et al, 2021;OECD, 2014a). The psychometric indices of the test proved to be good, independent of the cultures and nations (see Wüstenberg et al, 2014;Wu and Moln ar, 2021).…”

Exploring the Role of English as a Foreign Language Receptive Skills and Learning Strategy Usage in the Ability to Acquire and Apply Knowledge at the Beginning of Higher Education

“…Language learners often face complex, meaning-focused tasks that involve a cumulative sequence of cognitive processes, situation-specific thinking, and activating and reconstructing links between recent and new knowledge stored in memory. These complex tasks require higher-order skills that are not available at the beginning of the process and rather need to be constructed during it ( Molnár et al, 2021 ; Nicolay et al, 2021 ). Knowledge acquisition and knowledge application also entail the mobilization of certain motivational and creative skills to find a solution to the unfamiliar task ( Lesh and Zawojewski, 2007 ).…”