Assessing approaches to learning with nonparametric multidimensional scaling

Abstract

This article reports on a trace‐based assessment of approaches to learning used by middle school aged children who interacted with NASA Mars Mission science, technology, engineering and mathematics (STEM) games in Whyville , an online game environment with 8 million registered young learners. The learning objectives of two games included awareness and knowledge of NASA missions, developing knowledge and skills of measurement and scaling, applying measurement for planetary comparisons in the solar system. Trace data from 1361 interactions were analysed with nonparametric multidimensional scaling methods, which permitted visual examination and statistical validation, and provided an example and proof of concept for the multidimensional scaling approach to analysis of time‐based behavioural data from a game or simulation. Differences in approach to learning were found illustrating the potential value of the methodology to curriculum and game‐based learning designers as well as other creators of online STEM content for pre‐college youth. The theoretical framework of the method and analysis makes use of the Epistemic Network Analysis toolkit as a post hoc data exploration platform, and the discussion centres on issues of semantic interpretation of interaction end‐states and the application of evidence centred design in post hoc analysis. Practitioner notes What is already known about this topic Educational game play has been demonstrated to positively affect learning performance and learning persistence. Trace‐based assessment from digital learning environments can focus on learning outcomes and processes drawn from user behaviour and contextual data. Existing approaches used in learning analytics do not (fully) meet criteria commonly used in psychometrics or for different forms of validity in assessment, even though some consider learning analytics a form of assessment in the broadest sense. Frameworks of knowledge representation in trace‐based research often include concepts from cognitive psychology, education and cognitive science. What this paper adds To assess skills‐in‐action, stronger connections of learning analytics with educational measurement can include parametric and nonparametric statistics integrated with theory‐driven modelling and semantic network analysis approaches widening the basis for inferences, validity, meaning and understanding from digital traces. An expanded methodological foundation is offered for analysis in which nonparametric multidimensional scaling, multimodal analysis, epistemic network analysis and evidence‐centred design are combined. Implications for practice and policy The new foundations are suggested as a principled, theory‐driven, embedded data collection and analysis framework that provides structure fo...

Automatic Active Lesion Tracking in Multiple Sclerosis Using Unsupervised Machine Learning

“…With LDR, we assume that the subspace that best fits X is linear. LDR has been used in several healthcare applications because it is highly adaptable and effective in extracting features from large high dimensional datasets [ 27 , 28 , 29 ]. An example of LDR is multidimensional scaling (MDS).…”

“…Recently, Knezek et al [ 27 ] used DR to assess various learning approaches to improve motivation amongst young learners. These authors applied MDS to add meaning and understanding to educational game play, and its effect on children’s learning performance.…”

Construction and optimization of non-parametric analysis model for meter coefficients via back propagation neural network

“…Bernard et al (2022) 23 explored the impact of climate change on agricultural productivity in Africa using non-parametric analysis methods, which provided important insights for policy-making. In addition, Knezek et al (2023) 24 used non-parametric multidimensional scaling analysis to study students’ learning approaches and identified similarities and differences in learning styles across disciplines, which can inform educational quality evaluation. Overall, non-parametric analysis methods are indispensable tools for modern data analysis, and further development and refinement of these methods are necessary to better address practical challenges in different fields.…”

Learning analytics and assessment: Emerging research trends, promises and future opportunities

“…Park et al (2022) examined self‐regulatory learning behaviours of students studying with the adaptive learning analytics dashboard and identified groups of students relative to their learning strategies. Knezek et al (2022) utilised student trace‐data to unobtrusively assess learning behaviours of students studying in a game‐based learning environment. The authors identified several learning actions (eg, guess and page view ) and related those actions to course content to understand different learning approaches of individual students.…”