Collaborative Problem Solving and the Assessment of Cognitive Skills: Psychometric Considerations

Abstract

Collaboration is generally recognized as a core competency of today's knowledge economy and has taken a central role in recent theoretical and technological developments in education research. Yet, the methodology for assessing the learning benefits of collaboration continues to rely on educational tests designed for isolated individuals. Thus, what counts as evidence of learning does not correspond to current best practices for teaching, and it does not reflect what students are ultimately expected to be able to do with their knowledge. The goals of this paper are to give an overview of the research conducted in several fields of work related to collaboration, propose a framework for the assessment of cognitive skills (such as science or math) through collaborative problem‐solving tasks, and propose several statistical approaches to model the data collected from collaborative interactions. This research contributes to the knowledge needed to support a new generation of assessments based on collaboration.

Automated classification of collaborative problem solving interactions in simulated science tasks

“…Collaborative problem solving (CPS) is a complex activity that involves an interplay between cognitive processes, such as content understanding, knowledge acquisition, action planning and execution (Greiff, 2012;von Davier and Halpin, 2013), non-cognitive processes, such as adaptability, engagement, social regulation, and affect states, such as boredom, confusion, and frustration (Baker et al, 2010;Graesser et al, 2010). Collaborative learning techniques are used extensively in educational practices, from pre-school to higher education.…”

Mapping an Experiment‐Based Assessment of Collaborative Behavior Onto Collaborative Problem Solving in PISA 2015: A Cluster Analysis Approach for Collaborator Profiles

“…von Davier & Halpin, 2013). Moreover, there is no established statistical theory for the assessment of CPS to date (von Davier & Halpin, 2013). Moreover, there is no established statistical theory for the assessment of CPS to date (von Davier & Halpin, 2013).…”

Computational Psychometrics for the Measurement of Collaborative Problem Solving Skills

“…There are stochastic models (point processes, for example) that can be used to model the temporal dynamics of the CPS tasks (von Davier and Halpin, 2013), or hidden Markov models (Soller and Stevens, 2007); there are also models based on the cognitive or social theories such as Agent-based modeling (Bergner et al, 2015) and Markov Decision Process, which is a cognitive model with parameters that describe the goals or beliefs of the agents and which defines behavior as an optimization of expected rewards based on current beliefs about the world (LaMar, 2014). With the aid of data mining techniques we may reduce the dimensionality of the dataset by extracting interpretable patterns which allow research questions to be addressed that would otherwise not be feasible (Romero et al, 2009).…”