Computational Psychometrics Approach to Holistic Learning and Assessment Systems

Davier, Alina A. von; Deonovic, Benjamin; Yudelson, Michael; Polyak, Stephen T.; Woo, Ada

doi:10.3389/feduc.2019.00069

Cited by 25 publications

(16 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Let’s talk briefly about what we mean when we use the term “progress.” In recent years, I have been actively engaged in conducting research into changing students’ educational experiences through developing tools for learning and assessment that are in sync with the ways they actually learn and live: digital, on demand, mobile, interactive, and in real-time (see, e.g., von Davier et al, 2019). Attempts to include technology that would transform the testing experience, say by making it faster (automatic item generation, with real-time automatic test assembly), more accessible (say, online testing on multiple devices with accessibility tools), or more appropriate for the way people learn (new item types), have been received with concerns about comparability to the past.…”

Section: My Commentary On the Contextmentioning

confidence: 99%

Commentary on “Validation Methods for Aggregate-Level Test Scale Linking: A Case Study Mapping School District Test Score Distributions to a Common Scale”

Davier

2020

Journal of Educational and Behavioral Statistics

Self Cite

View full text Add to dashboard Cite

In this commentary, I share my perspective on the goals of assessments in general, on linking assessments that were developed according to different specifications and for different purposes, and I propose several considerations for the authors and the readers. This brief commentary is structured around three perspectives (1) the context of this research, (2) the methodology proposed here, and (3) the consequences for applied research.

show abstract

Section: My Commentary On the Contextmentioning

confidence: 99%

Commentary on “Validation Methods for Aggregate-Level Test Scale Linking: A Case Study Mapping School District Test Score Distributions to a Common Scale”

Davier

2020

Journal of Educational and Behavioral Statistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…The current model used in ACT Academy for computing the diagnostic profile is not the Master model but a simpler model. For details, see von Davier et al (2019).…”

Section: Act Academymentioning

confidence: 99%

“…This desire to blend models for assessment with models of learning has been brought up many times in the past (Cronbach 1957;Pellegrino et al 1999;Embretson and Gorin 2001;Gorin 2006;Deonovic et al 2018). In fact, more recently, von Davier et al (2019) took this idea one step further and, echoing sentiments from elsewhere in the literature (Savi, van der Maas, et al 2015), proposed a framework in which research in cognitive science, educational measurement, and navigation are all considered when designing an educational platform. The thesis of von Davier et al (2019) is simple: we must help students know where they are (assessment), where they want to go (navigation), and how to get there (learning).…”

Section: Introductionmentioning

confidence: 99%

The Ising on the Tree: The Master Model for Learning, Assessment, and Navigation

Deonovic¹,

Bechger²,

Maris³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Learning and assessment are intrinsically linked. However, the research, tools, and statistical models used within the two fields differ greatly. This has created a disconnect: The goals and missions of educational institutions are codified in the language and ideas of learning, but evaluated, monitored, and administered with the tools of assessment. We propose a novel statistical model, the Master model, capable of being the engine behind a modern learning and assessment system. The Master model combines three key concepts from the assessment and learning literature from the past century: A learning model should be multidimensional and hierarchical and should incorporate learning progressions. The Master model is a multidimensional latent variable model, more specifically a latent class model, that not only ranks learners from best to worst but also provides detailed diagnostic feedback to tell learners what they know, and more importantly, what they don't know. By incorporating a hierarchical structure of the latent variables, the Master model reproduces the positive manifold, a phenomenon that continues to be replicated in assessment data where scores between cognitive tests correlate positively. Finally, expert and data-driven annotation can incorporate learning progressions directly into the latent variables. With these three key concepts, the Master model can track the estimate of a learner’s latent skills, track the efficacy of various educational resources such as videos, and recommend which resources the learner should next focus on in order to maximize their learning.

show abstract

“…See Figure 12 for an illustration of the architecture for the Educational Companion App. More details about this prototype are given in von Davier et al (2019).…”

Section: Learning Measurement and Navigation Systemsmentioning

confidence: 99%

The Argument for a “Data Cube” for Large-Scale Psychometric Data

et al. 2019

Self Cite

View full text Add to dashboard Cite

In recent years, work with educational testing data has changed due to the affordances provided by technology, the availability of large data sets, and by the advances made in data mining and machine learning. Consequently, data analysis has moved from traditional psychometrics to computational psychometrics. Despite advances in the methodology and the availability of the large data sets collected at each administration, the way assessment data is collected, stored, and analyzed by testing organizations is not conducive to these real-time, data intensive computational methods that can reveal new patterns and information about students. In this paper, we propose a new way to label, collect, and store data from large scale educational learning and assessment systems (LAS) using the concept of the "data cube." This paradigm will make the application of machine-learning, learning analytics, and complex analyses possible. It will also allow for storing the content for tests (items) and instruction (videos, simulations, items with scaffolds) as data, which opens up new avenues for personalized learning. This data paradigm will allow us to innovate at a scale far beyond the hypothesis-driven, small-scale research that has characterized educational research in the past.

show abstract

Computational Psychometrics Approach to Holistic Learning and Assessment Systems

Cited by 25 publications

References 20 publications

Commentary on “Validation Methods for Aggregate-Level Test Scale Linking: A Case Study Mapping School District Test Score Distributions to a Common Scale”

Commentary on “Validation Methods for Aggregate-Level Test Scale Linking: A Case Study Mapping School District Test Score Distributions to a Common Scale”

The Ising on the Tree: The Master Model for Learning, Assessment, and Navigation

The Argument for a “Data Cube” for Large-Scale Psychometric Data

Contact Info

Product

Resources

About