How Developments in Psychology and Technology Challenge Validity Argumentation

Mislevy, Robert J.

doi:10.1111/jedm.12117

Cited by 29 publications

(34 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Third, advanced technologies, as they allow for interactive task designs (e.g., simulations, intelligent tutoring, adaptive testing), may improve the assessment of traditional constructs (e.g., literacy and numeracy) and enable the assessment novel constructs that require complex task designs (e.g., collaborative skills, adaptive problem solving). Together with the availability of process data, however, we believe that this potential requires the careful crafting of a validity argument to facilitate the interpretation of the scores and indicators resulting from advanced technologybased assessments (see also Katz et al, 2017;Mislevy, 2016). Overall, we encourage SPECIAL FEATURE ADVANCED TECHNOLOGIES 5 researchers in the field to not shy away from novel approaches to modeling and assessing complex constructs in education with the help of modern psychometrics and advanced technologies.…”

Section: Resultsmentioning

confidence: 97%

Special feature: advanced technologies in educational assessment

Scherer

Wiberg

2018

Behaviormetrika

View full text Add to dashboard Cite

Section: Resultsmentioning

confidence: 97%

Special feature: advanced technologies in educational assessment

Scherer

Wiberg

2018

Behaviormetrika

View full text Add to dashboard Cite

“…The transfer task was developed following evidence-centered design [48] with iterative refinements based on the feedback from expert teachers, think-aloud interviews, and extensive testing with students. The task, which took around 45 min, was centered around the question of "How do instant heat packs work?"…”

Section: The Transfer Taskmentioning

confidence: 99%

Transferring Knowledge in a Knowledge-in-Use Task—Investigating the Role of Knowledge Organization

et al. 2020

View full text Add to dashboard Cite

Knowledge-in-Use, i.e., the ability to apply what one has learned, is a major goal of education and involves the ability to transfer one’s knowledge. While some general principles of knowledge transfer have been revealed, the literature is full of inconclusive results and it remains hard to predict successful transfer. However, research into expertise suggests that how one organizes one’s knowledge is critical for successful transfer. Drawing on data from a larger study on the learning of energy, we employed network analysis to investigate how the organization of students’ knowledge about energy influenced their ability to transfer and what role achievement goal orientation may have played in this. We found that students that had more coherently organized knowledge networks were more successful in transfer. Furthermore, we also found a connection between mastery goal orientation and the organization of students’ knowledge networks. Our results extend the literature by providing evidence for a direct connection between the organization of students’ knowledge networks, their success in transfer, and their goal orientation and hint at the complexities in the relationship between mastery approach goal orientation and successful transfer beyond what is reported in the literature.

show abstract

“…Successful planning of an experiment to investigate the relationship between voltage over a resistor and the current through the resistor, for example, requires blending knowledge about how to correctly measure voltage and current, the ability to control potentially influencing variables (e.g., temperature; Schwichow et al 2016), and the skills to plan the measurements. underlying the demonstration of the different KSAs (and hence LCS patterns) are within student resources (Mislevy 2016)-in the simplest case, respective KSAs actually held by the individual student. In some cases, however, different resources (i.e., KSAs) may lead to the same demonstrations (i.e., LCS patterns).…”

Section: A Sociocognitive Perspective On Assessing Experimentation Comentioning

confidence: 99%

Assessing Complex Patterns of Student Resources and Behavior in the Large Scale

Neumann¹,

Schecker²,

Theyßen³

2019

The ANNALS of the American Academy of Political and Social Scie

View full text Add to dashboard Cite

Large-scale assessments still focus on those aspects of students' competence that can be evaluated using paper-and-pencil tests (or computer-administered versions thereof). Performance tests are considered costly due to administration and scoring, and, more importantly, they are limited in reliability and validity. In this article, we demonstrate how a sociocognitive perspective provides an understanding of these issues and how, based on this understanding, an argument-based approach to assessment design, interpretation, and use can help to develop comprehensive, yet reliable and valid, performance-based assessments of student competence. More specifically, we describe the development of a computer-administered, simulation-based assessment that can reliably and validly assess students' competence to plan, perform, and analyze physics experiments at a large scale. Data from multiple validation studies support the potential of adopting a sociocognitive perspective and assessments based on an argument-based approach to design, interpretation, and use. We conclude by discussing the potential of simulations and automated scoring methods for reliable and valid performance-based assessments of student competence.

show abstract

How Developments in Psychology and Technology Challenge Validity Argumentation

Cited by 29 publications

References 53 publications

Special feature: advanced technologies in educational assessment

Special feature: advanced technologies in educational assessment

Transferring Knowledge in a Knowledge-in-Use Task—Investigating the Role of Knowledge Organization

Assessing Complex Patterns of Student Resources and Behavior in the Large Scale

Contact Info

Product

Resources

About