Validating Classifications From Learning Progressions: Framework and Implementation

Attali, Yigal; Arieli‐Attali, Meirav

doi:10.1002/ets2.12253

Cited by 5 publications

(7 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, the system can record individual behaviors while interacting with the system as much as possible. These data sources can offer the possibility of capturing evidence for identifying a meaningful problem-solving process and learning patterns [36,37]. This analytic framework is based on computational psychometrics.…”

Section: Discussionmentioning

confidence: 99%

Learning Analytics Using Social Network Analysis and Bayesian Network Analysis in Sustainable Computer-Based Formative Assessment System

Choi

Cho

2020

Sustainability

View full text Add to dashboard Cite

The sustainable computer-based evaluation system (SCE) is a scenario-based formative evaluation system, in which students are assigned a task during a course. The tasks include the diversity conditions in real-world scenarios. The goals of this system are learning to think as a professional in a certain discipline. While the substantive, psychological, instructional, and task developmental aspects of the assessment have been investigated, few analytic methods have been proposed that allow us to provide feedback to learners in a formative way. The purpose of this paper is to introduce a framework of a learning analytic method including (1) an assessment design through evidence-centered design (ECD), (2) a data mining method using social network analysis, and (3) an analytic method using a Bayesian network. This analytic framework can analyze the learners’ performances based on a computational psychometric framework. The tasks were designed to measure 21st century learning skills. The 250 samples of data collected from the system were analyzed. The results from the social network analysis provide the learning path during a course. In addition, the 21st century learning skills of each learner were inferred from the Bayesian network over multiple time points. Therefore, the learning analytics proposed in this study can offer the student learning progression as well as effective feedback for learning.

show abstract

Section: Discussionmentioning

confidence: 99%

Learning Analytics Using Social Network Analysis and Bayesian Network Analysis in Sustainable Computer-Based Formative Assessment System

Choi

Cho

2020

Sustainability

View full text Add to dashboard Cite

show abstract

“…DCMs are psychometric models that classify students based on categorical latent variables related to the purpose of the assessment. This psychometric model is a promising means for students, instructors, and test developers to obtain more detailed diagnostic information [ 5 ]. This study conducted an analysis using the DINA model, which is one of the main types of DCMs, using response data from the KMLE.…”

Section: Discussionmentioning

confidence: 99%

“…Most score report in the health professions also provide a total score or pass-fail decisions based on classical test theory (CTT). DCMs are psychometric models that characterize examinee responses to test items through the use of categorical latent variables that represent the knowledge states of an examinee [5]. Thus, DCMs have become a popular area of psychometric research in education.…”

Section: Introductionmentioning

confidence: 99%

Estimation of item parameters and examinees’ mastery probability in each domain of the Korean Medical Licensing Examination using a deterministic inputs, noisy “and” gate (DINA) model

Choi

Seo

2020

J Educ Eval Health Prof

View full text Add to dashboard Cite

Purpose: Deterministic inputs, noisy and gate (DINA) model is one of the promising statistical means for providing useful diagnostic information about a student’ level of achievement. Diagnostics information is core element for improving learning instead of selection. Educators often want to be provided with diagnostic information which how a given examinees did on each content strand, called diagnostic profiles. The purpose of this paper is to classify examinees in different content domains using the DINA model.Methods: This paper analyzed data from the Korean medical licensing examination (KMLE) with 360 items and 3259 examinees. The application study estimate examinees parameters as well as item characteristics. The guessing and slipping parameters of each item were estimated. DINA model was conducted as a statistical analysis. Results: The output table shows the examples of some items, which can be used for the check of item quality. In addition, the probabilities of being mastery at each content domain were estimated, which indicates the mastery profile of each examinee. Classifications accuracy for 8 contents ranged from .849 to .972 and classification consistency for 8 contents ranged from .839 to .994. As a result, classification reliability in a CDM was very high for 8 contents in KMLEConclusion: This mastery profile can be useful diagnostic information for each examinee in terms of the content domains of KMLE. The master profile from KMLE provides each examinee’s mastery profile in terms of each content domain. The individual mastery profile allows educators and examinees to understand that which domain(s) should be improved for mastering all domains in KMLE. In addition, the results found that all items are reasonable level with respect to item parameters character.

show abstract

“…This simplification entails two related assumptions: that ideas can be grouped, with most students holding the ideas in a group simultaneously, such that coherent LP levels can be defined (Anderson, 2008) and that most students provide consistent evidence of the ideas in a single group (across variation in item context, item format, etc.) such that students can be reliably categorized using LP levels (e.g., Attali & Arieli‐Attali, 2019). …”

mentioning

confidence: 99%

“…that most students provide consistent evidence of the ideas in a single group (across variation in item context, item format, etc.) such that students can be reliably categorized using LP levels (e.g., Attali & Arieli‐Attali, 2019).…”

mentioning

confidence: 99%

Learning progressions as a simplified model: Examining teachers' reported uses to inform classroom assessment practices

2022

View full text Add to dashboard Cite

As models of how students' thinking may change over time, learning progressions (LPs) have been considered as supports for teachers' classroom assessment practices. However, like all models, LPs provide simplified representations of complex phenomena. One key simplification is the characterization of student thinking using levels—that is, the twin assumptions that student thinking is both coherent and consistent. While useful for the design of standards and curricula, the LP level simplification may threaten the basic premise that LPs could be used to diagnose a student's level and then provide tailored instruction in response. At the same time, our work with teachers suggests that, even with their simplifications, LPs may be useful in the classroom. Thus, rather than abandoning LPs, we sought to understand their potential affordances by exploring how teachers learn from LPs (knowledge‐for‐practice) and contribute to deeper understanding of LP use (knowledge‐of‐practice) as they identify and enact uses of these tools. To do so, we engaged high school physics teachers in a 2‐year, LP‐based professional development program. Based on qualitative analyses of planning meetings and interviews with the teachers, we describe how teachers used LPs to support classroom assessment with varying reliance on the LP level simplification. Although teachers used LPs in ways that relied on the coherence and consistency assumptions of the LP level simplification, uses of LPs that did not require these assumptions were more prevalent both within and across teachers. This study's findings have implications for research, teacher professional development, and the design of LPs.

show abstract

Validating Classifications From Learning Progressions: Framework and Implementation

Cited by 5 publications

References 35 publications

Learning Analytics Using Social Network Analysis and Bayesian Network Analysis in Sustainable Computer-Based Formative Assessment System

Learning Analytics Using Social Network Analysis and Bayesian Network Analysis in Sustainable Computer-Based Formative Assessment System

Estimation of item parameters and examinees’ mastery probability in each domain of the Korean Medical Licensing Examination using a deterministic inputs, noisy “and” gate (DINA) model

Learning progressions as a simplified model: Examining teachers' reported uses to inform classroom assessment practices

Contact Info

Product

Resources

About