Reliability and Attribute‐Based Scoring in Cognitive Diagnostic Assessment

Gierl, Mark J.; Cui, Ying; Zhou, Jiawen

doi:10.1111/j.1745-3984.2009.00082.x

Cited by 30 publications

(27 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, Gierl et al. () and Templin and Henson () did not examine the relationship between the reliability index and test‐retest consistency rate in their studies. What's more, their alpha reliability was adapted from Cronbach's coefficient alpha, whose underlying assumptions may not hold in practice (Green & Yang, ; Sijtsma, ).…”

mentioning

confidence: 97%

“…Some worthwhile attempts, though not many, about the reliability of attribute diagnostic scores have been made from different perspectives such as alpha reliability (Gierl, Cui, & Zhou, ), empirical reliability (Templin & Henson, ), correlation‐based reliability (Templin & Bradshaw, ), and test‐retest consistency rate (Roussos et al., ). However, Gierl et al.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Attribute‐Level and Pattern‐Level Classification Consistency and Accuracy Indices for Cognitive Diagnostic Assessment

Wang

Song

Chen

et al. 2015

J Educational Measurement

View full text Add to dashboard Cite

show abstract

mentioning

confidence: 97%

mentioning

confidence: 99%

Attribute‐Level and Pattern‐Level Classification Consistency and Accuracy Indices for Cognitive Diagnostic Assessment

Wang

Song

Chen

et al. 2015

J Educational Measurement

View full text Add to dashboard Cite

show abstract

“…A different approach may be used to provide additional information for more valid diagnostic inferences. A longer test may also improve the situation (Gierl, Cui, & Zhou, 2009). However, to overcome the limitations of the post hoc approach, a cognitive diagnostic assessment has to be designed where an attribute‐based hierarchy is created by, initially, defining the cognitive model and then generating items systematically using the Q r matrix from the AHM analysis to measure each attribute in the hierarchy.…”

Section: Discussionmentioning

confidence: 99%

Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees’ Cognitive Skills in Critical Reading

Wang

Gierl

2011

Journal of Educational Measurement

Self Cite

View full text Add to dashboard Cite

The purpose of this study is to apply the attribute hierarchy method (AHM) to a subset of SAT critical reading items and illustrate how the method can be used to promote cognitive diagnostic inferences. The AHM is a psychometric procedure for classifying examinees’ test item responses into a set of attribute mastery patterns associated with different components from a cognitive model. The study was conducted in two steps. In step 1, three cognitive models were developed by reviewing selected literature in reading comprehension as well as research related to SAT Critical Reading. Then, the cognitive models were validated by having a sample of students think aloud as they solved each item. In step 2, psychometric analyses were conducted on the SAT critical reading cognitive models by evaluating the model‐data fit between the expected and observed response patterns produced from two random samples of 2,000 examinees who wrote the items. The model that provided best data‐model fit was then used to calculate attribute probabilities for 15 examinees to illustrate our diagnostic testing procedure.

show abstract

“…Put differently; cognitive models help assessment specialists explain and predict test takers' performances. CDA employs cognitive models so as to i) generate items tapping on specific skills, knowledge and/or cognitive processes which are called attributes in CDA literature; ii) depict item-attribute alignments for the existing tests, and iii) make interpretations about test takers' performances on test tasks (Gierl, Cui, & Zhou, 2009). By utilizing cognitive models, CDA is assumed to be capable of capturing the existing deficiencies or gaps in the cognitive mechanisms and knowledge structures that are of great importance to perform on a test task.…”

Section: What Is Cognitive Diagnostic Assessment?mentioning

confidence: 99%

Eğitimde Ölçme ve Değerlendirme Uygulamalarına Bilişsel Tanılayıcı Bir Yaklaşım

Toprak¹,

Çakır²

2018

Kuramsal Eğitimbilim

View full text Add to dashboard Cite

A growing emphasis on the union of cognitive psychology with psychometrics has led to the inception of Cognitive Diagnostic Assessment (CDA). CDA can be defined as a cognitively-grounded assessment methodology which aims to detect examinees' strengths and weaknesses in a given domain, make reliable diagnostic classifications directly from the statistical models, and present stakeholders with fine-grained and pedagogically-meaningful diagnostic feedback. Although CDA holds great promise for educational assessment practices, it remains relatively unknown to many assessment specialists. Hence, this paper aims to describe the theoretical underpinnings and working principles of CDA by giving information about the developments that have led to the inception of CDA and elaborate on how CDA can be implemented in operational assessment settings either by using an inductive or retrofitted approach to foster learning and enhance accountability within educational programs. Finally, the potential that CDA bears for educational assessment is discussed and practical implications are made.Keywords: cognitive diagnostic assessment, diagnostic classification modelling, skills diagnosis, assessment for learning. ÖZ:Bilişsel psikolojinin psikometri ile harmanlanması Bilişsel Tanılayıcı Değerlendirme (BTD) adı verilen ölçme ve değerlendirme yaklaşımının ortaya çıkmasını sağlamıştır. BTD, bilişsel temelli, istatistiki açıdan sofistike ve alternatif bir ölçme ve değerlendirme yaklaşımıdır. Bireylerin belirli bir beceri ya da akademik alandaki güçlü ve zayıf yanlarının, eksiklerinin ve yanılgılarının saptanmasını ve bu hususlara yönelik paydaşlara (öğrenci, öğretmen, veli ve idarecilere) bireylerin halihazırdaki durumları hakkında detaylı dönüt verilmesini amaçlar. Sağlanan dönüt, pedagojik açıdan anlamlı ve öğrenme sürecini destekleyici boyutta olmalıdır. Bu değerlendirme yaklaşımının eğitim öğretim faaliyetleri için pek çok yararı olmasına karşın, BTD hem eğitim araştırmacıları hem de ölçme değerlendirme alanında çalışan araştırmacılar tarafından yeteri derecede tanınmamaktadır. Bu makalede, BTD yaklaşımının ortaya çıkmasına sebep veren eğitimsel akım ve gelişmeler ele alınmış, BTD'nin kuramsal temelleri, çalışma prensipleri, işlevleri hakkında detaylı bilgi verilmiştir. Ayrıca, BTD'nin öğrenme çıktılarını iyileştirme ve eğitim programlarının kalite ve hesap verebilirliğinin artırılması hedeflerine yönelik olarak, eğitim ve ölçme değerlendirme ortamlarında nasıl uygulanabileceği hususunda öneriler sunulmuştur.Anahtar kelimeler: eğitimde ölçme ve değerlendirme, bilişsel tanılayıcı değerlendirme, tanılayıcı sınıflama modellemesi.

show abstract

Reliability and Attribute‐Based Scoring in Cognitive Diagnostic Assessment

Cited by 30 publications

References 17 publications

Attribute‐Level and Pattern‐Level Classification Consistency and Accuracy Indices for Cognitive Diagnostic Assessment

Attribute‐Level and Pattern‐Level Classification Consistency and Accuracy Indices for Cognitive Diagnostic Assessment

Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees’ Cognitive Skills in Critical Reading

Eğitimde Ölçme ve Değerlendirme Uygulamalarına Bilişsel Tanılayıcı Bir Yaklaşım

Contact Info

Product

Resources

About