Polytomous Item Response Theory Models

CONTEXT A test score is a number which purportedly reflects a candidate's proficiency in some clearly defined knowledge or skill domain. A test theory model is necessary to help us better understand the relationship that exists between the observed (or actual) score on an examination and the underlying proficiency in the domain, which is generally unobserved. Common test theory models include classical test theory (CTT) and item response theory (IRT). The widespread use of IRT models over the past several decades attests to their importance in the development and analysis of assessments in medical education. Item response theory models are used for a host of purposes, including item analysis, test form assembly and equating. Although helpful in many circumstances, IRT models make fairly strong assumptions and are mathematically much more complex than CTT models. Consequently, there are instances in which it might be more appropriate to use CTT, especially when common assumptions of IRT cannot be readily met, or in more local settings, such as those that may characterise many medical school examinations.OBJECTIVES The objective of this paper is to provide an overview of both CTT and IRT to the practitioner involved in the development and scoring of medical education assessments.METHODS The tenets of CCT and IRT are initially described. Then, main uses of both models in test development and psychometric activities are illustrated via several practical examples. Finally, general recommendations pertaining to the use of each model in practice are outlined.DISCUSSION Classical test theory and IRT are widely used to address measurement-related issues that arise from commonly used assessments in medical education, including multiple-choice examinations, objective structured clinical examinations, ward ratings and workplace evaluations. The present paper provides an introduction to these models and how they can be applied to answer common assessment questions.item response theory

show abstract

“…Readers interested in obtaining an overview of common models and applications should refer to other sources. 21 …”

Section: Using Irt With Performance Assessmentsmentioning

confidence: 99%

A primer on classical test theory and item response theory for assessments in medical education

2010

View full text Add to dashboard Cite

show abstract

“…The model deals with the possibility of different thresholds between the categories (Andrich, 1988(Andrich, , 2005Ostini & Nering, 2006), and Rasch model helps users to develop scales with strong psychometric properties including greater generalizability (Embreston & Reise, 2000;Hambleton, Swaminathan, & Rogers, 1991). It enables users to create an interval scale of scores for both items and persons (ability) that are sample independent and intervally measured (Bond & Fox, 2001 …”

Section: Discussionmentioning

confidence: 99%

Differential Item Functioning in Students Rating of Teaching Effectiveness Surveys in Higher Education According to Academic Disciplines: Data from a Saudi University

Alquraan

Kuwaiti

2017

J. educ. psychol. stud.

View full text Add to dashboard Cite

This study explored academic discipline as a source of differential item functioning (DIF) in students' rating of teaching quality and effectiveness at higher education institutions. Data utilized in this study was collected by Imam Abudalruman Bin Faisal University -KSA. The total number of surveys analyzed for the purpose of this study is 36459 from three colleges: Education, Health, and Engineering. Using Extended Rasch model (Rating Scale Model), the results show that the instrument contains four DIF items. The content of these four items confirm the possibility of considering discipline as a source of DIF items in students evaluation of teaching in higher education. Moreover, the results of the current study show that removing DIF items from the instrument increases construct validity.

show abstract

“…Já o C-9 foi além: para cada questão incorreta, o sistema fornece dicas e conteúdos e o usuário tem outra chance de responder. Com isso, foi construído o Modelo Politômico de Crédito Parcial (do inglês, Parcial Credit Model -PCM) [Ostini and Nering 2006], que considera 0 para errou, 1 para acertou, sem feedback, e 2 para acertou, com feedback. O trabalho mostrou que os feedbacks influenciaram de forma positiva o comportamento das respostas.…”

Section: Tabela 2 Técnicas De Adaptação Empregadas Nos Sistemas De Aunclassified

Testes Adaptativos Computadorizados baseados na Teoria de Resposta ao Item em Sistemas e-learning: Uma revisão sistemática da literatura

Jatobá¹,

Valdivia-Delgado²,

Farías³

et al. 2017

Anais Do XXVIII Simpósio Brasileiro De Informática Na Educação (SBIE 2017)

View full text Add to dashboard Cite

Abstract. Computerized Adaptive Testing (CAT) based on Item Response Theory (IRT) allows more accurate assessments and shows advantages when incorporated into e-learning environments. However, due the solutions multiplicity, this knowledge is fragmented in the literature, what makes it difficult to understand the challenges faced in the area. In this sense, the goal of this work is to understand and to characterize e-learning systems based on CAT and IRT. To this, a Systematic Review of the literature was performed where four research questions were analyzed in 9 papers selected by the defined protocol criteria. Resumo. Testes Adaptativos Computadorizados (CAT) baseados na Teoria deResposta ao Item (IRT) permitem realizar avaliações mais precisas e mostram vantagens quando incorporados a ambientes de aprendizado virtual (elearning). Entretanto, devidoà multiplicidade de soluções, este conhecimento encontra-se fragmentado na literatura, o que dificulta o entendimento dos dissensos e desafios enfrentados naárea. Neste sentido, este trabalho buscou entender e caracterizar os estudos que usam CAT e IRT que estão inseridos em sistemas e-learning. Para tal, uma revisão sistemática da literatura foi realizada, na qual quatro questões de pesquisa foram analisadas em 9 artigos previamente selecionados pelos critérios estabelecidos no protocolo de revisão.

show abstract

Polytomous Item Response Theory Models

Cited by 230 publications

References 0 publications

A primer on classical test theory and item response theory for assessments in medical education

A primer on classical test theory and item response theory for assessments in medical education

Differential Item Functioning in Students Rating of Teaching Effectiveness Surveys in Higher Education According to Academic Disciplines: Data from a Saudi University

Testes Adaptativos Computadorizados baseados na Teoria de Resposta ao Item em Sistemas e-learning: Uma revisão sistemática da literatura

Contact Info

Product

Resources

About