An application of item response theory to psychological test development

Abstract

Item response theory (IRT) has become a popular methodological framework for modeling response data from assessments in education and health; however, its use is not widespread among psychologists. This paper aims to provide a didactic application of IRT and to highlight some of these advantages for psychological test development. IRT was applied to two scales (a positive and a negative affect scale) of a self-report test. Respondents were 853 university students (57 % women) between the ages of 17 and 35 and who answered the scales. IRT analyses revealed that the positive affect scale has items with moderate discrimination and are measuring respondents below the average score more effectively. The negative affect scale also presented items with moderate discrimination and are evaluating respondents across the trait continuum; however, with much less precision. Some features of IRT are used to show how such results can improve the measurement of the scales. The authors illustrate and emphasize how knowledge of the features of IRT may allow test makers to refine and increase the validity and reliability of other psychological measures.

Redução da Escala Tendência Empreendedora Geral (TEG-FIT) a partir do Coeficiente de Validade de Conteúdo (CVC) e Teoria da Resposta ao Item (TRI)

“…Tais modelos foram impulsionados por volta de 1950 e apresentam especial interesse na avaliação individual de cada um dos itens, implementando técnicas que tratam esses dados pelo nível de medida categórico (Pasquali et al, 2003). A TRI é vista como técnica de primeira escolha em pesquisas sobre atitudes (Foster et al, 2017), mas, no Brasil, ela ainda não apresenta a mesma popularidade que já alcançou em outros locais (Zanon et al, 2016), Os modelos da TRI lidam tanto com apenas uma habilidade psicológica (unidimensional), quanto com duas ou mais habilidades (multidimensionais) e assumem uma relação não-linear entre a habilidade psicológica de um respondente (traço latente, apresentado pela letra grega θ) e as características (parâmetros) dos itens (Furr & Bacharach, 2008;Reise & Revicki, 2015;Van Der Linden & Hambleton, 1997). A estimativa é feita regredindo a resposta aos itens (entendido como variável dependente) no traço latente, independentemente de quais itens tenham sido respondidos.…”

“…Para tanto, a modelagem foi realizada pela interface entre Teoria de Resposta ao Item (TRI) e o Coeficiente de Validade de Conteúdo. Tendo em vista a pouca popularidade da utilização de tais técnicas matemáticas no cenário acadêmico brasileiro, conforme descrito por Zanon, Hutz, Yoo, & Hambleton, (2016), a descrição mais extensiva de tais métodos foi elaborada junto ao referencial teórico. …”

Πλαίσιο Διαδικασίας Για Την Αντικειμενοποίηση Της Επιλογής Ερωτημάτων Τελικής Δοκιμασίας Μέσω Αξιοποίησης Των Δεδομένων Από Ενδιάμεσες Δοκιμασίες Αξιολόγησης

“…Αυτό το χαρακτηριστικό είναι ιδιαίτερα σημαντικό και μοναδικό στα μοντέλα IRT, επειδή επιτρέπει στους ερευνητές να αποφασίζουν εύκολα ποια ερωτήματα θα επιλέξουν, βασιζόμενοι στο αντικείμενο που ενδιαφέρει, αλλά και στην επίδραση των ερωτημάτων στη συνολική πληροφορία της δοκιμασίας. Έτσι, μπορούν να επιλεγούν ερωτήματα που παρουσιάζουν περισσότερη πληροφορία σε συγκεκριμένη περιοχή της κλίμακας της ικανότητας θ ώστε να μεγιστοποιηθεί η ακρίβεια των αποτελεσμάτων στην περιοχή αυτή (Zanon, Hutz, Yoo, & Hambleton, 2016).…”

Item response theory: A basic concept

“…Likewise, the higher the item difficulty level, the higher the test participants' ability to answer the item correctly. Despite a claim stating that modern theory cannot substitute classic theory (Zanon et al, 2016), based on the aforementioned description, the basic concept of item response theory is considered a strong theory compared to that of classic theory.…”