Controlling Bias in Both Constructed Response and Multiple‐Choice Items When Analyzed With the Dichotomous Rasch Model

Andrich, David; Marais, Ida

doi:10.1111/jedm.12176

Cited by 7 publications

(3 citation statements)

References 16 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These findings come in agreement to earlier claims by Bachman and Palmer (1996: 202) who indicated that: "an item would be significantly more difficult if the options were closer in meaning because that would make identifying the correct answer more demanding for the test-taker". In support, Andrich & Marais (2018), declared that the more difficult the item, the greater the degree of guessing and that persons with greater proficiency tended to correctly answer the more difficult items at a greater rate than the less proficient.…”

Section: Discussionmentioning

confidence: 99%

Fairness and Ethics in Multiple Choice (MC) Scoring: An Empirical Study

Ypsilandis¹,

Mouti²

2019

JLE

View full text Add to dashboard Cite

One among the main concerns of language testers in the design and implementation of tests is selecting the method of scoring for the tool used to perform the evaluation. This attribute indirectly reveals the tester’s ethical beliefs and personal stance on testing pedagogy. This is another study challenging the typical 1-0 method of scoring in Multiple Choice Tests (MCT) and implements, for experimental purposes, a simple polychotomous partial-credit scoring system on official tests administered for the National Foreign Language Exam System in Greece (NFLES-Gr). The study comes in support of earlier findings on the subject by the same authors in analogous smaller-scale studies. The MCT items chosen were completed by a total of 1,922 subjects in different levels of the NFLES-Gr test for Italian as an L2 in Greece. Results clearly indicate that the tested scoring procedure provides refined insights into students’ interlanguage levels, enhances sensitivity in scoring procedures, and may provide significant differences for testees found to be close to the pass/non-pass borderline without jeopardizing test reliability.

show abstract

Section: Discussionmentioning

confidence: 99%

Fairness and Ethics in Multiple Choice (MC) Scoring: An Empirical Study

Ypsilandis¹,

Mouti²

2019

JLE

View full text Add to dashboard Cite

show abstract

“…Distractor response options are an important part of test development and the quality of distractors selected determines the difficulty and discrimination of items (Andrich and Styles 2011;Rodriguez 2005). We chose to select distractors at the item level based on the confusions of emotion labels from Study 1.…”

Section: Selection Of Distractor Itemsmentioning

confidence: 99%

The Meso-Expression Test (MET): A Novel Assessment of Emotion Perception

et al. 2023

View full text Add to dashboard Cite

Emotion perception is a primary facet of Emotional Intelligence (EI) and the underpinning of interpersonal communication. In this study, we examined meso-expressions—the everyday, moderate-intensity emotions communicated through the face, voice, and body. We theoretically distinguished meso-expressions from other well-known emotion research paradigms (i.e., macro-expression and micro-expressions). In Study 1, we demonstrated that people can reliably discriminate between meso-expressions, and we created a corpus of 914 unique video displays of meso-expressions across a race- and gender-diverse set of expressors. In Study 2, we developed a novel video-based assessment of emotion perception ability: The Meso-Expression Test (MET). In this study, we found that the MET is psychometrically valid and demonstrated measurement equivalence across Asian, Black, Hispanic, and White perceiver groups and across men and women. In Study 3, we examined the construct validity of the MET and showed that it converged with other well-known measures of emotion perception and diverged from cognitive ability. Finally, in Study 4, we showed that the MET is positively related to important psychosocial outcomes, including social well-being, social connectedness, and empathic concern and is negatively related to alexithymia, stress, depression, anxiety, and adverse social interactions. We conclude with a discussion focused on the implications of our findings for EI ability research and the practical applications of the MET.

show abstract

“…Table 1 summarizes the fixed and manipulated characteristics of the simulated data. In all conditions, I generated ratings using a five-category rating scale (x = 0, 1, 2, 3, 4) because this scale length is popular in applied and methodological psychometric research related to both affective scales (e.g., Bozdag˘& Bilge, 2022;Haddad et al, 2021;Hagedoorn et al, 2018;Moors, 2008;Waugh, 2002) and performance assessments that include ordinal rating scales (e.g., Andrich, 2010;Wind & Walker, 2019). I generated person parameters (u) and overall item locations (d i ) from a standard normal distribution to reflect recent simulation and real data IRT research related to rating scales (Buchholz & Hartig, 2019;Finch, 2011; Wind & Guo, 2019;Wolfe et al, 2014).…”

Section: Simulation Studymentioning

confidence: 99%

Detecting Rating Scale Malfunctioning With the Partial Credit Model and Generalized Partial Credit Model

Wind

2022

Educational and Psychological Measurement

View full text Add to dashboard Cite

Rating scale analysis techniques provide researchers with practical tools for examining the degree to which ordinal rating scales (e.g., Likert-type scales or performance assessment rating scales) function in psychometrically useful ways. When rating scales function as expected, researchers can interpret ratings in the intended direction (i.e., lower ratings mean “less” of a construct than higher ratings), distinguish between categories in the scale (i.e., each category reflects a unique level of the construct), and compare ratings across elements of the measurement instrument, such as individual items. Although researchers have used these techniques in a variety of contexts, studies are limited that systematically explore their sensitivity to problematic rating scale characteristics (i.e., “rating scale malfunctioning”). I used a real data analysis and a simulation study to systematically explore the sensitivity of rating scale analysis techniques based on two popular polytomous item response theory (IRT) models: the partial credit model (PCM) and the generalized partial credit model (GPCM). Overall, results indicated that both models provide valuable information about rating scale threshold ordering and precision that can help researchers understand how their rating scales are functioning and identify areas for further investigation or revision. However, there were some differences between models in their sensitivity to rating scale malfunctioning in certain conditions. Implications for research and practice are discussed.

show abstract

Controlling Bias in Both Constructed Response and Multiple‐Choice Items When Analyzed With the Dichotomous Rasch Model

Cited by 7 publications

References 16 publications

Fairness and Ethics in Multiple Choice (MC) Scoring: An Empirical Study

Fairness and Ethics in Multiple Choice (MC) Scoring: An Empirical Study

The Meso-Expression Test (MET): A Novel Assessment of Emotion Perception

Detecting Rating Scale Malfunctioning With the Partial Credit Model and Generalized Partial Credit Model

Contact Info

Product

Resources

About