Global, Local, and Graphical Person-Fit Analysis Using Person-Response Functions.

Abstract

Person-fit statistics test whether the likelihood of a respondent's complete vector of item scores on a test is low given the hypothesized item response theory model. This binary information may be insufficient for diagnosing the cause of a misfitting item-score vector. The authors propose a comprehensive methodology for person-fit analysis in the context of nonparametric item response theory. The methodology (a) includes H. Van der Flier's (1982) global person-fit statistic U3 to make the binary decision about fit or misfit of a person's item-score vector, (b) uses kernel smoothing (J. O. Ramsay, 1991) to estimate the person-response function for the misfitting item-score vectors, and (c) evaluates unexpected trends in the person-response function using a new local person-fit statistic (W. H. M. Emons, 2003). An empirical data example shows how to use the methodology for practical person-fit analysis.

Item Randomized-Response Models for Measuring Noncompliance: Risk-Return Perceptions, Social Influences, and Self-Protective Responses

“…Even when respondents participate actively in the randomization process to protect their privacy, some of them may not be convinced that the protective measures are effective, and, as a consequence, they may not follow the randomization scheme and provide a truthful answer. If the number of items is sufficiently large (J ≥ 40), both global and local person-fit statistics (Emons, Sijtsma, & Meijer, 2005) can be developed for IRR models that allow identifying such respondents. However, these methods are of little use for a smaller number of items because they lack power for a satisfactory detection rate.…”

On the Usefulness of a Multilevel Logistic Regression Approach to Person-Fit Analysis

“…For dichotomously scored items, the PRF provides the relationship between an examinee's probability of having a 1 score on an item as a function of the item's location. Lumsden (1978), Ferrando (2004Ferrando ( , 2007, and Emons et al (2005) noticed that the PRF based on the 1PLM decreases. Emons et al (2005) argued that a PRF that increases locally indicates misfit to the 1PLM and that the location of the increase in the PRF on the latent scale and also the shape of the PRF provide diagnostic information about misfit.…”

“…Lumsden (1978), Ferrando (2004Ferrando ( , 2007, and Emons et al (2005) noticed that the PRF based on the 1PLM decreases. Emons et al (2005) argued that a PRF that increases locally indicates misfit to the 1PLM and that the location of the increase in the PRF on the latent scale and also the shape of the PRF provide diagnostic information about misfit. For example, for average-ability examinees low probabilities of correct responses on the first and easiest items might signal test anxiety, and for low-ability examinees high probabilities of correct responses on the most difficult items might signal cheating.…”

“…For example, Ferrando (2004Ferrando ( , 2007) developed a PRF model based on Lumsden's Thurstonian model (1977), and Strandmark and Linn (1987) formulated the PRF as a generalized logistic response model. In the context of nonparametric IRT, Sijtsma and Meijer (2001) and Emons et al (2004Emons et al ( , 2005 estimated PRFs using nonparametric regression methods such as binning and kernel smoothing, and for parametric IRT, Trabin and Weiss (1983) and Nering and Meijer (1998) used binning to estimate the PRF.…”

Personality Assessment of Global Talent: Conceptual and Methodological Issues

“…An alternative way of testing the applicability of norms may be the application of person-fit statistics. There is an extensive set of statistical techniques available that provide information on to what extent individual score profiles such as the vectors of item scores can be seen as belonging to a specific target population that has a known set of scores on the same items (Emons, Sijtsma, & Meijer, 2005). The use of norms to the heterogeneous applicant pool would then be restricted to those applicants who have a profile that does not differ significantly from the profile of the normative population.…”