A New Person‐Fit Statistic for the Lognormal Model for Response Times

Abstract

Response‐time models are of increasing interest in educational and psychological testing. This article focuses on the lognormal model for response times, which is one of the most popular response‐time models, and suggests a simple person‐fit statistic for the model. The distribution of the statistic under the null hypothesis of no misfit is proved to be a χ2 distribution. A simulation study and a real data example demonstrate the usefulness of the suggested statistic.

The Detection of Cheating on E-Exams in Higher Education—The Performance of Several Old and Some New Indicators

“…The Z 2 value of an examinee is the sum of the squared doubly standardized log response times over the questions. The Z 2 statistic is closely related to the log-normal model of van der Linden ( 2006 ) and very similar to the statistics proposed by Marianti et al ( 2014 ) and Sinharay ( 2018 ). Although the three statistics differ in the way they weight the response times, they are highly correlated.…”

“…We also considered indicators that are based on the response times. These indicators were the KL statistic (Man et al, 2018 ), a Z 2 statistic similar to the one proposed by Marianti et al ( 2014 ) and Sinharay ( 2018 ), and a new index—the KT statistic—that evaluates the Guttman homogeneity of an examinee's response time pattern. We furthermore considered indicators that were based on the number of response revisions and the corresponding response times (Qualls, 2005 ; Bishop and Egan, 2017 ).…”

Detection of Item Preknowledge Using Response Times

“…The LNMRT is arguably one of the most popular RTMs. The model was considered, either to analyze only the response times, or to analyze the response times and item scores, by, for example, Bolsinova and Tijmstra (2018), Boughton et al (2017), Glas and van der Linden (2010), Qian et al (2016), Sinharay (2018), van der Linden (2007, 2009, 2016), van der Linden and Glas (2010), and van der Linden and Guo (2008). Bolsinova and Tijmstra (2018, p. 13) commented that the LNMRT is used in most applications of RTMs.…”

“…The χ pf statistic can be used to detect item preknowledge. Marianti et al (2014) and Fox and Marianti (2017) suggested a Bayesian person-fit analysis approach that was found to perform very similarly, but slightly worse than the χ pf statistic by Sinharay (2018)—so their Bayesian approach is not considered henceforth.…”

Use of Data Mining Methods to Detect Test Fraud

“…Proposed methods to detect test fraud generally align with a particular type of cheating behavior. (Kling, 1979, cited in Saretsky, 1984; K 1 and K 2 (Sotaridona & Meijer, 2003); VM (Belov, 2011); S 2 (Sotaridona & Meijer, 2003); ω (Wollack, 1997); D (Trabin & Weiss, 1983); l z (Drasgow, Levine, & Williams, 1985); Hierarchical RT approach (van der Linden & Guo, 2008); Z c (Meijer & Sotaridona, 2006); KL (Man, Harring, Ouyang, & Thomas, 2018); RT residual analysis (H. Qian, Staniewska, Reckase, & Woo, 2016); Bivariate lognormal RT analysis (van der linden, 2009); l s (Marianti, Fox, Avetisyan, Veldkamp, & Tijmstra, 2014); l y s (Fox & Marianti, 2017); χ pt (Sinharay, 2018) Suspicious Answer Changing Linear regression analysis (Primoli, Liassou, Bishop, & Nhouyvanisvong, 2011); Generalized WR analysis (van der Linden & Jeon, 2012); GBT (van der Linden & ; EDI (Wollack, Cohen, & Eckerly, 2015); D(G||h) (Belov, 2015); PPD EDI (Sinharay & Johnson, 2017) Suspicious Gain Scores BHLM (Skorupski & Egan, 2011); EDI g (Wollack & Eckerly, 2017) Methods to detect unexpected gain scores, collusion, preknowledge of items, and other unspecified aberrant test-taking behaviors include the cumulative distribution method (Holland, 2002), l * z index (Drasgow, Levine, & McLaughlin, 1987;Snijders, 2001), H T index (Sijtsma, 1986;Sijtsma & Meijer, 1992), ω (Wollack, 1997), L s (Sinharay, 2017), erasure detection index (Wollack, Cohen, & Eckerly, 2015), and S (Belov, 2015), which are based on individuals' item scores.…”