Independent evaluation of Q‐Interactive: A paper equivalency comparison using the PPVT‐4 with preschoolers

Abstract

Pearson now uses a technology‐based testing platform, Q‐Interactive, to administer tests previously available in paper versions. The same norms are used for both versions; Pearson's in‐house equivalency studies indicated that both versions are equated. The goal of the current study is to independently evaluate equivalency findings. For the current study, equivalency was measured using the three‐part test set forth by American Psychological Association in 1986. First, the researchers examined rank order similarity; then, they examined mean score similarity; and finally, they examined score‐distribution similarity. One of these equivalency standards (rank order similarity) was not met, and one other standard is debatable (mean score similarity); therefore, the authors noted concerns about the use of Peabody Picture Vocabulary Test, Fourth Edition Q‐Interactive for preschoolers. New normative data should be collected.

Q-interactive: Training Implications for Accuracy and Technology Integration

“…Everything is included in one location, a tablet, rather than multiple materials (e.g., protocol, administration manual, notepad, or stopwatch). Krach et al ( 2019 ) reported that a set of tablets containing two complete tests may weigh 5 lbs, where the equivalent paper version with all of its materials might weigh 20 lbs or more. This is no small consideration for individuals who must daily transport test materials into and out of school buildings.…”

Meeting the COVID-19 Deadlines: Choosing Assessments to Determine Eligibility

“…Adaptation equivalency guidelines were set forward by the American Psychological Association (APA, 1986) in the Guidelines for Computer-Based Tests and Interpretations as well as by a joint commission in the Standards for Educational and Psychological Testing (AERA et al, 2014). Additional equivalency requirements were set forward by Grosch et al (2011), Hodge et al (2019), Krach, McCreery et al (2020), Cohen (1988), andFarmer et al (2020). Table 2 lists these guidelines as well as the parameters for how to evaluate each one when reading an equivalency study.…”

“…Use of any adapted, tele-assessment instrument lacking concurrent validity data between face-to-face and tele-assessment versions is not recommended. To feel truly comfortable with version equivalency, all of these standards should be met:research should be published in a reputable, academic source;between version correlations should be at or above .80 (Krach, McCreery et al, 2020). Concurrent validity standards are not sufficient for equivalency studies;no statistically significant differences between scores derived via different formats (AERA, APA, & NCME, 2014; APA, 1986);between version effect sizes should be at or below .2 (Daniel, Wahstrom, & Zang, 2014);normative dispersion of scores should have a statistically similar shape (AERA et al, 2014; APA, 1986);samples used in the study should match the U.S.…”

“…WJ-IV: COG = Woodcock–Johnson Tests of Cognitive Ability, Fourth Edition ; WJ-IV: ACH = Woodcock–Johnson Tests of Achievement, Fourth Edition ; RIASs = Reynolds Intellectual Assessment Scales ; TOGRA = Test of General Reasoning Ability ; RAIT = Reynolds Adaptable Intelligence Test ; WISC-V = Wechsler Intelligence Scales for Children, Fifth Edition ; CogAT = Cognitive Abilities Test. Note 2. Correlation requirement ≥ .80 established by Krach, McCreery et al (2020). Note 3. Effect size of less than .2 established by Daniel et al (2014). Note 4.…”

“…Effect size of less than .2 established by Daniel et al (2014). Note 4. Sample equivalency guidelines provided by Grosch et al (2011), Hodge et al (2019) and Krach, McCreery et al (2020). Note 5. Data on population breakdown derived from the U.S.…”