Enhancing the Efficiency of Confrontation Naming Assessment for Aphasia Using Computer Adaptive Testing

Abstract

Fergadiotis, 2015) created using item response theory (IRT) methods. Method: The full PNT and the PNT-CAT were administered to 47 participants with aphasia in counterbalanced order. Latent trait-naming ability estimates for the 2 PNT versions were analyzed in a Bayesian framework, and the agreement between them was evaluated using correlation and measures of constant, variable, and total error. We also evaluated the extent to which individual pairwise differences were credibly greater than 0 and whether the IRT measurement model provided an adequate indication of the precision of individual score estimates. Results: The agreement between the PNT and the PNT-CAT was strong, as indicated by high correlation (r = .95, 95% CI [.92, .97]), negligible bias, and low variable and total error. The number of statistically robust pairwise score differences did not credibly exceed the Type I error rate, and the precision of individual score estimates was reasonably well predicted by the IRT model. Discussion: The strong agreement between the full PNT and the PNT-CAT suggests that the latter is a suitable measurement of anomia in group studies. The relatively robust estimates of score precision also suggest that the PNT-CAT can be useful for the clinical assessment of anomia in individual cases. Finally, the IRT methods used to construct the PNT-CAT provide a framework for additional development to further reduce measurement error.

Rational Adaptation in Using Conceptual Versus Lexical Information in Adults With Aphasia

“…The posterior distributions are summarized by the estimated parameters and 95% highest density credible intervals (HDI). The HDI is comparable to the frequentist confidence interval and is determined as the narrowest interval containing the assigned proportion of the posterior distribution’s probability mass within which all values have a higher probability density than any values outside the interval (see Fergadiotis et al, 2019 for further explanation).…”

Empirical Evaluation of Computer-Adaptive Alternate Short Forms for the Assessment of Anomia Severity

“…Preliminary fitting and comparison of singlelevel and multilevel regression models were also conducted to test for effects of data collection site. Interrater reliability for scoring of the individual naming responses was assessed for a random subsample drawn from a larger sample that included the present participants and is reported by Fergadiotis et al (2019).…”

“…To generate conservative estimates of the posterior distributions, each parameter was assigned a vague prior (McElreath, 2018). The Stan code, including the prior specifications, is provided in the supplementary materials of Fergadiotis et al (2019). To allow the Bayesian estimation process to explore the full parameter space and assess whether they arrive at the same distribution, dispersed starting values were assigned to four Hamiltonian Markov chain Monte Carlo (MCMC) chains.…”

“…The PNT-CAT30 was administered in the first session, and the PNT-CATVL was administered approximately 2 weeks later. The 2-week test-retest interval was chosen to control threats to internal validity related to maturation and history while minimizing the potential for test practice effects on other parts of the protocol reported elsewhere (Fergadiotis, Hula, Swiderski, Lei, & Kellough, 2019;. For the first trial of the PNT-CAT30, each PWA was initially assigned a provisional ability score of 0 and was presented with the item "pumpkin," whose difficulty value was estimated to be very close to 0 on the same latent trait scale, making it one of the most informative items in the PNT item bank for that particular ability level (Fergadiotis et al, 2015; see Supplemental Material S1).…”

The Utility of Multiplex Closeness Centrality for Predicting Item Difficulty Parameters in Anomia Tests

“…The computer adaptive test engine was further validated in two follow up empirical studies. Specifically, Fergadiotis, Hula, Swiderski, Lei, and Kellough ( 2019) demonstrated that scores on the PNT -CAT from 47 PWA were in high agreement with their scores on the full PNT. In addition, Hula et al (2020) developed two equivalent, dynamic, test forms with non-overlapping items for repeated administrations.…”