Computer Adaptive Testing for the Assessment of Anomia Severity

Fergadiotis, Gerasimos; Casilio, Marianne; Hula, William D.; Swiderski, Alexander M.

doi:10.1055/s-0041-1727252

Cited by 7 publications

(1 citation statement)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further integration of digital technology can allow the STELA to assess language ability even more rapidly while keeping the granularity. For example, the employment of computer adaptive testing methodology [ 27 ] may contribute to further shorten the administration time of the test and reduce the stress of the patients who receive the assessment. Freeing severely impaired patients from distress caused by continually confronting them with challenging questions could be critical to support their motivation to engage in rehabilitation training and adherence.…”

Section: Discussionmentioning

confidence: 99%

A Tablet-Based Aphasia Assessment System “STELA”: Feasibility and Validation Study

Inamoto¹,

Mukaino²,

Imaeda³

et al. 2023

JMIR Form Res

View full text Add to dashboard Cite

Background There is an extensive library of language tests, each with excellent psychometric properties; however, many of the tests available take considerable administration time, possibly bearing psychological strain on patients. The Short and Tailored Evaluation of Language Ability (STELA) is a simplified, tablet-based language ability assessment system developed to address this issue, with a reduced number of items and automated testing process. Objective The aim of this paper is to assess the administration time, internal consistency, and validity of the STELA. Methods The STELA consists of a tablet app, a microphone, and an input keypad for clinician’s use. The system is designed to assess language ability with 52 questions grouped into 2 comprehension modalities (auditory comprehension and reading comprehension) and 3 expression modalities (naming and sentence formation, repetition, and reading aloud). Performance in each modality was scored as the correct answer rate (0-100), and overall performance expressed as the sum of modality scores (out of 500 points). Results The time taken to complete the STELA was significantly less than the time for the WAB (mean 16.2, SD 9.4 vs mean 149.3, SD 64.1 minutes; P<.001). The STELA’s total score was strongly correlated with the WAB Aphasia Quotient (r=0.93, P<.001), supporting the former’s concurrent validity concerning the WAB, which is a gold-standard aphasia assessment. Strong correlations were also observed at the subscale level; STELA auditory comprehension versus WAB auditory comprehension (r=0.75, P<.001), STELA repetition versus WAB repetition (r=0.96, P<.001), STELA naming and sentence formation versus WAB naming and word finding (r=0.81, P<.001), and the sum of STELA reading comprehension or reading aloud versus WAB reading (r=0.82, P<.001). Cronbach α obtained for each modality was .862 for auditory comprehension, .872 for reading comprehension, .902 for naming and sentence formation, .787 for repetition, and .892 for reading aloud. Global Cronbach α was .961. The average of the values of item-total correlation to each subscale was 0.61 (SD 0.17). Conclusions Our study confirmed significant time reduction in the assessment of language ability and provided evidence for good internal consistency and validity of the STELA tablet-based aphasia assessment system.

show abstract

Section: Discussionmentioning

confidence: 99%

A Tablet-Based Aphasia Assessment System “STELA”: Feasibility and Validation Study

Inamoto¹,

Mukaino²,

Imaeda³

et al. 2023

JMIR Form Res

View full text Add to dashboard Cite

show abstract

Item Response Theory Modeling of the Verb Naming Test

Fergadiotis

Casilio

Dickey

et al. 2023

J Speech Lang Hear Res

View full text Add to dashboard Cite

Purpose: Item response theory (IRT) is a modern psychometric framework with several advantageous properties as compared with classical test theory. IRT has been successfully used to model performance on anomia tests in individuals with aphasia; however, all efforts to date have focused on noun production accuracy. The purpose of this study is to evaluate whether the Verb Naming Test (VNT), a prominent test of action naming, can be successfully modeled under IRT and evaluate its reliability. Method: We used responses on the VNT from 107 individuals with chronic aphasia from AphasiaBank. Unidimensionality and local independence, two assumptions prerequisite to IRT modeling, were evaluated using factor analysis and Yen's Q 3 statistic (Yen, 1984), respectively. The assumption of equal discrimination among test items was evaluated statistically via nested model comparisons and practically by using correlations of resulting IRT-derived scores. Finally, internal consistency, marginal and empirical reliability, and conditional reliability were evaluated. Results: The VNT was found to be sufficiently unidimensional with the majority of item pairs demonstrating adequate local independence. An IRT model in which item discriminations are constrained to be equal demonstrated fit equivalent to a model in which unique discrimination parameters were estimated for each item. All forms of reliability were strong across the majority of IRT ability estimates. Conclusions: Modeling the VNT using IRT is feasible, yielding ability estimates that are both informative and reliable. Future efforts are needed to quantify the validity of the VNT under IRT and determine the extent to which it measures the same construct as other anomia tests. Supplemental Material: https://doi.org/10.23641/asha.22329235

show abstract

Response Latencies During Confrontation Picture Naming in Aphasia: Are Proxy Measurements Sufficient?

Mazumdar,

De la Mora,

Roberts

et al. 2024

J Speech Lang Hear Res

View full text Add to dashboard Cite

Purpose: Anomia, or word-finding difficulty, is a prevalent and persistent feature of aphasia, a neurogenic language disorder affecting millions of people in the United States. Anomia assessments are essential for measuring performance and monitoring outcomes in clinical settings. This study aims to evaluate the reliability of response time (RT) annotation based on spectrograms and assess the predictive utility of proxy RTs collected during computerized naming tests. Method: Archival data from 10 people with aphasia were used. Trained research assistants phonemically transcribed participants' responses, and RTs were generated from the onset of picture stimulus to the initial phoneme of the first complete attempt. RTs were measured in two ways: hand-generated RTs (from spectrograms) and proxy RTs (automatically extracted online). Interrater agreement was evaluated based on interclass correlation coefficients and generalizability theory tools including variance partitioning and the φ-coefficient. The predictive utility of proxy RTs was evaluated within a linear mixed-effects framework. Results: RT annotation reliability showed near-perfect agreement across research assistants (φ-coefficient = .93), and the variance accounted for by raters was negligible. Furthermore, proxy RTs significantly and strongly predicted hand-annotated RTs ( R 2 = ~0.82), suggesting their utility as an alternative measure. Conclusions: The study confirms the reliability of RT annotation and demonstrates the predictive utility of proxy RTs in estimating RTs during computerized naming tests. Incorporating proxy RTs can enhance clinical assessments, providing additional information for cognitive measurement. Further research with larger samples and exploring the impact of using proxy RTs in different psychometric models could optimize clinical protocols and improve communication interventions for individuals with aphasia.

show abstract

Computer Adaptive Testing for the Assessment of Anomia Severity

Cited by 7 publications

References 39 publications

A Tablet-Based Aphasia Assessment System “STELA”: Feasibility and Validation Study

A Tablet-Based Aphasia Assessment System “STELA”: Feasibility and Validation Study

Item Response Theory Modeling of the Verb Naming Test

Response Latencies During Confrontation Picture Naming in Aphasia: Are Proxy Measurements Sufficient?

Contact Info

Product

Resources

About