Philips IntelliSpace Cognition digital test battery: Equivalence and measurement invariance compared to traditional analog test versions

Abstract

Objective: To collect evidence of validity for a selection of digital tests on the Philips IntelliSpace Cognition (ISC) platform. Method: A total of 200 healthy participants (age 50-80) completed both the ISC battery and an analog version of the battery during separate visits. The battery included the following screeners and cognitive tests: Mini-Mental State Examination (2 nd edition), Clock Drawing Test, Trail-Making Test (TMT), Rey Auditory Verbal Learning Test (RAVLT), Rey-Osterrieth Complex Figure Test (ROCFT), Letter Fluency, Star Cancellation Test, and Digit Span Test. The ISC tests were administered on an iPad Pro and were automatically scored using designated algorithms. The analog tests were administered in line with existing guidelines and scored by trained neuropsychologists. Criterion validity was established through relative agreement coefficients and raw score equivalence tests. In addition,measurement invariance analysis was used to compare the factor structures of both versions. Finally, we explored effects of demographics and experience with digital devices on performance. Results: We found fair to excellent relative agreement between test versions. Absolute equivalence was found for RAVLT, Letter Fluency, Star Cancellation Test, and Digit Span Test. Importantly, we demonstrated equal loadings of the digital and analog test versions on the same set of underlying cognitive domains. Demographic effects were mostly comparable between modalities, and people's experience with digital devices was found to only influence performance on TMT B. Conclusions: This study provides several sources of evidence for the validity of the ISC test battery, offering an important step in validating ISC for clinical use.Digital cognitive testing has been gaining traction in recent years as a powerful and flexible alternative to traditional analog testing. Its potential benefits are well-documented, and include a high level of standardization, a significant increase © 2021 Koninklijke philips n.V. published by informa uK limited, trading as Taylor & Francis group CONTACT gijs van elswijk

Validation of At-Home Application of a Digital Cognitive Screener for Older Adults

“…More importantly, the lack of correlations in individual differences between RCM and PAP on both measures show that the participants experienced the challenge to demonstrate their working memory span very differently between the remote digital task and the in-clinic task with a live administrator. By contrast, a relevant study that compared digit span assessments using an administrator in-clinic found only small effects of older adults' higher performance on PAP than on digital tests, and more significantly, fair to good comparability between participants' raw scores in the two tests (Vermeent et al, 2021). Interestingly, our pilot data from a new cohort of participants in a follow-up study, who ran both the RCM and PAP verbal digit span tasks in the laboratory during a one-session experiment using counterbalanced order for the method, support the interpretation that a majority of older adults are more comfortable responding to an in-person than remote assessment (Germine et al, 2019).…”

“…Our design and analysis were concerned with testing the similarity in participants' performance between PAP and RCM versions of each task, and also the comparability of individual differences in participants' scores. Consistent with several recent and relevant reports that have compared digital methods of cognitive screening (Bjorngrim et al, 2019;Domen et al, 2019;Koenig et al, 2019;Sacco et al, 2019;Mackin et al, 2021;Vermeent et al, 2021), we use the similarity of mean PAP and RCM scores on a task to inform construct validity for the RCM version of that measure, and the comparability of participants' individual differences (PAP versus RCM) to inform inter-method reliability and validity of the RCM version of that measure. Our hypothesis was that pairwise comparisons of the data collected on each task (i.e., digital versus PAP versions) would not show statistically meaningful differences between versions, and we would interpret this finding as confirmation that participants experience the remotely collected RCM and inperson PAP tasks as being highly similar.…”

“…More recent projects have focused on the advantages of automation in administering neurocognitive testing (Koenig et al, 2019;Sacco et al, 2019). These proof-of-concept studies, which collected data using in-person testing with trained technicians, have reported fair correspondence between the digital and analog test versions (Bjorngrim et al, 2019;Sacco et al, 2019;Vermeent et al, 2021), although findings vary between particular cognitive domains. Results have shown strong correlations between digital and analog raw scores in measurements of older adults' performance on immediate verbal memory tests, but moderate to weak correlations in their performance on verbal tests of delayed recall and digit span, as well as executive control during set-shifting (Bjorngrim et al, 2019;Sacco et al, 2019;Mackin et al, 2021;Vermeent et al, 2021).…”

Long-Term Effects of Repeated Blast Exposure in United States Special Operations Forces Personnel: A Pilot Study Protocol

“…Assessments are distributed across the first and second days of the study. Some assessments are administered via a computer (e.g., Automated Neuropsychological Assessment Metric) 102 or iPad (e.g., Philips Intellispace Cognition) 103 to complement traditional tests and obtain granular performance metrics (e.g., reaction time). Pupillometry measurements are obtained to assess the integrity of pupil constriction and dilation—a metric of cognitive processing load tolerance.…”

Is the analogue cognitive test from the ISPOCD equivalent to the digital cognitive test Mindmore? A protocol for a randomised cross-over study including qualitative interviews with self-reported healthy seniors

“…The participants will not know on beforehand which test they will start with, only the test leaders will know. The chosen interval was guided by earlier studies comparing digitalised versus analogue tests of cognitive function in healthy participants people with a wash out period ranging from 30 min to 4 weeks between the studies 33 35 44. Data will be both electronically and manually, and the data forms will be kept at the university site in a locked cabinet.…”