Stability, reliability, and validity of the THINC‐it screening tool for cognitive impairment in depression: A psychometric exploration in healthy volunteers

Harrison, John; Barry, Harry; Baune, Bernhard T.; Best, Michael W.; Bowie, Christopher R.; Danielle, S.; Culpepper, Larry; Fossati, Philippe; Greer, Tracy L.; Harmer, Catherine J.; Klag, Esther; Lam, Raymond W.; Lee, Yena; Mansur, Rodrigo B.; Wittchen, H.‐U.; McIntyre, Roger S.

doi:10.1002/mpr.1736

Cited by 29 publications

(33 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…25 When the THINC-it psychological measurement was conducted, it was found that the convergence validity of Symbol Check task was low. 26 In our study, in order to reduce the test error, bipolar depression group, and healthy control group completed the testing in a strict order. However, variability in educational levels, depression severity, and cognitive function resulted in differences in the duration (1.5-4 hours), quality, and accuracy (about 50%) in completing the test.…”

Section: Discussionmentioning

confidence: 99%

<p>Reliability and Validity of THINC-it in Evaluating Cognitive Function of Patients with Bipolar Depression</p>

Zhang

Zhu

Lai

et al. 2020

NDT

View full text Add to dashboard Cite

The THINC-integrated tool (THINC-it) as a brief screening tool can assesses cognitive impairment in patients with major depressive depression (MDD). Here, we aim to evaluate the reliability and validity of the THINC-it in a bipolar depression (BD-D) group in comparison with a healthy control (HC) group. Materials and Methods: Both groups were matched according to age, gender, years of education, and IQ. All participants completed the THINC-it test, including Spotter, Symbol Check, Codebreaker, Trails, and the Perceived Deficits Questionnaire for Depression-5-item (PDQ-5-D). The concurrent validity and internal consistency of the THINC-it test were analyzed, and 30 healthy controls were randomly sampled to retest THINC-it to verify the reliability of the THINC-it retest. The correlation between THINC-it and Hamilton Depression Scale (HAMD-17) and Sheehan Disability Scale (SDS) was also analyzed. Results: Fifty-eight patients with BD-D and 61 HCs were included for final analysis. There were significant mean difference (MD) standard errors (SE) between two groups in PDQ-5-D, Spotter and Codebreaker (all P<0.01), Trails (P=0.015). There was no significant difference in Symbol Check (MD (SE)=−0.01 (0.18), P=0.938; 95% CI=−0.38 to 0.35). The Cronbach's α of PDQ-5-D was 0.640. The intraclass correlation coefficient (ICC) was between 0.440 and 0.757. The highest concurrent validity was PDQ-5-D (r=0.812, P<0.001). PDQ-5-D was positively correlated with HAMD-17 and SDS score (P<0.01). The objective test had no significant correlation with HAMD-17 and SDS scores (P>0.05). Conclusion: This study found that THINC-it can accurately present the cognitive impairment of patients with BD-D. It can be potentially applied in assessing the cognitive function of patients with BD-D although Symbol Check may not accurately reflect the level of cognitive function. The concurrent validity and retest reliability are lower than expected, we need to further increase the sample size to study.

show abstract

Section: Discussionmentioning

confidence: 99%

<p>Reliability and Validity of THINC-it in Evaluating Cognitive Function of Patients with Bipolar Depression</p>

Zhang

Zhu

Lai

et al. 2020

NDT

View full text Add to dashboard Cite

show abstract

“…The insignificant error rates of ASR, as calculated for the Danish and English responses (6.3% and 17.8%, respectively), indicate a promising future of ASR, particularly for Danish-speaking patients who will be the primary users of the ICAT. According to the results obtained from the recent THINC-it validity study on healthy subjects [10], the 2 cognitive games called Trails (executive function, r=0.74) and Codebreaker (attention, working memory, and executive function, r=0.63) revealed strong to moderate convergent validity, respectively, whereas Symbol Check (working memory, r=0.19) and Spotter (attention, r=0.44) showed low validity. In our study, the ICAT subtest for psychomotor speed also showed moderate concurrent validity, as did the subtest for verbal memory.…”

Section: Discussionmentioning

confidence: 99%

“…For example, the Trails game is adapted from the trail-making test part B. According to the latest study [10], 100 healthy controls were tested for temporal stability and reliability as well as the validity of the THINC-it. Overall, high stability and reliability and moderate validity were found.…”

Section: Introductionmentioning

confidence: 99%

The Internet-Based Cognitive Assessment Tool: System Design and Feasibility Study

Hafiz¹,

Miskowiak²,

Kessing³

et al. 2019

JMIR Form Res

View full text Add to dashboard Cite

Background Persistent cognitive impairment is prevalent in unipolar and bipolar disorders and is associated with decreased quality of life and psychosocial dysfunction. The screen for cognitive impairment in psychiatry (SCIP) test is a validated paper-and-pencil instrument for the assessment of cognition in affective disorders. However, there is no digital cognitive screening tool for the brief and accurate assessment of cognitive impairments in this patient group. Objective In this paper, we present the design process and feasibility study of the internet-based cognitive assessment tool (ICAT) that is designed based on the cognitive tasks of the SCIP. The aims of this feasibility study were to perform the following tasks among healthy individuals: (1) evaluate the usability of the ICAT, (2) investigate the feasibility of the ICAT as a patient-administered cognitive assessment tool, and (3) examine the performance of automatic speech recognition (ASR) for the assessment of verbal recall. Methods The ICAT was developed in a user-centered design process. The cognitive measures of the ICAT were immediate and delayed recall, working memory, and psychomotor speed. Usability and feasibility studies were conducted separately with 2 groups of healthy individuals (N=21 and N=19, respectively). ICAT tests were available in the English and Danish languages. The participants were asked to fill in the post study system usability questionnaire (PSSUQ) upon completing the ICAT test. Verbal recall in the ICAT was assessed using ASR, and the performance evaluation criterion was word error rate (WER). A Pearson 2-tailed correlation analysis significant at the .05 level was applied to investigate the association between the SCIP and ICAT scores. Results The overall psychometric factors of PSSUQ for both studies gave scores above 4 (out of 5). The analysis of the feasibility study revealed a moderate to strong correlation between the total scores of the SCIP and ICAT (r=0.63; P =.009). There were also moderate to strong correlations between the SCIP and ICAT subtests for immediate verbal recall (r=0.67; P =.002) and psychomotor speed (r=0.71; P =.001). The associations between the respective subtests for working memory, executive function, and delayed recall, however, were not statistically significant. The corresponding WER for English and Danish responses were 17.8% and 6.3%, respectively. Conclusions The ICAT is the first digital screening instrument modified from the SCIP using Web-based technology and ASR. There was good accuracy of the ASR for verbal memory assessment. The moderate correlation between the ICAT and SCIP scores suggests that the ICAT is a valid tool for assessing cognition, although this should be confirmed in a larger study with greater statistical power. Taken together, the ICAT seems to be a valid W...

show abstract

“…Two validated computer-based tools, PsyToolkit [ 36 , 37 ] and the THINC-it application [ 15 ], were run on a MacBook Pro (15-inch Retina display, Apple Inc) during the study. A Fitbit Ionic smartwatch (1.42-inch screen, 348 × 250 pixel resolution) was used to run the UbiCAT apps.…”

Section: Methodsmentioning

confidence: 99%

“…Digital cognitive screening tools have been designed for different technological platforms, targeting both mentally ill patients and healthy individuals. The Cambridge Neuropsychological Test Automated Battery (CANTAB) Mobile [ 13 ], the Internet-based Cognitive Assessment Tool (ICAT) [ 14 ], the THINC-integrated tool (THINC-it) [ 15 ], MyCognition Quotient (MyCQ) [ 16 ], CogState [ 17 ], and the Brief Assessment of Cognition in Schizophrenia (BACS) [ 18 ] are some examples of the validated cognitive test batteries administered on a computer or tablet. The existing cognitive test batteries are administered at a certain time in a controlled condition.…”

Section: Introductionmentioning

confidence: 99%

The Ubiquitous Cognitive Assessment Tool for Smartwatches: Design, Implementation, and Evaluation Study

Hafiz¹,

Bardram²

2020

JMIR Mhealth Uhealth

View full text Add to dashboard Cite

Background Cognitive functioning plays a significant role in individuals’ mental health, since fluctuations in memory, attention, and executive functions influence their daily task performance. Existing digital cognitive assessment tools cannot be administered in the wild and their test sets are not brief enough to capture frequent fluctuations throughout the day. The ubiquitous availability of mobile and wearable devices may allow their incorporation into a suitable platform for real-world cognitive assessment. Objective The aims of this study were threefold: (1) to evaluate a smartwatch-based tool for the assessment of cognitive performance, (2) to investigate the usability of this tool, and (3) to understand participants’ perceptions regarding the application of a smartwatch in cognitive assessment. Methods We built the Ubiquitous Cognitive Assessment Tool (UbiCAT) on a smartwatch-based platform. UbiCAT implements three cognitive tests—an Arrow test, a Letter test, and a Color test—adapted from the two-choice reaction-time, N-back, and Stroop tests, respectively. These tests were designed together with domain experts. We evaluated the UbiCAT test measures against standard computer-based tests with 21 healthy adults by applying statistical analyses significant at the 95% level. Usability testing for each UbiCAT app was performed using the Mobile App Rating Scale (MARS) questionnaire. The NASA-TLX (Task Load Index) questionnaire was used to measure cognitive workload during the N-back test. Participants rated perceived discomfort of wearing a smartwatch during the tests using a 7-point Likert scale. Upon finishing the experiment, an interview was conducted with each participant. The interviews were transcribed and semantic analysis was performed to group the findings. Results Pearson correlation analysis between the total correct responses obtained from the UbiCAT and the computer-based tests revealed a significant strong correlation (r=.78, P<.001). One-way analysis of variance (ANOVA) showed a significant effect of the N-back difficulty level on the participants' performance measures. The study also demonstrated usability ratings above 4 out of 5 in terms of aesthetics, functionality, and information. Low discomfort (<3 out of 7) was reported by our participants after using the UbiCAT. Seven themes were extracted from the transcripts of the interviews conducted with our participants. Conclusions UbiCAT is a smartwatch-based tool that assesses three key cognitive domains. Usability ratings showed that participants were engaged with the UbiCAT tests and did not feel any discomfort. The majority of the participants were interested in using the UbiCAT, although some preferred computer-based tests, which might be due to the widespread use of personal computers. The UbiCAT can be administered in the wild with mentally ill patients to assess their attention, working memory, and executive function.

show abstract

Stability, reliability, and validity of the THINC‐it screening tool for cognitive impairment in depression: A psychometric exploration in healthy volunteers

Cited by 29 publications

References 20 publications

<p>Reliability and Validity of THINC-it in Evaluating Cognitive Function of Patients with Bipolar Depression</p>

<p>Reliability and Validity of THINC-it in Evaluating Cognitive Function of Patients with Bipolar Depression</p>

The Internet-Based Cognitive Assessment Tool: System Design and Feasibility Study

The Ubiquitous Cognitive Assessment Tool for Smartwatches: Design, Implementation, and Evaluation Study

Contact Info

Product

Resources

About