Device-Embedded Cameras for Eye Tracking–Based Cognitive Assessment: Validation With Paper-Pencil and Computerized Cognitive Composites

Abstract

BackgroundAs eye tracking-based assessment of cognition becomes more widely used in older adults, particularly those at risk for dementia, reliable and scalable methods to collect high-quality data are required. Eye tracking-based cognitive tests that utilize device-embedded cameras have the potential to reach large numbers of people as a screening tool for preclinical cognitive decline. However, to fully validate this approach, more empirical evidence about the comparability of eyetracking-based paradigms to existing cognitive batteries is needed.ObjectiveUsing a population of clinically normal older adults, we examined the relationship between a 30-minute Visual Paired Comparison (VPC) recognition memory task and cognitive composite indices sensitive to a subtle decline in domains associated with Alzheimer disease. Additionally, the scoring accuracy between software used with a commercial grade eye tracking camera at 60 frames per second (FPS) and a manually scored procedure used with a laptop-embedded web camera (3 FPS) on the VPC task was compared, as well as the relationship between VPC task performance and domain-specific cognitive function.MethodsA group of 49 clinically normal older adults completed a 30-min VPC recognition memory task with simultaneous recording of eye movements by a commercial-grade eye-tracking camera and a laptop-embedded camera. Relationships between webcam VPC performance and the Preclinical Alzheimer Cognitive Composite (PACC) and National Institutes of Health Toolbox Cognitive Battery (NIHTB-CB) were examined. Inter-rater reliability for manually scored tests was analyzed using Krippendorff’s kappa formula, and we used Spearman’s Rho correlations to investigate the relationship between VPC performance scores with both cameras. We also examined the relationship between VPC performance with the device-embedded camera and domain-specific cognitive performance.ResultsModest relationships were seen between mean VPC novelty preference and the PACC (r=.39, P=.007) and NIHTB-CB (r=.35, P=.03) composite scores, and additional individual neurocognitive task scores including letter fluency (r=.33, P=.02), category fluency (r=.36, P=.01), and Trail Making Test A (–.40, P=.006). Robust relationships were observed between the 60 FPS eye tracker and 3 FPS webcam on both trial-level VPC novelty preference (r=.82, P<.001) and overall mean VPC novelty preference (r=.92 P<.001). Inter-rater agreement of manually scored web camera data was high (kappa=.84).ConclusionsIn a sample of clinically normal older adults, performance on a 30-minute VPC task correlated modestly with computerized and paper-pencil based cognitive composites that serve as preclinical Alzheimer disease cognitive indices. The strength of these relationships did not differ between camera devices. We suggest that using a device-embedded camera is a reliable and valid way to assess performance on VPC tasks accurately and that these tasks correlate with existing cognitive composites.

Validation of a digitally delivered visual paired comparison task: reliability and convergent validity with established cognitive tests

“…The VPC tests (VPC-30 and VPC-5) were completed on a laptop computer equipped with a factory-installed web camera. The VPC test construction is explained in detail elsewhere (Bott et al 2018). Briefly, VPC tasks use eyetracking data to assess declarative memory by quantifying the amount of time a participant spends viewing novel images compared to previously viewed images (Crutcher et al 2009;Zola et al 2013;Bott et al 2018).…”

“…The average age of MCI and/or AD detection is 70 years (Hanninen et al 2002;Ward et al 2012); however, negative neuronal and neurobiological changes begin approximately 15 years before apparent cognitive changes are recognized (Bott et al 2018;Kremen et al 2014). Thus, it is critical to administer reliable cognitive assessments that can detect cognitive deficits as early as possible in the disease's progression.…”

“…However, this equipment is expensive and not readily available outside of research or clinical facilities, limiting scalability of the assessment. Recently, Bott and colleagues (Bott et al 2018) validated the use of readily available web cameras for VPC tasks as an alternative to the traditional gold-standard eye-tracking equipment. With web cameras ubiquitously available on most standard computing/smart devices, this represents a more scalable method to gather population-level and longitudinal data on cognitive health.…”

Development of a new virtual reality test of cognition: assessing the test-retest reliability, convergent and ecological validity of CONVIRT

“…Laboratory based equipment is cumbersome, involves adjusting camera positions and chin rests and as such, may compromise the reliability of such assessment. Recent advances in camera-based eye-tracking technology within wearable headsets however, have led to high precision tests that are reliable and valid [22,23]. Using eye-tracking technolgy within a virtual-reality environment adds a further step towards 'normalising' the experience of such testing for the user.…”

Toward Kinecting cognition by behaviour recognition-based deep learning and big data

“…In terms of using new technologies for nonpharmacological interventions, Liappas and Cabrera-Umpierrez [15] recently proposed the treatment of cognitive deficits by the use of virtual reality (VR), while a novel virtual reality application was developed in [81] for screening of early dementia. In addition, a commercial-grade eye-tracking camera and a laptop-embedded camera were used for a 30-min visual paired comparison (VPC) recognition memory task and it was found that it accurately correlates with existing cognitive composites [82]. Another method for detection of early decline in cognitive function was suggested in [83] by continuously tracking basic aspects of internet search terms using some software on line.…”