Learning classification models of cognitive conditions from subtle behaviors in the digital Clock Drawing Test

Abstract

The Clock Drawing Test – a simple pencil and paper test – has been used for more than 50 years as a screening tool to differentiate normal individuals from those with cognitive impairment, and has proven useful in helping to diagnose cognitive dysfunction associated with neurological disorders such as Alzheimer’s disease, Parkinson’s disease, and other dementias and conditions. We have been administering the test using a digitizing ballpoint pen that reports its position with considerable spatial and temporal precision, making available far more detailed data about the subject’s performance. Using pen stroke data from these drawings categorized by our software, we designed and computed a large collection of features, then explored the tradeoffs in performance and interpretability in classifiers built using a number of different subsets of these features and a variety of different machine learning techniques. We used traditional machine learning methods to build prediction models that achieve high accuracy. We operationalized widely used manual scoring systems so that we could use them as benchmarks for our models. We worked with clinicians to define guidelines for model interpretability, and constructed sparse linear models and rule lists designed to be as easy to use as scoring systems currently used by clinicians, but more accurate. While our models will require additional testing for validation, they offer the possibility of substantial improvement in detecting cognitive impairment earlier than currently possible, a development with considerable potential impact in practice.

How technology is reshaping cognitive assessment: Lessons from the Framingham Heart Study.

“…Starting in 2011, using the academic version of proprietary software developed and patented by researchers at MIT and Lahey Hospital and Medical Center (U.S. patent number US 2008/9243933A1), FHS substituted use of a regular ballpoint pen with a digital pen for the Clock Drawing Test. The MIT/Lahey-owned software transforms the data captured by the digital pen into more than 1000 analyzable features that have been combined to reveal decision-making latencies and graphomotor characteristics that may serve as measures of underlying cognitive processes (Piers et al, under review; Souillard-Mandar et al, 2016; Cohen et al, 2014; Libon, Penney, et al, 2014), taking the BPA approach to a level of precision that hand scored BPA efforts cannot. For example, preliminary exploration of metrics obtained using the digital Clock Drawing Test (dCDT) suggests that information processing speed is not a single, all-encompassing construct (Piers, Devlin, et al, 2016).…”

Technology and Dementia: The Future is Now

“…Two touchscreen-based cognitive assessment batteries – Cambridge Neuropsychological Test Automated Battery [1] and Examen Cognitif par Ordinateur [2] were developed in Europe in the 1980s and 1990s. More recently multiple web-based and app-based cognitive assessments have emerged [3] including digitized versions of pen and paper tasks, such as clock drawing [4] including a digital clock-drawing pen [5]. In addition, the manner in which people use technology and changes in their patterns of use has been identified as an early indicator of emerging cognitive impairment [6, 7].…”

Marrying Past and Present Neuropsychology: Is the Future of the Process-Based Approach Technology-Based?

“…Binaco et al (2018) propose the use of machinelearning algorithms to establish a relationship between features of performance in the CDT (captured digitally) and the level of cognitive decline. This may be due in part to the fact that machine learning has the capability to automate time-consuming and subjective processes, analyzing data that is difficult for the clinicians to interpret manually and helps in detecting cognitive impairment at an earlier stage than is possible currently (Souillard-Mandar et al, 2016;Binaco et al, 2018). In this sense, the development of a Digital Clock Drawing Test (dCDT) has provided a more sophisticated and granular evaluation of the cognitive processes involved in the CDT.…”