Interpretable deep learning approach for extracting cognitive features from hand-drawn images of intersecting pentagons in older adults

Abstract

Hand drawing involves multiple neural systems for planning and precise control of sequential movements, making it a valuable cognitive test for older adults. However, conventional visual assessment of drawings may not capture intricate nuances that could help track cognitive states. To address this issue, we utilized a deep-learning model, PentaMind, to examine cognition-related features from hand-drawn images of intersecting pentagons. PentaMind, trained on 13,777 images from 3,111 participants in three aging cohorts, explained 23.3% of the variance in global cognitive scores, a comprehensive hour-long cognitive battery. The model's performance, which was 1.92 times more accurate than conventional visual assessment, significantly improved the detection of cognitive decline. The improvement in accuracy was due to capturing additional drawing features that we found to be associated with motor impairments and cerebrovascular pathologies. By systematically modifying the input images, we discovered several important drawing attributes for cognition, including line waviness. Our results demonstrate that hand-drawn images can provide rich cognitive information, enabling rapid assessment of cognitive decline and suggesting potential clinical implications in dementia.

A Three-Stage Algorithm for Quantification of the MMSE Interlocking Pentagon Areas

“…To improve efficiency and objectivity, automated scoring approaches employing Deep Learning (DL) techniques have been adopted 4 . On the other hand, Tasaki et al 6 developed a deep learning model correlating with cognitive scores and introduced eight features, several of which were novel features for PCT characteristics. The QIP algorithm presents a methodology for quantifying three features out of the aforementioned eight proposed by Tasaki et al 6 .…”

“…On the other hand, Tasaki et al 6 developed a deep learning model correlating with cognitive scores and introduced eight features, several of which were novel features for PCT characteristics. The QIP algorithm presents a methodology for quantifying three features out of the aforementioned eight proposed by Tasaki et al 6 . Another noteworthy aspect of the QIP algorithm is its applicability to interlocking polygons beyond pentagons, such as rectangles or triangles as long as objects maintain convexity.…”

“…A composite score, which was the sum of five domain scores based on the number of angles, distance/intersection between two pentagons, closure/opening of the image contour, rotation, and closing-in was proposed 3 . To enhance efficiency and objectivity, automatic scoring approaches utilizing Deep Learning (DL) techniques, such as U-Net and convolutional neural networks, have been proposed 4-6 . A study developed DL method for a mobile application, which employed a convolutional network known as U-net along with mobile sensor data 4 .…”

“…Another study adopted a convolutional neural network utilizing an object detection model to generate an automatic standard binary score 5 . While two preceding studies focused on training deep learning models to automate two well-established scoring systems, a study devised a deep learning approach to explore an optimal scoring method for the interlocking pentagons test’s correlation with cognition 6 . Additionally, eight crucial drawing characteristics were identified through simulations using synthetic interlocking pentagon images including differences in sizes between two pentagons, overall pentagon size, and distance between the pentagons.…”

“…In this study, we selected three attributes as highlighted in Tasaki et al 6 , and conducted quantification for the targeted attributes. We hypothesized that individuals with impaired visual construction skills, particularly older adults, may encounter difficulties accurately reproducing the sample interlocking pentagons.…”