Assessing Handwriting Skills in a Web Browser: Development and Validation of an Automated Online Test in Japanese Kanji

Abstract

Online assessments of language and literacy have become prevalent in research and practice across settings. However, a notable exception is the assessment of handwriting and spelling, which has traditionally been conducted in person with paper and pencil. In light of this, we developed an automated, browser-based handwriting test application (Online Assessment of Handwriting and Spelling: OAHaS) for Japanese Kanji (Study 1) and examined its psychometric properties (Study 2). The automated scoring function using Convolutional Neural Network (CNN) models achieved high recall (98.7%) and specificity (84.4%), as well as high agreement with manual scoring (95.4%). Additionally, behavioral validation with data from primary school children (N = 261, 49.0% female) indicated the high reliability and validity of our online test application, with a strong correlation between children’s scores on the online and paper-based tests (r = .86). Moreover, our analysis indicated the practical utility of implementing measures of writing fluency (latency and duration) that are automatically recorded by OAHaS. Taken together, our browser-based test application demonstrated the feasibility and viability of evaluating handwriting abilities remotely and automatically, offering a streamlined approach to research and practice on handwriting. The source code of the application and supporting materials are available on Open Science Framework (https://osf.io/gver2/).