Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) Techniques

Abstract

Air writing is one of the essential fields that the world is turning to, which can benefit from the world of the metaverse, as well as the ease of communication between humans and machines. The research literature on air writing and its applications shows significant work in English and Chinese, while little research is conducted in other languages, such as Arabic. To fill this gap, we propose a hybrid model that combines feature extraction with deep learning models and then uses machine learning (ML) and optical character recognition (OCR) methods and applies grid and random search optimization algorithms to obtain the best model parameters and outcomes. Several machine learning methods (e.g., neural networks (NNs), random forest (RF), K-nearest neighbours (KNN), and support vector machine (SVM)) are applied to deep features extracted from deep convolutional neural networks (CNNs), such as VGG16, VGG19, and SqueezeNet. Our study uses the AHAWP dataset, which consists of diverse writing styles and hand sign variations, to train and evaluate the models. Prepossessing schemes are applied to improve data quality by reducing bias. Furthermore, OCR character (OCR) methods are integrated into our model to isolate individual letters from continuous air-written gestures and improve recognition results. The results of this study showed that the proposed model achieved the best accuracy of 88.8% using NN with VGG16.

Dhad—A Children’s Handwritten Arabic Characters Dataset for Automated Recognition

“…Accuracy (Acc) is defined as the proportion of correctly predicted examples (1). The loss (Loss) quantifies the degree of misclassification by determining the proportion of incorrect predictions relative to the total predictions made by the model (2). Precision (P) is the fraction of correctly classified positive examples among all positively classified examples (3).…”

“…Arabic is a widely spoken language, with over 360 million people using it as their primary language [1]. In the domain of language processing and technological applications, the recognition of handwritten Arabic characters, especially in the realm of children's script, poses unique challenges [2,3]. Arabic, being a Semitic language, presents inherent complexities in its script, demanding advanced algorithms for precise recognition.…”

A TinyDL Model for Gesture-Based Air Handwriting Arabic Numbers and Simple Arabic Letters Recognition

“…Nahar et al conducted a comprehensive study on recognizing Arabic air-written letters using ML and deep CNNs combined with optical character recognition (OCR) [29]. They employed a hybrid approach incorporating feature extraction, DL models, and ML algorithms like neural networks, random forests, K-nearest neighbors, and support vector machines.…”

Research on a Web System Data-Filling Method Based on Optical Character Recognition and Multi-Text Similarity

“…After image pre-processing, feature extraction and character classification are performed. Features are the key information used to recognize text, and each different text can be distinguished from other texts through features [19]. Character classification is the process of passing the extracted features to the classifier, allowing the trained classifier to recognize the given features as corresponding text [20].…”