edATLAS: An Efficient Disambiguation Algorithm for Texting in Languages with Abugida Scripts

Abstract

Abugida refers to a phonogram writing system where each syllable is represented using a single consonant or typographic ligature, along with a default vowel or optional diacritic(s) to denote other vowels. However, texting in these languages has some unique challenges in spite of the advent of devices with soft keyboard supporting custom key layouts. The number of characters in these languages is large enough to require characters to be spread over multiple views in the layout. Having to switch between views many times to type a single word hinders the natural thought process. This prevents popular usage of native keyboard layouts. On the other hand, supporting romanized scripts (native words transcribed using Latin characters) with language model based suggestions is also set back by the lack of uniform romanization rules.To this end, we propose a disambiguation algorithm and showcase its usefulness in two novel mutually non-exclusive input methods for languages natively using the abugida writing system: (a) disambiguation of ambiguous input for abugida scripts, and (b) disambiguation of word variants in romanized scripts. We benchmark these approaches using public datasets, and show an improvement in typing speed by 19.49%, 25.13%, and 14.89%, in Hindi, Bengali, and Thai, respectively, using Ambiguous Input, owing to the human ease of locating keys combined with the efficiency of our inference method. Our Word Variant Disambiguation (WDA) maps valid variants of romanized words, previously treated as Out-of-Vocab, to a vocabulary of 100k words with high accuracy, leading to an increase in Error Correction F1 score by 10.03% and Next Word Prediction (NWP) by 62.50% on average.

LIDSNet: A Lightweight on-device Intent Detection model using Deep Siamese Network

“…This has led to the unsuitability of existing machine learning approaches for real-world dialog systems in low-resource edge devices due to their high latency and reliance on huge pre-trained models. Lately, there has been an increased academic and commercial interest in supporting AI solutions that can work directly on a user's device on local data [4], [5]. On-device AI models have the potential to support intent detection in real-time at low latency and also helps in enhancing the privacy of sensitive user data like smartphone messages.…”