Corpus studies in word prediction

Trnka, Keith; McCoy, Kathleen F.

doi:10.1145/1296843.1296877

Cited by 14 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where n c denotes the number of actual characters in test set and n k denotes the number of keystrokes which were needed to produce the test set in soft keyboard [15]. NWP refers to the probability that given a sequence of words from a sentence in test data, w k−1 1 , the next word in that sentence, w k will appear among the word prediction candidates.…”

Section: Experiments Resultsmentioning

confidence: 99%

Real-Time Optimized N-Gram for Mobile Devices

Mani

Gothe

Ghosh

et al. 2019

2019 IEEE 13th International Conference on Semantic Computing (ICSC)

View full text Add to dashboard Cite

With the increasing number of mobile devices, there has been continuous research on generating optimized Language Models (LMs) for soft keyboard. In spite of advances in this domain, building a single LM for low-end feature phones as well as high-end smartphones is still a pressing need. Hence, we propose a novel technique, Optimized N-gram (Op-Ngram), an end-to-end N-gram pipeline that utilises mobile resources efficiently for faster Word Completion (WC) and Next Word Prediction (NWP). Op-Ngram applies Stupid Backoff [1] and pruning strategies to generate a light-weight model. The LM loading time on mobile is linear with respect to model size. We observed that Op-Ngram gives 37% improvement in Language Model (LM)-ROM size, 76% in LM-RAM size, 88% in loading time and 89% in average suggestion time as compared to SORTED array variant of BerkeleyLM [2]. Moreover, our method shows significant performance improvement over KenLM[3] as well.

show abstract

Section: Experiments Resultsmentioning

confidence: 99%

Real-Time Optimized N-Gram for Mobile Devices

Mani

Gothe

Ghosh

et al. 2019

2019 IEEE 13th International Conference on Semantic Computing (ICSC)

View full text Add to dashboard Cite

show abstract

“…We use two metrics to benchmark the quality of LM as perceived by an end-user, namely Keystroke Saving Ratio (KSR) and Next Word Prediction (NWP) with a sample test set of 500 sentences [23], [24], [25]. The KSR metric [27] indicates the number of keystrokes saved due to effective suggestions provided by the engine. It is defined as:…”

Section: E Evaluationmentioning

confidence: 99%

edATLAS: An Efficient Disambiguation Algorithm for Texting in Languages with Abugida Scripts

Ghosh

Gothe

Sanchi

et al. 2021

2021 IEEE 15th International Conference on Semantic Computing (ICSC)

View full text Add to dashboard Cite

Abugida refers to a phonogram writing system where each syllable is represented using a single consonant or typographic ligature, along with a default vowel or optional diacritic(s) to denote other vowels. However, texting in these languages has some unique challenges in spite of the advent of devices with soft keyboard supporting custom key layouts. The number of characters in these languages is large enough to require characters to be spread over multiple views in the layout. Having to switch between views many times to type a single word hinders the natural thought process. This prevents popular usage of native keyboard layouts. On the other hand, supporting romanized scripts (native words transcribed using Latin characters) with language model based suggestions is also set back by the lack of uniform romanization rules.To this end, we propose a disambiguation algorithm and showcase its usefulness in two novel mutually non-exclusive input methods for languages natively using the abugida writing system: (a) disambiguation of ambiguous input for abugida scripts, and (b) disambiguation of word variants in romanized scripts. We benchmark these approaches using public datasets, and show an improvement in typing speed by 19.49%, 25.13%, and 14.89%, in Hindi, Bengali, and Thai, respectively, using Ambiguous Input, owing to the human ease of locating keys combined with the efficiency of our inference method. Our Word Variant Disambiguation (WDA) maps valid variants of romanized words, previously treated as Out-of-Vocab, to a vocabulary of 100k words with high accuracy, leading to an increase in Error Correction F1 score by 10.03% and Next Word Prediction (NWP) by 62.50% on average.

show abstract

“…An ngram model trained on text of a different topic and/or style may perform very poorly compared to a model trained and tested on similar text. Trnka and McCoy (2007) and Wandmacher and Antoine (2006) have demonstrated the domain sensitivity of ngram models for word prediction.…”

Section: Introductionmentioning

confidence: 99%