Truecasing German user-generated conversational text

Abstract

True-casing, the task of restoring proper case to (generally) lower case input, is important in downstream tasks and for screen display. In this paper, we investigate truecasing as an intrinsic task and present several experiments on noisy user queries to a voice-controlled dialog system. In particular, we compare a rulebased, an n-gram language model (LM) and a recurrent neural network (RNN) approaches, evaluating the results on a German Q&A corpus and reporting accuracy for different case categories. We show that while RNNs reach higher accuracy especially on large datasets, character n-gram models with interpolation are still competitive, in particular on mixedcase words where their fall-back mechanisms come into play.

Capitalization Normalization for Language Modeling with an Accurate and Efficient Hierarchical RNN Model

“…[11] first introduced character-based LSTM for this task and completely solved the mixed case word problem. Recently, [2] compared character-based n-gram (n up to 15) language models with the character LSTM of [11]. [12] advanced the state of the art with a character-based CNN-LSTM-CRF model.…”

“…The vast amount of online text powers language models for speech recognition, typing suggestions and many other language generation tasks. However user-generated texts, especially those from mobile applications such as Twitter Tweets [1], often violate the grammatical rules of casing in English and other western languages [2]. The process of restoring the proper case, often known as tRuEcasIng [3], provides a factorized solution with a dedicated model for case normalization.…”

Position-Invariant Truecasing with a Word-and-Character Hierarchical Recurrent Neural Network

“…Susanto et al (2016) first introduced character-based LSTM for this task and completely solved the mixed case word problem. Recently, Grishina et al (2020) compared character-based n-gram (n up to 15) language models with the character LSTM of Susanto et al (2016). Ramena et al (2020) advanced the state of the art with a characterbased CNN-LSTM-CRF model which introduced local output label dependencies.…”

“…Automatically generated texts such as speech recognition (ASR) transcripts as well as user-generated texts from mobile applications such as Twitter Tweets (Nebhi et al, 2015) often violate the grammatical rules of casing in English and other western languages (Grishina et al, 2020). The process of restoring the proper case, often known as tRuEcas-Ing (Lita et al, 2003), is not only important for the ease of consumption by end-users (e.g.…”