Automated Assessment of Noisy Crowdsourced Free-text Answers for Hindi in Low Resource Setting

Abstract

The requirement of performing assessments continually on a larger scale necessitates the implementation of automated systems for evaluation of the learners' responses to free-text questions. We target children of age group 8-14 years and use an ASR integrated assessment app to crowdsource learners' responses to free text questions in Hindi. The app helped collect 39641 user answers to 35 different questions of Science topics. Since the users are young children from rural India and may not be well-equipped with technology, it brings in various noise types in the answers. We describe these noise types and propose a preprocessing pipeline to denoise user's answers. We showcase the performance of different similarity metrics on the noisy and denoised versions of user and model answers. Our findings have large-scale applications for automated answer assessment for school children in India in low resource settings.

The Korean Morphologically Tight-Fitting Tokenizer for Noisy User-Generated Texts

“…Sjöblom et al (2018) defined noises for subtitles including misspellings, misalignments, and sentence segmentation errors. Agarwal et al (2020) summarized noise types in free-text answers written by Hindi children, such as punctuations, emojis, translated/transliterated text, missing space between words, and so on. Some other works focused on orthographical noises (Karpukhin et al, 2019;Kumar et al, 2020).…”