Integrating sentence- and word-level error identification for disfluency correction

Abstract

While speaking spontaneously, speakers often make errors such as self-correction or false starts which interfere with the successful application of natural language processing techniques like summarization and machine translation to this data. There is active work on reconstructing this errorful data into a clean and fluent transcript by identifying and removing these simple errors. Previous research has approximated the potential benefit of conducting word-level reconstruction of simple errors only on those sentences known to have errors. In this work, we explore new approaches for automatically identifying speaker construction errors on the utterance level, and quantify the impact that this initial step has on word-and sentence-level reconstruction accuracy.

Removing fillers to induce semantic classes for a Chinese dialogue system

“…Emphasis points of approaches removing disfluency are mostly repairs [8] while fillers received little attention. And approaches are most supervised methods which require annotated disfluencies [9], [10], [11], [12] and emphasis on the repairs. The previous approaches removing fillers are either rule-based methods which can remove all the fillers exhaustively or based on disfluency annotations [6].…”