Multimodality, interactivity, and crowdsourcing for document transcription

Abstract

Knowledge mining from documents usually use document engineering techniques that allow the user to access the information contained in documents of interest. In this framework, transcription may provide efficient access to the contents of handwritten documents. Manual transcription is a time‐consuming task that can be sped up by using different mechanisms. A first possibility is employing state‐of‐the‐art handwritten text recognition systems to obtain an initial draft transcription that can be manually amended. A second option is employing crowdsourcing to obtain a massive but not error‐free draft transcription. In this case, when collaborators employ mobile devices, speech dictation can be used as a transcription source, and speech and handwritten text recognition can be fused to provide a better draft transcription, which can be amended with even less effort. A final option is using interactive assistive frameworks, where the automatic system that provides the draft transcription and the transcriber cooperate to generate the final transcription. The novel contributions presented in this work include the study of the data fusion on a multimodal crowdsourcing framework and its integration with an interactive system. The use of the proposed solutions reduces the required transcription effort and optimizes the overall performance and usability, allowing for a better transcription process.

PANDORA: Uma Plataforma Colaborativa para Transcrição Semiautomática de Boletins de Ocorrência Manuscritos

“…É importante mencionar que as abordagens supracitadas não focam em colaborac ¸ão, somente na transcric ¸ão inicial realizada por métodos de Aprendizado de Máquina. Granell et al (2018) propõem uma plataforma para a transcric ¸ão de textos históricos. A abordagem proposta realiza uma fusão de dados considerando textos manuscritos e áudio de descric ¸ão desses textos.…”

“…A abordagem proposta realiza uma fusão de dados considerando textos manuscritos e áudio de descric ¸ão desses textos. A plataforma proposta por Granell et al (2018) trabalha de forma colaborativa e permite que diversos usuários especialistas em transcric ¸ões de documentos históricos possam ajustar a transcric ¸ão inicial identificada pelo modelo de HTR. Similarmente, Tomic et al (2021) propõem uma plataforma para transcric ¸ão de documentos históricos croatas que seguem o alfabeto glagolítico.…”

Improving Transcription of Manuscripts with Multimodality and Interaction

“…Therefore, the CATTI system should try to complete the validated prefix by searching for a most likely suffixŝ taking into account both sequences of feature vectors. Following the asumptions presented in [15], the CATTI problem can be formulatted as:…”

Proceedings of the 3rd International Conference on Historical Cryptology HistoCrypt 2020

“…During the last decade, several handwritten text recognition (HTR) methods have been developed and applied successfully to historical handwritten sources, allowing (semi-)automatic transcription (Kahle et al, 2017;Romero et al, 2017). Alternative approaches use word spotting (Santoro et al, 2017), speech recognition (Granell et al, 2018) or even gamification (Chen et al, 2018) for speeding up the manual transcription. However, all these tools have been developed to only deal with known scripts (e.g.…”