An Automatic Real-time Synchronization of Live speech with Its Transcription Approach

Abstract

Most studies in automatic synchronization of speech and transcription focus on the synchronization at the sentence level or the phrase level. Nevertheless, in some languages, like Thai, boundaries of such levels are difficult to linguistically define, especially in case of the synchronization of speech and its transcription. Consequently, the synchronization at a finer level like the syllabic level is promising. In this article, an approach to synchronize live speech with its corresponding transcription in real time at the syllabic level is proposed. Our approach employs the modified real-time syllable detection procedure from our previous work and the transcription verification procedure then adopts to verify correctness and to recover errors caused by the real-time syllable detection procedure. In our experiment, we empirically customized the acoustic features and the parameters to ensure that there is no inconsistency between the speech and its transcription. Results are compared with two baselines which have been applied to the Thai scenario. Experimental results indicate that, our approach outperforms two baselines with error rate reduction of 75.9% and 41.9% respectively and also can provide results in the real-time situation. Besides, our approach is applied to the practical application, namely ChulaDAISY. Practical experiments show that ChulaDAISY applied with our approach could reduce time consumption for producing audio books.

Automating Audio Description for Live Theater

“…Live audio alignment has been successful in music score following [1] and audio-transcription alignment [7] tasks. In this implementation, a reference recording is aligned to live input using an online time warping algorithm [4].…”