Paragraph-based prosodic cues for speech synthesis applications

Abstract

Speech synthesis has improved in both expressiveness and voice quality in recent years. However, obtaining full expressiveness when dealing with large multi-sentential synthesized discourse is still a challenge, since speech synthesizers do not take into account the prosodic differences that have been observed in discourse units such as paragraphs. The current study validates and extends previous work by analyzing the prosody of paragraph units in a large and diverse corpus of TED Talks using automatically extracted F0, intensity and timing features. In addition, a series of classification experiments was performed in order to identify which features are consistently used to distinguish paragraph breaks. The results show significant differences in prosody related to paragraph position. Moreover, the classification experiments show that boundary features such as pause duration and differences in F0 and intensity levels are the most consistent cues in marking paragraph boundaries. This suggests that these features should be taken into account when generating spoken discourse in order to improve naturalness and expressiveness.

Attentional Parallel RNNs for Generating Punctuation in Transcribed Speech

“…The corpus consists of 1046 talks by 884 English speakers, uttering a total amount of 156034 sentences. The corresponding transcripts, as well as audio and video files, are available on TED's website; they were created by volunteers and include punctuation and paragraph breaks [12]. The subtitle timings of TED transcripts do not always correspond to sentences in the transcript.…”

“…To overcome this limitation, precise word timings were first obtained through Viterbi forced alignment using an automatic speech recognition system. The word timings were then further used to automatically obtain sentence boundaries and thus sentence timings [12].…”

“…As for the prosodic features, three main prosodic elements were extracted following the methodology in [12] in order to analyze their influence on punctuation generation: pauses, fundamental frequency (f0), and intensity. Pause durations were extracted from the provided word timings, while f0 and intensity contours were extracted at 10 ms precision using Praat software [5] with linear interpolation and octave jump removal for fundamental frequency provided by Praat.…”

Automatic Paragraph Segmentation with Lexical and Prosodic Features

“…Based on previous analysis of paragraph prosody [18], we calculated aggregate statistics for each sentence: mean, standard deviation, maximum, minimum, median, slope, range (99th-1st quantiles). We also record the values for the previous and next sentences, as well as their differences to the target, and the difference between the first and last word of the target.…”

“…Similarly, prosodic features based on pitch, energy and timing have been used to perform topic segmentation on their own [13,14,15] or in conjunction with lexical features [8,12,16,17]. While pause duration appears to be the most robust segmentation cue, paragraphs also seem to follow general prosodic declination and reset patterns [18]. So, we expect prosody to be informative of paragraph breaks.…”