Automatic prosodic tone choice classification with Brazil’s intonation model

Johnson, David; Kang, Okim

doi:10.1007/s10772-015-9327-z

Cited by 9 publications

(3 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem of prosodic similarity evaluation arises within multiple research areas including automatic tone classification [44] and proficiency assessment [45]. Although in [46] it was shown that the difference between two given pitch contours may be perfectly grasped by Pearson correlation coefficient (PCC) and Root Mean Square (RMS), in the last two decades, prosodic similarities were successfully evaluated using dynamic time warping (DTW).…”

mentioning

confidence: 99%

Speech Processing for Language Learning: A Practical Approach to Computer-Assisted Pronunciation Teaching

et al. 2021

View full text Add to dashboard Cite

This article contributes to the discourse on how contemporary computer and information technology may help in improving foreign language learning not only by supporting better and more flexible workflow and digitizing study materials but also through creating completely new use cases made possible by technological improvements in signal processing algorithms. We discuss an approach and propose a holistic solution to teaching the phonological phenomena which are crucial for correct pronunciation, such as the phonemes; the energy and duration of syllables and pauses, which construct the phrasal rhythm; and the tone movement within an utterance, i.e., the phrasal intonation. The working prototype of StudyIntonation Computer-Assisted Pronunciation Training (CAPT) system is a tool for mobile devices, which offers a set of tasks based on a “listen and repeat” approach and gives the audio-visual feedback in real time. The present work summarizes the efforts taken to enrich the current version of this CAPT tool with two new functions: the phonetic transcription and rhythmic patterns of model and learner speech. Both are designed on a base of a third-party automatic speech recognition (ASR) library Kaldi, which was incorporated inside StudyIntonation signal processing software core. We also examine the scope of automatic speech recognition applicability within the CAPT system workflow and evaluate the Levenstein distance between the transcription made by human experts and that obtained automatically in our code. We developed an algorithm of rhythm reconstruction using acoustic and language ASR models. It is also shown that even having sufficiently correct production of phonemes, the learners do not produce a correct phrasal rhythm and intonation, and therefore, the joint training of sounds, rhythm and intonation within a single learning environment is beneficial. To mitigate the recording imperfections voice activity detection (VAD) is applied to all the speech records processed. The try-outs showed that StudyIntonation can create transcriptions and process rhythmic patterns, but some specific problems with connected speech transcription were detected. The learners feedback in the sense of pronunciation assessment was also updated and a conventional mechanism based on dynamic time warping (DTW) was combined with cross-recurrence quantification analysis (CRQA) approach, which resulted in a better discriminating ability. The CRQA metrics combined with those of DTW were shown to add to the accuracy of learner performance estimation. The major implications for computer-assisted English pronunciation teaching are discussed.

show abstract

mentioning

confidence: 99%

Speech Processing for Language Learning: A Practical Approach to Computer-Assisted Pronunciation Teaching

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Next, the rise and fall amplitudes are computed for each segmented verse. These parameters represent the strength of voice conveyed through a verse [38,39].…”

Section: Tilt-based Tarannum Shape Labelingmentioning

confidence: 99%

Melody Training With Segment-Based Tilt Contour for Quranic Tarannum

Hanum

Abas

Aziz

et al. 2021

MJCS

View full text Add to dashboard Cite

Tarannum, or melodic recitation of Quranic verses, employs the softness of the voice in reading the holy verses of the Quran. Melody training technology allows users to practise repetitively while also providing feedback on their performance. This paper describes an application that captures the pattern of tarannum melodies (from Quranic recitations) and provides feedback to the user. Recordings of Quranic verses are collected from an expert reciting Bayati tarannum. The samples are pre-processed into segmented tarannum verse-contours using pitch sequences. Using the k-Nearest Neighbor (kNN) classifier, the melody patterns are trained on 20 samples. Input vectors are formed by computing the melody verse-contour representation using mean, standard deviation, and slope values and combining them with an identified Tilt-based contour label. A tarannum training prototype is built to test similarity between a user’s recitation and the trained patterns. To identify similarity between a pair of verse-contours, the application employs a shape-based contour similarity algorithm. The proposed application also provides feedback in the form of a grade and a percentage of accuracy, as determined by a melody curve similarity algorithm. As results, the current samples have an overall shape-based weighted score of 66%. Some samples are successfully classified with a similarity score as high as 80% individually. The study provides an alternative interactive session for people who want to learn Tarannum, as well as a preliminary step toward understanding the melodic patterns for tarannum. The application provides a repetitive training experience and encourages users to improve their recitations in order to achieve the highest possible score.

show abstract

“…Similarly, the computer employed a number of supervised machine learning classifiers to evaluate the pitch contours of the prominent syllables to ascertain their tone choice (Johnson & Kang, 2015b). The same combinations of the Analyst X annotated TIMIT and BURNC corpora, which were drawn on to test and train the prominent syllable classifiers, were also made use of in testing and training the tone choice classifiers.…”

Section: Computer Annotationmentioning

confidence: 99%

Comparison of Inter-rater Reliability of Human and Computer Prosodic Annotation Using Brazil’s Prosody Model

Kang¹,

Johnson²

2015

ELR

Self Cite

View full text Add to dashboard Cite

The current study examined whether the computer annotations of prodody based on Brazil's (1997) framework were comparable with human annotations. A series of statistical tests were performed for each prosodic feature: tone unit (two accuracy scores and Pearson's correlation), prominent syllable (accuracy, F-measure, and Cohen's kappa), tone choice (accuracy and Fleiss' kappa), and relative pitch (accuracy, Fleiss' kappa, and Pearson's correlation). We considered one population to be the inter-rater reliability scores between the three human coders and the other population to be the inter-rater reliability scores between the computer and the three humans. If the differences between these two populations were significant, then the computer and human annotations were considered not comparable, but if the differences were not significant, then the computer and human annotations were considered comparable. The results indicated that the computer and human annotations were comparable for tone choice and not comparable for prominent syllable. For tone unit, two of the t-tests provided evidence that they were comparable and one did not. The relative pitch t-tests showed a significant disparity between the estimates of relative pitch by the humans and the computer's actual relative pitch calculation.

show abstract

Automatic prosodic tone choice classification with Brazil’s intonation model

Cited by 9 publications

References 35 publications

Speech Processing for Language Learning: A Practical Approach to Computer-Assisted Pronunciation Teaching

Speech Processing for Language Learning: A Practical Approach to Computer-Assisted Pronunciation Teaching

Melody Training With Segment-Based Tilt Contour for Quranic Tarannum

Comparison of Inter-rater Reliability of Human and Computer Prosodic Annotation Using Brazil’s Prosody Model

Contact Info

Product

Resources

About