Prosodic contex-based analysis of disfluencies.

Moniz, Helena; Batista, Fernando; Trancoso, Isabel; Mata, Ana Isabel

doi:10.21437/interspeech.2012-215

Cited by 4 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Previous work has found that disfluencies exhibit distinct acoustic cues [32], [45]. Acoustic approaches for disfluency detection benefit because they do not need to rely transcripts as input and as a result they would not be impacted by ASR errors.…”

Section: Acoustic-based Approaches For Disfluency Detectionmentioning

confidence: 99%

Toward A Multimodal Approach for Disfluency Detection and Categorization

Romana

2023

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Speech disfluencies, such as filled pauses or repetitions, are disruptions in the typical flow of speech. Stuttering is a speech disorder characterized by a high rate of disfluencies, but all individuals speak with some disfluencies and the rates of disfluencies may by increased by factors such as cognitive load. Clinically, automatic disfluency detection may help in treatment planning for individuals who stutter. Outside of the clinic, automatic disfluency detection may serve as a pre-processing step to improve natural language understanding in downstream applications. With this wide range of applications in mind, we investigate language, acoustic, and multimodal methods for frame-level automatic disfluency detection and categorization. Each of these methods relies on audio as an input. First, we evaluate several automatic speech recognition (ASR) systems in terms of their ability to transcribe disfluencies, measured using disfluency error rates. We then use these ASR transcripts as input to a language-based disfluency detection model. We find that disfluency detection performance is largely limited by the quality of transcripts and alignments. We find that an acoustic-based approach that does not require transcription as an intermediate step outperforms the ASR language approach. Finally, we present multimodal architectures which we find improve disfluency detection performance over the unimodal approaches. Ultimately, this work introduces novel approaches for automatic frame-level disfluency and categorization. In the long term, this will help researchers incorporate automatic disfluency detection into a range of applications.

show abstract

Section: Acoustic-based Approaches For Disfluency Detectionmentioning

confidence: 99%

Toward A Multimodal Approach for Disfluency Detection and Categorization

Romana

2023

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Nestes, a tarefa de classificação de disfluências e de marcas de pontuação, para delimitação de constituintes similares a frases (SUs), teve como base o seguinte conjunto de pistas prosódicas: presença e duração de pausas nas fronteiras dos eventos, declinação global de f0, reinicializações de f0 e de energia, alongamento final dos segmentos, ocorrência de pausas e ocorrência de fragmentos (Moniz et al, 2014b).…”

Section: Configuração Das Experiênciasunclassified

“…Analisando os resultados já disponíveis para disfluências, observa-se que as características diferem das encontradas entre as classes SU e marcadores discursivos. De acordo com Moniz et al (2014aMoniz et al ( , 2014b, sobre a caracterização prosódica de disfluências, as características mais informativas extraídas do openSMILE são MFCC e comprimento de audSpec. O que é específico das disfluências são as diferenças espectrais de áudio, em linha com Shriberg (2001), que mostram que as disfluências também apresentam perturbações fonético--fonológicas ao nível da fronteira de final de palavra.…”

Section: Propriedades Acústico-prosódicas Mais Relevantesunclassified

“…A tarefa de classificação multiclasse permitir-nos-á, num futuro próximo, incluir os marcadores discursivos nas transcrições automáticas do reconhecer de fala (Neto et al, 2008). A identificação de marcas de pontuação (Batista, 2011;Batista et al, 2012;Moniz, 2013) e de disfluências (e.g., pausas preenchidas lexicalizadas, como "aam" e/ou "mm", apagamentos, substituições, entre outros 1 ) nas transcrições já permitiu uma melhoria significativa do output do sistema, o que resultou numa diminuição da taxa de erro de reconhecimento (Moniz et al, 2014b). Com a recente disponibilização de uma grande quantidade de corpora de fala espontânea, foi possível analisar os eventos em falta, nomeadamente, os marcadores discursivos (Liu et al, 2006;Ostendorf et al, 2008), tópico do presente estudo.…”

Section: Introductionunclassified

See 1 more Smart Citation

Classificação prosódica de marcadores discursivos

Cabarrão,

Moniz,

Jaime Ferreira

et al. 2016

rapl

View full text Add to dashboard Cite

This work describes the discourse markers present in two corpora for European Portuguese, in different domains (university lectures and map-task dialogues). In this study, we also perform a multiclass automatic classification task based on prosodic features to verify in both corpora which words are discourse markers, which are disfluencies, and which are sentence like-units (SUs). Results show that the selection of discourse markers varies across domain and between speakers. As for the classification task, results show that the discourse markers are better classified in the lectures corpus (87%) than in the dialogue corpus (84%). However, cross-domain experiments evidenced that data trained with the dialogue corpus predicts better the events in the lecture corpus, since this domain displays more speakers and therefore complex patterns. In both corpora, markers are more easily classified as SUs than as disfluencies.

show abstract

“…[11,12] use the same university lectures corpus subset also used in the present study and concluded that the best features to identify whether an element should be rated as fluent or disfluent are: prosodic phrasing, contour shape, and presence/absence of silent pauses. Recently, [13] quence, pointing out to prosodic contrast strategy (pitch and energy increases) between the reparandum and the repair. The authors evidenced that, although prosodic contrast marking between those regions is a cross speaker and cross category strategy, there are degrees in doing so, namely: filled pauses exhibit the highest f0 increase, and repetitions exhibit the highest energy.…”

Section: Related Workmentioning

confidence: 99%

Disfluency detection based on prosodic features for university lectures

Medeiros¹,

Moniz

Batista

et al. 2013

Interspeech 2013

Self Cite

View full text Add to dashboard Cite

This paper focuses on the identification of disfluent sequences and their distinct structural regions, based on acoustic and prosodic features. Reported experiments are based on a corpus of university lectures in European Portuguese, with roughly 32h, and a relatively high percentage of disfluencies (7.6%). The set of features automatically extracted from the corpus proved to be discriminant of the regions contained in the production of a disfluency. Several machine learning methods have been applied, but the best results were achieved using Classification and Regression Trees (CART). The set of features which was most informative for cross-region identification encompasses word duration ratios, word confidence score, silent ratios, and pitch and energy slopes. Features such as the number of phones and syllables per word proved to be more useful for the identification of the interregnum, whereas energy slopes were most suited for identifying the interruption point.

show abstract

Prosodic contex-based analysis of disfluencies.

Cited by 4 publications

References 13 publications

Toward A Multimodal Approach for Disfluency Detection and Categorization

Toward A Multimodal Approach for Disfluency Detection and Categorization

Classificação prosódica de marcadores discursivos

Disfluency detection based on prosodic features for university lectures

Contact Info

Product

Resources

About