Linguistically Aided Speaker Diarization Using Speaker Role Information

Flemotomos, Nikolaos; Georgiou, Panayiotis G.; Narayanan, Shrikanth

doi:10.21437/odyssey.2020-17

Cited by 15 publications

(16 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The more recent speaker diarization systems that take advantage of the ASR transcript have employed a DNN model to capture the linguistic pattern in the given ASR output to enhance the speaker diarization result. The authors in [177] proposed a way of using the linguistic information for the speaker diarization task where participants have distinct roles that are known to the speaker diarization system. Fig.…”

Section: Using Lexical Information From Asrmentioning

confidence: 99%

A Review of Speaker Diarization: Recent Advances with Deep Learning

Park¹,

Kanda²,

Dimitriadis³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Speaker diarization is a task to label audio or video recordings with classes corresponding to speaker identity, or in short, a task to identify "who spoke when". In the early years, speaker diarization algorithms were developed for speech recognition on multispeaker audio recordings to enable speaker adaptive processing, but also gained its own value as a stand-alone application over time to provide speaker-specific meta information for downstream tasks such as audio retrieval. More recently, with the rise of deep learning technology that has been a driving force to revolutionary changes in research and practices across speech application domains in the past decade, more rapid advancements have been made for speaker diarization. In this paper, we review not only the historical development of speaker diarization technology but also the recent advancements in neural speaker diarization approaches. We also discuss how speaker diarization systems have been integrated with speech recognition applications and how the recent surge of deep learning is leading the way of jointly modeling these two components to be complementary to each other. By considering such exciting technical trends, we believe that it is a valuable contribution to the community to provide a survey work by consolidating the recent developments with neural methods and thus facilitating further progress towards a more efficient speaker diarization.

show abstract

Section: Using Lexical Information From Asrmentioning

confidence: 99%

A Review of Speaker Diarization: Recent Advances with Deep Learning

Park¹,

Kanda²,

Dimitriadis³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moreover, the architecture design followed, where the various modules are trained independently and are then connected to form a pipeline, inevitably leads to error propagation. There are indications that alternative frameworks could reduce errors in specific cases, if for example diarization is aware of the different speaker roles (Flemotomos, Georgiou, & Narayanan, 2020) or if the two tasks of diarization and role recognition are performed simultaneously (Flemotomos, Papadopoulos, et al, 2018).…”

Section: Limitations and Conclusionmentioning

confidence: 99%

Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies

Flemotomos,

Martinez,

Chen

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

With the growing prevalence of psychological interventions, it is vital to have measures which rate the effectiveness of psychological care, in order to assist in training, supervision, and quality assurance of services. Traditionally, quality assessment is addressed by human raters who evaluate recorded sessions along specific dimensions, often codified through constructs relevant to the approach and domain. This is however a cost-prohibitive and time-consuming method which leads to poor feasibility and limited use in real-world settings. To facilitate this process, we have developed an automated competency rating tool able to process the raw recorded audio of a session, analyzing who spoke when, what they said, and how the health professional used language to provide therapy. Focusing on a use case of a specific type of psychotherapy called Motivational Interviewing, our system gives comprehensive feedback to the therapist, including information about the dynamics of the session (e.g., therapist's vs. client's talking time), low-level psychological language descriptors (e.g., type of questions asked), as well as other high-level behavioral constructs (e.g., the extent to which the therapist understands the clients' perspective). We describe our platform and its performance, using a dataset of more than 5,000 recordings drawn from its deployment in a real-world clinical setting used to assist training of new therapists. We are confident that a widespread use of automated psychotherapy rating tools in the near future will augment experts' capabilities by providing an avenue for more effective training and skill improvement and will eventually lead to more positive clinical outcomes.

show abstract

“…Although speaker diarization is conventionally an audio-only task, the linguistic content carried by speech signals [219] and the speaker behaviors, e.g., the movement of lips, recorded by videos [220] offer valuable supplementary cues to the detection of active speakers. To incorporate the aforementioned knowledge, multimodal speaker diarization is emerging.…”

Section: Multimodal Speaker Diarizationmentioning

confidence: 99%

“…The first class is audio-linguistic speaker diarization, Park et al [221] integrated lexical cues and acoustic cues together by a gated recurrent unit-based sequence-to-sequence model, which improves the diarization performance by exploring linguistic variability deeply. The effectiveness of using both the linguistic and acoustic cues for diarization has been manifested further in structured scenarios [222,219] where the speakers are assumed to produce distinguishable linguistic patterns. For instance, a teacher is likely to speak in a more didactic style while a student tends to be more inquisitive; a doctor is likely to inquire on symptoms and prescribe while a patient describe symptoms, etc.…”

Section: Multimodal Speaker Diarizationmentioning

confidence: 99%

Speaker Recognition Based on Deep Learning: An Overview

Bai

Zhang

2020

Preprint

View full text Add to dashboard Cite

Speaker recognition is a task of identifying persons from their voices. Recently, deep learning has dramatically revolutionized speaker recognition. However, there is lack of comprehensive reviews on the exciting progress. In this paper, we review several major subtasks of speaker recognition, including speaker verification, identification, diarization, and robust speaker recognition, with a focus on deep-learning-based methods. Because the major advantage of deep learning over conventional methods is its representation ability, which is able to produce highly abstract embedding features from utterances, we first pay close attention to deep-learning-based speaker feature extraction, including the inputs, network structures, temporal pooling strategies, and objective functions respectively, which are the fundamental components of many speaker recognition subtasks. Then, we make an overview of speaker diarization, with an emphasis of recent supervised, end-to-end, and online diarization. Finally, we survey robust speaker recognition from the perspectives of domain adaptation and speech enhancement, which are two major approaches of dealing with domain mismatch and noise problems. Popular and recently released corpora are listed at the end of the paper.

show abstract

Linguistically Aided Speaker Diarization Using Speaker Role Information

Cited by 15 publications

References 42 publications

A Review of Speaker Diarization: Recent Advances with Deep Learning

A Review of Speaker Diarization: Recent Advances with Deep Learning

Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies

Speaker Recognition Based on Deep Learning: An Overview

Contact Info

Product

Resources

About