Multi-Channel End-to-End Neural Diarization with Distributed Microphones

Horiguchi, Shota; Takashima, Yuki; Garcia, Paola; Watanabe, Soichi; Kawaguchi, Yohei

doi:10.48550/arxiv.2110.04694

Cited by 3 publications

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The study partner was instructed to place 1 microphone facing themselves, and the other at the participant, however it was not possible to confirm if this was carried out. Future work could use more sophisticated diarization algorithms that incorporate both channels in the model training, thus resulting in an optimised dual-channel speaker diarization system 27,28 .…”

Section: Discussionmentioning

confidence: 99%

Automatic speaker diarization for natural conversation analysis in autism clinical trials

O’Sullivan

Bogaarts

Schoenenberger

et al. 2023

Preprint

View full text Add to dashboard Cite

Challenges in social communication is one of the core symptom domains in autism spectrum disorder (ASD). Novel therapies are under development to help individuals with these challenges, however the ability to show a benefit is dependent on a sensitive and reliable measure of treatment effect. Currently, measuring these deficits requires the use of time-consuming and subjective techniques. Objective measures extracted from natural conversations could be more ecologically relevant, and administered more frequently, perhaps giving them added sensitivity to change. While several studies have used automated analysis methods to study autistic speech, they require manual transcriptions. In order to bypass this time-consuming process, an automated speaker diarization algorithm must first be applied. In this paper, we are testing whether a speaker diarization algorithm can be applied to natural conversations between autistic individuals and their conversational partner in a natural setting at home over the course of a clinical trial. We calculated the average duration that a participant would speak for within their turn. We found a significant correlation between this feature and the Vineland Adaptive Behaviour Scales (VABS) expressive communication score (r=0.51, p=7x10-5). Our results show that natural conversations can be used to obtain measures of talkativeness, and that this measure can be derived automatically, thus showing the promise of objectively evaluating communication challenges in ASD.

show abstract

Section: Discussionmentioning

confidence: 99%

Automatic speaker diarization for natural conversation analysis in autism clinical trials

O’Sullivan

Bogaarts

Schoenenberger

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Most works following the EEND principle have focused on improvements on the architecture or modeling. Some by using self-attention layers [4] or conformer layers [5] instead of the original BLSTM layers for feature encoding; others have focused on more complex diarization scenarios such as its online fashion [6,7] or when more than one microphone is available [8] or by improving the model iteratively using pseudo-labels [9]. Some have used EEND together with more standard approaches by using EEND-inspired models to find overlaps among pairs of speakers in the output of a cascaded system [10] or leveraging EEND's VAD performance by us-ing an external VAD system [11] or combining short duration diarization outputs to produce better whole-utterance diarization [12,13,14,15].…”

Section: Introductionmentioning

confidence: 99%

From Simulated Mixtures to Simulated Conversations as Training Data for End-to-End Neural Diarization

Landini¹,

Lozano-Díez²,

Díez³

et al. 2022

Preprint

View full text Add to dashboard Cite

End-to-end neural diarization (EEND) is nowadays one of the most prominent research topics in speaker diarization. EEND presents an attractive alternative to standard cascaded diarization systems since a single system is trained at once to deal with the whole diarization problem. Several EEND variants and approaches are being proposed, however, all these models require large amounts of annotated data for training but available annotated data are scarce. Thus, EEND works have used mostly simulated mixtures for training. However, simulated mixtures do not resemble real conversations in many aspects. In this work we present an alternative method for creating synthetic conversations that resemble real ones by using statistics about distributions of pauses and overlaps estimated on genuine conversations. Furthermore, we analyze the effect of the source of the statistics, different augmentations and amounts of data. We demonstrate that our approach performs substantially better than the original one, while reducing the dependence on the fine-tuning stage. Experiments are carried out on 2-speaker telephone conversations of Callhome and DIHARD 3. Together with this publication, we release our implementations of EEND and the method for creating simulated conversations.

show abstract

Automatic speaker diarization for natural conversation analysis in autism clinical trials

O’Sullivan

Bogaarts

Schoenenberger

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Challenges in social communication is one of the core symptom domains in autism spectrum disorder (ASD). Novel therapies are under development to help individuals with these challenges, however the ability to show a benefit is dependent on a sensitive and reliable measure of treatment effect. Currently, measuring these deficits requires the use of time-consuming and subjective techniques. Objective measures extracted from natural conversations could be more ecologically relevant, and administered more frequently—perhaps giving them added sensitivity to change. While several studies have used automated analysis methods to study autistic speech, they require manual transcriptions. In order to bypass this time-consuming process, an automated speaker diarization algorithm must first be applied. In this paper, we are testing whether a speaker diarization algorithm can be applied to natural conversations between autistic individuals and their conversational partner in a natural setting at home over the course of a clinical trial. We calculated the average duration that a participant would speak for within their turn. We found a significant correlation between this feature and the Vineland Adaptive Behaviour Scales (VABS) expressive communication score (r = 0.51, p = 7 × 10–5). Our results show that natural conversations can be used to obtain measures of talkativeness, and that this measure can be derived automatically, thus showing the promise of objectively evaluating communication challenges in ASD.

show abstract

Multi-Channel End-to-End Neural Diarization with Distributed Microphones

Cited by 3 publications

References 26 publications

Automatic speaker diarization for natural conversation analysis in autism clinical trials

Automatic speaker diarization for natural conversation analysis in autism clinical trials

From Simulated Mixtures to Simulated Conversations as Training Data for End-to-End Neural Diarization

Automatic speaker diarization for natural conversation analysis in autism clinical trials

Contact Info

Product

Resources

About