Intelligibility Improvement of Esophageal Speech Using Sequence-to-Sequence Voice Conversion with Auditory Attention

Abstract

Laryngectomees are individuals whose larynx has been surgically removed, usually due to laryngeal cancer. The immediate consequence of this operation is that these individuals (laryngectomees) are unable to speak. Esophageal speech (ES) remains the preferred alternative speaking method for laryngectomees. However, compared to the laryngeal voice, ES is characterized by low intelligibility and poor quality due to chaotic fundamental frequency F0, specific noises, and low intensity. Our proposal to solve these problems is to take advantage of voice conversion as an effective way to improve speech quality and intelligibility. To this end, we propose in this work a novel esophageal–laryngeal voice conversion (VC) system based on a sequence-to-sequence (Seq2Seq) model combined with an auditory attention mechanism. The originality of the proposed framework is that it adopts an auditory attention technique in our model, which leads to more efficient and adaptive feature mapping. In addition, our VC system does not require the classical DTW alignment process during the learning phase, which avoids erroneous mappings and significantly reduces the computational time. Moreover, to preserve the identity of the target speaker, the excitation and phase coefficients are estimated by querying a binary search tree. In experiments, objective and subjective tests confirmed that the proposed approach performs better even in some difficult cases in terms of speech quality and intelligibility.

Special Issue on Applications of Speech and Language Technologies in Healthcare

“…Two papers address esophageal speech, both using speech conversion techniques to improve its quality and intelligibility. Ezzine et al [5] used a novel sequence-to-sequence model with an auditory attention mechanism, while Raman et al [6] used synthetic speech as the target of a voice conversion system to improve the quality and intelligibility of the original voice.…”

Cross-Lingual Propaganda Detection

“…Variational autoencoder (VAE) [12]- [15] is one such implementation, which learns a latent space for speakerindependent representation. Similarly, generative adversarial networks (GAN) [16] disentangle the speech attributes with an extra adversarial loss to guarantee a distribution match between the generated and true data [17], [18].…”