EPG2S: Speech Generation and Speech Enhancement Based on Electropalatography and Audio Signals Using Multimodal Learning

Chen, Li-Chin; Chen, Po-Hsun; Tsai, Richard Tzong-Han; Tsao, Yu

doi:10.1109/lsp.2022.3184636

Cited by 7 publications

(2 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, EPG information is also applied to get an accurate image registration by a CT scanner [ 35 ]. Furthermore, EPG can be combined with audio signals for speech generation and speech enhancement applications [ 48 ]. EPG uses a hard plate beneath the tongue to detect the contact between the tongue and the array of sensors in the plate.…”

Section: Introductionmentioning

confidence: 99%

Tongue Contour Tracking and Segmentation in Lingual Ultrasound for Speech Recognition: A Review

et al. 2022

View full text Add to dashboard Cite

Lingual ultrasound imaging is essential in linguistic research and speech recognition. It has been used widely in different applications as visual feedback to enhance language learning for non-native speakers, study speech-related disorders and remediation, articulation research and analysis, swallowing study, tongue 3D modelling, and silent speech interface. This article provides a comparative analysis and review based on quantitative and qualitative criteria of the two main streams of tongue contour segmentation from ultrasound images. The first stream utilizes traditional computer vision and image processing algorithms for tongue segmentation. The second stream uses machine and deep learning algorithms for tongue segmentation. The results show that tongue tracking using machine learning-based techniques is superior to traditional techniques, considering the performance and algorithm generalization ability. Meanwhile, traditional techniques are helpful for implementing interactive image segmentation to extract valuable features during training and postprocessing. We recommend using a hybrid approach to combine machine learning and traditional techniques to implement a real-time tongue segmentation tool.

show abstract

Section: Introductionmentioning

confidence: 99%

Tongue Contour Tracking and Segmentation in Lingual Ultrasound for Speech Recognition: A Review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Graph generation aims to provide models that can generate new graph samples from the desired data distributions. Thus, similar to generative methods in other data domains such as image [ 10 ], text [ 11 ], and speech [ 12 ], graph generative approaches can bring substantial capacity for graph data modeling to address various real-world problems such as drug design [ 13 ], understanding and modeling the interactions in social networks [ 14 ], and human diseases diagnosis [ 15 ].…”

Section: Introductionmentioning

confidence: 99%

SCGG: A deep structure-conditioned graph generative model

et al. 2022

View full text Add to dashboard Cite

Deep learning-based graph generation approaches have remarkable capacities for graph data modeling, allowing them to solve a wide range of real-world problems. Making these methods able to consider different conditions during the generation procedure even increases their effectiveness by empowering them to generate new graph samples that meet the desired criteria. This paper presents a conditional deep graph generation method called SCGG that considers a particular type of structural conditions. Specifically, our proposed SCGG model takes an initial subgraph and autoregressively generates new nodes and their corresponding edges on top of the given conditioning substructure. The architecture of SCGG consists of a graph representation learning network and an autoregressive generative model, which is trained end-to-end. More precisely, the graph representation learning network is designed to compute continuous representations for each node in a graph, which are not only affected by the features of adjacent nodes, but also by the ones of farther nodes. This network is primarily responsible for providing the generation procedure with the structural condition, while the autoregressive generative model mainly maintains the generation history. Using this model, we can address graph completion, a rampant and inherently difficult problem of recovering missing nodes and their associated edges of partially observed graphs. The computational complexity of the SCGG method is shown to be linear in the number of graph nodes. Experimental results on both synthetic and real-world datasets demonstrate the superiority of our method compared with state-of-the-art baselines.

show abstract

Speech synthesis from three-axis accelerometer signals using conformer-based deep neural network

Kwon,

Hwang,

Sung

et al. 2024

Computers in Biology and Medicine

View full text Add to dashboard Cite

EPG2S: Speech Generation and Speech Enhancement Based on Electropalatography and Audio Signals Using Multimodal Learning

Cited by 7 publications

References 37 publications

Tongue Contour Tracking and Segmentation in Lingual Ultrasound for Speech Recognition: A Review

Tongue Contour Tracking and Segmentation in Lingual Ultrasound for Speech Recognition: A Review

SCGG: A deep structure-conditioned graph generative model

Speech synthesis from three-axis accelerometer signals using conformer-based deep neural network

Contact Info

Product

Resources

About