Tongue Contour Tracking and Segmentation in Lingual Ultrasound for Speech Recognition: A Review

Al-hammuri, Khalid; Gebali, Fayez; Chelvan, Ilamparithi Thirumarai; Kanan, Awos

doi:10.3390/diagnostics12112811

Cited by 9 publications

(6 citation statements)

References 116 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tongue contour tracking has been explored using different techniques that are based on traditional computer vision techniques or deep learning-based algorithms [4]. Traditional techniques are based on two base models: The Snake algorithm, the active contour model, and the shape consistency model.…”

Section: Background and Related Workmentioning

confidence: 99%

“…The head and oral cavity picture was modified from the original picture for the case, courtesy of Associate Professor Frank Gaillard, Radiopaedia.org, rID: 35836, [25]. Right: Output of recorded ultrasound image, ultrasound image source [18].…”

Section: Image Ingestionmentioning

confidence: 99%

“…In the cited work [18], the researchers proposed a multi-stage algorithm that incorporates preprocessing to enhance noisy ultrasound images. The algorithm also extracts features using a combined shock filter and curvelet transform.…”

Section: Background and Related Workmentioning

confidence: 99%

“…The methods that are based on Convolutional neural networks (CNNs) were the most efficient compared to other techniques. In the era of deep learning, UNet architecture became the stateof-the-art architecture to segment medical images [2][3][4]. UNet architecture is structured based on a stack of convolutional layers to form encoders and decoders arrangement [5].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

TongueTransUNet: Toward Effective Tongue Contour Segmentation Using Small Dataset

Al-hammuri,

Gebali,

Kanan

2024

Preprint

Self Cite

View full text Add to dashboard Cite

Medical image segmentation is important for extracting desired objects among complex human structures to enable further analysis. In the case of lingual ultrasound, it is important to extract tongue contour to understand the language behaviour, which enables lingual ultrasound to act as a biofeedback. In order to segment tongue from ultrasound images, we need to train the deep-learning model on a large dataset, which made it challenging to generalize it using a wide variety of images as it is difficult to collect this huge data. In this research, we are proposing a strategy and generalized model that can work effectively using a well-managed small dataset. This article presents a hybrid architecture using UNet, Vision Transformer (ViT) and Contrastive loss to build a foundation model cumulatively. The process starts with building a reference representation in the embedding space using human experts to validate any new input for training data. UNet and ViT encoders are used to extract the input feature representations. The contrastive loss was then used to compare the new feature embedding with the reference in the embedding space. The UNet-based decoder is used to reconstruct the image to its original size. Before releasing the final results, a quality control process is used to assess the value of the segmented contour, and if rejected, the algorithm requests an action from a human expert to annotate it manually. The results show an improved accuracy over the traditional techniques and can be generalized as it contains only high-quality and relevant features related to the tongue in the embedding space.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Section: Image Ingestionmentioning

confidence: 99%

Section: Background and Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

TongueTransUNet: Toward Effective Tongue Contour Segmentation Using Small Dataset

Al-hammuri,

Gebali,

Kanan

2024

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…MRI has become the gold standard modality for speech imaging [1]. It demonstrates numerous advantages over other imaging techniques such as electromagnetic articulography [2], X-rays [3], and ultrasound imaging [4,5]. However, due to physical constraints, it remains relatively slow, which complicates imaging of fast movements and leads to the presence of obstructing motion artifacts.…”

Section: Introductionmentioning

confidence: 99%

Super-Resolved Dynamic 3D Reconstruction of the Vocal Tract during Natural Speech

Isaieva,

Odille,

Laprie

et al. 2023

J. Imaging

View full text Add to dashboard Cite

MRI is the gold standard modality for speech imaging. However, it remains relatively slow, which complicates imaging of fast movements. Thus, an MRI of the vocal tract is often performed in 2D. While 3D MRI provides more information, the quality of such images is often insufficient. The goal of this study was to test the applicability of super-resolution algorithms for dynamic vocal tract MRI. In total, 25 sagittal slices of 8 mm with an in-plane resolution of 1.6 × 1.6 mm2 were acquired consecutively using a highly-undersampled radial 2D FLASH sequence. The volunteers were reading a text in French with two different protocols. The slices were aligned using the simultaneously recorded sound. The super-resolution strategy was used to reconstruct 1.6 × 1.6 × 1.6 mm3 isotropic volumes. The resulting images were less sharp than the native 2D images but demonstrated a higher signal-to-noise ratio. It was also shown that the super-resolution allows for eliminating inconsistencies leading to regular transitions between the slices. Additionally, it was demonstrated that using visual stimuli and shorter text fragments improves the inter-slice consistency and the super-resolved image sharpness. Therefore, with a correct speech task choice, the proposed method allows for the reconstruction of high-quality dynamic 3D volumes of the vocal tract during natural speech.

show abstract

Research in methodologies for modelling the oral cavity

Shahid,

French,

Valstar

et al. 2024

Biomed. Phys. Eng. Express

View full text Add to dashboard Cite

The paper aims to explore the current state of understanding surrounding in silico oral modelling. This involves exploring methodologies, technologies and approaches pertaining to the modelling of the whole oral cavity; both internally and externally visible structures that may be relevant or appropriate to oral actions. Such a model could be referred to as a “complete model” which includes consideration of a full set of facial features (i.e. not only mouth) as well as synergistic stimuli such as audio and facial thermal data. 3D modelling technologies capable of accurately and efficiently capturing a complete representation of the mouth for an individual have broad applications in the study of oral actions, due to their cost-effectiveness and time efficiency.This review delves into the field of clinical phonetics to classify oral actions pertaining to both speech and non-speech movements, identifying how the various vocal organs play a role in the articulatory and masticatory process. Vitaly, it provides a summation of 12 articulatory recording methods, forming a tool to be used by researchers in identifying which method of recording is appropriate for their work. After addressing the cost and resource-intensive limitations of existing methods, a new system of modelling is proposed that leverages external to internal correlation modelling techniques to create a more efficient models of the oral cavity.The vision is that the outcomes will be applicable to a broad spectrum of oral functions related to physiology, health and wellbeing, including speech, oral processing of foods as well as dental health. The applications may span from speech correction, designing foods for the aging population, whilst in the dental field we would be able to gain information about patient’s oral actions that would become part of creating a personalised dental treatment plan.

show abstract

Tongue Contour Tracking and Segmentation in Lingual Ultrasound for Speech Recognition: A Review

Cited by 9 publications

References 116 publications

TongueTransUNet: Toward Effective Tongue Contour Segmentation Using Small Dataset

TongueTransUNet: Toward Effective Tongue Contour Segmentation Using Small Dataset

Super-Resolved Dynamic 3D Reconstruction of the Vocal Tract during Natural Speech

Research in methodologies for modelling the oral cavity

Contact Info

Product

Resources

About