Text-to-Speech for Low-Resource Agglutinative Language With Morphology-Aware Language Model Pre-Training

Liu, Rui; Hu, Yifan; Zuo, Haolin; Luo, Zhaojie; Wang, Longbiao; Gao, Guanglai

doi:10.1109/taslp.2023.3348762

Cited by 6 publications

(3 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, morphological segmentation, which divides words into their smallest semantic units while maintaining semantic information, effectively alleviates the data sparsity issue caused by rich morphology. Therefore, morphological segmentation and stemming are widely used in various downstream natural language processing tasks such as named entity recognition [8], keyword extraction [4], question answering [9], speech recognition [10], machine translation [11,12], and language modeling [3].…”

Section: ‫نىڭكى‬mentioning

confidence: 99%

“…Models are trained on a 3090 GPU. The supervised experiment parameters are as follows: character embedding dimension is 128, CNN window sizes are [1,3,5], and BiLSTM hidden layer dimension is 128. The input dimension of the Transformer is 512, with a single-layer Transformer structure and an 8-head attention mechanism.…”

Section: Hyperparametersmentioning

confidence: 99%

“…The stem expresses the meaning of the word, whereas the affix expresses grammatical information. Uyghur and Kazakh are typical agglutinative languages with extremely rich morphology [3] and a rich set of affixes expressing derivation or inflection. The lexical and grammatical structures of the two languages are very similar [4], and their word formation rules can be represented as [5] pre f ix 1 + ... + pre f ix n + stem + su f f ix 1 + ... + su f f ix m .…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Benchmark for Morphological Segmentation in Uyghur and Kazakh

Abudouwaili,

Ruzmamat,

Abiderexiti

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Morphological segmentation and stemming are foundational tasks in natural language processing. They have become effective ways to alleviate data sparsity in agglutinative languages because of the nature of agglutinative language word formation. Uyghur and Kazakh, as typical agglutinative languages, have made significant progress in morphological segmentation and stemming in recent years. However, the evaluation metrics used in previous work are character-level based, which may not comprehensively reflect the performance of models in morphological segmentation or stemming. Moreover, existing methods avoid manual feature extraction, but the model’s ability to learn features is inadequate in complex scenarios, and the correlation between different features has not been considered. Consequently, these models lack representation in complex contexts, affecting their effective generalization in practical scenarios. To address these issues, this paper redefines the morphological-level evaluation metrics: F1-score and accuracy (ACC) for morphological segmentation and stemming tasks. In addition, two models are proposed for morpheme segmentation and stem extraction tasks: supervised model and unsupervised model. The supervised model learns character and contextual features simultaneously, then feature embeddings are input into a Transformer encoder to study the correlation between character and context embeddings. The last layer of the model uses a CRF or softmax layer to determine morphological boundaries. In unsupervised learning, an encoder–decoder structure introduces n-gram correlation assumptions and masked attention mechanisms, enhancing the correlation between characters within n-grams and reducing the impact of characters outside n-grams on boundaries. Finally, comprehensive comparative analyses of the performance of different models are conducted from various points of view. Experimental results demonstrate that: (1) The proposed evaluation method effectively reflects the differences in morphological segmentation and stemming for Uyghur and Kazakh; (2) Learning different features and their correlation can enhance the model’s generalization ability in complex contexts. The proposed models achieve state-of-the-art performance on Uyghur and Kazakh datasets.

show abstract