Highly Efficient Parts of Speech Tagging in Low Resource Languages with Improved Hidden Markov Model and Deep Learning

Baishya, Diganta; Baruah, Rupam

doi:10.14569/ijacsa.2021.0121011

Cited by 4 publications

(1 citation statement)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The quality of the data used for training and testing the proposed POS tagger plays a crucial role in determining its performance. Also, the size of the dataset used in the proposed POS tagger, i.e., 50,000 words, is higher than the previously existing work (12) which used a dataset consisting of 25,000 words. The importance of dataset size and quality cannot be overstated, as they directly impact the effectiveness of the POS tagger.…”

Section: Recall = T P T P + Fnmentioning

confidence: 95%

Parts-of-Speech Tagging for Unknown Words in Assamese using Viterbi Algorithm

Phukan,

Baruah,

Sarma

et al. 2023

IJST

View full text Add to dashboard Cite

Objectives: This study aims to explore the application of the Viterbi algorithm for Part-of-Speech (POS) tagging in the Assamese language, focusing on tagging out-of-vocabulary words. The objective of this paper is to assess the algorithm's performance using various training and testing data ratios of 50:50, 70:30, and 90:10. Methods: The study utilizes the dynamic programming capabilities of the Viterbi algorithm to determine the most likely sequence of hidden states based on observable events. A corpus comprising approximately 50,000 words is employed to train the algorithm, with different ratios of this data utilized for training and testing purposes. Findings: The Viterbi algorithm achieves an accuracy of 86.34%, surpassing the state-of-the-art POS taggers for the Assamese language. The experimental evaluation demonstrates that the proposed approach outperforms previously existing research work by achieving 6.14% higher accuracy in tagging out-of-vocabulary words, highlighting its effectiveness in addressing the challenges associated with less-resourced languages like Assamese. Novelty: The results of this study contribute to the understanding and development of POS tagging techniques in less-resourced languages like Assamese. The proposed approach not only achieves superior performance in terms of accuracy but also showcases its potential for improving POS tagging in similar linguistic contexts, surpassing the achievements of previous research efforts.

show abstract

Section: Recall = T P T P + Fnmentioning

confidence: 95%