Part of Speech Tagging in Urdu: Comparison of Machine and Deep Learning Approaches

Khan, Wahab; Daud, Ali; Khan, Khairullah; Nasir, Jamal Abdul; Basheri, Mohammed; Aljohani, Naif Radi; Alotaibi, Fahd Saleh

doi:10.1109/access.2019.2897327

Cited by 31 publications

(14 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Khan et al [21] dealt with a morphologically rich language, they examined and evaluated the effectiveness of machine learning and deep learning for morphologically rich language part-of-speech tagging.…”

Section: Related Workmentioning

confidence: 99%

Analysis of Edit Operations for Post-editing Systems

Kapusta

Benko

Munková

et al. 2021

Int J Comput Intell Syst

View full text Add to dashboard Cite

Post-editing has become an important part not only of translation research but also in the global translation industry. While computer-aided translation tools, such as translation memories, are considered to be part of a translator's work, lately, machine translation (MT) systems have also been accepted by human translators. However, many human translators are still adopting the changes brought by translation technologies to the translation industry. This paper introduces a novel approach for seeking suitable pairs of n-grams when recommending n-grams (corresponding n-grams between MT and post-edited MT) based on the type of text (manual or administrative) and MT system used for machine translation. A tool that recommends and speeds up the correction of MT was developed to help the post-editors with their work. It is based on the analysis of words with the same lemmas and analysis of n-gram recommendations. These recommendations are extracted from sequence patterns of the mismatched words (MisMatch) between MT output and post-edited MT output. The paper aims to show the usage of morphological analysis for recommending the post-edit operations. It describes the usage of mismatched words in the n-gram recommendations for the post-edited MT output. The contribution consists of the methodology for seeking suitable pairs of words, n-grams and additionally the importance of taking into account metadata (the type of the text and/or style and MT system) when recommending post-edited operations.

show abstract

Section: Related Workmentioning

confidence: 99%

Analysis of Edit Operations for Post-editing Systems

Kapusta

Benko

Munková

et al. 2021

Int J Comput Intell Syst

View full text Add to dashboard Cite

show abstract

“…The labeling of natural language text with POS tags can be a complicated task, requiring much effort, even for trained annotators (Rane et al, 2020). A large number of LRs are publicly available for high-resource languages such as English (Marcus and Marcinkiewicz), Chinese, Indian languages (Baskaran Sankaran and Subbarao, 2008;Khan et al, 2019) and others (Petrov et al, 2012). Unlike rich-resourced languages such as English and Chinese with abundant publicly accessible LRs, Sindhi is relatively low-resource , which lacks the POS tagged dataset that can be utilized to train a supervised or statistical algorithm.…”

Section: Related Workmentioning

confidence: 99%

SiPOS: A Benchmark Dataset for Sindhi Part-of-Speech Tagging

Ali¹,

Xu²

2021

Proceedings of the Student Research Workshop Associated With RANLP 2021

View full text Add to dashboard Cite

In this paper, we introduce the SiPOS dataset for part-of-speech tagging in the low-resource Sindhi language with quality baselines. The dataset consists of more than 293K tokens annotated with sixteen universal part-of-speech categories. Two experienced native annotators annotated the SiPOS using the Doccano text annotation tool with an inter-annotation agreement of 0.872. We exploit the conditional random field, the popular bidirectional long-short-term memory neural model, and self-attention mechanism with various settings to evaluate the proposed dataset. Besides pre-trained GloVe and fastText representation, the character-level representations are incorporated to extract character-level information using the bidirectional long-short-term memory encoder. The high accuracy of 96.25% is achieved with the task-specific joint wordlevel and character-level representations. The SiPOS dataset is likely to be a significant resource for the low-resource Sindhi language.

show abstract

“…We apply a deep learning approach to improve the performance of our biomedical NER system. Recent studies [39]- [42] using deep learning techniques have demonstrated the high performance for sequential labeling tasks contain the NER task. In particular, LSTM-CRF is known as a model with high performance in NER tasks [43], [44].…”

Section: ) Bidirectional Lstm-crf Network Based Biomedical Nermentioning

confidence: 99%

A Bootstrapping Approach With CRF and Deep Learning Models for Improving the Biomedical Named Entity Recognition in Multi-Domains

Kim

Seo

2019

IEEE Access

View full text Add to dashboard Cite

Biomedical named entity recognition (biomedical NER) is a core component to build biomedical text processing systems, such as biomedical information retrieval and question answering systems. Recently, many studies based on machine learning have been developed for a biomedical NER. The machine learning-based approaches generally require significant amounts of annotated corpora to achieve high performance. However, it is expensive to manually create a large number of high-quality corpora due to the demand for biomedical experts. In addition, most existing corpora have focused on several specific sub-domains, such as disease, protein, and species. It is difficult for a biomedical NER system trained with these corpora to provide much information for biomedical text processing systems. In this paper, we propose a method for automatically generating the machine-labeled biomedical NER corpus that covers various sub-domains by using proper categories from the semantic groups of a unified medical language system (UMLS). We use a bootstrapping approach with a small amount of manually annotated corpus to automatically generate a significant amount of corpus and then construct a biomedical NER system trained with the machine-labeled corpus. At last, we train two machine learning-based classifiers, conditional random fields (CRFs) and long short-term memory (LSTM), with the machine-labeled data to improve performance. The experimental results show that the proposed method is effective to improve performance. As a result, the proposed one obtains higher performance in 23.69% than the model that trained only a small amount of manually annotated corpus in F1-score.

show abstract

Part of Speech Tagging in Urdu: Comparison of Machine and Deep Learning Approaches

Cited by 31 publications

References 50 publications

Analysis of Edit Operations for Post-editing Systems

Analysis of Edit Operations for Post-editing Systems

SiPOS: A Benchmark Dataset for Sindhi Part-of-Speech Tagging

A Bootstrapping Approach With CRF and Deep Learning Models for Improving the Biomedical Named Entity Recognition in Multi-Domains

Contact Info

Product

Resources

About