Chunking Arabic texts using Conditional Random Fields

Khoufi, Nabil; Aloulou, Chafik; Belguith, Lamia Hadrich

doi:10.1109/aiccsa.2014.7073230

Cited by 6 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other NLP libraries (SpaCy and Natural Language Tool Kit) also provide users with supervised nounphrase chunkers trained on the CoNLL-2000 corpus. Supervision limits the parsing model to the available annotated corpora, and therefore a supervised approach for text chunking has been used for context biased text with domain-and language-specific annotated corpora: for instance, chunkers for the bio-medical and medical fields [36], chunkers for software engineering [37], Chinese, Thai, and Arabic language chunkers [38][39][40], and social media text chunkers [41][42][43]. Table 1 groups the text chunking methods found in the literature, whereas Table 2 shows an overview of supervised approaches for text chunking with their respective performances on specific annotated datasets.…”

Section: Supervised and Semi-supervised Approachesmentioning

confidence: 99%

Unsupervised acquisition of idiomatic units of symbolic natural language: An n-gram frequency-based approach for the chunking of news articles and tweets

2020

View full text Add to dashboard Cite

Symbolic sequential data are produced in huge quantities in numerous contexts, such as text and speech data, biometrics, genomics, financial market indexes, music sheets, and online social media posts. In this paper, an unsupervised approach for the chunking of idiomatic units of sequential text data is presented. Text chunking refers to the task of splitting a string of textual information into non-overlapping groups of related units. This is a fundamental problem in numerous fields where understanding the relation between raw units of symbolic sequential data is relevant. Existing methods are based primarily on supervised and semi-supervised learning approaches; however, in this study, a novel unsupervised approach is proposed based on the existing concept of n-grams, which requires no labeled text as an input. The proposed methodology is applied to two natural language corpora: a Wall Street Journal corpus and a Twitter corpus. In both cases, the corpus length was increased gradually to measure the accuracy with a different number of unitary elements as inputs. Both corpora reveal improvements in accuracy proportional with increases in the number of tokens. For the Twitter corpus, the increase in accuracy follows a linear trend. The results show that the proposed methodology can achieve a higher accuracy with incremental usage. A future study will aim at designing an iterative system for the proposed methodology.

show abstract

Section: Supervised and Semi-supervised Approachesmentioning

confidence: 99%

Unsupervised acquisition of idiomatic units of symbolic natural language: An n-gram frequency-based approach for the chunking of news articles and tweets

2020

View full text Add to dashboard Cite

show abstract

“…The Information generated by this task can be helpful for many purposes, including automatic summarizing or question answering, information extraction. It can be either shallow parsing or deep parsing [2].…”

Section: Introductionmentioning

confidence: 99%

“…Today different researchers investigated text-chunking for the various language of the world using a different methodology, for instance, Arabic [2], Chinese [6], Bengali [3], Amharic [7], Indonesian [8], Urdu [9]. As far as the researchers' knowledge concerned, only one work was done for Amharic.…”

Section: Introductionmentioning

confidence: 99%

“…The main reason is the Amharic language is highly inflectional, morphologically rich, and has its own letters and grammatical rules. Conditional Random Fields (CRFs) are one class of supervised ML, it was applied for different IR and NLP applications, where they are used for classification of sequential data [2].it were first introduced by [11] for the different sequential annotation tasks. Some of them were named entity recognition, POS tag, text chunking, etc., where they get excellent results ( [7], [3], [2]).…”

Section: Introductionmentioning

confidence: 99%

“…Conditional Random Fields (CRFs) are one class of supervised ML, it was applied for different IR and NLP applications, where they are used for classification of sequential data [2].it were first introduced by [11] for the different sequential annotation tasks. Some of them were named entity recognition, POS tag, text chunking, etc., where they get excellent results ( [7], [3], [2]). Due to the above reason, we are motivated to investigate Text chunker for Amharic language using them (i.e., CRFs).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Amharic Text Chunker Using Conditional Random Fields

Getaneh¹,

Achamyeleh²,

Belay³

2021

Com, Engine, Sys, Sci

View full text Add to dashboard Cite

This paper introduces Amharic text chunker using conditional random fields. To get the optimal feature set of the chunker; the researchers’ conduct different experiments using different scenarios until a promising result obtained. In this study different sentences are collected from Amharic grammar books, new articles, magazines and news of Walta Information Center (WIC) for the training and testing datasets. Thus, these datasets were analyzed and tagged manually and used as a corpus for our model training and testing. The entire datasets were chunk tagged manually for the training dataset and approved by linguistic professionals. For the identification of the boundary of the phrases IOB2 chunk specification is selected and used in this study. The result of all experiments is reported with the maximum overall accuracy off 97.26%, with a window size of two on both sides, with their corresponding POS tag of each token and the worst performance achieved is 84.57%, with only the window size of one word on both the left and right sides.

show abstract