A Malay Text Corpus Analysis for Sentence Compression Using Pattern-Growth Method

Alias, Suraya; Mohammad, Siti Khaotijah; Gan, Keng Hoon; Ping, Tan Tien

doi:10.11113/jt.v78.7413

Cited by 6 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First of all, there are limited publicly available Malay text corpuses as most are private or for academic usage (Alias et al [6]). [3] claimed that the Dewan Bahasa and Pustaka, Pangkalan Data Korpus (UKM-DBP) could possibly be the most comprehensive corpus within Malaysia.…”

Section: Morphological and Lexical Analysis General Challengesmentioning

confidence: 99%

“…Most researchers have no choice but to resort to compile their own corpus specific to their domain. [6]and Hazaa et al [7] for instance, web crawled news articles from Malaysian National News Agency (BERNAMA) news while Saloot et al [4] compiled tweets using Tweeter API.…”

Section: Morphological and Lexical Analysis General Challengesmentioning

confidence: 99%

“…There is a scarcity of Malay NLP resources such as open source Part of Speech (POS) [6] and credible Named Entity Recognition (NER) system [2].…”

Section: Named Entity Recognition (Ner) Ner In Generalmentioning

confidence: 99%

“…Frequent Eliminated Pattern, a Pattern-Growth based model, was experimented with in [6] to study the heuristic linguistic rules deployed by human summarizers in compressing news articles. Three Malay language experts contributed to the research by summarising 100 tragedy related news and conducted Shallow POS tagging on the resulting summary dataset.…”

Section: Text Summarizationmentioning

confidence: 99%

See 3 more Smart Citations

Challenges and Development in Malay Natural Language Processing

2020

jcr

View full text Add to dashboard Cite

This paper aims to review thechallenges and current development in the Malay Natural Language Processing (NLP). The incongruence of the spoken and written Malay language presents a NLP challenge and is attributable to both (1) the cultural, state level dialects and (2) the mixed usage of English-Malay languages, being part of the South East Asia region's colonization legacy. There is also inadequate publicly available Malay NLP resources in the form of domain specific text corpus, open source Part of Speech (POS) and Named Entity Recognition (NER) systems. That said, significant progress is being made in Malay lexicon, information retrieval (stop words and NER), text normalization, machine translation, sentiment analysis and test summarization. These present potential commercial opportunity in reaching out to the 215 million native speakers in the region, not counting theglobal Malay diaspora community. The Malay NLP is also crucial in protecting national and cross border as digital counter-terrorism arsenal.

show abstract

Section: Morphological and Lexical Analysis General Challengesmentioning

confidence: 99%

Section: Morphological and Lexical Analysis General Challengesmentioning

confidence: 99%

“…There is a scarcity of Malay NLP resources such as open source Part of Speech (POS) [6] and credible Named Entity Recognition (NER) system [2].…”

Section: Named Entity Recognition (Ner) Ner In Generalmentioning

confidence: 99%

Section: Text Summarizationmentioning

confidence: 99%

See 2 more Smart Citations

Challenges and Development in Malay Natural Language Processing

2020

jcr

View full text Add to dashboard Cite

show abstract

“…In the Malay language, the exploration of text simplification is a relatively new area of study. Recent years have witnessed extensive research in Malay language studies, particularly in the domains of text summarization and sentence compression [8][9][10][11]. Researchers have been keen on enhancing the quality and cohesiveness of generated summaries.…”

Section: Introductionmentioning

confidence: 99%

Text Simplification using Hybrid Semantic Compression and Support Vector Machine for Troll Threat Sentences

Bakar,

Yusoff,

Harun

et al. 2023

IJACSA

View full text Add to dashboard Cite

Text Simplification (TS) is an emerging field in Natural Language Processing (NLP) that aims to make complex text more accessible. However, there is limited research on TS in the Malay language, known as Bahasa Malaysia, which is widely spoken in Southeast Asia. The challenges in this domain revolve around data availability, feature engineering, and the suitability of methods for text simplification. Previous studies predominantly employed single methods such as semantic compression, or machine learning with the Support Vector Machine (SVM) classifier consistently achieving an accuracy of approximately 70% in identifying troll sentences-statements containing threats from online trolls notorious for their disruptive online behavior. This study combines semantic compression and machine learning methods across lexical, syntactic, and semantic levels, utilizing frequency dictionaries as semantic features. Support Vector Machine and Decision Tree classifiers are applied and tested on 6,836 datasets, divided into training and testing sets. When comparing SVM and Decision Tree with and without semantic features, SVM with semantics achieves an average accuracy of 92.37%, while Decision Tree with semantics reaches 91.21%. The proposed TS method is evaluated on troll sentences, which are often associated with cyberbullying. Furthermore, it is worth noting that cyberbullying has been reported to be a significant issue, with Malaysia ranking as the second worst out of the 28 countries surveyed in Asia. Therefore, the outcomes of the study could potentially offer means, such as machine translation and relation extraction, to help prevent cyberbullying in Malaysia.

show abstract