Evaluating the Morphological and Capitalization Features for Word Embedding-Based POS Tagger in Bahasa Indonesia

Manik, Lindung Parningotan; Syafiandini, Arida Ferti; Mustika, Hani Febri; Abka, Achmad F.; Rianto, Yan

doi:10.1109/ic3ina.2018.8629519

Cited by 6 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The work in [13,15] pre-trained the Long-Short-Term Memory (LSTM) network from scratch for neural sentence embedding. GloVe, Word2Vec, and FastText are popular methods in this context to create word embeddings [9]. However, this approach requires vast resources and high costs.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Out-of-Scope Intent Detection on A Knowledge-Based Chatbot

Manik¹

2021

IJIES

View full text Add to dashboard Cite

Knowledge-based chatbot (KBC) has grown in popularity in recent years and has been widely used for various use cases. Building KBC from scratch using deep learning (DL) is challenging since no prior historical data exists. Meanwhile, DL systems need a vast Volume of data to be trained. This paper proposes a novel framework to create an intent classifier of the KBC used to detect in-scope (IS) and out-of-scope (OOS) intents. We introduce an automated queries generator to create IS intents employed as the training data from an ontology input. We utilize Bidirectional Encode Representations from Transformers (BERT) fine-tuning as the backbone of our DL system. Moreover, we present a Bayesian approach as an extension of the BERT to classify OOS queries with minimal OOS training data. The experiments result show that the proposed method manages to achieve an F1 score of 100% for IS intents and 86% for OOS queries.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Without the data, training the intent classifier that utilizes machine learning (ML) algorithms is challenging. Moreover, it is well-known that deep learning (DL) systems need a vast Volume of data to be trained [9]. In addition, determining a user's unknown intents is problematic since there is no prior information [10].…”

Section: Introductionmentioning

confidence: 99%

Out-of-Scope Intent Detection on A Knowledge-Based Chatbot

Manik¹

2021

IJIES

View full text Add to dashboard Cite

show abstract

“…Di tahun 2018, Manik dkk menggunakan fitur morfologis dan kapitalisasi [21] untuk meningkatkan performa pos tagger dengan word embedding yang sudah ada. Arsitektur meural network yang digunakan adalah dua layer masukan, satu layer gabungan dan dua layer tersembunyi.…”

Section: Neural Networkunclassified

Perkembangan Part-of-Speech Tagger Bahasa Indonesia

Kamayani¹

2019

JLK

View full text Add to dashboard Cite

Tujuan dari artikel ini adalah membuat kajian literatur terhadap metode pelabelan part-of-speech (POS tagger) untuk Bahasa Indonesia yang telah dilakukan selama 11 tahun terakhir (sejak tahun 2008). Artikel ini dapat menjadi roadmap POS tagger Bahasa Indonesia dan juga dasar pertimbangan untuk pengembangan selanjutnya agar menggunakan dataset dan tagset yang standar sebagai benchmark metode. Terdapat 15 publikasi yang dibahas, pembahasan meliputi dataset, tagset dan metode yang digunakan untuk POS tag Bahasa Indonesia. Dataset yang paling banyak digunakan dan paling mungkin menjadi corpus standar adalah IDN Tagged Corpus terdiri dari lebih dari 250.000 token. Tagset Bahasa Indonesia hingga saat ini belum terstandarisasi dengan jumlah label bervariasi dari 16 tag hingga 37 tag. Metode yang paling banyak dikembangkan dan berpotensi menjadi state-of-the-art adalah neural network, dengan varian metode biLSTM dan CRF dan sejauh ini memberikan skor F1 dan akurasi tertinggi (>96%).

show abstract

“…It is considered one of the breakthroughs in deep learning. Studies in [5], [6], [7] suggested the Word2Vec approach to extract text features, while others suggested Glove [8]. Nevertheless, both approaches are context-independent, and they could not catch all semantic information such as Out-Of-Vocabulary (OOV) and some opposite word pairs.…”

Section: Introductionmentioning

confidence: 99%

On the Role of Text Preprocessing in BERT Embedding-based DNNs for Classifying Informal Texts

Kurniasih¹,

Manik²

2022

IJACSA

Self Cite

View full text Add to dashboard Cite

Due to highly unstructured and noisy data, analyzing society reports in written texts is very challenging. Classifying informal text data is still considered a difficult task in natural language processing since the texts could contain abbreviated words, repeating characters, typos, slang, et cetera. Therefore, text preprocessing is commonly performed to remove the noises and make the texts more structured. However, we argued that most tasks of preprocessing are no longer required if suitable word embeddings approach and deep neural network (DNN) architecture are correctly chosen. This study investigated the effects of text preprocessing in fine-tuning a pre-trained Bidirectional Encoder Representations from Transformers (BERT) model using various DNN architectures such as multilayer perceptron (MLP), long short-term memory (LSTM), bidirectional long-short term memory (Bi-LSTM), convolutional neural network (CNN), and gated recurrent unit (GRU). Various experiments were conducted using numerous learning rates and batch sizes. As a result, text preprocessing had insignificant effects on most models such as LSTM, Bi-LSTM, and CNN. Moreover, the combination of BERT embeddings and CNN produced the best classification performance.

show abstract

Evaluating the Morphological and Capitalization Features for Word Embedding-Based POS Tagger in Bahasa Indonesia

Cited by 6 publications

References 6 publications

Out-of-Scope Intent Detection on A Knowledge-Based Chatbot

Out-of-Scope Intent Detection on A Knowledge-Based Chatbot

Perkembangan Part-of-Speech Tagger Bahasa Indonesia

On the Role of Text Preprocessing in BERT Embedding-based DNNs for Classifying Informal Texts

Contact Info

Product

Resources

About