Developing an Adaptive Language Model for Bahasa Indonesia

Hidayatullah, Satria Nur; Suyanto, Suyanto

doi:10.14569/ijacsa.2019.0100163

Cited by 8 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Again, this word is assigned to another topic, and its probabilistic score is calculated; after this iterative process, the list of words of each topic with a probability of belonging to a particular topic is obtained. For example [29] argues that these words tend to coexist in the same context, and words with high frequency have more significant positions in each topic. In the LDA model, a set of documents share the same topics, but the proportions are different for each document.…”

Section: Data Tagging and Trainingmentioning

confidence: 99%

Search and classify topics in a corpus of text using the latent dirichlet allocation model

Iparraguirre-Villanueva¹,

Sierra-Liñan

Salazar³

et al. 2023

IJEECS

View full text Add to dashboard Cite

This work aims at discovering topics in a text corpus and classifying the most relevant terms for each of the discovered topics. The process was performed in four steps: first, document extraction and data processing; second, labeling and training of the data; third, labeling of the unseen data; and fourth, evaluation of the model performance. For processing, a total of 10,322 "curriculum" documents related to data science were collected from the web during 2018-2022. The latent dirichlet allocation (LDA) model was used for the analysis and structure of the subjects. After processing, 12 themes were generated, which allowed ranking the most relevant terms to identify the skills of each of the candidates. This work concludes that candidates interested in data science must have skills in the following topics: first, they must be technical, they must have mastery of structured query language, mastery of programming languages such as R, Python, java, and data management, among other tools associated with the technology.

show abstract

Section: Data Tagging and Trainingmentioning

confidence: 99%

Search and classify topics in a corpus of text using the latent dirichlet allocation model

Iparraguirre-Villanueva¹,

Sierra-Liñan

Salazar³

et al. 2023

IJEECS

View full text Add to dashboard Cite

show abstract

“…In Indonesian, the same intonation can give different meanings depending on the topic domain of the word or term. For example, the Indonesian greeting "kemeja" with the same intonation can be written as "ke meja" (go to the table) or "kemeja" (a dress) [28].…”

Section: ) Stemmingmentioning

confidence: 99%

Indonesian Words Error Detection System using Nazief Adriani Stemmer Algorithm

Yudhana¹,

Fadlil²,

Rosidin³

2019

IJACSA

View full text Add to dashboard Cite

Stemming in each language has a different process and is determined according to the structure of the language. Stemming is mostly used as a complete step in the processing of words and phrases. There are many stemming algorithms available, and some used as a process for word processing. One function of stemming is to detect word errors in Indonesian. In this study, researchers created the Indonesian words error detection system using Nazief and Adriani algorithm. In the trials conducted, the system will accept text input obtained from the user. Then the system will preprocess the text. In this study, there are three stages of preprocessing, namely tokenization, case folding, and filtering. After the stages in preprocessing are finished, the system will call each word for the process of stemming. The results of the stemming will be compared with the base words available in the database. If it does not match, then the word is highlighted and is considered an error word. The first finding is the Nazief Adriani's algorithm can be able to detect words error until 100%. The second finding is the Nazief Adriani's algorithm also detect non-words error, the accuracy of detecting is 97.464%.

show abstract

“…It is important not just in some researches but also in many linguistics-based applications. It is generally used in speech recognition [1] [2], speech synthesis [3] [4] [5], emotion classification [6] [7], speaker's dialect identification [8], speaking rate estimation [9], speaking proficiency scoring [10], word count estimation [11], phonemicization [12] [13], collecting a minimum sentence set in developing speech corpus, as described in [14] [15] [16], etc.…”

Section: Introductionmentioning

confidence: 99%

Data Augmentation Methods for Low-Resource Orthographic Syllabification

et al. 2020

View full text Add to dashboard Cite

An n-gram syllabification model generally produces a high error rate for a low-resource language, such as Indonesian, because of the high rate of out-of-vocabulary (OOV) n-grams. In this paper, a combination of three methods of data augmentations is proposed to solve the problem, namely swapping consonant-graphemes, flipping onsets, and transposing nuclei. An investigation on 50k Indonesian words shows that the combination of three data augmentation methods drastically increases the amount of both unigrams and bigrams. A previous procedure of flipping onsets has been proven to enhance the standard bigram-syllabification by relatively decreasing the syllable error rate (SER) by up to 18.02%. Meanwhile, the previous swapping consonant-graphemes has been proven to give a relative decrement of SER up to 31.39%. In this research, a new transposing nuclei-based augmentation method is proposed and combined with both flipping and swapping procedures to tackle the drawback of bigram syllabification in handling the OOV bigrams. An evaluation based on k-fold cross-validation (k-FCV), using k = 5, for 50 thousand Indonesian formal words concludes that the proposed combination of the three procedures relatively decreases the mean SER produced by the standard bigram model by up to 37.63%. The proposed model is comparable to the fuzzy k-nearest neighbor in every class (FkNNC)-based model. It is worse than the state-of-the-art model, which is developed using a combination of bidirectional long short-term memory (BiLSTM), convolutional neural networks (CNN), and conditional random fields (CRF), but it offers a low complexity.

show abstract

Developing an Adaptive Language Model for Bahasa Indonesia

Cited by 8 publications

References 11 publications

Search and classify topics in a corpus of text using the latent dirichlet allocation model

Search and classify topics in a corpus of text using the latent dirichlet allocation model

Indonesian Words Error Detection System using Nazief Adriani Stemmer Algorithm

Data Augmentation Methods for Low-Resource Orthographic Syllabification

Contact Info

Product

Resources

About