RoBERTa: language modelling in building Indonesian question-answering systems

Suwarningsih, Wiwin; Pramata, Raka Aditya; Rahadika, Fadhil Yusuf; Purnomo, Mochamad Havid Albar

doi:10.12928/telkomnika.v20i6.24248

Cited by 2 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, the paper on Bidirectional Encoder Representations from Transformers (BERT) tested learning rates of 1E-5 and 5e-5 for both languages, as per the reference. The results obtained are an accuracy of 91.7% and an F1score of 86.2% with the conclusion RoBERTa model was better to implement the Indonesian questionand-answer system [11].…”

Section: Implementation Of Recurrent Neural Network (Rnn) For Questio...mentioning

confidence: 83%

Implementation of Recurrent Neural Network (RNN) for Question Similarity Identification in Indonesian Language

Iqbal,

Hasmawati,

Romadhony

2023

join

View full text Add to dashboard Cite

In a question-and-answer forum, the identification of question similarity is used to determine how similar two questions are. This procedure makes sure that user-submitted questions are compared to the questions in a database for matches to improve system performance on the online Q&A platform. Currently, question similarity is mostly done in foreign languages. The purpose of this research is to identify question similarities and evaluate the effectiveness of the methods used in Indonesian language questions. The data used is a public dataset with labeled pairs of questions as 0 and 1 where label 0 for different pairs of questions and label 1 for the same pairs of questions. The method used is a Recurrent Neural Network (RNN) with the Manhattan Distance approach to calculate the similarity distance between two questions. The question pairs are taken as two inputs with a reference label to identify the similarity distance between the two question inputs. We evaluated the model using three different optimizers namely RMSprop, Adam, and Adagrad. The best results were obtained using the Adam optimizer with 80:20 ratio split-data and overall accuracy is 76%, precision is 74%, recall is 98.8%, and F1-score is 85.1%.

show abstract

Section: Implementation Of Recurrent Neural Network (Rnn) For Questio...mentioning

confidence: 83%

Implementation of Recurrent Neural Network (RNN) for Question Similarity Identification in Indonesian Language

Iqbal,

Hasmawati,

Romadhony

2023

join

View full text Add to dashboard Cite

show abstract

Document Information Extraction: An Analysis of Invoice Anatomy

Hamri,

Devanne,

Weber

et al. 2024

Applied Computational Intelligence and Soft Computing

View full text Add to dashboard Cite

In this paper, we present a new approach of document information extraction by studying the document anatomy where we investigated the possible variants and forms it could have for each document component. To overcome the lack of publicly available document datasets, we used a generated invoice database where we conceived 9 different templates and generated 100 samples for each one where the documents were annotated automatically during the generation process. We analysed the following invoice components: dates block (invoice date and invoice due date), address block, amounts block (tax‐free amount, tax amount, and total amount), and lines block (lines table) by investigating the impact of training our model on various block variants. We conducted several experiments where we compared the results obtained when we tested on templates that included variants not encountered during the training phase versus when we introduced them to the training dataset. This allowed us to analyse the improvement in results after adding these previously unseen variants. The obtained results have shown that the model generalises better when trained on a large variety of cases and achieves remarkable performance. We conducted experiments on various models to highlight the model‐agnostic character of our proposed approach. This methodology allows to have great performance, even with models that have significantly fewer parameters, especially in comparison to recently published models with millions of parameters.

show abstract

RoBERTa: language modelling in building Indonesian question-answering systems

Cited by 2 publications

References 19 publications

Implementation of Recurrent Neural Network (RNN) for Question Similarity Identification in Indonesian Language

Implementation of Recurrent Neural Network (RNN) for Question Similarity Identification in Indonesian Language

Document Information Extraction: An Analysis of Invoice Anatomy

Contact Info

Product

Resources

About