Improving Bi-LSTM Performance for Indonesian Sentiment Analysis Using Paragraph Vector

Abstract

Bidirectional Long Short-Term Memory Network (Bi-LSTM) has shown promising performance in sentiment classification task. It processes inputs as sequence of information. Due to this behavior, sentiment predictions by Bi-LSTM were influenced by words sequence and the first or last phrases of the texts tend to have stronger features than other phrases. Meanwhile, in the problem scope of Indonesian sentiment analysis, phrases that express the sentiment of a document might not appear in the first or last part of the document that can lead to incorrect sentiment classification. To this end, we propose the using of an existing document representation method called paragraph vector as additional input features for Bi-LSTM. This vector provides information context of the document for each sequence processing. The paragraph vector is simply concatenated to each word vector of the document. This representation also helps to differentiate ambiguous Indonesian words. Bi-LSTM and paragraph vector were previously used as separate methods. Combining the two methods has shown a significant performance improvement of Indonesian sentiment analysis model. Several case studies on testing data showed that the proposed method can handle the sentiment phrases position problem encountered by Bi-LSTM.

Improving Indonesian Text Classification Using Multilingual Language Model

“…However, there are cases where the addition of excessive English data decreases classification performance. Using a fine-tuning approach, we further improve the result of previous research on sentiment analysis [2,3] and hate speech detection [4] on the Indonesian language.…”

“…Through this result, we can see that adding English data can help the performance of the model. On [2] & [3] dataset, adding English language data consistently improves the performance. However, on [4], there is a point where the added English data results in worse performance.…”

“…Evaluation of their dataset yields 0.8521 F1-score using a neural network with TF-IDF as its feature. Crisdayanti and Purwarianti [3] conducted experiments on various text representations and neural network topology models. Evaluation of their dataset yields 0.9369 F1-score using Bi-LSTM with word embedding enhanced with paragraph vector.…”

“…The experiment was reduced to only using the maximum total data and an added English data multiplier up to 3. For the Indonesian language sentiment analysis dataset, we used data originated from previous related work by [2,3]. Farhan and Khodra [2] used reviews crawled from TripAdvisor.…”

“…For example, the tasks of sentiment analysis and hate speech detection have English dataset like Yelp Review 1 (598K) and Jigsaw Toxic Comment 2 (1902K) respectively. In contrast with the Indonesian language dataset from recent publications such as [2] (12K) & [3] (11K) for sentiment analysis, and [4] (12K) for hate speech detection.…”