Q-Learning for Shift-Reduce Parsing in Indonesian Tree-LSTM-Based Text Generation

Abstract

Tree-LSTM algorithm accommodates tree structure processing to extract information outside the linear sequence pattern. The use of Tree-LSTM in text generation problems requires the help of an external parser at each generation iteration. Developing a good parser demands the representation of complex features and relies heavily on the grammar of the corpus. The limited corpus results in an insufficient number of vocabs for a grammar-based parser, making it less natural to link the text generation process. This research aims to solve the problem of limited corpus by proposing the use of a Reinforcement Learning algorithm in the formation of constituency trees, which link the sentence generation process given a seed phrase as the input in the Tree-LSTM model. The tree production process is modeled as a Markov’s decision process, where a set of states consists of word embedding vectors, and a set of actions of {Shift, Reduce}. The Deep Q-Network model as an approximator of the Q-Learning algorithm is trained to obtain optimal weights in representing the Q-value function. The test results on perplexity-based evaluation show that the proposed Tree-LSTM and Q-Learning combination model achieves values 9.60 and 4.60 for two kinds of corpus with 205 and 1,000 sentences, respectively, better than the Shift-All model. Human evaluation of Friedman test and posthoc analysis showed that all five respondents tended to give the same assessment for the combination model of Tree-LSTM and Q-Learning, which on average outperforms two other nongrammar models, i.e., Shift-All and Reduce-All.

Application and Effect Analysis of Think Tank Information Processing Technology in Cultivating Strategic Research Talents in Enterprises

“…RP Hastuti and his team proposed a method to solve the limited corpus problem in their research [3], which involves using reinforcement learning algorithms in the formation of selection trees. This algorithm associates the given seed phrase with the input of the tree LSTM model during sentence generation, effectively addressing the challenges of limited corpus.…”

Child_Sum EATree-LSTMs: Enhanced Attentive Child_Sum Tree-LSTMs for Biomedical Event Extraction

“…In recent years, Tree-LSTM have been widely applied to many fields of NLP, such as text generation, neural machine translation, sentiment analysis, sentence semantic modeling, and event extraction tasks [6][7][8][9] . Compared with the sequence structure, the tree-structured neural network is a better alternative for extracting text information [10][11][12] .…”

Child-Sum EATree-LSTMs: enhanced attentive Child-Sum Tree-LSTMs for biomedical event extraction

“…In recent years, Tree-LSTM has been widely applied to many fields of NLP, such as text generation, neural machine translation, sentiment analysis, sentence semantic modelling, and event extraction tasks [ 12 – 15 ]. Compared with an sequence structure, a tree-structured neural network is a better alternative for extracting text information [ 16 , 17 ]. In tree-structured neural network, words contribute unevenly to building a syntactic dependency tree, and it will bring noise and affect model training when all hidden states of the child nodes are transferred to the parent nodes.…”