Choosing the right word: Using bidirectional LSTM tagger for writing support systems

Abstract

Scientific writing is difficult. It is even harder for those for whom English is a second language (ESL learners). Scholars around the world spend a significant amount of time and resources proofreading their work before submitting it for review or publication.In this paper we present a novel machine learning based application for proper word choice task. Proper word choice is a generalization the lexical substitution (LS) and grammatical error correction (GEC) tasks. We demonstrate and evaluate the usefulness of applying bidirectional Long Short Term Memory (LSTM) tagger, for this task. While state-of-the-art grammatical error correction uses error-specific classifiers and machine translation methods, we demonstrate an unsupervised method that is based solely on a high quality text corpus and does not require manually annotated data. We use a bidirectional Recurrent Neural Network (RNN) with LSTM for learning the proper word choice based on a word's sentential context. We demonstrate and evaluate our application on both a domain-specific (scientific), writing task and a general-purpose writing task. We show that our domain-specific and general-purpose models outperform state-of-the-art general context learning. As an additional contribution of this research, we also share our code, pretrained models, and a new ESL learner test set with the research community.

Agent-Based Semantic Role Mining for Intelligent Access Control in Multi-Domain Collaborative Applications of Smart Cities

“…The recent successes of neural language machine translations used by [ 29 , 30 , 31 , 32 , 46 ] convinced us to use Word Embedding (Word2Vec) and LSTM for the solution of our problem of converting textual policies into structural knowledge. The automated population of organizational ontology and its matching is shown in Figure 2 .…”

“…In [ 30 ], LSTM was again used to create ontology for the physics domain by converting the text of a physics book. The research in [ 31 ] used bidirectional LSTM for proper word choice, based on its sentential context, in a domain-specific scientific writing task as well as a general-purpose writing task. In [ 32 ], stock market trend classification is conducted through text data by using LSTM, which automatically populates ontologies from text data of the stock market.…”

Deep Learning Model for Shield Tunneling Advance Rate Prediction in Mixed Ground Condition Considering Past Operations

“…Hence, the AR predictive model is supposed to have the ability to consider past data. According to the plentiful applications of LSTM in the area of the NLP [53], the LSTM based neural network is suitable for the AR prediction.. Figure 5 shows the typical architectures of the feedforward network and the recurrent neural network (RNN). The major difference between RNN and feedforward network is the feedback in the hidden layers of RNN [26].…”

Comparative Study of Data Clustering Algorithms and Analysis of The Keywords Extraction Efficiency: Learner Corpus Case

“…Corpora consisting of texts produced by non-native speakers present an invaluable source of linguistic data for researchers. Various studies have been conducted on the basis of learner corpora: automatic language scoring (Vajjala, 2018), identifying text complexity (Kurdi, 2020), automatic text classification within different, proper word choice task (Makarenkov et al, 2019), semantic collocation correction (Dahlmeier and Ng, 2011), lexical substitution (McCarthy and Navigli, 2009), paraphrase generation (Madnani and Dorr, 2010), grammatical error correction (Ng et al, 2014), sentence completion (Zweig and Burges, 2011), to name just a few. Also, there are many papers devoted to obtaining document embeddings ((Salton and Buckley, 1988), (Whissell and Clarke, 2011), (Mikolov and Le, 2014)), clustering algorithms ( (Steinhaus, 1956), (Ester at al., 1996), (Merris, 1994)), and various techniques for keywords extraction ( (Mihalcea and Tarau, 2004), (Rose et al, 2010), (Sterckx et al, 2016)).…”