Deep Learning for User Comment Moderation

Abstract

Experimenting with a new dataset of 1.6M user comments from a Greek news portal and existing datasets of English Wikipedia comments, we show that an RNN outperforms the previous state of the art in moderation. A deep, classification-specific attention mechanism improves further the overall performance of the RNN. We also compare against a CNN and a word-list baseline, considering both fully automatic and semi-automatic moderation.

Proceedings of the 2017 EMNLP Workshop: Natural Language Processing meets Journalism

“…RNN: This is the RNN-based method of our previous work (Pavlopoulos et al, 2017a). It is a chain of GRU cells (Cho et al, 2014) that transforms the tokens w 1 .…”

“…Each user type t is mapped to a user type embedding v t ∈ R d . The 6 In our previous work (Pavlopoulos et al, 2017a), we also considered a variant of RNN, called a-RNN, with an attention mechanism. We do not consider a-RNN here to save space.…”

“…User and user type embeddings are randomly initialized and updated by backpropagation. Word embeddings are initialized to the WORD2VEC embeddings of our previous work (Pavlopoulos et al, 2017a), which were pretrained on 5.2M Gazzetta comments. Out of vocabulary words, meaning words not encountered or encountered only once in the training set and/or words with no initial embeddings, are mapped (during both training and testing) to a single randomly initialized word embedding, updated by backpropagation.…”

“…The dataset contains Greek comments from Gazzetta (Pavlopoulos et al, 2017a). There are approximately 1.45M training comments (covering Jan. 1, 2015 to Oct. 6, 2016); we call them G-TRAIN ( Table 1).…”

“…Smaller news portals may be unable to employ moderators, and some are forced to shut down their comments. 2 In previous work (Pavlopoulos et al, 2017a), we introduced a new dataset of approx. 1.6M manually moderated user comments from a Greek sports news portal, called Gazzetta, which we made publicly available.…”