Deep learning for detecting inappropriate content in text

Abstract

Today, there are a large number of online discussion fora on the internet which are meant for users to express, discuss and exchange their views and opinions on various topics. For example, news portals, blogs, social media channels such as youtube. typically allow users to express their views through comments. In such fora, it has been often observed that user conversations sometimes quickly derail and become inappropriate such as hurling abuses, passing rude and discourteous comments on individuals or certain groups/communities. Similarly, some virtual agents or bots have also been found to respond back to users with inappropriate messages. As a result, inappropriate messages or comments are turning into an online menace slowly degrading the effectiveness of user experiences. Hence, automatic detection and filtering of such inappropriate language has become an important problem for improving the quality of conversations with users as well as virtual agents. In this paper, we propose a novel deep learning-based technique for automatically identifying such inappropriate language. We especially focus on solving this problem in two application scenarios-(a) Query completion suggestions in search engines and (b) Users conversations in messengers. Detecting inappropriate language is challenging due to various natural language phenomenon such as spelling mistakes and variations, polysemy, contextual ambiguity and semantic variations. For identifying inappropriate query suggestions, we propose a novel deep learning architecture called "Convolutional Bi-Directional LSTM (C-BiLSTM)" which combines the strengths of both Convolution Neural Networks (CNN) and Bi-directional LSTMs (BLSTM). For filtering inappropriate conversations, we use LSTM and Bi-directional LSTM (BLSTM) sequential models. The proposed models do not rely on hand-crafted features, are trained end-end as a single model, and effectively capture both local features as well as their global semantics. Evaluating C-BiLSTM, LSTM and BLSTM models on real-world search queries and conversations reveals that they significantly outperform both pattern-based and other hand-crafted feature-based baselines.

Deep learning for misinformation detection on online social networks: a survey and new perspectives

“…They claimed that their model can achieve an extremely high accuracy exceeding 0.96 AUC. Yenala et al (2018) identified that the automatic detection and filtering of inappropriate messages or comments have become an important problem for improving the quality of conversations with users as well as virtual agents. They proposed a novel hybrid DL model to automatically identify the inappropriate language.…”

“…However, the automated detection of misinformation is difficult to accomplish as it requires the advanced model to understand how related or unrelated the reported information is when compared to real information (Wu et al 2019). Also, to solve many complex MID problems, academia and industry researchers have applied DL to a large number of applications to make decisions (Xu et al 2019;Yenala et al 2018;Yin et al 2020). Therefore, this survey seeks to provide such a systematic review of current research on MID based on DL techniques.…”

“… 2019 ; Yenala et al. 2018 ; Yin et al 2020 ). Therefore, this survey seeks to provide such a systematic review of current research on MID based on DL techniques.…”

Novel Approach for Generating Hybrid Features Set to Effectively Identify Hate Speech

“…Then they have applied the multi-step Deep learning methods CNN+GRU(LSTM) followed by a dense layer with different embedding techniques like GloVe [11], word2vec [13] etc., for feature learning and classification. Yenala H et al [9] have worked on detecting inappropriate text in web search queries and chat data using Deep Learning Techniques. They have used Convolution-BiLSTM and BiLSTM respectively for their experiments.…”

“…In our experiments, we have introduced the concept called Hybrid Features, which are learned from deep learning models and are used with other semantic features [9][25] [28] like Parts of Speech (POS), Term Frequency -Inverse Document Frequency(TF-IDF), and tweet specific syntactic features for an effective feature representation. We have explored different kinds of neural networks for improving feature learning, starting with single architecture model like Convolution Neural Network (CNN), Long Short Term Memory (LSTM), Bidirectional LSTM (BiLSTM) and hybrid models like Convolution Neural Network + Long Short Term Memory (CNN+LSTM), Convolution Neural Network + Gated Recurrent Unit (CNN+GRU) and also focused on improving preprocessing steps to reduce the number of missing embeddings and to increase the vocabulary for efficient feature learning.…”