N-Gram Representations For Comment Filtering

Abstract

Accurate classifiers for short texts are valuable assets in many applications. Especially in online communities, where users contribute to content in the form of posts and comments, an effective way of automatically categorising posts proves highly valuable. This paper investigates the use of Ngrams as features for short text classification, and compares it to manual feature design techniques that have been popular in this domain. We find that the N-gram representations greatly outperform manual feature extraction techniques.