Deviation detection in text using conceptual graph interchange format and error tolerance dissimilarity function

Abstract

The rapid increase in the amount of textual data has brought forward a growing research interest towards mining text to detect deviations. Specialized methods for specific domains have emerged to satisfy various needs in discovering rare patterns in text. This paper focuses on a graph-based approach for text representation and presents a novel error tolerance dissimilarity algorithm for deviation detection. We resolve two non-trivial problems, i.e. semantic representation of text and the complexity of graph matching. We employ conceptual graphs interchange format (CGIF)-a knowledge representation formalism to capture the structure and semantics of sentences. We propose a novel error tolerance dissimilarity algorithm to detect deviations in the CGIFs. We evaluate our method in the context of analyzing real world financial statements for identifying deviating performance indicators. We show that our method performs better when compared with two related text based graph similarity measuring methods. Our proposed method has managed to identify deviating sentences and it strongly correlates with expert judgments. Furthermore, it offers error tolerance matching of CGIFs and retains a linear complexity with the increasing number of CGIFs.

Towards an outlier detection model in text data stream

“…Outlier detection in the field of data stream is considered one of the most difficult issues [9]. Therefore, in the machine learning community, it has drawn the attention of many scientists [10].…”

Algorithms Comparison for Non-Requirements Classification using the Semantic Feature of Software Requirement Statements

“…Several studies focus on adapting unsupervised machine learning to detect noise [9][10][11][12][13] . Another study focused on detecting irregularities in documents using conceptual charts [14] . The study provides a graphic visualization of the deviations that occur in text data.…”