A System for Affect Analysis of Utterances in Japanese Supported with Web Mining

Abstract

We propose a method for affect analysis of textual input in Japanese supported with Web mining. The method is based on a pragmatic reasoning that emotional states of a speaker are conveyed by emotional expressions used in emotive utterances. It means that if an emotive expression is used in a sentence in a context described as emotive, the emotion conveyed in the text is revealed by the used emotive expression. The system ML-Ask (Emotive Elements / Expressions Analysis System) is constructed on the basis of this idea. An evaluation of the system is performed in which two evaluation methods are compared. To choose the most objective evaluation method we compare the most popular method in the field and a method proposed by us. The proposed evaluation method was shown to be more objective and revealed the strong and weak points of the system in detail. In the evaluation experiment ML-Ask reached human level in recognizing the general emotiveness of an utterance (0.83 balanced F-score) and 63% of human level in recognizing the specific types of emotions. We support the system with a Web mining technique to improve the performance of emotional state types extraction. In the Web mining technique emotive associations are extracted from the Web using co-occurrences of emotive expressions with morphemes of causality. The Web mining technique improved the performance of the emotional states types extraction to 85% of human performance

Towards Computational Fronesis

“…Ptaszynski et al (2009a) already showed that ML-Ask and Shi's technique are compatible and can be used as complementary means to improve the emotion recognition task. However, these two methods are based on different assumptions.…”

“…On the other hand, Tokuhisa et al (2008) and Shi et al (2008) used a large number of examples gathered from the Web to estimate user emotions. Furthermore, Ptaszynski et al (2009a) proposed a Web-based supported affect analysis system for Japanese text-based utterances.…”

“…It is a collection of over two thousand expressions describing emotional states collected manually from a wide range of literature. It is not a tool per se, but was converted into an emotive expression database by Ptaszynski et al (2009a; in their research on affect analysis of utterances in Japanese. In our task we needed to choose a lexicon proved to be the most appropriate for the Japanese language.…”

ML-Ask: Open Source Affect Analysis Software for Textual Input in Japanese

“…Evaluations ML-Ask has been evaluated a number of times on different datasets and frameworks. In first evaluations, Ptaszynski et al [12,20,21] focused on evaluating the system on separate sentences. For example, in [20], there were 90 sentences (45 emotive and 45 non-emotive) annotated by authors of the sentences (first-person standpoint annotations).…”

“…On this dataset ML-Ask achieved 83% of balanced F-score for determining whether a sentence is emotive, 63% of human level of unanimity score for determining emotive value and 45% of balanced F-score for detecting particular emotion types. In [12] Ptaszynski et al added annotations of third-party annotators and performed additional evaluation from the third-person standpoint. The evaluation showed that ML-Ask achieves better performance when supported by additional Web-mining procedure (not included in the OpenSource version) for extracting emotive associations from the Internet.…”