Automating Intention Mining

Abstract

Developers frequently discuss aspects of the systems they are developing online. The comments they post to discussions form a rich information source about the system. Intention mining, a process introduced by Di Sorbo et al., classifies sentences in developer discussions to enable further analysis. As one example of use, intention mining has been used to help build various recommenders for software developers. The technique introduced by Di Sorbo et al. to categorize sentences is based on linguistic patterns derived from two projects. The limited number of data sources used in this earlier work introduces questions about the comprehensiveness of intention categories and whether the linguistic patterns used to identify the categories are generalizable to developer discussion recorded in other kinds of software artifacts (e.g., issue reports). To assess the comprehensiveness of the previously identified intention categories and the generalizability of the linguistic patterns for category identification, we manually created a new dataset, categorizing 5,408 sentences from issue reports of four projects in GitHub. Based on this manual effort, we refined the previous categories. We assess Di Sorbo et al.'s patterns on this dataset, finding that the accuracy rate achieved is low (0.31). To address the deficiencies of Di Sorbo et al.'s patterns, we propose and investigate a convolution neural network (CNN)-based approach to automatically classify sentences into different categories of intentions. Our approach optimizes CNN by integrating batch normalization to accelerate the training speed, and an automatic hyperparameter tuning approach to tune appropriate hyperparameters of CNN. Our approach achieves an accuracy of 0.84 on the new dataset, improving Di Sorbo et al.'s approach by 171%. We also apply our approach to improve an automated software engineering task, in which we use our proposed approach to rectify misclassified issue reports, thus reducing the bias introduced by such data to other studies. A case study on four open source projects with 2,076 issue reports shows that our approach achieves an average AUC score of 0.687, which improves other baselines by at least 16%.

Aspect-Level Sentiment Analysis Based on Bidirectional-GRU in SIoT

“…ℎ is the maximum length of the sentence, and v is the maximum length of the aspect. Each word of a sentence and [40]. With our prior experience with training deep learning models, we used the manual method.…”

Social media intention mining for sustainable information systems: categories, taxonomy, datasets and challenges

“…This study distinguished to related studies in such a context that other SLRs focused on detecting users' goals using process mining. The studies [2][3][4][5] classified the intention into categories, but there is no significant contribution to present techniques and approaches used to infer intentions as well as there is no comprehensive literature available on the classification of datasets used to extract intentions of the user. This manuscript's primary focus is to discover and define the intention mining categories, taxonomy of approaches and highlight research challenges and gaps in intension mining domain.…”

“…As an example, few expressions are categorized in different intentions as described in Table 1. When a user puts any query in the search engine to retrieve related content, it depicts his search query intention [4]. A comment or query expressing the need or wish to buy a product is called purchase intention [5,6].…”

Learning to extract transaction function from requirements: an industrial case on financial software

“…Naive Bayes (NB), Logistic Regression (LR), J48, and Random Forest (RF). These four classifiers are also widely used in the previous studies [20,21,23]. We use the TF-IDF method to represent each word in functions, in which the frequencies of words are multiplied by their inverse document-frequency.…”