Replaying IDE interactions to evaluate and improve change prediction approaches

Abstract

Abstract-Change prediction helps developers by recommending program entities that will have to be changed alongside the entities currently being changed. To evaluate their accuracy, current change prediction approaches use data from versioning systems such as CVS or SVN. These data sources provide a coarse-grained view of the development history that flattens the sequence of changes in a single commit. They are thus not a valid basis for evaluation in the case of developmentstyle prediction, where the order of the predictions has to match the order of the changes a developer makes.We propose a benchmark for the evaluation of change prediction approaches based on fine-grained change data recorded from IDE usage. Moreover, the change prediction approaches themselves can use the more accurate data to fine-tune their prediction. We present an evaluation procedure and use it on several change prediction approaches, both novel and from the literature, and report on the results.

Slicing Fine-Grained Code Change History

“…Our goal is to provide a mechanism and its tool that makes replaying past code changes more efficient. This agrees with the goal of several studies [8], [9], [13], [14], [38], [39], including the work mentioned in Sect. 2.1, all of which focus on replaying fine-grained code changes and help programmers understand the code and its evolution.…”

“…These changes consist of the finest-level atomic change operations on the abstract syntax tree (AST) of the program and the higher-level composite change operations that abstract the atomic change operations. They also presented a benchmarking procedure to evaluate the change predictions based on the replay of the code changes actually recorded from IDE interactions [13].…”

“…For example, SpyWare [8], Syde [9], Fluorite [10], CodingTracker [11], and OperationRecorder [12] are embedded into modern integrated development environments (IDEs) and capture all of the fine-grained code changes performed on the editors provided by their respective IDEs. In addition, some of these recording tools collaborate with tools that visualize, filter, and/or replay recorded code changes [13]- [16].…”

A Survey on Mining Software Repositories

“…In conclusion, a simple model, such as VSM or Unigram shows better performance than complex models like LDA, LSA, and CBDM. To support Change-related activity Many studies have been conducted on change prediction area [69]- [71]. Canfora and Cerulo defined impact analysis techniques based on information retrieval, which notify the set of changeable files using the textual descriptions about newly introduced bugs in the bug repository [69].…”

“…However, execution time changed from second level to hour level. Robbes et al presented a benchmark that is able to evaluate the change production technique with fine-grained change data recorded in IDE, and showed the procedure to estimate existing prediction techniques [71].…”