Diff/TS: A Tool for Fine-Grained Structural Change Analysis

Abstract

This paper reports on a tool for fine-grained analysis of structural changes made between revisions of programs. The tool, called Diff/TS, calculates, visualizes and classifies edit operations including "moves" that will change one revision into another by means of detailed tree structural analysis on source code. Such analysis tends to be time consuming and inflexible. We have extended a general tree comparison algorithm with heuristics driven control configurable for multiple programming languages and have achieved both processing speed and analysis precision needed for investigating large-scale software projects. The tool is capable of processing Python, Java, C and C++ projects. We present several applications including software "archaeology" on a widely known open source software project and automated "phylogenetic" malware classification based on control flows. These examples suggest that tree differencing is useful for measuring distance or dissimilarity between tree structured artifacts, and offer good precision tests of the method.

Type-directed diffing of structured data

“…The canonical solution for the untyped linear scenario is the well known Unix diff (Hunt and McIlroy 1976). For the tree-structured variation, though, a variety of implementations (Falleri et al 2014;Farinier et al 2015;Hashimoto and Mori 2008) has arisen in the last few years. In this paper, however, we have explored how to exploit the type structure of trees to give a more precise account of our di algorithm.…”

Towards Automatically Estimating Porting Effort between Web Service APIs

“…Baxter et al introduced the notion of syntactic similarity (based on the Dice coefficient), as opposed to exact matches, as a more practical means of finding program segments with similar functionality and behavior. Other researchers employ similar methods to detect source code plagiarism [25], to analyze differences between programs written in different languages [26], and to analyze how program code evolves over time [27].…”

A Method for Analyzing Code Homology in Genealogy of Evolving Software

“…In this study, we used a tree differencing system called Diff/TS [9]. Diff/TS extends a general tree comparison algorithm with heuristics driven control configurable for multiple programming languages.…”

“…However, without any development history or any explicit record of tagging or version copying operations, it may be difficult to determine the origin of branching, notably the version from which a development branch is duplicated. In such situations, we can reconstruct the genealogies by utilizing tree differencing and tools for phylogeny [9].…”

“…The method for tracking cloned code is relatively less explored with several exceptions [5,3,[6][7][8] despite its practical importance. In this paper, we propose an automated method for detecting and tracking homologues in genealogy of evolving software using a fine-grained tree differencing tool called Diff/TS [9] for source code. The method also reconstructs semantic change histories from raw edit sequences computed by differencing on abstract syntax trees and identifies inconsistent changes by comparing change histories of homologues.…”