On the Soundness of Call Graph Construction in the Presence of Dynamic Language Features - A Benchmark and Tool Evaluation

Sui, Li; Dietrich, Jens; Emery, Michael Ray.; Rasheed, Shawn; Tahir, Amjed

doi:10.1007/978-3-030-02768-1_4

Cited by 16 publications

(8 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sui et al [37] present a similar study that investigates how the soundness of call graph construction algorithms is compromised by dynamic language features such as reflection and dynamic loading, proxies, serialization, the invokedynamic instruction, and the unsafe native API. They define a micro-benchmark consisting of a set of programs that exercise each of the dynamic features.…”

Section: Empirical Studiesmentioning

confidence: 99%

A Study of Call Graph Construction for JVM-Hosted Languages

Ali

Lai

Luo

et al. 2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Call graphs have many applications in software engineering, including bug-finding, security analysis, and code navigation in IDEs. However, the construction of call graphs requires significant investment in program analysis infrastructure. An increasing number of programming languages compile to the Java Virtual Machine (JVM), and program analysis frameworks such as WALA and SOOT support a broad range of program analysis algorithms by analyzing JVM bytecode. This approach has been shown to work well when applied to bytecode produced from Java code. In this paper, we show that it also works well for diverse other JVM-hosted languages: dynamically-typed functional Scheme, statically-typed object-oriented Scala, and polymorphic functional OCaml. Effectively, we get call graph construction for these languages for free, using existing analysis infrastructure for Java, with only minor challenges to soundness. This, in turn, suggests that bytecode-based analysis could serve as an implementation vehicle for bug-finding, security analysis, and IDE features for these languages. We present qualitative and quantitative analyses of the soundness and precision of call graphs constructed from JVM bytecodes for these languages, and also for Groovy, Clojure, Python, and Ruby. However, we also show that implementation details matter greatly. In particular, the JVM-hosted implementations of Groovy, Clojure, Python, and Ruby produce very unsound call graphs, due to the pervasive use of reflection, invokedynamic instructions, and run-time code generation. Interestingly, the dynamic translation schemes employed by these languages, which result in unsound static call graphs, tend to be correlated with poor performance at run time.

show abstract

Section: Empirical Studiesmentioning

confidence: 99%

A Study of Call Graph Construction for JVM-Hosted Languages

Ali

Lai

Luo

et al. 2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…Dietrich et al [5] proposed automated and manual techniques to generate unsoundness oracles to test static analysis. Sui et al [32] present the causes of unsoundness in static analysis frameworks (Soot, Wala, and Doop) due to the dynamic features of languages. Rief et al [21] did a comprehensive study, focussed on features in Java 9, for call-graph generation algorithms and expose the problems in the state-of-the-art esp.…”

Section: Related Workmentioning

confidence: 99%

Effects of Program Representation on Pointer Analyses — An Empirical Study

Prakash

Tiwari

Hammer

2021

Fundamental Approaches to Software Engineering

View full text Add to dashboard Cite

Static analysis frameworks, such as Soot and Wala, are used by researchers to prototype and compare program analyses. These frameworks vary on heap abstraction, modeling library classes, and underlying intermediate program representation (IR). Often, these variations pose a threat to the validity of the results as the implications of comparing the same analysis implementation in different frameworks are still unexplored. Earlier studies have focused on the precision, soundness, and recall of the algorithms implemented in these frameworks; however, little to no work has been done to evaluate the effects of program representation. In this work, we fill this gap and study the impact of program representation on pointer analysis. Unfortunately, existing metrics are insufficient for such a comparison due to their inability to isolate each aspect of the program representation. Therefore, we define two novel metrics that measure these analyses’ precision after isolating the influence of class-hierarchy and intermediate representation. Our results establish that the minor differences in the class hierarchy and IR do not impact program analysis significantly. Besides, they reveal the sources of unsoundness that aid researchers in developing program analysis.

show abstract

“…The authors then proceed to compare Java call graph generators, namely Soot [48], [49], WALA [42], DOOP [41] and OPAL [45]. Sui et al [53], also present a test suite of Java benchmarks, and they use it to evaluate and compare Soot [48], [49], WALA [42], and DOOP [41]. The above benchmark suites are very similar, leading to Judge consolidating them into one benchmark suite.…”

Section: Related Workmentioning

confidence: 99%

PyCG: Practical Call Graph Generation in Python

Salis¹,

Sotiropoulos²,

Λουρίδας³

et al. 2021

Preprint

View full text Add to dashboard Cite

Call graphs play an important role in different contexts, such as profiling and vulnerability propagation analysis. Generating call graphs in an efficient manner can be a challenging task when it comes to high-level languages that are modular and incorporate dynamic features and higher-order functions.Despite the language's popularity, there have been very few tools aiming to generate call graphs for Python programs. Worse, these tools suffer from several effectiveness issues that limit their practicality in realistic programs. We propose a pragmatic, static approach for call graph generation in Python. We compute all assignment relations between program identifiers of functions, variables, classes, and modules through an inter-procedural analysis. Based on these assignment relations, we produce the resulting call graph by resolving all calls to potentially invoked functions. Notably, the underlying analysis is designed to be efficient and scalable, handling several Python features, such as modules, generators, function closures, and multiple inheritance.We have evaluated our prototype implementation, which we call PyCG, using two benchmarks: a micro-benchmark suite containing small Python programs and a set of macrobenchmarks with several popular real-world Python packages.Our results indicate that PyCG can efficiently handle thousands of lines of code in less than a second (0.38 seconds for 1k LoC on average). Further, it outperforms the state-of-the-art for Python in both precision and recall: PyCG achieves high rates of precision ∼99.2%, and adequate recall ∼69.9%. Finally, we demonstrate how PyCG can aid dependency impact analysis by showcasing a potential enhancement to GitHub's "security advisory" notification service using a real-world example.

show abstract

On the Soundness of Call Graph Construction in the Presence of Dynamic Language Features - A Benchmark and Tool Evaluation

Cited by 16 publications

References 30 publications

A Study of Call Graph Construction for JVM-Hosted Languages

A Study of Call Graph Construction for JVM-Hosted Languages

Effects of Program Representation on Pointer Analyses — An Empirical Study

PyCG: Practical Call Graph Generation in Python

Contact Info

Product

Resources

About