Ariadne: analysis for machine learning programs

Dolby, Julian; Shinnar, Avraham; Allain, Allison; Reinen, Jenna

doi:10.1145/3211346.3211349

Cited by 35 publications

(22 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, 93.88% projects in our corpus have not adopted type annotations, while Jedi only requires all the external dependencies and the entire source code to infer the binding information. Jedi is a popular (4,000 GitHub stars), widely adopted (47,300 users) tool and used in previous studies [41,44]. Therefore we use Jedi for inferring type binding information.…”

Section: Analysing the Api Usagementioning

confidence: 99%

Understanding Software-2.0

Dilhara

Ketkar

Dig

2021

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Enabled by a rich ecosystem of Machine Learning (ML) libraries, programming using learned models , i.e., Software-2.0 , has gained substantial adoption. However, we do not know what challenges developers encounter when they use ML libraries. With this knowledge gap, researchers miss opportunities to contribute to new research directions, tool builders do not invest resources where automation is most needed, library designers cannot make informed decisions when releasing ML library versions, and developers fail to use common practices when using ML libraries. We present the first large-scale quantitative and qualitative empirical study to shed light on how developers in Software-2.0 use ML libraries, and how this evolution affects their code. Particularly, using static analysis we perform a longitudinal study of 3,340 top-rated open-source projects with 46,110 contributors. To further understand the challenges of ML library evolution, we survey 109 developers who introduce and evolve ML libraries. Using this rich dataset we reveal several novel findings. Among others, we found an increasing trend of using ML libraries: The ratio of new Python projects that use ML libraries increased from 2% in 2013 to 50% in 2018. We identify several usage patterns including the following: (i) 36% of the projects use multiple ML libraries to implement various stages of the ML workflows, (ii) developers update ML libraries more often than the traditional libraries , (iii) strict upgrades are the most popular for ML libraries among other update kinds, (iv) ML library updates often result in cascading library updates, and (v) ML libraries are often downgraded (22.04% of cases). We also observed unique challenges when evolving and maintaining Software-2.0 such as (i) binary incompatibility of trained ML models and (ii) benchmarking ML models. Finally, we present actionable implications of our findings for researchers, tool builders, developers, educators, library vendors, and hardware vendors.

show abstract

Section: Analysing the Api Usagementioning

confidence: 99%

Understanding Software-2.0

Dilhara

Ketkar

Dig

2021

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

“…Dolby et al [220] extended WALA to support static analysis of the behaviour of tensors in Tensorflow learning programs written in Python. They defined and tracked tensor types for machine learning, and changed WALA to produce a dataflow graph to abstract possible program behavours.…”

Section: Bug Detection In Learning Programmentioning

confidence: 99%

“…Similar to mltest, there is a testing framework for writing unit tests for pytorch-based ML systems, named torchtest 14 . Dolby et al [220] extended WALA to enable static analysis for machine learning code using TensorFlow.…”

Section: Open-source Tool Support In ML Testingmentioning

confidence: 99%

Machine Learning Testing: Survey, Landscapes and Horizons

Zhang

Harman

Ма

et al. 2022

IIEEE Trans. Software Eng.

501

318

View full text Add to dashboard Cite

This paper provides a comprehensive survey of Machine Learning Testing (ML testing) research. It covers 128 papers on testing properties (e.g., correctness, robustness, and fairness), testing components (e.g., the data, learning program, and framework), testing workflow (e.g., test generation and test evaluation), and application scenarios (e.g., autonomous driving, machine translation). The paper also analyses trends concerning datasets, research trends, and research focus, concluding with research challenges and promising research directions in ML testing. Index Terms-machine learning, software testing, deep neural network, ! • Jie M. Zhang and Mark Harman are with CREST, University College London, United Kingdom. Mark Harman is also with Facebook London.

show abstract

“…Various tools to analyse the programs have been discussed in [14] for detecting the vulnerabilities. In fact, there are static tools as well as dynamic tools to serve this purpose such as Python Taint and WALA.…”

Section: Calculate the Line By Line Valuesmentioning

confidence: 99%

“…The authors [12,13,14] used the models for the bytecode information which may help researchers to retrieve back-end code with visualization. That means clear understanding of bytecode in a visual forms, which seems to be very promising work in the dynamic analysis of bytecode.…”

Section: Comparative Analysismentioning

confidence: 99%

A Framework for Debugging Java Programs in a Bytecode

Soomro

Belgaum

Alansari

et al. 2018

2018 International Conference on Computing, Electronics &Amp; Communications Engineering (iCCECE)

View full text Add to dashboard Cite

In the domain of Software Engineering, program analysis and understanding has been considered to be a very challenging task since decade, as it demands dedicated time and efforts. The analysis of source code may occasionally be comparatively easier due to its static nature, however, the back-end code (Bytecode), especially in terms of Java programming, is complicated to be analysed. In this paper, we present a methodological approach towards understanding the Bytecode of Java programs. We put forward a framework for the debugging process of Java Bytecode. Furthermore, we discuss the debugging process of Bytecode understanding from simple to multiple statements with regards to data flow analysis. Finally, we present a comparative analysis of Bytecode along with the simulation of the proposed framework for the debugging process.

show abstract

Ariadne: analysis for machine learning programs

Cited by 35 publications

References 18 publications

Understanding Software-2.0

Understanding Software-2.0

Machine Learning Testing: Survey, Landscapes and Horizons

A Framework for Debugging Java Programs in a Bytecode

Contact Info

Product

Resources

About