Model-based testing of breaking changes in Node.js libraries

Møller, Anders; Torp, Martin Toldam

doi:10.1145/3338906.3338940

Cited by 23 publications

(14 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They compute operational abstractions based on input/output behaviour to test whether a new component can replace an old one. M0ller and Torp [12] proposed a model-based testing approach to identify type-regression problems that result in breaking changes in JavaScript libraries. These works deal with improvements and repairs applied to a package repository or library, and thus have a different focus compared to SHIPWRIGHT, which is on repairing broken Dockerfiles.…”

Section: Empirical Studies On Docker (And Devops)mentioning

confidence: 99%

Shipwright: A Human-in-the-Loop System for Dockerfile Repair

Henkel

Silva

Teixeira

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Docker is a tool for lightweight OS-level virtualization. Docker images are created by performing a build, controlled by a source-level artifact called a Dockerfile. We studied Dockerfiles on GitHub, and-to our great surprisefound that over a quarter of the examined Dockerfiles failed to build (and thus to produce images). To address this problem, we propose Sh i p w r i g h t , a human-in-the-loop system for finding repairs to broken Dockerfiles. Sh i p w r i g h t uses a modified version of the BERT language model to embed build logs and to cluster broken Dockerfiles. Using these clusters and a searchbased procedure, we were able to design 13 rules for making automated repairs to Dockerfiles. With the aid of Sh i p w r i g h t , we submitted 45 pull requests (with a 42.2% acceptance rate) to GitHub projects with broken Dockerfiles. Furthermore, in a "time-travel" analysis of broken Dockerfiles that were later fixed, we found that S h i p w r i g h t proposed repairs that were equivalent to human-authored patches in 22.77% of the cases we studied. Finally, we compared our work with recent, state-of-theart, static Dockerfile analyses, and found that, while static tools detected possible build-failure-inducing issues in 20.6-33.8% of the files we examined, Sh i p w r i g h t was able to detect possible issues in 73.25% of the files and, additionally, provide automated repairs for 18.9% of the files.

show abstract

Section: Empirical Studies On Docker (And Devops)mentioning

confidence: 99%

Shipwright: A Human-in-the-Loop System for Dockerfile Repair

Henkel

Silva

Teixeira

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

show abstract

“…Updating library dependencies in projects. To assist developers with updating dependencies in projects, researchers have studied practices around updating dependencies [3,16,2,60,26,27] and proposed tools leveraging both static-and dynamic analysis [6,4,61]. Kula et al [3] empirical study of 2, 700 library dependencies in 4, 600 Java project found that 81.5% remain outdated, even with security problems.…”

Section: Related Workmentioning

confidence: 99%

Can We Trust Tests To Automate Dependency Updates? A Case Study of Java Projects

Hejderup,

Gousios

2021

Preprint

View full text Add to dashboard Cite

Developers are increasingly using services such as Dependabot to automate dependency updates. However, recent research has shown that developers perceive such services as unreliable, as they heavily rely on test coverage to detect conflicts in updates. To understand the prevalence of tests exercising dependencies, we calculate the test coverage of direct and indirect uses of dependencies in 521 well-tested Java projects. We find that tests only cover 58% of direct and 20% of transitive dependency calls. By creating 1,122,420 artificial updates with simple faults covering all dependency usages in 262 projects, we measure the effectiveness of test suites in detecting semantic faults in dependencies; we find that tests can only detect 47% of direct and 35% of indirect artificial faults on average. To increase reliability, we investigate the use of change impact analysis as a means of reducing false negatives; on average, our tool can uncover 74% of injected faults in direct dependencies and 64% for transitive dependencies, nearly two times more than test suites. We then apply our tool in 22 real-world dependency updates, where it identifies three semantically conflicting cases and five cases of unused dependencies. Our findings indicate that the combination of static and dynamic analysis should be a requirement for future dependency updating systems.

show abstract

“…Inspired by previous work to detect breaking changes in npm package updates [29,31], we propose using an access path mechanism for specifying contact points. An access path, or short ap, can be described as an S-expression, which is read from the innermost expression outwards.…”

Section: Specifying Contact Pointsmentioning

confidence: 99%

“…Our hypothesis is that these inputs are representative for most of the library usages in the wild. Related work employs similar assumptions [29,31].…”

Section: Limitationsmentioning

confidence: 99%

See 1 more Smart Citation

Extracting taint specifications for JavaScript libraries

Staicu

Torp

Schäfer³

et al. 2020

Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering

Self Cite

View full text Add to dashboard Cite

Modern JavaScript applications extensively depend on third-party libraries. Especially for the Node.js platform, vulnerabilities can have severe consequences to the security of applications, resulting in, e.g., cross-site scripting and command injection attacks. Existing static analysis tools that have been developed to automatically detect such issues are either too coarse-grained, looking only at package dependency structure while ignoring dataflow, or rely on manually written taint specifications for the most popular libraries to ensure analysis scalability.In this work, we propose a technique for automatically extracting taint specifications for JavaScript libraries, based on a dynamic analysis that leverages the existing test suites of the libraries and their available clients in the npm repository. Due to the dynamic nature of JavaScript, mapping observations from dynamic analysis to taint specifications that fit into a static analysis is non-trivial. Our main insight is that this challenge can be addressed by a combination of an access path mechanism that identifies entry and exit points, and the use of membranes around the libraries of interest.We show that our approach is effective at inferring useful taint specifications at scale. Our prototype tool automatically extracts 146 additional taint sinks and 7 840 propagation summaries spanning 1 393 npm modules. By integrating the extracted specifications into a commercial, state-of-the-art static analysis, 136 new alerts are produced, many of which correspond to likely security vulnerabilities. Moreover, many important specifications that were originally manually written are among the ones that our tool can now extract automatically. CCS CONCEPTS• Software and its engineering → Software notations and tools.

show abstract

Model-based testing of breaking changes in Node.js libraries

Cited by 23 publications

References 22 publications

Shipwright: A Human-in-the-Loop System for Dockerfile Repair

Shipwright: A Human-in-the-Loop System for Dockerfile Repair

Can We Trust Tests To Automate Dependency Updates? A Case Study of Java Projects

Extracting taint specifications for JavaScript libraries

Contact Info

Product

Resources

About