On the impact of using trivial packages: an empirical case study on npm and PyPI

Oda, Vinicius; Mujahid, Suhaib; Shihab, Emad

doi:10.1007/s10664-019-09792-9

Cited by 34 publications

(24 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Abdalkareem et al [17] recently found that around 10% to 16% of the packages within the prevalent package managers are trivial. Trivial packages are largely preferred by software engineers, and their utility is observed in large organizations, including Facebook and Netflix.…”

Section: Discussionmentioning

confidence: 99%

PyTorrent: A Python Library Corpus for Large-scale Language Models

Bahrami,

Liu,

Mizobuchi

et al. 2021

View full text Add to dashboard Cite

A large scale collection of both semantic and natural language resources is essential to leverage active Software Engineering research areas such as code reuse and code comprehensibility. Existing machine learning models ingest data from Open Source repositories (like GitHub projects) and forum discussions (like Stackoverflow.com), whereas, in this showcase, we took a step backward to orchestrate a corpus titled PyTorrent that contains 218,814 Python package libraries from PyPI and Anaconda environment. This is because earlier studies have shown that much of the code is redundant and Python packages from these environments are better in quality and are well-documented. PyTorrent enables users (such as data scientists, students, etc.) to build off the shelf machine learning models directly without spending months of effort on large infrastructure. The dataset, schema and a pretrained language model is available at: https://github.com/fla-sil/PyTorrent Preprint. Under review.

show abstract

Section: Discussionmentioning

confidence: 99%

PyTorrent: A Python Library Corpus for Large-scale Language Models

Bahrami,

Liu,

Mizobuchi

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Trivial packages are packages with low cyclomatic complexity and few LOC and should be considered with care due to lack of tests and high dependency count (ABDALKA-REEM et al, 2017). They later extended this study and included the PyPI ecosystem with similar results (ABDALKAREEM et al, 2020).…”

Section: Project Tacticsmentioning

confidence: 91%

Module Integration Using Graph Grammars (MIGRATE)

Cravo

Ribeiro

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Software, be it desktop, mobile or web, is becoming more and more connected. Software development is also becoming more connected with ecosystems comprised of networks of millions of packages. Engineering software today involves writing code that weaves together libraries, services and applications. Such processes are under constant changes due to both internal requests (e.g. new features) or external demands (e.g. dependency updates). Avoiding integration bugs in this scenario can be quite a challenge regardless of common strategies such as testing and versioning. We studied graph grammars to find a set of grammars (verification grammars) that represent how software modules integrate and leveraged existing graph grammar analysis, specifically critical pair analysis, to point out possible integration problems in such grammars automatically. Furthermore, we created a formalism (module nets) to represent how software modules share information and leveraged graph grammars, by the fact that they can be proven to have functional behavior (confluence), to translate instances of module nets to verification grammars, enabling developers to create and modify module nets and then have warnings concerning integration problems automatically generated. We summarized this process in a framework we call module integration using graph grammars (MIGRATE), which we illustrate in this work through a case study with a fictitious search engine for research articles. Our approach demonstrates how to leverage critical pair analysis of graph grammars to automatically uncover a few integration bugs. It also serves as a pathway for future research exercising other graph grammar analyses to full extent.

show abstract

“…Nevertheless, in future work, we need to explore the external validity of our results to other SE tasks (other than defect prediction) and for other kinds of data. For example, Abdalkareem et al [6 ] show that up to 16% of Python and JavaScript packages are "trivially small" (their terminology); i.e., have less than 250 lines of code. It is an open issue if our methods work for other kinds of software such as those trivial Javascript and Python packages.…”

Section: (A Claim He Supports With Numerous Examples From Vision Research)mentioning

confidence: 99%

Early Life Cycle Software Defect Prediction. Why? How?

Shrikanth

Majumder

2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Many researchers assume that, for software analytics, "more data is better." We write to show that, at least for learning defect predictors, this may not be true.To demonstrate this, we analyzed hundreds of popular GitHub projects. These projects ran for 84 months and contained 3,728 commits (median values). Across these projects, most of the defects occur very early in their life cycle. Hence, defect predictors learned from the first 150 commits and four months perform just as well as anything else. This means that, at least for the projects studied here, after the first few months, we need not continually update our defect prediction models.We hope these results inspire other researchers to adopt a "simplicity-first" approach to their work. Some domains require a complex and data-hungry analysis. But before assuming complexity, it is prudent to check the raw data looking for "short cuts" that can simplify the analysis.

show abstract

On the impact of using trivial packages: an empirical case study on npm and PyPI

Cited by 34 publications

References 50 publications

PyTorrent: A Python Library Corpus for Large-scale Language Models

PyTorrent: A Python Library Corpus for Large-scale Language Models

Module Integration Using Graph Grammars (MIGRATE)

Early Life Cycle Software Defect Prediction. Why? How?

Contact Info

Product

Resources

About