Papers with code or without code? Impact of GitHub repository usability on the diffusion of machine learning research

Kang, Donghyun; Kang, TaeYoung; Jang, Junkyu

doi:10.1016/j.ipm.2023.103477

Cited by 6 publications

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Towards more sustainable and trustworthy reporting in machine learning

Fischer,

Liebig,

Morik

2024

Data Min Knowl Disc

View full text Add to dashboard Cite

With machine learning (ML) becoming a popular tool across all domains, practitioners are in dire need of comprehensive reporting on the state-of-the-art. Benchmarks and open databases provide helpful insights for many tasks, however suffer from several phenomena: Firstly, they overly focus on prediction quality, which is problematic considering the demand for more sustainability in ML. Depending on the use case at hand, interested users might also face tight resource constraints and thus should be allowed to interact with reporting frameworks, in order to prioritize certain reported characteristics. Furthermore, as some practitioners might not yet be well-skilled in ML, it is important to convey information on a more abstract, comprehensible level. Usability and extendability are key for moving with the state-of-the-art and in order to be trustworthy, frameworks should explicitly address reproducibility. In this work, we analyze established reporting systems under consideration of the aforementioned issues. Afterwards, we propose STREP, our novel framework that aims at overcoming these shortcomings and paves the way towards more sustainable and trustworthy reporting. We use STREP’s (publicly available) implementation to investigate various existing report databases. Our experimental results unveil the need for making reporting more resource-aware and demonstrate our framework’s capabilities of overcoming current reporting limitations. With our work, we want to initiate a paradigm shift in reporting and help with making ML advances more considerate of sustainability and trustworthiness.

show abstract

Towards more sustainable and trustworthy reporting in machine learning

Fischer,

Liebig,

Morik

2024

Data Min Knowl Disc

View full text Add to dashboard Cite

show abstract

Leveraging multiple control codes for aspect-controllable related paper recommendation

Long,

Li,

Tang

et al. 2025

Information Processing & Management

View full text Add to dashboard Cite

“Paper, Meet Code”: A Deep Learning Approach to Linking Scholarly Articles With GitHub Repositories

Puangjaktha,

Choetkiertikul,

Tuarob

2024

IEEE Access

View full text Add to dashboard Cite

Computer scientists often publish their source code accompanying their publications, prominently using code repositories across various domains. Despite the concurrent existence of scholarly articles and their associated official code repositories, explicit references linking the two are often missing. Traditionally, identifying whether scholarly content and code repositories pertain to the same research project requires manual inspection, a time-consuming task. This paper proposes a deep learning-based algorithm for automatically matching scholarly articles with their corresponding official code repositories. Our findings indicate that the most common linking information includes the paper title and BibTeX entries, typically found in the repository's readme document. In this study, we employed SPECTER for vector embedding of paper and repository metadata. Utilizing these embedding representations with the Light Gradient Boosting Machine (LGBM), our method achieved an F1 score of 0.94. Moreover, combining our best model with a rule-based approach improved performance by 5.31%. This study successfully delineates a connection between academic papers and associated official code repositories, minimizing reliance on explicit bibliographic information in repositories.

show abstract

Papers with code or without code? Impact of GitHub repository usability on the diffusion of machine learning research

Cited by 6 publications

References 61 publications

Towards more sustainable and trustworthy reporting in machine learning

Towards more sustainable and trustworthy reporting in machine learning

Leveraging multiple control codes for aspect-controllable related paper recommendation

“Paper, Meet Code”: A Deep Learning Approach to Linking Scholarly Articles With GitHub Repositories

Contact Info

Product

Resources

About