Code4Bench: A multidimensional benchmark of Codeforces data for different program analysis techniques

Majd, Amirabbas; Vahidi-Asl, Mojtaba; Khalilian, Alireza; Baraani-Dastjerdi, Ahmad; Zamani, Bahman

doi:10.1016/j.cola.2019.03.006

Cited by 24 publications

(16 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To answer RQ3, we use Code4Bench for C/C++ code [ 16 ], which contains code written by different developers for different problems. Even for every pair of questions and users, there are many versions and defective and clean versions.…”

Section: Empirical Study and Analysismentioning

confidence: 99%

See 1 more Smart Citation

Attention based GRU-LSTM for software defect prediction

et al. 2021

View full text Add to dashboard Cite

Software defect prediction (SDP) can be used to produce reliable, high-quality software. The current SDP is practiced on program granular components (such as file level, class level, or function level), which cannot accurately predict failures. To solve this problem, we propose a new framework called DP-AGL, which uses attention-based GRU-LSTM for statement-level defect prediction. By using clang to build an abstract syntax tree (AST), we define a set of 32 statement-level metrics. We label each statement, then make a three-dimensional vector and apply it as an automatic learning model, and then use a gated recurrent unit (GRU) with a long short-term memory (LSTM). In addition, the Attention mechanism is used to generate important features and improve accuracy. To verify our experiments, we selected 119,989 C/C++ programs in Code4Bench. The benchmark tests cover various programs and variant sets written by thousands of programmers. As an evaluation standard, compared with the state evaluation method, the recall, precision, accuracy and F1 measurement of our well-trained DP-AGL under normal conditions have increased by 1%, 4%, 5%, and 2% respectively.

show abstract

Section: Empirical Study and Analysismentioning

confidence: 99%

“…To verify our experiment, in Code4Bench, we selected 119,989 C/C++ programs [ 16 ]. The Code4Bench benchmark contains every version of every program we wrote to introduce metrics.…”

Section: Introductionmentioning

confidence: 99%

Attention based GRU-LSTM for software defect prediction

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Researchers typically need to test their approaches [183]. For a long time there was a lack of available microservice benchmarks [173] and references are often hard to find.…”

Section: Appendix B: List Of Microservice Benchmarksmentioning

confidence: 99%

On Code Analysis Opportunities and Challenges for Enterprise Systems and Microservices

et al. 2020

View full text Add to dashboard Cite

Code analysis brings excellent benefits to software development, maintenance, and quality assurance. Various tools can uncover code defects or even software bugs in a range of seconds. For many projects and developers, the code analysis tools became essential in their daily routines. However, how can code analysis help in an enterprise environment? Enterprise software solutions grow in scale and complexity. These solutions no longer involve only plain objects and basic language constructs but operate with various components and mechanisms simplifying the development of such systems. Enterprise software vendors have adopted various development and design standards; however, there is a gap between what constructs the enterprise frameworks use and what current code analysis tools recognize. This manuscript aims to challenge the mainstream research directions of code analysis and motivate for a transition towards code analysis of enterprise systems with interesting problems and opportunities. In particular, this manuscript addresses selected enterprise problems apparent for monolithic and distributed enterprise solutions. It also considers challenges related to the recent architectural push towards a microservice architecture. Along with open-source proof-of-concept prototypes to some of the challenges, this manuscript elaborates on code analysis directions and their categorization. Furthermore, it suggests one possible perspective of the problem area using aspect-oriented programming.

show abstract

“…Code4Bench [71] is another cross-language benchmark comprising C/C++, Java, Python, and Kotlin programs among others. Code4Bench also features a coarse-grained bug classification based on an automatic fault localization process for which faults were classified only in three groups, namely addition, modifications, and deletion.…”

Section: Threats To Validitymentioning

confidence: 99%

BUGSJS: a benchmark and taxonomy of JavaScript bugs

Gyimesi

Vancsics

Stocco

et al. 2020

Software Testing Verif & Rel

View full text Add to dashboard Cite

JavaScript is a popular programming language that is also error-prone due to its asynchronous, dynamic, and loosely typed nature. In recent years, numerous techniques have been proposed for analyzing and testing JavaScript applications. However, our survey of the literature in this area revealed that the proposed techniques are often evaluated on different datasets of programs and bugs. The lack of a commonly used benchmark limits the ability to perform fair and unbiased comparisons for assessing the efficacy of new techniques. To fill this gap, we propose BUGSJS, a benchmark of 453 real, manually validated JavaScript bugs from 10 popular JavaScript server-side programs, comprising 444k lines of code (LOC) in total. Each bug is accompanied by its bug report, the test cases that expose it, as well as the patch that fixes it. We extended BUGSJS with a rich web interface for visualizing and dissecting the bugs' information, as well as a programmable API to access the faulty and fixed versions of the programs and to execute the corresponding test cases, which facilitates conducting highly reproducible empirical studies and comparisons of JavaScript analysis and testing tools. Moreover, following a rigorous procedure, we performed a classification of the bugs according to their nature. Our internal validation shows that our taxonomy is adequate for characterizing the bugs in BUGSJS. We discuss several ways in which the resulting taxonomy and the benchmark can help direct researchers interested in automated testing of JavaScript applications.

show abstract

Code4Bench: A multidimensional benchmark of Codeforces data for different program analysis techniques

Cited by 24 publications

References 8 publications

Attention based GRU-LSTM for software defect prediction

Attention based GRU-LSTM for software defect prediction

On Code Analysis Opportunities and Challenges for Enterprise Systems and Microservices

BUGSJS: a benchmark and taxonomy of JavaScript bugs

Contact Info

Product

Resources

About