Open Problems in Fuzzing RESTful APIs: A Comparison of Tools

Zhang, Man; Arcuri, Andrea

doi:10.48550/arxiv.2205.05325

Cited by 2 publications

(3 citation statements)

References 21 publications

(42 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With recent studies [32,52] conducted for studying existing fuzzers (including the fuzzers described above and EvoMaster), white-box EvoMaster achieved the best performance on open-source APIs. In addition, to the best of our knowledge, there does not exist in the literature any industrial evaluation for white-box system testing of microservices.…”

Section: Related Workmentioning

confidence: 99%

“…We are authors of this fuzzer. In the context of REST API testing, EvoMaster has been studied by different groups [33,53] to compare it with various existing open-source fuzzers on open-source REST APIs. Results of these studies demonstrate that EvoMaster can be considered as the most performant tool in fuzzing these selected open-source APIs.…”

Section: Common Challengesmentioning

confidence: 99%

“…Currently, the performance of EvoMaster has been mainly assessed on open-source APIs (such as APIs collected in the GitHub repository EMB [2]). With recent studies on REST API fuzzers [32,52], EvoMaster has been evaluated as the most performing tool on such open-source APIs. But, most of these open-source APIs are web services that work in isolation, and not part of a microservice architecture [2].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fuzzing Microservices In Industry: Experience of Applying EvoMaster at Meituan

Zhang¹,

Arcuri²,

Li³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

With several microservice architectures comprising of thousands of web services in total, used to serve 630 million customers, companies like Meituan face several challenges in the verification and validation of their software. The use of automated techniques, especially advanced AI-based ones, could bring significant benefits here. EvoMaster is an open-source test case generation tool for web services, that exploits the latest advances in the field of Search-Based Software Testing research. This paper reports on our experience of integrating the EvoMaster tool in the testing processes at Meituan. Experiments were carried out to evaluate its performance in detail on two industrial web services which are parts of a large e-commerce microservices. A questionnaire and interviews were carried out with the engineers and managers at Meituan, to better understand the applicability and usability of fuzzing tools like EvoMaster in real industrial settings. On the one hand, the results of these analyses clearly show that existing tools like EvoMaster are already of benefits for practitioners in industry, e.g., EvoMaster detected 21 real faults and achieved an average of 71.3% coverage for code defining endpoints and 51.7% coverage for code implementing business logic, on two industrial APIs. On the other hand, there are still many critical challenges that the research community has to investigate.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Common Challengesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fuzzing Microservices In Industry: Experience of Applying EvoMaster at Meituan

Zhang¹,

Arcuri²,

Li³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Exploring behaviours of RESTful APIs in an industrial setting

Karlsson,

Jongeling,

Čaušević

et al. 2024

Software Qual J

View full text Add to dashboard Cite

A common way of exposing functionality in contemporary systems is by providing a Web-API based on the REST API architectural guidelines. To describe REST APIs, the industry standard is currently OpenAPI-specifications. Test generation and fuzzing methods targeting OpenAPI-described REST APIs have been a very active research area in recent years. An open research challenge is to aid users in better understanding their API, in addition to finding faults and to cover all the code. In this paper, we address this challenge by proposing a set of behavioural properties, common to REST APIs, which are used to generate examples of behaviours that these APIs exhibit. These examples can be used both (i) to further the understanding of the API and (ii) as a source of automatic test cases. Our evaluation shows that our approach can generate examples deemed relevant for understanding the system and for a source of test generation by practitioners. In addition, we show that basing test generation on behavioural properties provides tests that are less dependent on the state of the system, while at the same time yielding a similar code coverage as state-of-the-art methods in REST API fuzzing in a given time limit.

show abstract

Open Problems in Fuzzing RESTful APIs: A Comparison of Tools

Cited by 2 publications

References 21 publications

Fuzzing Microservices In Industry: Experience of Applying EvoMaster at Meituan

Fuzzing Microservices In Industry: Experience of Applying EvoMaster at Meituan

Exploring behaviours of RESTful APIs in an industrial setting

Contact Info

Product

Resources

About