Automatic Generation of Test Cases for REST APIs: A Specification-Based Approach

Ed-douibi, Hamza; Izquierdo, Javier Luis Cánovas; Cabot, Jordi

doi:10.1109/edoc.2018.00031

Cited by 60 publications

(67 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, there has been an increase in research on black-box automated test generation based on REST API schemas defined with OpenAPI (Atlidakis et al 2019;Karlsson et al 2020;Viglianisi et al 2020;Ed-douibi et al 2018). Atlidakis et al (2019) developed RESTler to generate test sequences based on dependencies inferred from OpenAPI specifications and analysis on dynamic feedback from responses (e.g., status code) during test execution.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Resource and dependency based test case generation for RESTful Web services

2021

View full text Add to dashboard Cite

Nowadays, RESTful web services are widely used for building enterprise applications. REST is not a protocol, but rather it defines a set of guidelines on how to design APIs to access and manipulate resources using HTTP over a network. In this paper, we propose an enhanced search-based method for automated system test generation for RESTful web services, by exploiting domain knowledge on the handling of HTTP resources. The proposed techniques use domain knowledge specific to RESTful web services and a set of effective templates to structure test actions (i.e., ordered sequences of HTTP calls) within an individual in the evolutionary search. The action templates are developed based on the semantics of HTTP methods and are used to manipulate the web services’ resources. In addition, we propose five novel sampling strategies with four sampling methods (i.e., resource-based sampling) for the test cases that can use one or more of these templates. The strategies are further supported with a set of new, specialized mutation operators (i.e., resource-based mutation) in the evolutionary search that take into account the use of these resources in the generated test cases. Moreover, we propose a novel dependency handling to detect possible dependencies among the resources in the tested applications. The resource-based sampling and mutations are then enhanced by exploiting the information of these detected dependencies. To evaluate our approach, we implemented it as an extension to the EvoMaster tool, and conducted an empirical study with two selected baselines on 7 open-source and 12 synthetic RESTful web services. Results show that our novel resource-based approach with dependency handling obtains a significant improvement in performance over the baselines, e.g., up to + 130.7% relative improvement (growing from + 27.9% to + 64.3%) on line coverage.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Then, tests are generated by ordering the operations based on the graph and considering the semantics of the operations. Ed-douibi et al (2018) proposed an approach to first generate test models based on OpenAPI specifications, then produce tests with the models.…”

Section: Related Workmentioning

confidence: 99%

Resource and dependency based test case generation for RESTful Web services

2021

View full text Add to dashboard Cite

show abstract

“…There are several recent approaches regarding the functional evaluation of REST services, which are based on diverse techniques (e.g., genetic programming, fuzzing, modelbased testing). Regarding black-box approaches, most notably, Ed-douibi et al [45] use a model of the OpenAPI specification to generate test cases that are based on invalid and boundary conditions. The tool is offered as a plugin for Eclipse and is based on a relatively small set of faults, that do not account for security issues.…”

Section: Related Workmentioning

confidence: 99%

A Black Box Tool for Robustness Testing of REST Services

2021

View full text Add to dashboard Cite

REST services are nowadays being used to support many businesses, with most major companies exposing their services via REST interfaces (e.g., Google, Amazon, Instagram, and Slack). In this type of scenarios, heterogeneity is prevalent and software is sometimes exposed to unexpected conditions that may activate residual bugs, leading service operations to fail. Such failures may lead to financial or reputation losses (e.g., information disclosure). Although techniques and tools for assessing robustness have been thoroughly studied and applied to a large diversity of domains, REST services still lack practical approaches that specialize in robustness evaluation. In this paper, we present a tool (named bBOXRT) for performing robustness tests over REST services, solely based on minimal information expressed in their interface descriptions. We used bBOXRT to evaluate an heterogeneous set of 52 REST services that comprise 1,351 operations and fit in distinct categories (e.g., public, private, in-house). We were able to disclose several different types of robustness problems, including issues in services with strong reliability requirements and also a few security vulnerabilities. The results show that REST services are being deployed preserving software defects that harm service integration, and also carrying security vulnerabilities that can be exploited by malicious users.

show abstract

“…In the recent years, different techniques have been proposed for black-box testing of RESTful APIs (Atlidakis et al, 2019;Ed-douibi et al, 2018;Karlsson et al, 2020;Viglianisi et al, 2020). Those present different variants of random testing, enhanced with heuristics based on the information provided in the API schemas.…”

Section: Related Workmentioning

confidence: 99%