Deep Learning-Based Prediction of Test Input Validity for RESTful APIs

Mirabella, Angelo; Martin-Lopez, Alberto; Segura, Sergio; Valencia-Cabrera, Luis; Ruiz–Cortés, Antonio

doi:10.1109/deeptest52559.2021.00008

Cited by 17 publications

(13 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If a function is simple, there is likely little need for a predictive model in the first place. Several recent studies feature more thorough evaluations (e.g., [39,5,54]), even on industrial systems (e.g., [32,16]). However, it largely remains to be seen whether the proposed techniques can be used on real-world production code.…”

Section: Rq6: Limitations and Open Challengesmentioning

confidence: 99%

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

Fontes¹,

Gay²

2022

Preprint

View full text Add to dashboard Cite

Context: Machine learning (ML) may enable effective automated test generation.Objectives: We characterize emerging research, examining testing practices, researcher goals, ML techniques applied, evaluation, and challenges.Methods: We perform a systematic literature review on a sample of 97 publications.Results: ML generates input for system, GUI, unit, performance, and combinatorial testing or improves the performance of existing generation methods. ML is also used to generate test verdicts, property-based, and expected output oracles. Supervised learning-often based on neural networks-and reinforcement learning-often based on Q-learning-are common, and some publications also employ unsupervised or semi-supervised learning. (Semi-/Un-)Supervised approaches are evaluated using both traditional testing metrics and ML-related metrics (e.g., accuracy), while reinforcement learning is often evaluated using testing metrics tied to the reward function. Conclusion:Work-to-date shows great promise, but there are open challenges regarding training data, retraining, scalability, evaluation complexity, ML algorithms employed-and how they are applied-benchmarks, and replicability. Our findings can serve as a roadmap and inspiration for researchers in this field.

show abstract

Section: Rq6: Limitations and Open Challengesmentioning

confidence: 99%

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

Fontes¹,

Gay²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Deep neural networks (DNNs) [38] have been increasingly adopted in many fields, including computer vision [5], natural language processing [19], software engineering [13,18,32,39,45,48], etc. However, one of the crucial factors hindering DNNs from further serving applications with social impact is the unintended individual discrimination [44,47,55].…”

Section: Introductionmentioning

confidence: 99%

NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identification

Zheng¹,

Chen²,

Du³

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep neural networks (DNNs) have demonstrated their outperformance in various domains. However, it raises a social concern whether DNNs can produce reliable and fair decisions especially when they are applied to sensitive domains involving valuable resource allocation, such as education, loan, and employment. It is crucial to conduct fairness testing before DNNs are reliably deployed to such sensitive domains, i.e., generating as many instances as possible to uncover fairness violations. However, the existing testing methods are still limited from three aspects: interpretability, performance, and generalizability. To overcome the challenges, we propose NeuronFair, a new DNN fairness testing framework that differs from previous work in several key aspects: (1) interpretable -it quantitatively interprets DNNs' fairness violations for the biased decision; (2) effective -it uses the interpretation results to guide the generation of more diverse instances in less time; (3) generic -it can handle both structured and unstructured data. Extensive evaluations across 7 datasets and the corresponding DNNs demonstrate NeuronFair's superior performance. For instance, on structured datasets, it generates much more instances (∼ ×5.84) and saves more time (with an average speedup of 534.56%) compared with the state-of-the-art methods. Besides, the instances of NeuronFair can also be leveraged to improve the fairness of the biased DNNs, which helps build more fair and trustworthy deep learning systems. The code of NeuronFair is open-sourced at https:// github.com/ haibinzheng/ NeuronFair. CCS CONCEPTS• Computing methodologies → Artificial intelligence; • Software and its engineering → Software reliability.

show abstract

“…The tool also introduces the concept of operation dependency graphs, which is a method for detecting dependencies between different parameters and only running certain test cases when all mandatory dependencies are met. RestTestGen was validated against 116 real-world APIs and was able to uncover many different defects in them Mirabella et al (2021). introduces an approach for predicting the validity of generated API test inputs using Artificial Intelligence techniques.…”

mentioning

confidence: 99%

“…It is very common for API endpoints to include parameters that are interdependent (i.e., some combinations of values for them produce invalid requests). Google Maps API, for instance, requires that requests that inform a value for the location parameter also include the radius parameter as well(MIRABELLA et al, 2021). As a consequence of these interdependencies, it becomes more challenging to randomly generate valid test inputs Mirabella et al (2021).…”

mentioning

confidence: 99%

“…Google Maps API, for instance, requires that requests that inform a value for the location parameter also include the radius parameter as well(MIRABELLA et al, 2021). As a consequence of these interdependencies, it becomes more challenging to randomly generate valid test inputs Mirabella et al (2021). proposes to train a deep learning model on previous API request / response pairs.…”

mentioning

confidence: 99%

See 1 more Smart Citation

OAS DB: a shared infrastructure to support OpenAPI research

Stoll¹

View full text Add to dashboard Cite

It is common knowledge the great success achieved by the Web in the last decades. Together with the rise of Web systems in general, it came the increase of the number of Web APIs. There are many specifications used to describe an Web API. One of the most popular ones is OpenAPI. This specification allows one to describe all the resources that can be accessed and manipulated through a REST Web API. An OpenAPI specification can be used to perform different kinds of analysis and verification of the service implementing the described API. A common challenge faced by researchers, however, is the lack of shared validation infrastructure or a standard benchmark. The main contribution of our research is a software artifact -called OAS DB (OpenAPI Specifications Database) -that aims to provide researchers and industry practitioners with a complete solution to streamline the validation of new OpenAPI related techniques and tools. OAS DB is able to generate complete OpenAPI specifications and their corresponding mock implementations. It is also both capable of injecting faults and anti-patterns in these generated specifications/mock implementations and of indicating -through machine-readable files -which issues and anti-patterns are present in the generated assets. We use OAS DB to assess tools relying on both static and dynamic techniques to detect faults and anti-patterns in OpenAPI specifications. Our results indicate that these tools fail to detect relevant faults and anti-patterns in the synthetic APIs generated by OAS DB, indicating that there is room to improve these tools and the ways in which they are applying static and dynamic analysis techniques. The present work also has as contributions: a) a proof of concept REST API anti-pattern detector (which we call Oasis) and b) the description of a novel REST anti-pattern not described in the literature so far.

show abstract

Deep Learning-Based Prediction of Test Input Validity for RESTful APIs

Cited by 17 publications

References 15 publications

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identification

OAS DB: a shared infrastructure to support OpenAPI research

Contact Info

Product

Resources

About