Benchmarking Web API Quality

Abstract

Modern applications increasingly interact with web APIs -reusable components, deployed and operated outside the application, and accessed over the network. Their existence, arguably, spurs application innovations, making it easy to integrate data or functionalities. While previous work has analyzed the ecosystem of web APIs and their design, little is known about web API quality at runtime. This gap is critical, as qualities including availability, latency, or security can severely impact applications and user experience. In this paper, we revisit a 3-month, geo-distributed benchmark of popular web APIs, originally performed in 2015. We repeat this benchmark in 2018 and compare results from these two benchmarks regarding availability and latency. We furthermore introduce new results from assessing web API security measurements, collected both in 2015 and 2018, and results from our attempts to reach out to API providers with the results from our 2015 experiments. Our extensive experiments show that web API qualities vary 1.) based on the geo-distribution of clients, 2.) during our individual experiments, and 3.) between the two experiments. Our findings provide evidence to foster the discussion around web API quality, and can act as a basis for the creation of tools and approaches to mitigate quality issues.Index Terms-Web APIs, Benchmarking, Quality of Service ! 2. We denote an endpoint to be the combination of a resource, identified by a URL, and an HTTP method as proposed in [7].

Using application benchmark call graphs to quantify and improve the practical relevance of microbenchmark suites

“…Current work focuses on application-specific benchmarks. To name a few, there are benchmarks which evaluate database or storage systems (e.g., Bermbach et al, 2014 ; Cooper et al, 2010 ; Bermbach et al, 2017 ; Kuhlenkamp, Klems & Röss, 2014 ; Müller et al, 2014 ; Pallas et al, 2017 ; Pallas, Günther & Bermbach, 2017 ; Pelkonen et al, 2015 ; Difallah et al, 2013 ), benchmark microservices ( Villamizar et al, 2015 ; Grambow et al, 2020 ; Grambow, Wittern & Bermbach, 2020 ; Ueda, Nakaike & Ohara, 2016 ; Do et al, 2017 ), determine the quality of web APIs ( Bermbach & Wittern, 2016 , 2020 ), specifically tackle web sites ( Menascé, 2002 ), or evaluate other large-scale software systems (e.g., Jiang & Hassan, 2015 ; Hasenburg et al, 2020 ; Hasenburg & Bermbach, 2020 ). Our approach can use all of these application benchmarks as a baseline.…”

An API-first methodology for designing a microservice-based Backend as a Service platform

“…Another advantage of using the API gateway design pattern is that it allows the distribution of platform interaction with clients, depending on the type of client application. For example, processing APIs from web and mobile applications may result in dif-ferent performance indicators [6]. This is mainly because mobile applications present the same data less elaborate than web applications, given their physical limitations in screen size.…”

“…In the conducted analysis, the following three metrics were selected to evaluate the complexity and size of the developed web application: _ Lines of Code (LOC) -the number of lines of code in a given class or file [6]. _ Response for Class (RFC) -the number of methods that are called when a certain operation within a class is invoked [11].…”

An exploratory study on faults in web API integration in a large-scale payment company

“…Bermbach and Wittern [5] performed a geo-distributed benchmark to assess the quality of Web APIs, in terms of performance and availability. The authors ind a great variety in quality between diferent APIs.…”