The Behavioral Diversity of Java JSON Libraries

Abstract

JSON is an essential file and data format in domains that span scientific computing, web APIs or configuration management. Its popularity has motivated significant software development effort to build multiple libraries to process JSON data. Previous studies focus on performance comparison among these libraries and lack a software engineering perspective.We present the first systematic analysis and comparison of the input / output behavior of 20 JSON libraries, in a single software ecosystem: Java/Maven. We assess behavior diversity by running each library against a curated set of 473 JSON files, including both well-formed and ill-formed files. The main design differences, which influence the behavior of the libraries, relate to the choice of data structure to represent JSON objects and to the encoding of numbers. We observe a remarkable behavioral diversity with ill-formed files, or corner cases such as large numbers or duplicate data. Our unique behavioral assessment of JSON libraries paves the way for a robust processing of illformed files, through a multi-version architecture.

Automatic Diversity in the Software Supply Chain

“…We leverage one key observation to limit the development effort for adapters: the distributions of usages among libraries in a reservoir [26], among versions [27] and among the API members [28], are skewed towards a handful of popular elements. If the designers of a Library Substitution Framework accept to build a solution that is effective for a majority of client applications in a given domain, instead of all of them, then, the complexity of the framework can be significantly reduced.…”

“…This makes JSON libraries good targets for supply chain attacks on third-party packages. Therefore, the reservoir of JSON libraries [26] is a relevant target to experiment the concept of Library Substitution Framework.…”

“…These libraries have been designed by different people with different motivations. We have previously analyzed the behavior of this reservoir in depth [26]. In our previous work, we make three points that are of interest for this work: (i) these libraries make a large variety of design decisions to represent JSON data; (ii) these libraries exhibit a large diversity of behaviors regarding what JSON inputs they consider as well-formed or ill-formed; (iii) there is more diversity of behaviors observed on JSON inputs that do not conform to the standard grammar [40].…”

“…We rely both on knowledge of the JSON subject, and on the manual analysis of the libraries in the reservoir to identify three key, core features, which we specify in the facade: parsing JSON text into Java structures, serializing these structures to JSON text, and representing JSON types with Java structures. We observe that 15 out of 20 libraries propose a type for JSON object and 14 for JSON arrays [26]. The other libraries represent these data as types of the Java standard library, implementing either a Map or a List (or in one case a primitive array).…”

“…Our analysis of the test failures reveals that libraries can fail for several reasons. Firs, a standard may leave room for diversity among libraries implementing it [26]. In the case of JSON, the RFC [40] leaves JSON parsers free to decide whether to accept ill-formed JSON inputs.…”