LiDetector : License Incompatibility Detection for Open Source Software

Abstract

Open-source software (OSS) licenses dictate the conditions which should be followed to reuse, distribute, and modify software. Apart from widely-used licenses such as the MIT License, developers are also allowed to customize their own licenses, whose descriptions are more flexible. The presence of such various licenses imposes challenges to understand licenses and their compatibility. To avoid financial and legal risks, it is essential to ensure license compatibility when integrating third-party packages. In this work, we propose LiDetector , an effective tool that extracts and interprets OSS licenses, and detects license incompatibility. Specifically, LiDetector introduces a learning-based method to automatically identify meaningful license terms from an arbitrary license, and employs Probabilistic Context-Free Grammar (PCFG) to infer rights and obligations for incompatibility detection. Experiments demonstrate that LiDetector outperforms existing methods with 93.28% precision for term identification, and 91.09% accuracy for right and obligation inference, and can effectively detect incompatibility with 10.06% FP rate and 2.56% FN rate. Furthermore, with LiDetector , our large-scale empirical study on 1,846 projects reveals that 72.91% of the projects are suffering from license incompatibility, including popular ones such as the MIT License and the Apache License. We highlighted lessons learned from perspectives of different stakeholders and made all related data and the replication package publicly available to facilitate follow-up research.

LiResolver: License Incompatibility Resolution for Open Source Software

“…2.3.1 Fine-grained Entity Extraction. Unlike previous works that regarded each right/obligation as a single entity [45], in this paper, we observe that each right/obligation could be decomposed into four types of entities (i.e., action, object, attitude, and condition) and construct a fine-grained structure to model it. Each entity, regardless of its type, is a flexible expression that describes a specific action, object, attitude, or condition in the form of a word, a phrase, or even a sentence in a license.…”

“…The hierarchical structure of OSS accompanied with licenses leads to the hierarchical structure of licenses. Previous studies regarded all licenses in a project independently and equally [25,27,45], regardless of their licensing scopes. For instance, Figure 2 illustrates a real-world example from Flask JSONDash [9], a popular OSS to create chart dashboards.…”

“…However, manually extracting the detailed information from long and complicated licenses is time-consuming and labor-intensive. (2) A previous study investigated 1,846 projects and obtained 5,777 unique licenses, among which 24.56% were custom licenses with flexible expressions [45]. Motivated by them, it is valuable to automatically understand licenses to facilitate further analysis.…”

“…Each entity, regardless of its type, is a flexible expression that describes a specific action, object, attitude, or condition in the form of a word, a phrase, or even a sentence in a license. For instance, from "you are allowed to distribute modified works", the previous study [45] identified the term "Distribute" and inferred an attitude CAN. However, LiResolver extracts the action ("distribute"), the object ("modified works"), the attitude ("are allowed to"), and the condition if there exists.…”

“…License incompatibility represents a serious threat to all stakeholders from a legal perspective. An empirical study on 1,846 GitHub projects shows that 72.91% of the investigated projects suffered from license incompatibility [45]. Despite being problematic, fixing license incompatibility issues requires substantial efforts to either replace the reused code or migrate licenses.…”