Mining large software compilations over time

Abstract

With the success of libre (free, open source) software, a new type of software compilation has become increasingly common. Such compilations, often referred to as 'distributions', group hundreds, if not thousands, of software applications and libraries written by independent parties into an integrated system. Software compilations raise a number of questions that have not been targeted so far by software evolution, which usually focuses on the evolution of single applications. Undoubtedly, the challenges that software compilations face differ from those found in single software applications. Nevertheless, it can be assumed that both, the evolution of applications and that of software compilations, have similarities and dependencies.In this sense, we identify a dichotomy, common to that in economics, of software evolution in the small (microevolution) and in the large (macro-evolution). The goal of this paper is to study the evolution of a large software compilation, mining the publicly available repository of a well-known Linux distribution, Debian. We will therefore investigate changes related to hundreds of millions of lines of code over seven years. The aspects that will be covered in this paper are size (in terms of number of packages and of number of lines of code), use of programming languages, maintenance of packages and file sizes.

Open Source Software Evolution

“…This section discusses the studies which considered the changes, in code and other artifacts, as recorded in the version control systems. Robles et al (2006b) categorize files on the basis of the different aspects of software development they cater to such as documentation, user-interface design, multimedia content, and international language support. They analyze the evolutionary behavior of distinct file types, by using the concept of atomic commits, to understand the change patterns.…”

Open Source Software Evolution

“…Several software artifacts co-evolve with source code. Robles et al (2006b) analyzes the evolution of two Java based open source build systems ANT and Maven. Build systems are important for software development process as all changes reflect only after a build system is used to run the changed system.…”

“…It remains almost constant for some of them. Robles et al (2006b) analyzed Debian distribution for 7 years starting in 1998. The size (of stable releases), measured using SLOC and number of packages, doubled after every two years.…”

Tools for the Study of the Usual Data Sources found in Libre Software Projects

“…This is the case for packaging systems such as the .deb format used in Debian and Debian-based distributions or the .rpm Red Hat package system in use in Red Hat and other distributions. But beyond this, we can find project-related information in other places such as the Debian Popularity Contest (Robles et al, 2006b) or the Debian Developer database (Robles et al, 2005). Other data sources may also be considered; for instance, in KDE there is a file that is used to list all the ones who have write access to their SCM repository.…”

“…Some of the previous discrimination techniques are already in use in some tools, most notably in SLOCCount (see (Wheeler, 2001, Robles et al, 2006b). As SLOCCount counts the number of lines of code it is only concerned with identifying source code files and identifying the programming language in which they are written, not considering all other file types that we have taken into consideration in this work (documentation, translations, and other).…”