Standing on Shoulders or Feet? The Usage of the MSR Data Papers

Kotti, Zoe; Spinellis, Diomidis

doi:10.1109/msr.2019.00085

Cited by 12 publications

(9 citation statements)

References 97 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data from GitHub, Stackoverflow etc. is harvested into repositories such as GHTorrent to be employed in research [27]. Thus, the FAIR principles can also be applied to software, which can be stored and treated as data.…”

Section: The Fair Principles For Research Softwarementioning

confidence: 99%

From FAIR research data toward FAIR and open research software

Hasselbring

Carr

Hettrick

et al. 2020

It - Information Technology

View full text Add to dashboard Cite

The Open Science agenda holds that science advances faster when we can build on existing results. Therefore, research data must be FAIR (Findable, Accessible, Interoperable, and Reusable) in order to advance the findability, reproducibility and reuse of research results. Besides the research data, all the processing steps on these data – as basis of scientific publications – have to be available, too.For good scientific practice, the resulting research software should be both open and adhere to the FAIR principles to allow full repeatability, reproducibility, and reuse. As compared to research data, research software should be both archived for reproducibility and actively maintained for reusability.The FAIR data principles do not require openness, but research software should be open source software. Established open source software licenses provide sufficient licensing options, such that it should be the rare exception to keep research software closed.We review and analyze the current state in this area in order to give recommendations for making research software FAIR and open.

show abstract

Section: The Fair Principles For Research Softwarementioning

confidence: 99%

From FAIR research data toward FAIR and open research software

Hasselbring

Carr

Hettrick

et al. 2020

It - Information Technology

View full text Add to dashboard Cite

show abstract

“…The data sets associated with our study (data papers, citing papers, categorizations, MSR papers, citations, survey questionnaire and responses) are made available online. 4 A shorter version of this study appeared in the 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR '19) (Kotti and Spinellis 2019). This work extends the conference paper by using multiple raters and established research methods for the manual clustering of data papers and the classification of strong citations.…”

Section: Introductionmentioning

confidence: 99%

Standing on shoulders or feet? An extended study on the usage of the MSR data papers

et al. 2020

Self Cite

View full text Add to dashboard Cite

The establishment of the Mining Software Repositories (MSR) data showcase conference track has encouraged researchers to provide data sets as a basis for further empirical studies. Objective: Examine the usage of data papers published in the MSR proceedings in terms of use frequency, users, and use purpose. Method: Data track papers were collected from the MSR data showcase track and through the manual inspection of older MSR proceedings. The use of data papers was established through manual citation searching followed by reading the citing studies and dividing them into strong and weak citations. Contrary to weak, strong citations truly use the data set of a data paper. Data papers were then manually clustered based on their content, whereas their strong citations were classified by hand according to the knowledge areas of the Guide to the Software Engineering Body of Knowledge. A survey study on 108 authors and users of data papers provided further insights regarding motivation and effort in data paper production, encouraging and discouraging factors in data set use, and future desired direction regarding data papers. Results: We found that 65% of the data papers have been used in other studies, with a longtail distribution in the number of strong citations. Weak citations to data papers usually refer to them as an example. MSR data papers are cited in total less than other MSR papers. A considerable number of the strong citations stem from the teams that authored the data papers. Publications providing Version Control System (VCS) primary and derived data are the most frequent data papers and the most often strongly cited ones. Enhanced developer data papers are the least common ones, and the second least frequently strongly cited. Data paper authors tend to gather data in the context of other research. Users of data sets appreciate high data quality and are discouraged by lack of replicability of data set construction. Data related to machine learning or derived from the manufacturing sector are two suggestions of the respondents for future data papers. Conclusions: Data papers have provided the foundation for a significant number of studies,

show abstract

“…Kotti and Spinellis [12] looked at the use of pre-prepared datasets, particularly those described in MSR's Data Showcase. As these datasets often include a time component (see Section III-A), their definitions and methods are important.…”

Section: Previous Studiesmentioning

confidence: 99%

“…We also noticed another mixed VCS, namely hg-git 12 , which allows a Mercurial user to manipulate a Git repository using Mercurial commands. In particular, we note the addition of metadata to commits, with the rebase_source footer (seen 390 times), which is likely a result of rebases on Git repositories using Mercurial or similar tools.…”

Section: B Finding Out-of-order Commitsmentioning

confidence: 99%

Escaping the Time Pit: Pitfalls and Guidelines for Using Time-Based Git Data

Flint¹,

Chauhan²,

Dyer³

2021

Preprint

View full text Add to dashboard Cite

Many software engineering research papers rely on time-based data (e.g., commit timestamps, issue report creation/update/close dates, release dates). Like most real-world data however, time-based data is often dirty. To date, there are no studies that quantify how frequently such data is used by the software engineering research community, or investigate sources of and quantify how often such data is dirty. Depending on the research task and method used, including such dirty data could affect the research results. This paper presents the first survey of papers that utilize time-based data, published in the Mining Software Repositories (MSR) conference series. Out of the 690 technical track and data papers published in MSR 2004-2020, we saw at least 35% of papers utilized time-based data. We then used the Boa and Software Heritage infrastructures to help identify and quantify several sources of dirty commit timestamp data. Finally we provide guidelines/best practices for researchers utilizing time-based data from Git repositories.

show abstract

Standing on Shoulders or Feet? The Usage of the MSR Data Papers

Cited by 12 publications

References 97 publications

From FAIR research data toward FAIR and open research software

From FAIR research data toward FAIR and open research software

Standing on shoulders or feet? An extended study on the usage of the MSR data papers

Escaping the Time Pit: Pitfalls and Guidelines for Using Time-Based Git Data

Contact Info

Product

Resources

About