Support mechanisms to conduct empirical studies in software engineering

Borges, Alex; Ferreira, Waldemar; Barreiros, Emanoel; Almeida, Anderson Rodrigues de; Fonseca, Liliane; Teixeira, Eudis; Silva, Diogo; Alencar, Aline; Soares, Sérgio

doi:10.1145/2745802.2745823

Cited by 12 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering the previous section, we conclude that there is a lack of standard specification for SE experiments and software development processes. Aiming at fulfilling this gap, we analyzed coding experiments in literature and models for SE experiments and development processes Borges et al [25] . Following up from this result, (1) we elicited meaning and developed knowledge into concepts used within both areas and (2) we identified patterns combining the identified concepts.…”

Section: Methodsmentioning

confidence: 99%

Codex: A metamodel ontology to guide the execution of coding experiments

Ferreira¹,

Baldassarre²,

Soares³

2018

Computer Standards & Interfaces

Self Cite

View full text Add to dashboard Cite

Section: Methodsmentioning

confidence: 99%

Codex: A metamodel ontology to guide the execution of coding experiments

Ferreira¹,

Baldassarre²,

Soares³

2018

Computer Standards & Interfaces

Self Cite

View full text Add to dashboard Cite

“…However, only four out of 14 publications offer their data at all, with only one publication [15] using a public data repository [71], [72]. The other three publications only offer links that no longer work [5], [12], [13].…”

Section: Related Workmentioning

confidence: 99%

“…Until 2000, the most common research methods were conceptual analysis and concept implementation [7]. Between 2000 and 2015, the most commonly used research methods changed to case studies and experiments [3], [5], [8]- [11], which were expanded after 2015 to also include surveys and systematic literature reviews [12]- [15]. While this change shows an evolution of research methods used, we also note that experiments and case studies have been the two main research methods for empirical research in RE and SE for more than 20 years.…”

Section: Related Workmentioning

confidence: 99%

“…While this change shows an evolution of research methods used, we also note that experiments and case studies have been the two main research methods for empirical research in RE and SE for more than 20 years. Although these two research methods have been used for a long time, seven publications concluded that there is a need to develop, expand, and use standardized terminology and theories (from other disciplines) to more consistently represent the empirical research conducted and better explain the results found [1], [2], [6]- [8], [12], [13]. In this regard, seven publications also analyzed the information reported for a comprehensive description of a research design.…”

Section: Related Workmentioning

confidence: 99%

“…object of study [1], population/sample [3], [10], threats to validity [13], data collection [15], measurements/metrics [11], and data analysis [10], [13]. Overall, six publications [2], [3], [6], [10], [12], [13] explicitly state that the use of empirical research in RE and SE is constantly increasing. This increase is accompanied by the need to have a comprehensive, up-to-date, and long-term available overview of the state and evolution of empirical research [1], [5], [6].…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Divide and Conquer the EmpiRE: A Community-Maintainable Knowledge Graph of Empirical Research in Requirements Engineering

Karras,

Wernlein,

Klünder

et al. 2023

2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

View full text Add to dashboard Cite

Background.] Empirical research in requirements engineering (RE) is a constantly evolving topic, with a growing number of publications. Several papers address this topic using literature reviews to provide a snapshot of its "current" state and evolution. However, these papers have never built on or updated earlier ones, resulting in overlap and redundancy. The underlying problem is the unavailability of data from earlier works. Researchers need technical infrastructures to conduct sustainable literature reviews. [Aims.] We examine the use of the Open Research Knowledge Graph (ORKG) as such an infrastructure to build and publish an initial Knowledge Graph of Empirical research in RE (KG-EmpiRE) whose data is openly available. Our long-term goal is to continuously maintain KG-EmpiRE with the research community to synthesize a comprehensive, up-todate, and long-term available overview of the state and evolution of empirical research in RE. [Method.] We conduct a literature review using the ORKG to build and publish KG-EmpiRE which we evaluate against competency questions derived from a published vision of empirical research in software (requirements) engineering for 2020 -2025. [Results.] From 570 papers of the IEEE International Requirements Engineering Conference (2000 -2022), we extract and analyze data on the reported empirical research and answer 16 out of 77 competency questions. These answers show a positive development towards the vision, but also the need for future improvements. [Conclusions.] The ORKG is a ready-to-use and advanced infrastructure to organize data from literature reviews as knowledge graphs. The resulting knowledge graphs make the data openly available and maintainable by research communities, enabling sustainable literature reviews.

show abstract