Sampling in software engineering research: a critical review and guidelines

Baltes, Sebastian; Ralph, Paul

doi:10.1007/s10664-021-10072-8

Cited by 110 publications

(55 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The target population of our survey is software practitioners involved in RE-related activities, and therefore, we used a non-probabilistic purposive sampling technique in the study as our selection is based on speciic characteristics other than availability (as our target participants are software practitioners involved in RE-related activities) [8]. We used the Qualtrics platform to design the survey and after obtaining the required ethics approval (Reference Number: 26219), we advertised the survey as an anonymous survey link.…”

Section: Survey Sampling and Data Collectionmentioning

confidence: 99%

The Influence of Human Aspects on Requirements Engineering-related Activities: Software Practitioners’ Perspective

Hidellaarachchi

Grundy

Hoda

et al. 2023

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Requirements Engineering (RE)-related activities require high collaboration between various roles in software engineering (SE), such as requirements engineers, stakeholders, developers, etc. Their demographics, views, understanding of technologies, working styles, communication and collaboration capabilities make RE highly human dependent. Identifying how ”human aspects” – such as motivation, domain knowledge, communication skills, personality, emotions, culture, etc.– might impact RE-related activities would help us improve RE and SE in general. This study aims to better understand current industry perspectives on the influence of human aspects on RE-related activities, specifically focusing on motivation and personality, by targeting software practitioners involved in RE-related activities. Our findings indicate that software practitioners consider motivation, domain knowledge, attitude, communication skills and personality as highly important human aspects when involved in RE-related activities. A set of factors were identified as software practitioners’ key motivational factors when involved in RE-related activities, along with important personality characteristics to have when involved in RE. We also identified factors that made individuals less effective when involved in RE-related activities and obtained some feedback on measuring individuals’ performance when involved in RE. The findings from our study suggest various areas needing more investigation, and we summarise a set of key recommendations for further research.

show abstract

Section: Survey Sampling and Data Collectionmentioning

confidence: 99%

The Influence of Human Aspects on Requirements Engineering-related Activities: Software Practitioners’ Perspective

Hidellaarachchi

Grundy

Hoda

et al. 2023

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

“…Ad hoc reviews use purposive sampling [3]; that is, researchers purposefully select papers or studies that are useful, relevant, or support their arguments. Ad hoc reviews are often appropriate, for example, to support theory development, identify promising research topics, or prepare for a comprehensive exam.…”

Section: Review Types 21 Ad Hoc Reviewsmentioning

confidence: 99%

“…1 For example, Stol et al [31]'s critical review of the use of grounded theory in software engineering criticizes method slurring; that is, claiming to have used a research methodology that was not actually used to create the illusion of legitimacy. Similarly, Baltes and Ralph [3] used a critical review to describe and criticize how SE researchers overstate sample representativeness and often conflate random sampling with representative sampling. Indeed, critical reviews in software engineering often investigate methodological topics, including how ethnography is reported [33] or how qualitative research is synthesized [13].…”

Section: Critical Reviewsmentioning

confidence: 99%

“…Ad hoc reviews, case surveys, critical reviews, meta-analysis, meta-synthesis and scoping reviews are all types of secondary research. Secondary research is central to evidencebased practice because (1) practitioners cannot read every study; (2) important decisions typically should be made based on the balance of evidence rather than a single study; (3) in many fields, individual studies are too small to produce accurate estimates of population parameters (e.g. the strength of the relationship between two variables).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Paving the Way for Mature Secondary Research: The Seven Types of Literature Review

Ralph,

Baltes

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Confusion over different kinds of secondary research, and their divergent purposes, is undermining the effectiveness and usefulness of secondary studies in software engineering. This short paper therefore explains the differences between ad hoc reviews, case surveys, critical reviews, meta-analyses (aka systematic literature reviews), meta-synthesis (aka thematic analysis), rapid reviews and scoping reviews (aka systematic mapping studies). We envision that these guidelines will help researchers better select and describe their literature reviews, while helping reviewers select more appropriate evaluation criteria.

show abstract

“…In ESE studies the results obtained must be, at some degree, replicable and generalizable. Replicability is important to validate the results of a scientific study and can be tackled by providing the datasets and scripts of the conducted research [Baltes and Ralph 2020].…”

Section: Related Workmentioning

confidence: 99%

Open-Source Software Projects Curating Model for Empirical Software Engineering Studies

Carruthers

2022

Anais Do XXV Congresso Ibero-Americano Em Engenharia De Software (CIbSE 2022)

View full text Add to dashboard Cite

Software projects are common inputs in Empirical Software Engineering (ESE), and they are often selected without following a specific strategy, leading to biased samples. To avoid this problem, researchers choose to use publicly available datasets instead of picking the projects themselves. However, some datasets are not maintained, containing old versions of projects, or even deprecated ones. This may raise some representativeness issues due to major changes in development practices and technologies over time. The main goal of this research is to develop a procedures model to construct and maintain a software project dataset with their product quality metrics, to support the development of ESE studies.

show abstract

Sampling in software engineering research: a critical review and guidelines

Cited by 110 publications

References 66 publications

The Influence of Human Aspects on Requirements Engineering-related Activities: Software Practitioners’ Perspective

The Influence of Human Aspects on Requirements Engineering-related Activities: Software Practitioners’ Perspective

Paving the Way for Mature Secondary Research: The Seven Types of Literature Review

Open-Source Software Projects Curating Model for Empirical Software Engineering Studies

Contact Info

Product

Resources

About