Towards a unified checklist for empirical research in software engineering: first proposal

Abstract

Abstract-Background: Current checklists for empirical software engineering cover either experimental research or case study research but ignore the many commonalities that exist across all kinds of empirical research. Identifying these commonalities, and explaining why they exist, would enhance our understanding of empirical research in general and of the differences between experimental and case study research in particular. Aim: In this short paper we design a unified checklist for empirical research, that identify commonalities and differences between experimental and case study research. Method: We design the unified checklist as a specialization of the general engineering cycle, which itself is a special case of the rational choice cycle. The unified checklist is based on an analysis and integration of a number of existing checklists. Results: The current version of the checklist exhibits a shared structure of experimental and case study research. Conclusions: Although the checklist exhibits a shared underlying structure of empirical research, its limitations are that it ignores other research methods such as meta-research or surveys and that very little empirical validation of usability and utility of the checklist has been performed so far. We are currently performing one validation, and are planning additional ones. These validations will likely lead to improvements of the current proposal.

Dynamic Homecare Service Provisioning: A Field Test and Its Results

“…The field test is an action case study [19], in which we aim to improve the current situation of providing care by using a DHSP platform. We follow the guidelines described by Wieringa in [18] to perform the experiments systematically.…”

Care-giver Tailoring of IT-based Healthcare Services for Elderly at Home: A Field Test and its Results

“…The field test is an action case study [19], in which we aim to improve the current situation of providing care by using a tailorable IT system. We follow the guidelines described by Wieringa in [20] to perform the experiments systematically.…”

Lessons learned from evaluating a checklist for reporting experimental and observational research

“…RH was selected because it is the current state of the art of checklists for case study reporting in software engineering. Version 1 is basically the union of all three checklists [13] and contains more questions than Version 2. 1 The following observations about Version 1 are also valid for Version 2.1.…”

“…• The Population question does not occur explicitly as an item in these other checklists [13]. The two checklists for experiment reports consider this to be part of the description of the sampling procedure.…”

“…The first author has used this as a framework for teaching empirical methods since 2007 [14]. Its use to define a checklist for empirical research based on extant checklists is described in [12]. In section 3, we review a number of lessons learned from evaluating the checklist.…”