How to Write and Read a Scientific Evaluation Paper

Abstract

Abstract-Scientific evaluation papers investigate existing problem situations or validate proposed solutions with scientific means, such as by experiment or case study. There is a growing amount of literature about how to report about empirical research in software engineering, but there is still some confusion about the difference between a scientific evaluation paper and other kinds of research papers. This is related to lack of clarity about the relation between empirical research, engineering, and industrial practice. In this minitutorial we give a brief rundown on how to structure a scientific evaluation papers as a special kind of research paper, using experiment reports and case study reports as examples. We give checklists of items that a reader should be able to find in these papers, and sketch the dilemmas that writers and readers of these papers face when applying these checklists.

Effective application of process improvement patterns to business processes

“…For example, the parallel execution of formerly sequential tasks constitutes a PIP that may contribute to shorter cycle times as an exemplary PIO. Note that the concept of PIOs corresponds to the identification of stakeholders' goals, which has been proposed as a requirement for empirical IS research in [24]. 3.…”

“…To maintain scientific rigor, their assessment should take into account requirements known from the empirical evaluation of propositions in software engineering or IS research. In [24], the authors subsume requirements in terms of scientific methodology for evaluation approaches in IS research. Figure 6 provides an overview on the basic concepts described there.…”

“…Figure 6 provides an overview on the basic concepts described there. In the following, we align our approach to [24]. We describe how each component is reflected in our proposition.…”

“…The first four components we address constitute the problem statement according to [24]. Research question ("What do we want to know?")…”

“…Hevner et al [52] summarized empirical "design evaluation methods" for information systems research. As a basis of this paper, we chose the requirements summary by Wieringa et al [24] due to its concise, checklist-based character, which makes it readily applicable to research as well as to discussion with practitioners.…”