Lessons learned from evaluating a checklist for reporting experimental and observational research

Wieringa, Roel; Condori-Fernández, Nelly; Daneva, Maya; Mutschler, Bela; Pastor, Oscar

doi:10.1145/2372251.2372279

Cited by 11 publications

(10 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Looking at the ways in which checklists were built, researchers most often based the construction of a new checklist upon existing ones (cf. [22,48]).…”

Section: Checklists In Software Engineeringmentioning

confidence: 99%

An empirically evaluated checklist for surveys in software engineering

Molléri

Petersen

Mendes

2020

Information and Software Technology

View full text Add to dashboard Cite

Context: Over the past decade Software Engineering research has seen a steady increase in survey-based studies, and there are several guidelines providing support for those willing to carry out surveys. The need for auditing survey research has been raised in the literature. Checklists have been used to assess different types of empirical studies, such as experiments and case studies.Objective: This paper proposes a checklist to support the design and assessment of survey-based research in software engineering grounded in existing guidelines for survey research. We further evaluated the checklist in the research practice context.Method: To construct the checklist, we systematically aggregated knowledge from 14 methodological papers supporting survey-based research in software engineering. We identified the key stages of the survey process and its recommended practices through thematic analysis and vote counting. To improve our initially designed checklist we evaluated it using a mixed evaluation approach involving experienced researchers.Results: The evaluation provided insights regarding limitations of the checklist in relation to its understanding and objectivity. In particular, 19 of the 38 checklist items were improved according to the feedback received from its evaluation. Finally, a discussion on how to use the checklist and what its implications are for research practice is also provided.Conclusion: The proposed checklist is an instrument suitable for auditing survey reports as well as a support tool to guide ongoing research with regard to the survey design process.

show abstract

“…Looking at the ways in which checklists were built, researchers most often based the construction of a new checklist upon existing ones (cf. [22,48]).…”

Section: Checklists In Software Engineeringmentioning

confidence: 99%

An empirically evaluated checklist for surveys in software engineering

Molléri

Petersen

Mendes

2020

Information and Software Technology

View full text Add to dashboard Cite

show abstract

“…What should a case study look like? There are different guidelines on this, but a consensus seems to be emerging [1], [2], [3], [4]. First, all case studies should present and motivate their conceptual framework, identify the case to be studied, list their research questions, and summarize the current state of knowledge to which the case study aims to contribute.…”

Section: Industrial Case Studiesmentioning

confidence: 99%

Requirements engineering conferences: Wither industry tracks?

Wieringa

Eck

Mylopoulos

2013

2013 21st IEEE International Requirements Engineering Conference (RE)

Self Cite

View full text Add to dashboard Cite

Abstract-This position paper argues that industry tracks have no place in any research conference. Instead, a research conference should always have room for industrial case studies, evaluated according to criteria for empirical research. Such case studies would not be acceptable at a practitioners' industrial conference, just as papers presented at such conferences would not be acceptable at research conferences. It follows as corollary that if researchers want to become familiar with problems and solutions of RE practice, they should visit industrial conferences.

show abstract

“…We addressed the research questions of this thesis according to the Technical Action Research (TAR) methodology [17][18][19][20], which describes the interaction between an artifact and a problem context for producing effects, as shown in Figure 1-1. In the context of this thesis, we defined the artifact, context and effects as follow: -Artifact: Reusable architecture for the exchange of context-aware messages in pervasive healthcare environment; -Context: Several stakeholders, such as physicians, nurses, assistants and patients, and the hospital environment in which the stakeholders use the legacy systems; -Effect: Message exchange between legacy systems using the reusable architecture, which satisfy the requirements to support the interoperability and cooperation between these systems.…”

Section: Methodsmentioning

confidence: 99%

“…An openEHR archetype consists of three sections: a header contains an identifier for the archetype; a definition expresses the restrictions in a treelike structure that classify the archetypes in terms of entries of the RM, and constrains the cardinality and the content of the information model instances that comply with the archetype; and an ontology contains the codes associated to nodes, the constraints on text and terms, and the bindings to terminologies, such as, e.g., SNOMED-CT. [14][15][16][17][18][19][20] includes the terminological definitions, and the linguistic expression 'Pacemaker Implantation' is associated with the code 'at0000'. The archetypes designed in this work are explained in more detail in Chapter 5.…”

Section: Archetype Modelmentioning

confidence: 99%

“…The Agent Communication Language (ACL) proposed by the Foundation for Intelligent Physical Agents (FIPA) 20 is often used for this purpose, since it is a standard language for communication between agents. FIPA-ACL is based on the theory of speech acts and it is beneficial due to the formal definition of its performative library.…”

Section: Agent Communication Languagementioning

confidence: 99%

See 1 more Smart Citation

Methodological support to develop interoperable applications for pervasive healthcare

Moraes¹

View full text Add to dashboard Cite

The healthcare model currently being used in most countries will soon be inadequate, due to the increasing care costs of a growing population of elderly people, the rapid increase of chronic diseases, the growing demand for new treatments and technologies, and the relative decrease in the number of healthcare professionals with respect to the population increase. This healthcare model, which is centered on highly specialized people, located in large hospitals, needs to change into a distributed model, in order to produce faster responses and to allow patients to manage their own health. This centralized healthcare model also implies that patients and healthcare professionals have to move to the same place (a hospital or clinic) for the healthcare services to be delivered, and it is often expensive and inefficient. A distributed healthcare model that pervades the daily lives of the citizens is more appropriate to provide less expensive and more effective and timely healthcare, and characterizes Pervasive Healthcare. In Pervasive Healthcare, novel information and communication technologies are applied to support the provision of health services anywhere, at anytime and to anyone.This thesis aims at demonstrating the feasibility of using healthcare standards, ubiquitous computing technologies, service-oriented architecture principles, and software agents, to develop interoperable applications for exchanging messages in Pervasive Healthcare environments. This thesis presents a Reusable Architecture and a Message Generator that employ these technologies for achieving its aim. Our Reusable Architecture and Message Generator also meet the requirements of interoperability between heterogeneous healthcare information systems that are usually found in Pervasive Healthcare environments.In this work, a case study was performed in a realistic distributed healthcare environment, where three usage scenarios were defined in the cardiology domain: delivery of laboratory analysis results, schedule of an appointment for pacemaker evaluation, and support of a medical staff II ABSTRACT meeting to prepare for a cardiac surgery. Using the Reusable Architecture and the Message Generator, three applications were built for these scenarios, and they were employed and evaluated by both healthcare professionals and patients. The experiments were conducted in a distributed environment that consists of three cardiology clinics, one analysis laboratory, and the cardiology department of the Santa Casa Hospital of Marília (São Paulo, Brazil). The evaluation of these applications was carried out using the Technology Acceptance Model (TAM), since this model is generally considered as appropriate to explain technology acceptance. The evaluation involved a limited but representative set of healthcare professionals and patients.We showed that the methodology proposed in this thesis truly facilitates the development of Pervasive Healthcare. The proposed methodology preserves the investment in the legacy healthcare information systems and allow...

show abstract

Lessons learned from evaluating a checklist for reporting experimental and observational research

Abstract: This short paper summarizes and discusses the result of an iterative construction and evaluation of a checklist for writing and reading reports about experimental and observational research.

Cited by 11 publications

References 12 publications

An empirically evaluated checklist for surveys in software engineering

An empirically evaluated checklist for surveys in software engineering

Requirements engineering conferences: Wither industry tracks?

Methodological support to develop interoperable applications for pervasive healthcare

Contact Info

Product

Resources

About