Investigating the Use of a Hybrid Search Strategy for Systematic Reviews

Mourão, Érica; Kalinowski, Marcos; Murta, Leonardo; Mendes, Emilia; Wohlin, Claes

doi:10.1109/esem.2017.30

Cited by 46 publications

(62 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Our tertiary study was conducted following the methodology stated in [7,8] and the recommendations in [9,10,11,17]. In particular, our search strategy was hybrid, as recommended in [11]. First, we conducted an automated search in Scopus and Web of Science engines to select a relevant seed set of secondary studies.…”

Section: Methodsmentioning

confidence: 99%

“…Several trials and validations were performed on the search string by multiple researchers, in order to have a stable and final search string. We only used the Scopus and Web of Science search engines, but to counteract, we applied snowballing, as recommended by [11]. The snowballing process was a backward process based only on titles, implying the risk of missing relevant studies.…”

Section: Threats To Validitymentioning

confidence: 99%

“…To address the lack of tertiary studies on model-based testing, our research goal was to identify and characterize secondary studies in model-based testing, in terms of the areas, tools, and challenges investigated 1 . We conducted a tertiary study based on the guidelines described in [7,8] as well as the recommendations stated in [9,10,11]. The studies were characterized in terms of specific MBT areas, model types, test selection criteria, test generation and test execution techniques, testing tools, and challenges.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Model-based testing areas, tools and challenges: A tertiary study

Villalobos-Arias

Quesada-López

Martinez

et al. 2019

CLEIej

View full text Add to dashboard Cite

Context: Model-based testing is one of the most studied approaches by secondary studies in the area of software testing. Aggregating knowledge from secondary studies on model- based testing can be useful for both academia and industry. Objective: The goal of this study is to characterize secondary studies in model-based testing, in terms of the areas, tools and challenges they have investigated. Method: We conducted a tertiary study following the guidelines for systematic mapping studies. Our mapping included 22 secondary studies, of which 12 were literature surveys and 10 systematic reviews, over the period 1996–2016. Results: A hierarchy of model-based testing areas and subareas was built based on existing taxonomies as well as data that emerged from the secondary studies themselves. This hierarchy was then used to classify studies, tools, challenges and their tendencies in a unified classification scheme. We found that the two most studied areas are UML models and transition-based notations, both being modeling paradigms. Regarding tendencies of areas in time, we found two areas with constant activity through time, namely, test objectives and model specification. With respect to tools, we only found five studies that compared and classified model-based testing tools. These tools have been classified into common dimensions that mainly refer to the model type and phases of the model-based testing process they support. We reclassified all the tools into the hierarchy of model-based testing areas we proposed, and found that most tools were reported within the modeling paradigm area. With regard to tendencies of tools, we found that tools for testing the functional behavior of software have prevailed over time. Another finding was the shift from tools that support the generation of abstract tests to those that support the generation of executable tests. For analyzing challenges, we used six categories that emerged from the data (based on a grounded analysis): efficacy, availability, complexity, professional skills, investment, cost & effort, and evaluation & empirical evidence. We found that most challenges were related to availability. Besides, we too classified challenges according to our hierarchy of model-based testing areas, and found that most challenges fell in the model specification area. With respect to tendencies in challenges, we found they have moved from complexity of the approaches to the lack of approaches for specific software domains. Conclusions: Only a few systematic reviews on model-based testing could be found, therefore some areas still lack secondary studies, particularly, test execution aspects, language types, model dynamics, as well as some modeling paradigms and generation methods. We thus encourage the community to perform further systematic reviews and mapping studies, following known protocols and reporting procedures, in order to increase the quality and quantity of empirical studies in model-based testing.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Threats To Validitymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Model-based testing areas, tools and challenges: A tertiary study

Villalobos-Arias

Quesada-López

Martinez

et al. 2019

CLEIej

View full text Add to dashboard Cite

show abstract

“…[14] and Mourão et al. [10] and for objectivity reasons, secondary studies examined in this paper are selected through a rigour systematic search process. Moreover, the proposed search strategy adapts and extends the one proposed by Mourão et al.…”

Section: Related Workmentioning

confidence: 99%

Searching relevant papers for software engineering secondary studies: Semantic Scholar coverage and identification role

Hannousse

2021

IET Software

View full text Add to dashboard Cite

Searching relevant papers is a fundamental task for the elaboration of secondary studies. This task is known to be tedious and time‐consuming when it is made manually, especially with the presence of several academic repositories. Recently, Semantic Scholar has emerged as a new artificial intelligence‐based search engine enabling a set of valuable features. The present study investigates the role of Semantic Scholar in retrieving relevant papers for performing secondary studies in software engineering. For this sake, an examination is performed to check the ability of Semantic Scholar to locate included papers in recent and well‐established secondary studies. Afterwards, a hybrid and automatic search strategy is introduced making use of Semantic Scholar as a sole search engine and it incorporates: automatic search, snowballing, and use of Computer Science Ontology (CSO) and Software Engineering Body of Knowledge (SWEBOK) for refining queries. The proposed strategy is validated by replicating the search of high‐quality secondary studies in the software engineering field. To guarantee objectivity, a systematic search is conducted of recent secondary studies published in the field since 2015. For the coverage test, Semantic Scholar is examined to locate primary papers of selected secondary studies and identify missing venues. The proposed search strategy is used to check the ability to retrieve primary papers of each secondary study. The systematic search yielded 20 high‐quality secondary studies with 1337 distinct primary papers. The coverage test revealed that Semantic Scholar covers 98.88% of the papers. The proposed search strategy enabled the full replication of 13 studies and more than 90% for the 7 remaining studies.

show abstract

“…This decision was made due to the sheer volume of publications retrieved from just one source. Scopus was nonetheless chosen since the previous systematic reviews have shown that the searches in this platform alone result in a far reaching number of papers that would otherwise just be duplicated papers if other databases such as IEEE Xplore, ACM and PubMed are being searched independently [67]- [69]. Furthermore, the mapping studies are extendable and other scientific databases could be considered provided that the other researchers have enough resources to do so.…”

Section: B Threat II -Credibility Of the Literature Search Processmentioning

confidence: 99%

Security and Privacy for mHealth and uHealth Systems: A Systematic Mapping Study

2020

View full text Add to dashboard Cite

An increased adoption of mobile health (mHealth) and ubiquitous health (uHealth) systems empower users with handheld devices and embedded sensors for a broad range of healthcare services. However, m/uHealth systems face significant challenges related to data security and privacy that must be addressed to increase the pervasiveness of such systems. This study aims to systematically identify, classify, compare, and evaluate state-of-the-art on security and privacy of m/uHealth systems. We conducted a systematic mapping study (SMS) based on 365 qualitatively selected studies to (i) classify the types, frequency, and demography of published research, (ii) synthesize and categorize research themes, (iii) recurring challenges, (iv) prominent solutions (i.e., research outcomes) and their (v) reported evaluations (i.e., practical validations). Results suggest that the existing research on security and privacy of m/uHealth systems primarily focuses on select group of control families (compliant with NIST800-53), protection of systems and information, access control, authentication, individual participation, and privacy authorisation. In contrast, areas of data governance, security and privacy policies, and program management are underrepresented, although these are critical to most of the organizations that employ m/uHealth systems. Most research proposes new solutions with limited validation, reflecting a lack of evaluation of security and privacy of m/uHealth in the real world. Empirical research, development, and validation of m/uHealth security and privacy is still incipient, which may discourage practitioners from readily adopting solutions from the literature. This SMS facilitates knowledge transfer, enabling researchers and practitioners to engineer security and privacy for emerging and next generation of m/uHealth systems.

show abstract

Investigating the Use of a Hybrid Search Strategy for Systematic Reviews

Cited by 46 publications

References 16 publications

Model-based testing areas, tools and challenges: A tertiary study

Model-based testing areas, tools and challenges: A tertiary study

Searching relevant papers for software engineering secondary studies: Semantic Scholar coverage and identification role

Security and Privacy for mHealth and uHealth Systems: A Systematic Mapping Study

Contact Info

Product

Resources

About