Web Test Automation: Insights from the Grey Literature

Ricca, Filippo; Stocco, Andrea

doi:10.1007/978-3-030-67731-2_35

“…While there is extensive prior work on web bugs and web testing, there is limited prior work that investigates <canvas> bugs. As a discussion of related work on non-<canvas> web testing is outside the scope of our paper, we refer to the systematic literature review by Do gan et al [10] and the grey literature review by Ricca and Stocco [27] for an overview of related work about web testing.…”

Section: Related Workmentioning

confidence: 99%

A Taxonomy of HTML5 Canvas Bugs

Macklon¹,

Viggiato²,

Bezemer³

et al. 2022

Preprint

0

View full text Add to dashboard Cite

The HTML5

show abstract

“…This problem, much studied in the literature (e.g., Olianas et al and Morán et al 31,32 ), is a plague in particular for the testing of web and mobile applications. 33 Most flaky test bugs occur when the test code does not wait properly for asynchronous calls. 5,33 Some scientific studies (e.g., the paper by Nanda et al 34 ) report that static code analysis tools can detect a good percentage of bugs (estimated at 5-15%) in software.…”

Section: Related Workmentioning

confidence: 99%

“…33 Most flaky test bugs occur when the test code does not wait properly for asynchronous calls. 5,33 Some scientific studies (e.g., the paper by Nanda et al 34 ) report that static code analysis tools can detect a good percentage of bugs (estimated at 5-15%) in software. However, the same is not valid in the test code.…”

Section: Related Workmentioning

confidence: 99%

“…Instead, a test is flaky when its execution on the same application yields different outcomes due to environmental factors such as, for example, the network traffic. This problem, much studied in the literature (e.g., Olianas et al and Morán et al 31,32 ), is a plague in particular for the testing of web and mobile applications 33 . Most flaky test bugs occur when the test code does not wait properly for asynchronous calls 5,33 .…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Fight silent horror unit test methods by consulting a TestWizard

Cerioli

¹

,

Lagorio

²

,

Leotta

³

et al. 2021

J Software Evolu Process

Self Cite

2

0

View full text Add to dashboard Cite

Tests, when not correctly implemented, can pass on incorrect system implementations rather than fail. In this case, they are named silent horrors or false‐negative tests. They make releasing low‐quality (buggy) versions of the software system more probable. Furthermore, faithfully implementing test specifications is crucial when they play the role of documentation, like when documenting components or services or driving legacy systems' re‐engineering. This paper presents TestWizard, a novel approach and tool for automatically assessing individual tests' quality from the point of view of their coherence to specifications. TestWizard automatically assesses the quality of each individual test case w.r.t. its specification, providing detailed reports on why a single test is a false negative, hence helping testers fix them. Thus, TestWizard can help to automate the test code review process, which is still mainly manual today. The analysis of 1012 test implementations, developed by 123 students in three experiments, shows that TestWizard is (1) by far more accurate than code review performed by multiple students, (2) slightly better than code review performed by three senior experts, and (3) always able to detect a significant percentage of false‐negative test methods (up to 21.22%).

show abstract

“…Our reason to use the MLR approach is the industrial relevance of our topic. Based on prior scholars [15,16], GL (e.g., white papers, magazines, blog posts, wikis, and technical reports) is 'still an unexplored gold mine of guidelines for test automation', as practitioners exist in large numbers and they are sharing their experience matured on the field and propose test automation guidelines. Surveying the guidelines for improving test automation maturity from both AL and GL is critical to enable knowledge transfer between industry and research and identify the gaps.…”

mentioning

confidence: 99%

Improving test automation maturity: A multivocal literature review

Wang

¹

,

Mäntylä

²

,

Liu

³

et al. 2022

Software Testing Verif & Rel

View full text Add to dashboard Cite

Mature test automation is key for achieving software quality at speed. In this paper, we present a multivocal literature review with the objective to survey and synthesize the guidelines given in the literature for improving test automation maturity. We selected and reviewed 81 primary studies, consisting of 26 academic literature and 55 grey literature sources. From primary studies, we extracted 26 test automation best practices (e.g., Define an effective test automation strategy, Set up good test environments, Develop high-quality test scripts) and collected many pieces of advice (e.g., in forms of implementation/improvement approaches, technical techniques, concepts, experience-based heuristics) on how to conduct these best practices. We made main observations: (1) There are only 6 best practices whose positive effect on maturity improvement have been evaluated by academic studies using formal empirical methods; (2) Several technical related best practices in this MLR were not presented in test maturity models; (3) Some best practices can be linked to success factors and maturity impediments proposed by other scholars; (4) Most pieces of advice on how to conduct proposed best practices were identified from experience studies and their effectiveness need to be further evaluated with cross-site empirical evidence using formal empirical methods; (5) In the literature, some advice on how to conduct certain best practices are conflicting, and some advice on how to conduct certain best practices still need further qualitative analysis.

show abstract

Web Test Automation: Insights from the Grey Literature

Cited by 16 publications

References 30 publications

A Taxonomy of HTML5 Canvas Bugs

A Taxonomy of HTML5 Canvas Bugs

Fight silent horror unit test methods by consulting a TestWizard

Improving test automation maturity: A multivocal literature review

Contact Info

Product

Resources

About