Augmented Testing: Industry Feedback To Shape a New Testing Technology

Nass, Michel; Alégroth, Emil; Feldt, Robert

doi:10.1109/icstw.2019.00048

Cited by 9 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper focused on refactoring code smells for mobile applications. Aggregation of the resource usage using manual testing which focused on Graphical User Interface (GUI) events only [22], [23]. Thus, our study used ObreusDroid which was designed and developed using bash scripting and Android Debug Bridge (ADB).…”

Section: Methodsmentioning

confidence: 99%

“…For several reasons, we made a test case, as shown in Table II, to collect data on resource usage, i.e., CPU and memory, through ObreusDroid with ADB. First, another research [21]- [23] uses the top commands to measure the CPU usage and memory usage; we believe the result is to evaluate each application. Second, we exported the result of resource usage to a CSV file.…”

Section: B Step 2: Manual Testing Applications With Code Smellsmentioning

confidence: 99%

See 1 more Smart Citation

An Empirical Evaluation on the Effect Refactoring Code Smells Mobile Applications Android with ASATs on Resource Usage

Syawanodya,

Anggraini,

Al-Hijri

et al. 2024

Int. J. Adv. Sci. Eng. Inf. Technol.

View full text Add to dashboard Cite

The Application is closely connected to mobile devices designed for many people and has maintenance, but even maintenance can contain violations such as code smells that effect non-functional requirements, specifically the use of CPU and memory resources. When the software has a rapid use of resources, it gives rise to the phenomenon that the user may switch or uninstall the software. The solution to this phenomenon is to explore resource-related code smells and fix them by refactoring. Developments to explore code smells came with ASATs, namely SonarQube, which is already being used by 85,000 organizations to speed up analyzing code in software. This topic is related to code smells, and the research objective is to analyze and compare the performance of the original versions and single or cumulative refactored versions of Android mobile software using the Design Research Methodology (DRM) approach. Types of code smells are represented based on the classification on SonarQube, namely Blocker, Critical, Major, and Minor, with code smells such as HashMap Usage, Member Ignoring Method, and Slow Loop. Aspects tested include Fixed Detection Ratio (FDR), improvement, CPU, and memory usage. Based on the results of the research, it shows the depreciation of code smells which is proven to significantly increase CPU performance in a single refactoring, namely Member Ignoring Method and Critical by 7.7% and 9.90%, respectively. Moreover, single refactoring offers developers advantages reducing high costs, diminished exertion, and truncated maintenance duration. However, the cumulative refactoring occasionally endeavors hold the potential be high improvements.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: B Step 2: Manual Testing Applications With Code Smellsmentioning

confidence: 99%

An Empirical Evaluation on the Effect Refactoring Code Smells Mobile Applications Android with ASATs on Resource Usage

Syawanodya,

Anggraini,

Al-Hijri

et al. 2024

Int. J. Adv. Sci. Eng. Inf. Technol.

View full text Add to dashboard Cite

show abstract

“…However, a novel technique to automated GUI-based testing is Augmented Testing (AT), where a visual layer between the tester and the SUT's GUI superimpose information on top of the GUI is provided [10]. In more detail, in an AT workfow, information about a GUI's widgets or bitmaps is gathered to augment widgets and bitmaps with actions, checks, defects, and suggestions to build a layer between the user and the SUT.…”

Section: A Augmented Testing With Scoutmentioning

confidence: 99%

We Tried and Failed: An Experience Report on a Collaborative Workflow for GUI-based Testing

Bauer

Alégroth

2023

2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

View full text Add to dashboard Cite

Modern software development is a team-based effort supported by tools, processes, and practices. One integral part is automated testing, where developers incorporate automated tests on multiple levels of system abstraction, from low-level unit tests to high-level system tests and Graphical User Interface (GUI) tests. Furthermore, the common practices of code reviews allow collaboration on artifacts based on discussions that improve the artifact's quality and to share information within the team. However, the characteristics of GUI-based tests, due to the level of abstraction and visual elements, introduce additional requirements and complexities compared to code or lower-level test code review, delimiting the practice benefts.The objective of this work is to propose a tool-supported workfow that enables active collaboration among stakeholders and improves the effciency and effectiveness of team-based development of GUI-based tests.To evaluate the workfow, and show proof of concept, a technical demonstrator for merging of GUI-based tests was to be developed. However, during its development, we encountered several unforeseen challenges that forced us to halt its development. We report the negative results from this development and the main challenges we encountered, as well as the rationale and the decisions we took towards this workfow.In conclusion, this work presents a negative research result on a failed attempt to propose a tool-supported workfow that enables active collaboration on GUI-based tests. The outcome and learnings of this work are intended to guide future research and prevent researchers from falling into the same pitfalls we did.

show abstract

“…To address these challenges, a novel technique that we call Augmented Testing (AT) [6], has been proposed as a possible solution that enhances and increases the communication speed between the human tester and the machine. The intention is to make it easier for the tester to create effective tests and more easily co-evolve them with the SUT and its requirements.…”

Section: Introductionmentioning

confidence: 99%

“…Previous work on AT has, however, not explored the technique's applicability in industrial practice, only it's perceived applicability [6]. To evaluate the actual applicability, this paper presents an industrial empirical study where Scout, a prototype tool for AT, is compared against two commonly used frameworks, in the industry, for GUI test automation.…”

Section: Introductionmentioning

confidence: 99%

On the Industrial Applicability of Augmented Testing: An Empirical Study

Nass

Alégroth

Feldt

2020

2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)

Self Cite

View full text Add to dashboard Cite

Testing applications with graphical user Interfaces (GUI) is an important but also a time-consuming task in practice. Tools and frameworks for GUI test automation can make the test execution more efficient and lower the manual labor required for regression testing. However, the test scripts used for automated GUI-based testing still require a substantial development effort and are often reported as sensitive to change, leading to frequent and costly maintenance. The efficiency of development, maintenance, and evolution of such tests are thereby dependent on the readability of scripts and the ease-of-use of test tools/frameworks in which the test scripts are defined. To address these shortcomings in existing state-of-practice techniques, a novel technique referred to as Augmented Testing (AT) has been proposed. AT is defined as testing the System Under Test (SUT) through an Augmented GUI that superimposes information on top of the SUT GUI. The Augmented GUI can provide the user with hints, test data, or other support while also observing and recording the tester's interactions. For this study, a prototype tool, called Scout, has been used that adheres to the AT concept that is evaluated in an industrial empirical study. In the evaluation, quasi-experiments and questionnaire surveys are performed in two workshops, with 12 practitioners from two Swedish companies (Ericsson and Inceptive). Results show that Scout can be used to create equivalent test cases faster, with statistical significance, than creating automated scripts in two popular state-of-practice tools. The study concludes that AT has cost-value benefits, applies to industrial-grade software, and overcomes several deficiencies of state-of-practice GUI testing technologies in terms of ease-of-use.

show abstract

Augmented Testing: Industry Feedback To Shape a New Testing Technology

Cited by 9 publications

References 10 publications

An Empirical Evaluation on the Effect Refactoring Code Smells Mobile Applications Android with ASATs on Resource Usage

An Empirical Evaluation on the Effect Refactoring Code Smells Mobile Applications Android with ASATs on Resource Usage

We Tried and Failed: An Experience Report on a Collaborative Workflow for GUI-based Testing

On the Industrial Applicability of Augmented Testing: An Empirical Study

Contact Info

Product

Resources

About