Hybrid monkey testing: enhancing automated GUI tests with random test generation

Wetzlmaier, Thomas; Ramler, Rudolf

doi:10.1145/3121245.3121247

Cited by 14 publications

(6 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are many research works on automated GUI testing [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16] by dynamically exploring different pages with random actions (e.g., clicking, scrolling, filling in the text) until triggering the crash bugs or explicit exceptions. Some practical automated testing tools like Monkey [17], [18], Dynodroid [19] are also widely used in industry. However, these automated tools can only spot critical crash bugs, rather than UI display issues which cannot be captured by the system.…”

Section: Fig 1: Examples Of Ui Display Issuesmentioning

confidence: 99%

Nighthawk: Fully Automated Localizing UI Display Issues via Visual Understanding

Liu

Chen

Wang

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Graphical User Interface (GUI) provides a visual bridge between a software application and end users, through which they can interact with each other. With the upgrading of mobile devices and the development of aesthetics, the visual effects of the GUI are more and more attracting, and users pay more attention to the accessibility and usability of applications. However, such GUI complexity posts a great challenge to the GUI implementation. According to our pilot study of crowdtesting bug reports, display issues such as text overlap, component occlusion, missing image always occur during GUI rendering on different devices due to the software or hardware compatibility. They negatively influence the app usability, resulting in poor user experience. To detect these issues, we propose a fully automated approach, Nighthawk, based on deep learning for modelling visual information of the GUI screenshot. Nighthawk can detect GUIs with display issues and also locate the detailed region of the issue in the given GUI for guiding developers to fix the bug. At the same time, training the model needs a large amount of labeled buggy screenshots, which requires considerable manual effort to prepare them. We therefore propose a heuristic-based training data auto-generation method to automatically generate the labeled training data. The evaluation demonstrates that our Nighthawk can achieve average 0.84 precision and 0.84 recall in detecting UI display issues, average 0.59 AP and 0.60 AR in localizing these issues. We also evaluate Nighthawk with popular Android apps on Google Play and F-Droid, and successfully uncover 151 previously-undetected UI display issues with 75 of them being confirmed or fixed so far.

show abstract

Section: Fig 1: Examples Of Ui Display Issuesmentioning

confidence: 99%

Nighthawk: Fully Automated Localizing UI Display Issues via Visual Understanding

Liu

Chen

Wang

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…There are many research works on automated GUI testing [3,17,20,24,39,42,44,45,48,49,60] by dynamically exploring different pages with random actions (e.g., clicking, scrolling, filling in the text) until triggering the crash bugs or explicit exceptions. Some practical automated testing tools like Monkey [21,65],…”

Section: Figure 1: Examples Of Ui Display Issuesmentioning

confidence: 99%

Discovering UI display issues with visual understanding

Liu

2020

Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering

View full text Add to dashboard Cite

Graphical User Interface (GUI) provides a visual bridge between a software application and end users, through which they can interact with each other. With the development of technology and aesthetics, the visual effects of the GUI are more and more attracting. However, such GUI complexity posts a great challenge to the GUI implementation. According to our pilot study of crowdtesting bug reports, display issues such as text overlap, blurred screen, missing image always occur during GUI rendering on different devices due to the software or hardware compatibility. They negatively influence the app usability, resulting in poor user experience. To detect these issues, we propose a novel approach, OwlEye, based on deep learning for modelling visual information of the GUI screenshot. Therefore, OwlEye can detect GUIs with display issues and also locate the detailed region of the issue in the given GUI for guiding developers to fix the bug. We manually construct a large-scale labelled dataset with 4,470 GUI screenshots with UI display issues and develop a heuristics-based data augmentation method for boosting the performance of our OwlEye. The evaluation demonstrates that our OwlEye can achieve 85% precision and 84% recall in detecting UI display issues, and 90% accuracy in localizing these issues. We also evaluate OwlEye with popular Android apps on Google Play and F-droid, and successfully uncover 57 previously-undetected UI display issues with 26 of them being confirmed or fixed so far.

show abstract

“…It also provides a biased strategy to take into consideration the relevancy of the events to the context. In another work [9], a hybrid technique which seeks to enhance an existing GUI regression test suite by injecting interactions randomly generated by the monkey. This is intended to improve bug-finding capability of the regression tests, since in the lack of the random interactions, the regression tests always execute the same sequences of steps, and hence, cannot challenge the newly added behaviors, but only confirm that the previous behaviors are not altered.…”

Section: Why Is It New?mentioning

confidence: 99%

Making Android apps monkey-friendly

Paydar

2020

Proceedings of the IEEE/ACM 7th International Conference on Mobile Software Engineering and Systems

View full text Add to dashboard Cite

Monkey testing is a random testing technique in which a stream of pseudo-random events are automatically fired on the GUI of the application under test, usually with the purpose of robustness testing or responsiveness analysis. A line of research is dedicated to addressing the limitations of monkey testing for Android apps. However, all the existing works try to improve the underlying algorithms or techniques used by the monkey testing tools. In this vision paper, we propose the idea of improving the effectiveness of monkey testing by automatically refactoring the application under test. We provide two sample scenarios in which this idea can be used to address limitations of monkey testing for Android applications. CCS CONCEPTS • Software and its engineering → Software verification and validation.

show abstract

Hybrid monkey testing: enhancing automated GUI tests with random test generation

Cited by 14 publications

References 5 publications

Nighthawk: Fully Automated Localizing UI Display Issues via Visual Understanding

Nighthawk: Fully Automated Localizing UI Display Issues via Visual Understanding

Discovering UI display issues with visual understanding

Making Android apps monkey-friendly

Contact Info

Product

Resources

About