The identification of interface elements is the first step in mobile application automated testing and the key to smooth testing. However, existing object detection algorithms have a low accuracy rate, and some tiny elements are missed in the recognition of graphical user interface (GUI) elements. To address this limitation, this paper proposes the YOLOv5-MGC algorithm, a robot vision-based interface element recognition algorithm for mobile applications. The algorithm improves the network by using K-means++ algorithm for target anchor box generation, applying the attention mechanism, adding a microscale detection layer, and introducing the Ghost bottleneck module. The proposed approach enhances the recognition accuracy of the elements through the target anchor box and attention mechanism. Moreover, it enhances the network’s ability to detect tiny elements, which improves the shortcomings of the current target detection algorithm and is conducive to further promoting mobile application robot testing and enhancing robot testing automation. Experimental results show that the YOLOv5-MGC algorithm is superior to the YOLOv5 for object detection in the recognition of GUI elements, with the mean average precision (mAP_0.5) reaching 89.8% and the recognition precision reaching 80.8%.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.