We Don’t Need No Real Users?! Surveying the Adoption of User-less Automation Tools by UI Design Practitioners

Bakaev, Maxim; Speicher, Maximilian; Jagow, Johanna; Heil, Sebastian; Gaedke, Martin

doi:10.1007/978-3-031-09917-5_28

Cited by 3 publications

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Improved UI design enhances user satisfaction and contributes to increased retention and revenue [2]. Fundamental principles in UI design involve exploring solutions, identifying specific attributes, and actively engaging users in the design process [3]. Despite these principles, the manual testing of UIs remains cumbersome due to the hands-on verification of requirements [4].…”

Section: Introductionmentioning

confidence: 99%

Enhancing Software User Interface Testing Through Few Shot Deep Learning: A Novel Approach for Automated Accuracy and Usability Evaluation

Widodo,

Wibowo,

Kurniawan

2023

IJACSA

View full text Add to dashboard Cite

Traditional user interface (UI) testing methods in software development are time-consuming and prone to human error, requiring more efficient and accurate approaches. Moreover, deep learning requires extensive data training to develop accurate automated UI software testing. This paper proposes an efficient and accurate method for automating UI software testing using Deep learning with training data limitations. We propose a novel deep learning-based framework suitable for UI element analysis in data-scarce situations, focusing on Few-shot learning. Our framework initiates with several robust feature extraction modules that employ and compare sophisticated encoder models to be adept at capturing complex patterns from a sparse dataset. The methodology employs the Enrico and UI screen mistake datasets, overcoming training data limitations. Utilizing encoder models, including CNN, VGG-16, ResNet-50, MobileNet-V3, and EfficientNet-B1, the EfficientNet-B1 model excelled in the setting of Few-Shot learning with five-shot with an average accuracy of 76.05%. Our proposed model's accuracy was improved and compared to the state-of-the-art method. Our findings demonstrate the effectiveness of few-shot learning in UI screen classification, setting new benchmarks in software testing and usability evaluation, particularly in limited data scenarios.

show abstract

Section: Introductionmentioning

confidence: 99%

Enhancing Software User Interface Testing Through Few Shot Deep Learning: A Novel Approach for Automated Accuracy and Usability Evaluation

Widodo,

Wibowo,

Kurniawan

2023

IJACSA

View full text Add to dashboard Cite

show abstract

A Taxonomy of User Behavior Model (UBM) Tools for UI Design and User Research

Bakaev

Heil

Jagow³

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A Reasonable Effectiveness of Features in Modeling Visual Perception of User Interfaces

Bakaev

Heil

Gaedke

2023

BDCC

View full text Add to dashboard Cite

Training data for user behavior models that predict subjective dimensions of visual perception are often too scarce for deep learning methods to be applicable. With the typical datasets in HCI limited to thousands or even hundreds of records, feature-based approaches are still widely used in visual analysis of graphical user interfaces (UIs). In our paper, we benchmarked the predictive accuracy of the two types of neural network (NN) models, and explored the effects of the number of features, and the dataset volume. To this end, we used two datasets that comprised over 4000 webpage screenshots, assessed by 233 subjects per the subjective dimensions of Complexity, Aesthetics and Orderliness. With the experimental data, we constructed and trained 1908 models. The feature-based NNs demonstrated 16.2%-better mean squared error (MSE) than the convolutional NNs (a modified GoogLeNet architecture); however, the CNNs’ accuracy improved with the larger dataset volume, whereas the ANNs’ did not: therefore, provided that the effect of more data on the models’ error improvement is linear, the CNNs should become superior at dataset sizes over 3000 UIs. Unexpectedly, adding more features to the NN models caused the MSE to somehow increase by 1.23%: although the difference was not significant, this confirmed the importance of careful feature engineering.

show abstract

We Don’t Need No Real Users?! Surveying the Adoption of User-less Automation Tools by UI Design Practitioners

Cited by 3 publications

References 12 publications

Enhancing Software User Interface Testing Through Few Shot Deep Learning: A Novel Approach for Automated Accuracy and Usability Evaluation

Enhancing Software User Interface Testing Through Few Shot Deep Learning: A Novel Approach for Automated Accuracy and Usability Evaluation

A Taxonomy of User Behavior Model (UBM) Tools for UI Design and User Research

A Reasonable Effectiveness of Features in Modeling Visual Perception of User Interfaces

Contact Info

Product

Resources

About