StoryDroid: Automated Generation of Storyboard for Android Apps

ACM Trans. Softw. Eng. Methodol.

Xing

et al. 2020

Self Cite

UI design is an integral part of software development. For many developers who do not have much UI design experience, exposing them to a large database of real-application UI designs can help them quickly build up a realistic understanding of the design space for a software feature and get design inspirations from existing applications. However, existing keyword-based, image-similarity-based, and component-matching-based methods cannot reliably find relevant high-fidelity UI designs in a large database alike to the UI wireframe that the developers sketch, in face of the great variations in UI designs. In this article, we propose a deep-learning-based UI design search engine to fill in the gap. The key innovation of our search engine is to train a wireframe image autoencoder using a large database of real-application UI designs, without the need for labeling relevant UI designs. We implement our approach for Android UI design search, and conduct extensive experiments with artificially created relevant UI designs and human evaluation of UI design search results. Our experiments confirm the superior performance of our search engine over existing image-similarity or component-matching-based methods and demonstrate the usefulness of our search engine in real-world UI design tasks.

Section: Related Workmentioning

confidence: 99%

Wireframe-based UI Design Search through Image Autoencoder

ACM Trans. Softw. Eng. Methodol.

Xing

et al. 2020

Self Cite

“…Swearngin et al [72] adopt the image processing method to help designs with converting the mobile UI screenshots into editable files in Photoshop, so that designers can take it as a starting point for further customization. To render inspirations to the designer, Chen et al [27] propose a program-analysis method to efficiently generate the storyboard with UI screenshots, given one app executable file. Fischer et al [34] transfer the style from fine art to GUI.…”

Section: Ui Designmentioning

confidence: 99%

From Lost to Found

Proc. ACM Hum.-Comput. Interact.

Feng

Liu

et al. 2020

Self Cite

Design sharing sites provide UI designers with a platform to share their works and also an opportunity to get inspiration from others' designs. To facilitate management and search of millions of UI design images, many design sharing sites adopt collaborative tagging systems by distributing the work of categorization to the community. However, designers often do not know how to properly tag one design image with compact textual description, resulting in unclear, incomplete, and inconsistent tags for uploaded examples which impede retrieval, according to our empirical study and interview with four professional designers. Based on a deep neural network, we introduce a novel approach for encoding both the visual and textual information to recover the missing tags for existing UI examples so that they can be more easily found by text queries. We achieve 82.72% accuracy in the tag prediction. Through a simulation test of 5 queries, our system on average returns hundreds more results than the default Dribbble search, leading to better relatedness, diversity and satisfaction. CCS Concepts: • Human-centered computing → User interface design; User interface management systems; Graphical user interfaces.

“…Fig. 2 is consist of three violin plots, 4 representing our statistics on app names, package names and package sizes, respectively. In each "violin", the white dot represents the median, the thick bar in the middle represents the interquartile range while the thin bar represents 95% confidence interval.…”

Section: A Fake Sample Characteristicsmentioning

confidence: 99%

A Large-Scale Empirical Study on Industrial Fake Apps

Tang

2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)

Fan

et al. 2019

Self Cite

While there have been various studies towards Android apps and their development, there is limited discussion of the broader class of apps that fall in the fake area. Fake apps and their development are distinct from official apps and belong to the mobile underground industry. Due to the lack of knowledge of the mobile underground industry, fake apps, their ecosystem and nature still remain in mystery.To fill the blank, we conduct the first systematic and comprehensive empirical study on a large-scale set of fake apps. Over 150,000 samples related to the top 50 popular apps are collected for extensive measurement. In this paper, we present discoveries from three different perspectives, namely fake sample characteristics, quantitative study on fake samples and fake authors' developing trend. Moreover, valuable domain knowledge, like fake apps' naming tendency and fake developers' evasive strategies, is then presented and confirmed with case studies, demonstrating a clear vision of fake apps and their ecosystem.