Reverse-engineering user interfaces to facilitateporting to and across mobile devices and platforms

Shah, Eeshan; Tilevich, Eli

doi:10.1145/2095050.2095093

Cited by 15 publications

(14 citation statements)

References 5 publications

(4 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the prevalence of GUI-centric software, there has been a large body of work dedicated to building advanced tools to aid in the construction of GUIs and related code [44], [45], [46], [47], [48], [49], [50] and to reverse engineer GUIs [51], [52], [53], [54], [55], [56]. While these approaches are aimed at various goals, they all attempt to reason logical, or programatic info from graphical representations of GUIs.…”

Section: Other Gui-design and Reverse Engineering Toolsmentioning

confidence: 99%

Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps

Moran

Bernal-Cárdenas

Curcio

et al. 2020

IIEEE Trans. Software Eng.

157

120

View full text Add to dashboard Cite

It is common practice for developers of user-facing software to transform a mock-up of a graphical user interface (GUI) into code. This process takes place both at an application's inception and in an evolutionary context as GUI changes keep pace with evolving features. Unfortunately, this practice is challenging and time-consuming. In this paper, we present an approach that automates this process by enabling accurate prototyping of GUIs via three tasks: detection, classification, and assembly. First, logical components of a GUI are detected from a mock-up artifact using either computer vision techniques or mock-up metadata. Then, software repository mining, automated dynamic analysis, and deep convolutional neural networks are utilized to accurately classify GUI-components into domain-specific types (e.g., toggle-button). Finally, a data-driven, K-nearest-neighbors algorithm generates a suitable hierarchical GUI structure from which a prototype application can be automatically assembled. We implemented this approach for Android in a system called REDRAW. Our evaluation illustrates that REDRAW achieves an average GUI-component classification accuracy of 91% and assembles prototype applications that closely mirror target mock-ups in terms of visual affinity while exhibiting reasonable code structure. Interviews with industrial practitioners illustrate ReDraw's potential to improve real development workflows.

show abstract

Section: Other Gui-design and Reverse Engineering Toolsmentioning

confidence: 99%

Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps

Moran

Bernal-Cárdenas

Curcio

et al. 2020

IIEEE Trans. Software Eng.

157

120

View full text Add to dashboard Cite

show abstract

“…Androider [28] is a tool to reverse-engineer application GUIs. Androider uses aspect-oriented programming techniques to extract a platform-independent GUI model, which can then be used to port GUIs across different platforms.…”

Section: R Wmentioning

confidence: 99%

Inferring likely mappings between APIs

Gokhale

Ganapathy

Padmanaban

2013

2013 35th International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Abstract-Software developers often need to port applications written for a source platform to a target platform. In doing so, a key task is to replace an application's use of methods from the source platform API with corresponding methods from the target platform API. However, this task is challenging because developers must manually identify mappings between methods in the source and target APIs, e.g., using API documentation.We develop a novel approach to the problem of inferring mappings between the APIs of a source and target platform. Our approach is tailored to the case where the source and target platform each have independently-developed applications that implement similar functionality. We observe that in building these applications, developers exercised knowledge of the corresponding APIs. We develop a technique to systematically harvest this knowledge and infer likely mappings between the APIs of the source and target platform. The output of our approach is a ranked list of target API methods or method sequences that likely map to each source API method or method sequence. We have implemented this approach in a prototype tool called Rosetta, and have applied it to infer likely mappings between the Java2 Platform Mobile Edition and Android graphics APIs. I. ISoftware developers often wish to make their applications available on a wide variety of computing platforms. The developer of a gaming app, for instance, may wish to make his app available on smart phones and tablets manufactured by various vendors, on desktops, and via the cloud. The key hurdle that he faces in doing so is to port his app to these software and hardware platforms.Why is porting software a difficult problem? Consider an example: suppose that we wish to port a Java2 Platform Mobile Edition (JavaME)-based game to an Android-powered device. Among other tasks, we must modify the game to use Android's API [10] (the target platform) instead of JavaME's API [21] (the source platform). Unfortunately, the process of identifying the API methods in the target platform that implement the same functionality corresponding to that of a source platform API method is cumbersome. We must manually examine the SDKs of the source and target APIs to determine the right method (or sequence of methods) to use. To add complexity, there could be multiple ways in which a source API method can be implemented using the target's API methods. For example, the fillRect() method in JavaME's graphics API, which fills a specified rectangle with color, can be implemented using either one of these two sequences of methods in Android's graphics API: setStyle();drawRect() or as moveTo();lineTo(); lineTo();lineTo();lineTo();drawPath() (we have omitted class names and the parameters to these method calls).One way to address this problem is to populate a database of mappings between the APIs of the source and target platforms. In this database, each source API method (or method sequence) is mapped to a target API method (or method sequence) that implements its f...

show abstract

“…Then it examines the runtime UI tree to detect the constituent interface components and the listeners attached to them. This step of the approach extends on our prior work [1]. The interception is accomplished by means of Aspect-Oriented Programming (AOP) (we use AspectJ [12]), and the UI tree is walked by means of reflection.…”

Section: A Preprocessingmentioning

confidence: 99%

“…Cloud Twin builds on our prior work in which we used aspect-oriented programming and reflection to reverseengineer UIs at runtime with the purpose of subsequently translating them to other platforms [1]. Cloud Twin employs the same strategy for extracting UI elements.…”

Section: Related Workmentioning

confidence: 99%

Cloud Twin: Native execution of android applications on the Windows Phone

Holder

Shah

Davoodi

et al. 2013

2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE)

Self Cite

View full text Add to dashboard Cite

Abstract-To successfully compete in the software marketplace, modern mobile applications must run on multiple competing platforms, such as Android, iOS, and Windows Phone. Companies producing mobile applications spend substantial amounts of time, effort, and money to port applications across platforms. Creating individual program versions for different platforms further exacerbates the maintenance burden. This paper presents Cloud Twin, a novel approach to natively executing the functionality of a mobile application written for another platform. The functionality is accessed by means of dynamic cross-platform replay, in which the source application's execution in the cloud is mimicked natively on the target platform. The reference implementation of Cloud Twin natively emulates the behavior of Android applications on a Windows Phone. Specifically, Cloud Twin transmits, via web sockets, the UI actions performed on the Windows Phone to the cloud server, which then mimics the received actions on the Android emulator. The UI updates on the emulator are efficiently captured by means of Aspect Oriented Programming and sent back to be replayed on the Windows Phone. Our case studies with third-party applications indicate that the Cloud Twin approach can become a viable solution to the heterogeneity of the mobile application market.

show abstract

Reverse-engineering user interfaces to facilitateporting to and across mobile devices and platforms

Cited by 15 publications

References 5 publications

Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps

Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps

Inferring likely mappings between APIs

Cloud Twin: Native execution of android applications on the Windows Phone

Contact Info

Product

Resources

About