A multi-screen refactoring system for video-centric web applications

Abstract

The ubiquity of web applications and the user possession and utilization of multiple devices are major factors for the increased demand for multi-screen applications. Multi-screen applications impose challenges on the application developer and designer especially if existing single-screen applications have to be transformed to the multiscreen environment. Designers should plan the user interface distribution and should adapt the layout for various devices. Developers should reorganize the application logic and associate it to the distributed user interface. They should preserve the application functionality and nally they need to adapt it to the underlying multiscreen platform. In this work, we propose an end-to-end refactoring system. The system allows the re-use of existing single-screen applications to automatically create multi-screen applications. The components of the multi-screen applications have their layout adapted to small and large device and they are ready to operate synchronously to provide a complementary usage experience. Our system is quantitatively evaluated on dierent sets of applications containing at least one video element and interactive content. The content division of our system corresponds to a ground truth division with an average recall of 0.84. In addition, our layout refactoring approach obtains 60% accuracy on the tested applications. In addition, we evaluate the performance of the run-time behavior of one application and we compute the delays that are caused by our system and by the network in a real physical environment: with a total delay of 5 ms, our solution is realistic.

A Methodology for User Interface Adaptation of Multi-Device Broadcast-Broadband Services

“…Webstrates [24] synchronises elements across devices operating on the level of the Document Object Model (DOM) while maintaining exact copies on different devices. Sarkis et al [25] allowed for the re-use of existing single-screen applications to automatically create synchronous multi-screen applications that analyse the code of the application and classifies different elements, such as HTML tags or event listeners. Other frameworks such as HuddleLamp [26] and Connichiwa [27] work with graphical aspects of UIs, for instance, when spanning one image across multiple screens.…”

“…[31] [32] [33] [34] [35], for current multi-device platforms that distribute the content among devices only under user interaction, e.g. [25] [36] and finally for multi-screen solutions in other fields such as industry [16] [37] or crisis management [38]. However, the definition of the process itself is not straightforward, which is why this paper proposes a methodology to adapt the UI of multi-device media services.…”

“…Every component in the multi-device experience is labelled with one of the aforementioned types. Then, some properties have been identified in such a way that every component can be evaluated in terms of them [42] [25], such as:…”

Challenges of Migrating Legacies Web to Mobile: A Systematic Literature Review

“…Sarkis et al "A multi-screen refactoring system for videocentric web applications" [83] Sistema de refactoring automático que crea Sistemas de Multi-Pantallas (SMP). Resuelve la caracterización del ambiente, la división y distribución de la UI, y el funcionamiento de la aplicación en múltiples dispositivos.…”

A Gals Approach for Pogramming Distributed Interactive Multimedia Applications

“…Devices running part of a distributed UI share data whose state must be synchronized. In literature, there are centralized [118,120,126] and decentralized [127,128,129] approaches for handling consistency in cross-device applications.…”