Automated Software Testing Based on Hierarchical State Transition Matrix for Smart TV

Macík

IEEE Trans. Consumer Electron.

et al. 2020

As the popularity of Smart Televisions (TVs) and interactive Smart TV applications (apps) has recently grown, the usability of these apps has become an important quality characteristic. Previous studies examined Smart TV apps from a usability perspective. However, these methods are mainly manual, and the potential of automated model-based testing methods for usability testing purposes has not yet been fully explored. In this paper, we propose an approach to test the usability of Smart TV apps based on the automated generation of a Smart TV user interaction model from an existing app by a specialized automated crawler. By means of this model, defined user tasks in the Smart TV app can be evaluated automatically in terms of their feasibility and estimated user effort, which reflects the usability of the analyzed app. This analysis can be applied in the context of regular users and users with various specific needs. The findings from this model-based automated analysis approach can be used to optimize the user interface of a Smart TV app to increase its usability, accessibility, and quality.

Section: E Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Testing the Usability and Accessibility of Smart TV Applications Using an Automated Model-Based Approach

Macík

IEEE Trans. Consumer Electron.

et al. 2020

“…However, this type of model is useful when model optimization and reduction are needed. In fact, Cui et al [28] used a white-box crawler approach to scan the code of an app to construct a preliminary model. This initial model contains many obsolete nodes because the crawler detects all views present in the UI code.…”

Section: Background and Related Workmentioning

confidence: 99%

“…However, this could be a time-consuming task and may lead to deviation from the primary research focus while also adding overhead to the model construction process. In contrast to the contribution of Cui et al [28], our approach does not scan the app code. Our EvoCreeper strategy instead explores the UI of a smart TV app by examining each element and observing its reaction.…”

Section: Background and Related Workmentioning

confidence: 99%

EvoCreeper: Automated Black-Box Model Generation for Smart TV Applications

IEEE Trans. Consumer Electron.

2019

Smart TVs are coming to dominate the television market. This accompanied by an increase in the use of the smart TV applications (apps). Due to the increasing demand, developers need modeling techniques to analyze these apps and assess their comprehensiveness, completeness, and quality. In this paper, we present an automated strategy for generating models of smart TV apps based on a black-box reverse engineering. The strategy can be used to cumulatively construct a model for a given app by exploring the user interface in a manner consistent with the use of a remote control device and extracting the runtime information. The strategy is based on capturing the states of the user interface to create a model during runtime without any knowledge of the internal structure of the app. We have implemented our strategy in a tool called EvoCreeper. The evaluation results show that our strategy can automatically generate unique states and a comprehensive model that represents the real user interactions with an app using a remote control device. The models thus generated can be used to assess the quality and completeness of smart TV apps in various contexts, such as the control of other consumer electronics in smart houses.

“…For example, in the mobile app, the distance between two icons (states) does not make sense in the transition, while this is very important in the smart TV application, and that will lead to a different model. An important effort to formulate this model is done recently by Cui et al [9]. Here, the Hierarchical State Transition Matrix (HSTM) is proposed as a model for the Android smart TV applications.…”

Section: Motivation and Literaturementioning

confidence: 99%

Testing of Smart TV Applications: Key Ingredients, Challenges and Proposed Solutions

Advances in Intelligent Systems and Computing

2018

Smart TV applications are software applications that have been designed to run on smart TVs which are televisions with integrated Internet features. Nowadays, the smart TVs are going to dominate the television market, and the number of connected TVs is growing exponentially. This growth is accompanied by the increase of consumers and the use of smart TV applications that drive these devices. Due to the increasing demand for smart TV applications especially with the rise of the Internet of Things (IoT) services, it is essential to building an application with a certain level of quality. Despite the analogy between the smart TV and mobile apps, testing smart TV applications is different in many aspects due to the different nature of user interaction and development environment. To develop the field and formulate the concepts of smart TV application testing, this paper aims to provide the essential ingredients, solutions, answers to the most critical questions, and open problems. In addition, we offer initial results and proof of concepts for a creeper algorithm to detect essential views of the applications. This paper serves as an effort to report the key ingredients and challenges of the smart TV application testing systematically to the research community.