AdapTV: A Model-Based Test Adaptation Approach for End-to-End User Interface Testing of Smart TVs

Abstract

We introduce a model-based feedback-driven test adaptation approach for end-to-end user interface testing of smart TVs. From the perspective of the TV software, the proposed approach is a nonintrusive and completely black-box approach, which operates by interpreting the screen images. Given a test suite, which is known to work in an older version of the TV, and a new version of the TV, to which the test suite should be adapted, the proposed approach first automatically discovers user interface models for both the older and the new version of TV by opportunistically crawling the TVs. Then, each test case in the test suite is executed on the old version, and the path traversed by the test case in the respective UI model is found. Finally, a semantically equivalent path in the UI model discovered for the new version of the TV is determined and dynamically executed on the new version in a feedback-driven manner. We empirically evaluate the proposed approach in a setup that closely mimics the industrial setup used by a large consumer electronics company. While the proposed approach successfully adapted all the test cases, the alternative approaches used in the experiments could not adapt any of them. INDEX TERMSConsumer electronics testing, model-based testing, smart TV testing, test adaptation