Combinatorial methods for testing Internet of Things smart home systems

Abstract

In this paper, we report on applying combinatorial testing to Internet of Things (IoT) home automation hub systems. We detail how to create a dedicated input parameter model of an IoT home automation hub system for use with combinatorial test case generation strategies. Further, we developed an automated test execution framework and two test oracles for evaluation purposes. We applied and evaluated our proposed methodological approach to a real-world IoT system and analysed the obtained results of various combinatorial test sets with different properties generated based on the derived input model. Additionally, we compare these results to a random testing approach. Our empirical testing evaluations revealed multiple errors in the tested devices and also showed that all considered approaches performed nearly equally well. K E Y W O R D S combinatorial testing, IoT, software testingThis includes, but is not limited to, the Internet.

Testing Real-World Healthcare IoT Application: Experiences and Lessons Learned

“…Testing IoT Applications. Several studies target different aspects of testing IoT applications [16] such as model-based conformance testing of IoT [6], testing heterogeneity of IoT devices in loop [8], identifying faults in devices integration with IoT application [41], combinatorial testing and coverage criteria for IoT systems [25], combinatorial testing for IoT-based smart home applications [21], and simulation of health monitoring activities of healthcare IoT applications [39]. Our work focuses on testing REST APIs of a real-world healthcare IoT application using one of the latest REST APIs testing approaches.…”

Implementation of a Hybrid Intelligence System Enabling the Effectiveness Assessment of Interaction Channels Use in HMI

“…With the progress of Artificial Intelligence (AI), as well as NLP (Natural Language Processing), which is clearly seen in modern conversational {systems [6,7], the man-machine collaboration becomes an urgent and promising trend in modern industry. Nevertheless, despite recent significant advancements [8][9][10][11] in the field of Human-Computer Interaction (HCI) (namely: the Amazon Echo available since 2015 [12][13][14], Microsoft Cortana massively expanding in 2015 to numerous platforms [15], Google Speech announced in 2016, upgraded in 2018 [16] and 2022 [17], Google Assistant announced in 2016 [18,19], Google Nest introduced as Google Home also in 2016 [20][21][22][23][24], and the Apple Siri updated in 2017 [25,26]) and undoubtful progress in Artificial Intelligence and Machine Learning [27][28][29][30], conversational AI systems usually still tend to disappoint rather than to amaze their interlocutors (and there are some specific reasons for that, already diagnosed and described [31]). In the case of industrial solutions, an appropriate sensor network with insight into every aspect of the man-machine communication with regard to its purpose is required (allowing later to provide insight into the quality and efficiency of every stage of the process for a given worker/trainee).…”