Design and Implementation of the Sense Egypt Platform for Real-Time Analysis of IoT Data Streams

Abstract

Nowadays, we experience an abundance of Internet of Things middleware solutions that make the sensors and the actuators are able to connect to the Internet. These solutions, referred to as platforms to gain a widespread adoption, have to meet the expectations of different players in the IoT ecosystem, including devices [1]. Low cost devices are easily able to connect wirelessly to the Internet, from handhelds to coffee machines, also known as Internet of Things (IoT). This research describes the methodology and the development process of creating an IoT platform. This paper also presents the architecture and implementation for the IoT platform. The goal of this research is to develop an analytics engine which can gather sensor data from different devices and provide the ability to gain meaningful information from IoT data and act on it using machine learning algorithms. The proposed system is introducing the use of a messaging system to improve the overall system performance as well as provide easy scalability.

A Cloud-based Analytics-Platform for User-centric Internet of Things Domains – Prototype and Performance Evaluation

“…The key requirements for IoT analytics architectures are the ability to handle big data problems in terms of different 'v's such as velocity, variety and volume [3,4,5,6,7] in real-time [3,4,6,7,8,13]. While the precise definition of real-time computing is rather subjective and varies depending on the use case [16], we found that in terms of IoT analytics it is closely linked to the value of the information to be derived from data processing.…”

“…Looking at smart home environments, different use cases such as home security as well as disaster detection and prevention require at least low latency [17]. In this regard, time criticalness of analytics scenarios and the need for low latency of data processing in IoT environments was also frequently named in previous research [3,4,6,7]. Furthermore, the integration of the data from a multitude of sources must be possible [3,7,18].…”

“…In this regard, time criticalness of analytics scenarios and the need for low latency of data processing in IoT environments was also frequently named in previous research [3,4,6,7]. Furthermore, the integration of the data from a multitude of sources must be possible [3,7,18]. This includes the integration of historical and real-time data [6,18].…”

“…Data transmission and processing must be secure [3,4,6] and the privacy of users concerning their data has to be considered at all times [3,4,8]. Moreover, the data processing capabilities must be scalable [9,10,11,12] and handle input from a multitude of sources [3,6,7]. This input data may arrive asynchronously, e.g.…”

“…In addition, all ingested data as well as the analytics results need to be stored [15,18] and made available for other applications [6,18]. Furthermore, analytics architectures for IoT use cases need to offer the tools to visualize data [7]. Ultimately, the ability to flexibly extend and modify the data processing capabilities of an IoT analytics architecture is important [4,6,9].…”