Online unsupervised state recognition in sensor data

Abstract

Abstract-Smart sensors, such as smart meters or smart phones, are nowadays ubiquitous. To be "smart", however, they need to process their input data with limited storage and computational resources. In this paper, we convert the stream of sensor data into a stream of symbols, and further, to higher level symbols in such a way that common analytical tasks such as anomaly detection, forecasting or state recognition, can still be carried out on the transformed data with almost no loss of accuracy, and using far fewer resources. We identify states of a monitored system and convert them into symbols (thus, reducing data size), while keeping "interesting" events, such as anomalies or transition between states, as it is. Our algorithm is able to find states of various length in an online and unsupervised way, which is crucial since behavior of the system is not known beforehand. We show the effectiveness of our approach using real-world datasets and various application scenarios.

Online Sensor Drift Compensation for E-Nose Systems Using Domain Adaptation and Extreme Learning Machine

“…proposed online detection for outliers in wireless sensor network to ensure the high quality of the data [ 32 ]. Between 2015 to 2017, researches related to online processing in sensors has expanded from decentralized model for resource limited environment [ 33 ] and time-efficient monitoring and detection [ 34 , 35 ] to more complicated tasks such as gesture recognition [ 36 ], source location [ 37 ] and fault diagnosis [ 38 ] in specific applications. Therefore, it is also imperative to combine advanced algorithms with online processing for gas sensor drift compensation.…”

Towards a Semantic Web of Things: A Hybrid Semantic Annotation, Extraction, and Reasoning Framework for Cyber-Physical System

“…The observed system runs and switches among certain states, and when the internal state of the observed system is changed, an event will be generated and published. The high-level state could be extracted by pattern discovery and prediction methods based on streaming mining algorithms, such as unsupervised cluster models [ 41 ], hidden Markov models (HMM) [ 24 ], or deep learning models [ 42 ] with temporal feature representations. In Figure 2 , the processing of raw streaming data into high-level states and events are represented with dotted lines, and the line does not represent the exact predicts/relations between the entities but only describes reference methods of how data streams could be transformed into states or events.…”

DIsCO: DynamIc Data COmpression in Distributed Stream Processing Systems

“…Our work differs in, that, we examine the problem in a distributed stream processing setting trying to maximize the observed system throughput. Authors in [8] propose the use of an unsupervised learning technique for detecting patterns in the incoming data flow and minimize the amount of emitted tuples by not transmitting tuples that will not contribute on the query's results. In our approach we decided to avoid such load shedding techniques to minimize the information loss.…”

“…Elasticity schemes like [9] and [19], automatically increase the amount of system resources (i.e., stream processing operators or components) in order to adapt to sudden load spikes. Such techniques have significant shortcomings: (a) often DSPSs do not support this feature so applications need to be stopped and re-uploaded in the cluster with the new configurations, and (b) approaches such as load shedding [16], [8], automatically drop incoming data when load spikes occur; this penalizes the results' accuracy as many tuples will not be processed and thus important events of interest may be lost.…”