Survey of real-time processing systems for big data

Liu, Xiufeng; Iftikhar, Nadeem; Xie, Xike

doi:10.1145/2628194.2628251

Cited by 110 publications

(74 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…and manageability of the system. [1]. Flume is composed of the source, channel, and sink as above figure.…”

Section: A Apache Flumementioning

confidence: 99%

Motion Prediction in Rock Scissor Paper Game based on Machine Learning

Ryeol¹,

Kim²,

Yeol³

et al. 2017

Sixth International Conference on Advances in Computing, Control and Networking - ACCN 2017

View full text Add to dashboard Cite

Abstract-The main aim of this system is to predict and analyze gesture pattern from a user based on machine learning. The system is adopted in the rock-paper-scissors game which is suggested as always win at the game by predicting user gesture. Quantization and processing of user gesture from EMG sensor are implemented to generate training data in order that disciplining machine. As suggested procedure, an enormous amount of user gesture data will be collected and training model will be implemented with machine learning. By adopting the implemented system into the game, the research will verify that it is feasible to predict user gesture during playing game. The manner of a game is that computer shows the result when the user starts the rock-paper-scissors game in front of the monitor and the system always shows winning result that is the main purpose of it.

show abstract

“…and manageability of the system. [1]. Flume is composed of the source, channel, and sink as above figure.…”

Section: A Apache Flumementioning

confidence: 99%

Motion Prediction in Rock Scissor Paper Game based on Machine Learning

Ryeol¹,

Kim²,

Yeol³

et al. 2017

Sixth International Conference on Advances in Computing, Control and Networking - ACCN 2017

View full text Add to dashboard Cite

show abstract

“…Apache flume is a framework for data collection, which can collect various log data and transmit data in real time [3]. Apache flume is designed with the core of reliability, scalability and manageability of the system.…”

Section: A Apache Flumementioning

confidence: 99%

A Noble Gestures Control System Using Machine Learning

Nam¹,

Ryeol²,

Kim³

et al. 2017

Sixth International Conference on Advances in Computing, Control and Networking - ACCN 2017

View full text Add to dashboard Cite

Abstract-In this paper, we propose a method to improve the recognition rate of gestures motion and scalability problem which can occur in gestures when operate drone using machine learning. For these purposes, the gestures data transmitted from the drone controller are used for machine learning on real time, and a new learning model is periodically created using the gestures data stored in the HDFS(Hadoop Distributed File System). The goal of our proposed system is to increase the recognition rate of the gestures motion when new learning model is created. In addition, it is possible to expect enhanced scalability through recognition of gestures motion, and that drone is able to recognize a new gestures motion which is not defined in the server.

show abstract

“…Besides, the proposed approach can handle scalable data sets, and consumption pattern changing owing to its use of the PARX model. processing systems, and discuss the potential technologies used for lambda architecture [20]. Cheng et al propose a smart city data platform that supports both batch and real-time data processings [7], and they suggest that anomaly detection should be implemented as the chief component of any platform for processing sensor data.…”

Section: Related Workmentioning

confidence: 99%

“…Martnez-Prieto et al adapts the lambda architecture in semantic data processing [26]; Liu et al applies it to smart grid complex event processing (CEP) [19]; Villari et al proposes AllJoyn Lambda, the platform for managing embedded devices of smart homes [32]; and Hasani et al use it for real-time big data analytics [12]. Besides, the works [5,20] both give an extensive review of the technologies of the lambda architecture. Although there are various use cases of the lambda architecture, we focus on its use in the particular use case, anomalous energy consumption detection.…”

Section: Related Workmentioning

confidence: 99%

Online Anomaly Energy Consumption Detection Using Lambda Architecture

Liu

Iftikhar

Nielsen

et al. 2016

Big Data Analytics and Knowledge Discovery

Self Cite

View full text Add to dashboard Cite

Abstract. With the widely use of smart meters in the energy sector, anomaly detection becomes a crucial mean to study the unusual consumption behaviors of customers, and to discover unexpected events of using energy promptly. Detecting consumption anomalies is, essentially, a real-time big data analytics problem, which does data mining on a large amount of parallel data streams from smart meters. In this paper, we propose a supervised learning and statistical-based anomaly detection method, and implement a Lambda system using the in-memory distributed computing framework, Spark and its extension Spark Streaming. The system supports not only iterative refreshing the detection models from scalable data sets, but also real-time anomaly detection on scalable live data streams. This paper empirically evaluates the system and the detection algorithm, and the results show the effectiveness and the scalability of the lambda detection system.

show abstract

Survey of real-time processing systems for big data

Cited by 110 publications

References 15 publications

Motion Prediction in Rock Scissor Paper Game based on Machine Learning

Motion Prediction in Rock Scissor Paper Game based on Machine Learning

A Noble Gestures Control System Using Machine Learning

Online Anomaly Energy Consumption Detection Using Lambda Architecture

Contact Info

Product

Resources

About