Machine Learning Solutions for Investigating Streams Data using Distributed Frameworks: Literature Review

Kumar, Kota Arun; Sharma, Neeraj; Ali, A. B. M. Shawkat

doi:10.1109/csde53843.2021.9718391

Cited by 4 publications

(1 citation statement)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Streaming data lets systems analyze data in real-time and make predictions. Streaming data are usually ingested in real-time but are processed in small amounts, which is sufficient to train machine learning models [46][47][48][49][50].…”

Section: Apache Sparkmentioning

confidence: 99%

An Effective Ensemble Learning-Based Real-Time Intrusion Detection Scheme for an In-Vehicle Network

Alalwany,

Mahgoub

2024

Electronics

View full text Add to dashboard Cite

The emergence of connected and autonomous vehicles has led to complex network architectures for electronic control unit (ECU) communication. The controller area network (CAN) enables the transmission of data inside vehicle networks. However, although it has low latency and enjoys data broadcast capability, it is vulnerable to attacks on security. The lack of effectiveness of conventional security mechanisms in addressing these vulnerabilities poses a danger to vehicle safety. This study presents an intrusion detection system (IDS) that accurately detects and classifies CAN bus attacks in real-time using ensemble techniques and the Kappa Architecture. The Kappa Architecture enables real-time attack detection, while ensemble learning combines multiple machine learning classifiers to enhance the accuracy of attack detection. The scheme utilizes ensemble methods with Kappa Architecture’s real-time data analysis to detect common CAN bus attacks. This study entails the development and evaluation of supervised models, which are further enhanced using ensemble techniques. The accuracy, precision, recall, and F1 score are used to measure the scheme’s effectiveness. The stacking ensemble technique outperformed individual supervised models and other ensembles with accuracy, precision, recall, and F1 of 0.985, 0.987, and 0.985, respectively.

show abstract

Section: Apache Sparkmentioning

confidence: 99%

An Effective Ensemble Learning-Based Real-Time Intrusion Detection Scheme for an In-Vehicle Network

Alalwany,

Mahgoub

2024

Electronics

View full text Add to dashboard Cite

show abstract

A Methodology for the Development of Soft Sensors with Kafka-ML

Chaves,

Martín,

Torres

et al. 2023

Internet of Things

View full text Add to dashboard Cite

Machine learning based data collection protocol for intelligent transport systems: a real-time implementation on Dublin M50, Ireland

Gillani,

Niaz

2023

Complex Intell. Syst.

View full text Add to dashboard Cite

The continuous global urbanization with rapid and dynamic transitioning in traffic situations among highly populated cities results in difficulty for data collection and communication. Data collection for millions of vehicles hinders by various problems, i.e., higher cost of energy, time, space, and storage resources. Moreover, higher data traffic results in higher delays, larger throughput, excessive bottlenecks, and frequent repetition of data. To better facilitate the aforementioned challenges and to provide a solution, we have proposed a lightweight Machine Learning based data collection protocol named ML-TDG to effectively deal with higher data volumes in a real-time traffic environment capable of bringing the least burden on the network while utilizing less space, time, and energy. ML-TDG is functional based on Apache Spark, an effective data processing engine that indexes the data based on two logs, i.e., old commuters or frequent/daily commuters and second new/occasional commuters. The proposed protocol’s main idea is to utilize real-time traffic, distinguish the indexes in parallel based on two assigned logs criteria to train the network, and collect data with the least sources. For energy and time optimization, dynamic segmentation switching is introduced which is an intelligent road segments division and switching for reducing bottlenecks and replication. ML-TDG is tested and verified on Dublin, Ireland’s busiest motorway M50. ML-TDG performs the data collection, data sorting, and network training to decide the next execution altogether for better optimization every time. The experimental results verify that our proposed protocol is attaining higher performance with lower resource requirements along with rich and time-efficient sustainable data collection clusters in comparison with baseline protocols.

show abstract

Machine Learning Solutions for Investigating Streams Data using Distributed Frameworks: Literature Review

Cited by 4 publications

References 31 publications

An Effective Ensemble Learning-Based Real-Time Intrusion Detection Scheme for an In-Vehicle Network

An Effective Ensemble Learning-Based Real-Time Intrusion Detection Scheme for an In-Vehicle Network

A Methodology for the Development of Soft Sensors with Kafka-ML

Machine learning based data collection protocol for intelligent transport systems: a real-time implementation on Dublin M50, Ireland

Contact Info

Product

Resources

About