Data preprocessing techniques: emergence and selection towards machine learning models - a practical review using HPA dataset

Rao, K. M. Ch. Appa; Saikrishna, Ghanta; Kundrapu, Supriya,

doi:10.1007/s11042-023-15087-5

Cited by 11 publications

(2 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When preprocessing data from a connected vehicle sensor system, classical steps like data cleaning, integrating data from multiple vehicles in a time-sorted manner, and normalizing parameters under consideration may suffice [104]. However, processing highway video monitoring data poses challenges in obtaining vehicle characteristics directly, which are crucial for many V2X ML applications.…”

Section: Methodology For Analyzing Video Stream Datamentioning

confidence: 99%

Machine Learning-Driven Calibration of Traffic Models Based on a Real-Time Video Analysis

Lopukhova,

Abdulnagimov,

Voronkov

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Accurate traffic simulation models play a crucial role in developing intelligent transport systems that offer timely traffic information to users and efficient traffic management. However, calibrating these models to represent real-world traffic conditions accurately poses a significant challenge due to the dynamic nature of traffic flow and the limitations of traditional calibration methods. This article introduces a machine learning-based approach to calibrate macroscopic traffic simulation models using real-time traffic video stream data. The proposed method for creating and calibrating a traffic simulation model has significantly improved the statistical correspondence between the generated vehicle characteristics and real data about cars on the simulated road section. The correspondence has increased from 37% to 73%. Machine learning models trained on generated data and tested on real data show improved accuracy rates. Mean absolute error, mean square error, and mean absolute percentage error decreased by more than two orders of magnitude. The coefficient of determination has also increased, approaching 1. This method eliminates the need to deploy wireless sensor networks, which can reduce the cost of implementing intelligent transport systems.

show abstract

Section: Methodology For Analyzing Video Stream Datamentioning

confidence: 99%

Machine Learning-Driven Calibration of Traffic Models Based on a Real-Time Video Analysis

Lopukhova,

Abdulnagimov,

Voronkov

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Feature selection techniques and machine learning classifiers cannot operate on such noisy datasets. Therefore, the raw data is first cleaned and preprocessed [26] to make it suitable for the operation of classifiers and feature selection techniques. Data preprocessing (data cleaning) is described by Algorithm 1, and this stage yields results in the elimination and reduction of noise in a dataset.…”

Section: Data Preprocessingmentioning

confidence: 99%

Enhanced PSO Algorithm for Detecting DRDoS Attacks on LDAP Servers

2023

IJIES

View full text Add to dashboard Cite

In recent years, there has been an increase in distributed reflective denial of service (DRDoS) attacks, particularly those that target open lightweight directory access protocol (LDAP) servers. These attacks involve transmitting a small request to a large number of available LDAP servers, seeking information from all users. Consequently, the servers respond with significantly more data than the original request, amplifying the traffic and overwhelming the target with massive amounts of data. Therefore, this paper proposes a novel model for detecting LDAP-based DRDoS attacks by utilizing an enhanced particle swarm optimization (PSO) algorithm based on an adaptive weighted threshold (AWTPSO) model. The proposed AWTPSO model incorporates network traffic features and LDAP protocol characteristics to identify attack patterns. It further employs an adaptive weighted threshold model to dynamically adjust the threshold value for each feature. The enhanced PSO algorithm optimizes the threshold values, thereby improving the detection accuracy of the proposed model. The proposed AWTPSO detection model has been validated using the recent CICDDoS2019 dataset (LDAP sub-dataset). The experimental results demonstrate that the AWTPSO model effectively detects LDAP-based DRDoS attacks with exceptional accuracy of 99.99% and minimal false positives of 0.01%, surpassing other state-of-the-art techniques. Consequently, the proposed model presents a highly promising and robust solution for detecting the threat of LDAP-based DRDoS attacks on enterprise networks.

show abstract