An Automated Word Embedding with Parameter Tuned Model for Web Crawling

Abstract

In recent years, web crawling has gained a significant attention due to the drastic advancements in the World Wide Web. Web Search Engines have the issue of retrieving massive quantity of web documents. One among the web crawlers is the focused crawler, that intends to selectively gather web pages from the Internet. But the efficiency of the focused crawling can easily be affected by the environment of web pages. In this view, this paper presents an Automated Word Embedding with Parameter Tuned Deep Learning (AWE-PTDL) model for focused web crawling. The proposed model involves different processes namely pre-processing, Incremental Skip-gram Model with Negative Sampling (ISGNS) based word embedding, bidirectional long short-term memory-based classification and bird swarm optimization based hyperparameter tuning. The SGNS training desires to go over the complete training data to pre-compute the noise distribution before performing Stochastic Gradient Descent (SGD) and the ISGNS technique is derived for the word embedding process. Besides, the cosine similarity is computed from the word embedding matrix to generate a feature vector which is fed as input into the Bidirectional Long Short-Term Memory (BiLSTM) for the prediction of website relevance. Finally, the Birds Swarm Optimization-Bidirectional Long Short-Term Memory (BSO-BiLSTM) based classification model is used to classify the webpages and the BSO algorithm is employed to determine the hyperparameters of the BiLSTM model so that the overall crawling performance can be considerably enhanced. For validating the enhanced outcome of the presented model, a comprehensive set of simulations are carried out and the results are examined in terms of different measures. The Automated Word Embedding with Parameter Tuned Deep Learning (AWE-PTDL) technique has attained a higher harvest rate of 85% when compared with the other techniques. The experimental results highlight the enhanced web crawling performance of the proposed model over the recent state of art web crawlers.

Improved Metaheuristic-Driven Energy-Aware Cluster-Based Routing Scheme for IoT-Assisted Wireless Sensor Networks

“…Kavitha, A. et al [11] proposed a cluster-based routing with simulated annealing and genetic algorithm-based hybrid (SAGA-H) method. The presented technique was simulated and explained by utilizing MATLAB.…”

LSGDM with Biogeography-Based Optimization (BBO) Model for Healthcare Applications

“…Similarly, the MR-LSDGM method classified 495 instances as Walk, 464 instances as Up, 416 instances as Down, 450 instances as Sit, 508 instances as Stand, and 536 instances as Lay under run-4. Furthermore, under run-5, the MR-LSDGM algorithm classified 495 instances as Walk, 464 instances as Up, 415 instances as Down, 453, Sit, 506, Stand, and 534 instances as Lay [ 26 ].…”

Capsule Network-Based Deep Transfer Learning Model for Face Recognition

“…The overall accuracy outcome analysis of the GWOECN-FR technique under four datasets is portrayed in Figure 4. The results demonstrated that the GWOECN-FR algorithm has accomplished improved validation accuracy compared to training accuracy [24][25][26][27][28][29][30]. It can be also observable that the accuracy values get saturated with the count of epochs.…”