An Enhanced Semantic Focused Web Crawler Based on Hybrid String Matching Algorithm

Abstract

Topic precise crawler is a special purpose web crawler, which downloads appropriate web pages analogous to a particular topic by measuring cosine similarity or semantic similarity score. The cosine based similarity measure displays inaccurate relevance score, if topic term does not directly occur in the web page. The semantic-based similarity measure provides the precise relevance score, even if the synonyms of the given topic occur in the web page. The unavailability of the topic in the ontology produces inaccurate relevance score by the semantic focused crawlers. This paper overcomes these glitches with a hybrid string-matching algorithm by combining the semantic similarity-based measure with the probabilistic similarity-based measure. The experimental results revealed that this algorithm increased the efficiency of the focused web crawlers and achieved better Harvest Rate (HR), Precision (P) and Irrelevance Ratio (IR) than the existing web focused crawlers achieve.

Analysis of Chinese Image Discourse Based on Crawler Algorithms

“…Combined with the hot events of Chinese people during the epidemic prevention and control period, seven of the top 20 events were related to the image of "noncooperators," and "Australian-Chinese woman who returns to Beijing and refuses to go out for a run in isolation" ranked first among all events with 77,499 messages. e "Australian-Chinese woman refusing to run outside in isolation" ranked first with 77,499 messages [12]. It is clear that the negative image of Chinese people has a profound impact.…”

Seed URL Selection and Performance Analysis in Web Crawlers: A Comprehensive Review

“…Hasat Oranı Kesinlik/ Hassasiyet(%) Geri Çağırma Oranı Taranan Sayfa Sayısı [43] 0.389 36.00 0.611 5000 [44] 0.411 ------1000 [45] 0.810 ------5000 [46] 0.850 ------5000 [47] 0.500 ---0.600 6500 [48] ---92.01 0.590 495 [49] 0.890 ------1000 [50] 0.500 32.00 ---3200 [51] 0.830 ------1200 [26] ---92.00 0.300 2000 [25] 0.750 ---0.400 10000 [52] ------0.820 400 [53] 0.850 ------6000 [54] 0.700 ---0.470 13377 Tablo 2' de görüldüğü gibi odaklı web tarayıcılarında en çok kullanılan performans ölçütü hasat oranıdır. Hasat oranı taranan sayfa sayıları arttıkça genel olarak artmaktadır.…”