Horizontally Scalable Web Crawler using Containerization and a Graphical user Interface

Prusty, Ansuman; Kancherlapalli, Pavan; Schiebel, Roland; Shah, Aditya Suresh Aniket

doi:10.17577/ijertv9is050268

Cited by 1 publication

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of these simulations demonstrate that the processes provide beneficial performance, which is the conclusion that can be drawn from the simulations. Prusty et al (2020) in reference [40] argue that because to the complex structure of the internet, it is inefficient for "single" crawlers to effectively crawl and index the content of the internet. This is because of the nature of the internet itself.…”

Section: [22]mentioning

confidence: 99%

“…This means that read-write and stored functions must be obtained before the actual python raw code is run. [40] They have come up with a fresh idea for a Crawler named Inference Crawler by semantic similarity, paradigmatic similarity, and rules of inference.…”

Section: Python Programming Languagementioning

confidence: 99%

“…This testbed is flexible and enables the modeling of the whole web, using either artificially constructed pages or a stored partial snapshot of the web. [40].…”

Section: Future Scope Of Web Crawlingmentioning

confidence: 99%

See 2 more Smart Citations

Distributed Fundamentals based Conducting the Web Crawling Approaches and Types (Focused, Incremental, Distributed, Parallel, Hidden Web, Form Focused and Breadth First) Crawlers

Mehyadin,

Abdulrahman,

Ahmed

et al. 2022

Journal of Smart Internet of Things

View full text Add to dashboard Cite

Over the last several years, there has been a significant rise in the number of people getting online and using the internet. Individual hypertext links are available, and any one of them may be used to get access to the resource. There is a variety of hypertext links available. It has been feasible to construct new websites as a result of the growth of crawlers, which has been facilitated by the rise in the number of people who use the internet. Web crawlers are highly evolved search engines that make it simpler for customers to get the information they are searching for on the internet. Web crawlers are also known as web crawlers. In a similar vein, these web crawlers have the potential to be used for more research endeavours in the months and years to come. Furthermore, the information that has been gathered may be used to detect and uncover any connections that are absent, as well as to assess the possibility for expansion inside complicated networks. This can be done by discovering any connections that are missing. The analysis of web crawlers is the primary topic of this study. Topics covered include the architecture of web crawlers, the many types of web crawlers, and the challenges that search engines have while using web crawlers.

show abstract

Section: [22]mentioning

confidence: 99%

Section: Python Programming Languagementioning

confidence: 99%