HAIF: A Hierarchical Attention-Based Model of Filtering Invalid Webpage

Zhou, Chaoran; Zhao, Jianping; Tai, Miaomiao; Zhou, Xin

doi:10.1587/transinf.2020ntp0001

Cited by 2 publications

(2 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We used a convolutional neural network (CNN), the state-of-the-art tool in computer vision, to detect error Web pages and then separate them into a dedicated folder, all automatically. In this sense, Web pages that do not contain useful information are called "ERROR" Web pages, whereas Web pages that contain valuable information are called "VALID" Web pages [5]. Here, we present an automatic detection of error Web pages based exclusively on their webshots.…”

Section: Use Case 2: Automatic Recognition Of Error Web Pagesmentioning

confidence: 99%

“…Te efective management of such a quantity and variety of information is an increasingly difcult task for traditional techniques. For example, organizing valid content and fltering invalid content (purifying the Internet) are challenges facing the current Web [5]. Te immense presence on the Internet of error Web pages (e.g., under construction, maintenance, domain ofer, suspended account, page not found, browser incompatibility, virus, phishing, or service failure), which continue to be indexed and returned by search engines, afecting webmasters and users in general.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Large Visual, Qualitative, and Quantitative Dataset for Web Intelligence Applications

Mejia-Escobar,

Cazorla,

Martinez-Martin

2023

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The Web is the communication platform and source of information par excellence. The volume and complexity of its content have grown enormously, with organizing, retrieving, and cleaning Web information becoming a challenge for traditional techniques. Web intelligence is a novel research area to improve Web-based services and applications using artificial intelligence and automatic learning algorithms, for which a large amount of Web-related data are essential. Current datasets are, however, limited and do not combine visual representation and attributes of Web pages. Our work provides a large dataset of 49,438 Web pages, composed of webshots, along with qualitative and quantitative attributes. This dataset covers all the countries in the world and a wide range of topics, such as art, entertainment, economics, business, education, government, news, media, science, and the environment, addressing different cultural characteristics and varied design preferences. We use this dataset to develop three Web Intelligence applications: knowledge extraction on Web design using statistical analysis, recognition of error Web pages using a customized convolutional neural network (CNN) to eliminate invalid pages, and Web categorization based solely on screenshots using a CNN with transfer learning to assist search engines, indexers, and Web directories.

show abstract

Section: Use Case 2: Automatic Recognition Of Error Web Pagesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%