Purely URL-based topic classification

Baykan, Eda; Henzinger, Monika; Marian, Ludmila; Weber, Ingmar

doi:10.1145/1526709.1526880

Cited by 93 publications

(71 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We then use these lists to search for indicative words in the visible text of <a> elements, and in normalized text extracted from the URLs. We use a token extraction technique from [18] to extract the normalized text from URLs. For example, /watch/baseball/foxsports.html would be split into the tokens watch, baseball, foxsports, and html.…”

Section: A Feature Extractionmentioning

confidence: 99%

It's Free for a Reason: Exploring the Ecosystem of Free Live Streaming Services

Rafique

Goethem

Joosen

et al. 2016

Proceedings 2016 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

Abstract-Recent years have seen extensive growth of services enabling free broadcasts of live streams on the Web. Free live streaming (FLIS) services attract millions of viewers and make heavy use of deceptive advertisements. Despite the immense popularity of these services, little is known about the parties that facilitate it and maintain webpages to index links for free viewership.This paper presents a comprehensive analysis of the FLIS ecosystem by mapping all parties involved in the anonymous broadcast of live streams, discovering their modus operandi, and quantifying the consequences for common Internet users who utilize these services. We develop an infrastructure that enables us to perform more than 850,000 visits by identifying 5,685 free live streaming domains, and analyze more than 1 Terabyte of traffic to map the parties that constitute the FLIS ecosystem. On the one hand, our analysis reveals that users of FLIS websites are generally exposed to deceptive advertisements, malware, malicious browser extensions, and fraudulent scams. On the other hand, we find that FLIS parties are often reported for copyright violations and host their infrastructure predominantly in Europe and Belize. At the same time, we encounter substandard advertisement set-ups by the FLIS parties, along with potential trademark infringements through the abuse of domain names and logos of popular TV channels.Given the magnitude of the discovered abuse, we engineer features that characterize FLIS pages and build a classifier to identify FLIS pages with high accuracy and low false positives, in an effort to help human analysts identify malicious services and, whenever appropriate, initiate content-takedown requests.

show abstract

Section: A Feature Extractionmentioning

confidence: 99%

It's Free for a Reason: Exploring the Ecosystem of Free Live Streaming Services

Rafique

Goethem

Joosen

et al. 2016

Proceedings 2016 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

show abstract

“…aspx, u2: http://allrecipes.com/RECIPE/Smores/ default.aspx}. Rule generated from these URLs with u1 as source URL and u2 as target URL, has context c(k (1,5) …”

Section: Deep Token Componentsmentioning

confidence: 99%

“…In contrast with efforts in literature for extracting semantic features from URLs [16,5], we discuss a technique which extracts syntactic features from URLs. We employ an unsupervised technique to learn custom URL encodings used by webmasters.…”

Section: Deep Tokenizationmentioning

confidence: 99%

Learning URL patterns for webpage de-duplication

Koppula

Leela

Agarwal³

et al. 2010

Proceedings of the Third ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

Presence of duplicate documents in the World Wide Web adversely affects crawling, indexing and relevance, which are the core building blocks of web search. In this paper, we present a set of techniques to mine rules from URLs and utilize these rules for de-duplication using just URL strings without fetching the content explicitly. Our technique is composed of mining the crawl logs and utilizing clusters of similar pages to extract transformation rules, which are used to normalize URLs belonging to each cluster. Preserving each mined rule for de-duplication is not efficient due to the large number of such rules. We present a machine learning technique to generalize the set of rules, which reduces the resource footprint to be usable at web-scale. The rule extraction techniques are robust against web-site specific URL conventions. We compare the precision and scalability of our approach with recent efforts in using URLs for de-duplication. Experimental results demonstrate that our approach achieves 2 times more reduction in duplicates with only half the rules compared to the most recent previous approach. Scalability of the framework is demonstrated by performing a large scale evaluation on a set of 3 Billion URLs, implemented using the MapReduce framework.

show abstract

“…When classifying Web documents, another source of information that can be used for classification is their Uniform Resource Locator (URL). Previous research has shown that classifiers built from features based solely on document URLs can achieve surprisingly good results on tasks such as language identification [24] or topic attribution [23]. Intuitively, URLs contain information that can be used to discriminate between local and global pages, such as top level domains or words such as local or regional.…”

Section: Classification Featuresmentioning

confidence: 99%

“…For instance, a document whose URL has a top level domain .uk is more likely to be local than a document with a top level domain such as .com. Taking inspiration on the experiments reported by Baykan et al [23], the following features were considered:…”

Section: Classification Featuresmentioning

confidence: 99%

Classifying Documents According to Locational Relevance

Anastácio

Martins

Calado

2009

Progress in Artificial Intelligence

View full text Add to dashboard Cite

Abstract. This paper presents an approach for categorizing documents according to their implicit locational relevance. We report a thorough evaluation of several classifiers designed for this task, built by using support vector machines with multiple alternatives for feature vectors. Experimental results show that using feature vectors that combine document terms and URL n-grams, with simple features related to the locality of the document (e.g. total count of place references) leads to high accuracy values. The paper also discusses how the proposed categorization approach can be used to help improve tasks such as document retrieval or online contextual advertisement.

show abstract

Purely URL-based topic classification

Cited by 93 publications

References 6 publications

It's Free for a Reason: Exploring the Ecosystem of Free Live Streaming Services

It's Free for a Reason: Exploring the Ecosystem of Free Live Streaming Services

Learning URL patterns for webpage de-duplication

Classifying Documents According to Locational Relevance

Contact Info

Product

Resources

About