Improving Tor Hidden Service Crawler Performance

Park, Jong Hyeon; Mun, Hyunsu; Lee, Youngseok

doi:10.1109/desec.2018.8625103

Cited by 11 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In the work of [5], they presented a TOR hidden service crawling method that can monitor the status of the onion pages in the TOR network for content analysis. They developed TOR crawling software with Docker support.…”

Section: Literaturementioning

confidence: 99%

“…In addition, it is necessary to monitor Tor traffic in order to uncover illegal activities and to create related measures. For this purpose, numerous crawler software are being developed [3,4,5]. This softwares send requests to the seed onion address which is given them as a parameter and download the content of that address if the connection is provided.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

TOR Gizli Servis Tarayıcılarının Performans Karşılaştırması

Arısoy

Küçüksille

2019

Bilecik Şeyh Edebali Üniversitesi Fen Bilimleri Dergisi

View full text Add to dashboard Cite

TOR (The Onion Routing), kullanıcısına anonimliği sağlaması sebebiyle son zamanlarda popülerliği artan ve onion uzantılı gizli servisler tarafından sıklıkla tercih edilen bir ağ yapısıdır. Gizliliğin esas olması nedeniyle dikkatleri üzerine çeken bu ağda, her geçen gün depolanan veri miktarı artmakta bu da verilerin taranma ve analiz edilme durumlarını zorlaştırmaktadır. Bu ağda yer alan servislerin (onion uzantılı web sayfaları) taranması için çeşitli crawler yazılımları geliştirilmiştir. Yalnız, burada yapılan tarama yüzey ağında yapılan taramadan farklıdır. Çünkü TOR ağı, yüzey ağının alt katmanlarında yer almakta ve buradaki sayfalara yalnızca TOR tarayıcısı aracılığıyla ulaşılmaktadır. Geliştirilen crawler yazılımlarında bu durum dikkate alınmış ve gizliliği korumak adına, adreslere yapılan her istekte farklı düğümler üzerinden yol seçimi yapılarak veri edinimine dikkat edilmiştir. TOR ağında kullanıcıların gönderdiği her istekte farklı düğümler üzerinden geçilerek hedef adrese ulaşılması bu ağı yavaşlatmaktadır. Ayrıca, TOR üzerinden bilgi getirmeye çalışan bir crawler yazılımının performansının düşük olması da uzun süreler beklemeyi beraberinde getirir. Bu yüzden tarama ve bilgi elde etme hızı yüksek crawler yazılımları ile çalışmak, araştırmacıların analiz süreçlerini de iyileştirecektir. Bu alanda araştırma yapacak olan kişileri yönlendirmesi ve crawler yazılımlarının birbirlerine karşı olan üstün ve zayıf yönlerinin değerlendirilmesi açısından 4 farklı crawler yazılımı çeşitli kriterlere göre değerlendirilmiştir. Gerçekleştirilen çalışma, TOR web servislerinin analizini yapmak isteyen araştırmacıların ilk çıkış noktaları anlamında doğru bir crawler yazılımını seçmeleri hususunda önemli bir bakış açısı sunmaktadır.

show abstract

Section: Literaturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

TOR Gizli Servis Tarayıcılarının Performans Karşılaştırması

Arısoy

Küçüksille

2019

Bilecik Şeyh Edebali Üniversitesi Fen Bilimleri Dergisi

View full text Add to dashboard Cite

show abstract

“…Later, the growing risk of the suspicious activities practiced on the Darknet called the attention of researchers to dive into this network and explore its content (Graczyk and Kinningham, 2015;Moore and Rid, 2016;Park et al, 2018;Dalins et al, 2018;Dong et al, 2018;Takaaki and Atsuo, 2019;Al-Nabki et al, 2019c). Concerning the Deep Web, Xu et al (2007) presented a supervised text classification framework and used Information Gain (IG) to extract features from the text.…”

Section: Supervised Classification Techniquesmentioning

confidence: 99%

“…Recently, Dalins et al (2018) presented a novel model, named Tor-use Motivation Model (TMM), to classify the hidden services of the Tor network with greater granularity. Park et al (2018) build a parallel Tor browsers framework to improve the crawling process of the Tor domains. He et al (2019) proposed a supervised classifier for the illegal activities in the Darknet of the Tor network using the TF-IDF technique and NB classifier.…”

Section: Supervised Classification Techniquesmentioning

confidence: 99%

“…La variedad de actividades y el creciente uso de la red Tor para actividades de carácter ilegal en determinados países ha atraído recientemente la atención de los investigadores (Graczyk and Kinningham, 2015;Moore and Rid, 2016;Park et al, 2018;Dalins et al, 2018;Dong et al, 2018;Takaaki and Atsuo, 2019;Al-Nabki et al, 2019c). Con respecto a la Web Profunda, Xu et al (2007) presentó un clasificador de texto supervisado y utilizó Ganancia de Información (IG del inglés, Information Gain) para extraer características del texto.…”

Section: Clasificación De Textounclassified

See 1 more Smart Citation

Supervised machine learning for classification mining and ranking of illegal web contents = Aprendizaje automático supervisado para la clasificación, extracción de conocimiento y ordenación de contenidos web ilegales

Nabki¹,

Wesam²

View full text Add to dashboard Cite

In this thesis, we propose new algorithms, methods, and datasets that can be used to classify, to mine information, and rank web domains or similar resources containing text. Motivated by our joint work with INCIBE, we focus our efforts on detecting web resources which content could indicate illegal activities. Most of these textual web pages are hosted in a darknet, and, because of that, we centered our analysis in The Onion Router (Tor) Darknet, based on the common belief that this net hosts plenty of criminal activities. Additionally, we also addressed the same problem in Online Notepad Services (ONS), in particular, Pastebin service.Several of the contributions that we present here are already incorporated in tools developed by INCIBE that help Spanish Law Enforcement Agencies (LEAs) to monitor the contents of the Tor Darknet. Our work relies on the application of machine learning, both classical and deep, using most of the time supervised learning. This approach required the creation of different datasets, naming the first of them as Darknet Usage Text Addresses (DUTA), which contained 6, 831 labeled samples distributed over 26 classes. Posteriorly, we extended this dataset up to 10, 367 samples, naming it as DUTA-10K.Using DUTA, we evaluated the combination of two text representation techniques with three well-known classifiers to categorize the Tor domains. The combination of TF-IDF words representation with Logistic Regression achieved a 93.7% macro F1 score, in a subset of DUTA where eight categories of illegal activities were selected. To classify Pastebin contents, we use Active Learning to select and label only the most informative samples, reducing in this way, the cost of building a labeled dataset. Our design requires three cascade classifiers, saying the last one whether a sample belongs to one out of six categories related to criminal activities, obtaining an average class recall of 95.24% as binary, and 80.33% as multiclass.To enrich the information that we provide to LEAs, we develop first a semi-automatic algorithm to identify emerging products in Tor marketplaces. Using Graph Theory, we build a Products Correlations Graph (PCG), in which the nodes are the markets' products, and the edges reflect the simultaneous offering of two products in the same market. Our algorithm decomposes the PCG, using the k-shell algorithm, and analyzes the connectivity of the products in the core-shell. We apply this method to drug Hidden Services (HS) in DUTA, finding that MDMA and Ecstasy were the most emerging drug products during the analyzed period. Second, we used Named Entity Recognition (NER) to recognize rare and emerging named entities in noisy user-generated text. We overcome the use of gazetteers to incorporate external resources to neural network architectures, presenting a novel feature that we named Local Distance Neighbor (LDN), obtaining in this way the state-of-the-art F1 score on three categories of the W-NUT-2017 dataset: Group, Person, and Product. Furthermore, we present an application of NER...

show abstract

Surveilling Systems Used to Detect Lingering Threats on Dark Web

Vithanage,

Niroshika

2023

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

Improving Tor Hidden Service Crawler Performance

Cited by 11 publications

References 9 publications

TOR Gizli Servis Tarayıcılarının Performans Karşılaştırması

TOR Gizli Servis Tarayıcılarının Performans Karşılaştırması

Supervised machine learning for classification mining and ranking of illegal web contents = Aprendizaje automático supervisado para la clasificación, extracción de conocimiento y ordenación de contenidos web ilegales

Surveilling Systems Used to Detect Lingering Threats on Dark Web

Contact Info

Product

Resources

About