Hadoop-based Dark Web Threat Intelligence Analysis Framework

Yang, Ying; Yu, Huanhuan; Yang, Lina; Yang, Ming; Chen, Lijuan; Zhu, Guichun; Wen, Liqiang

doi:10.1109/imcec46724.2019.8984106

Cited by 12 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The encrypted information is placed between the relays. Tor traffic as a whole passes through three relays and then it is forwarded to the final destination [14]. This mechanism ensures perfect forward secrecy between the nodes and the hidden services of Tor, while at the same time it routinely communicates through Tor nodes (consensus) operated by volunteers around the world.…”

Section: Literature Reviewsmentioning

confidence: 99%

“…Yang et al [26] introduce the current mainstream dark network communication system TOR and develop a visual dark web forum post association analysis system to graphically display the relationship between various forum messages and posters, which helps analysts to explore deep levels. In addition, another paper [14] designs a framework based on Hadoop in hidden threat intelligence. The framework uses a Hadoop database-based (HBase-based) distributed database to store and manage threat intelligence information, and a web crawler is used to collect data through the anonymous TOR tool in order to identify the characteristics of key dark network criminal networks, which is the basis for the later dark network research.…”

Section: Literature Reviewsmentioning

confidence: 99%

See 1 more Smart Citation

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

et al. 2021

View full text Add to dashboard Cite

Attackers are perpetually modifying their tactics to avoid detection and frequently leverage legitimate credentials with trusted tools already deployed in a network environment, making it difficult for organizations to proactively identify critical security risks. Network traffic analysis products have emerged in response to attackers’ relentless innovation, offering organizations a realistic path forward for combatting creative attackers. Additionally, thanks to the widespread adoption of cloud computing, Device Operators (DevOps) processes, and the Internet of Things (IoT), maintaining effective network visibility has become a highly complex and overwhelming process. What makes network traffic analysis technology particularly meaningful is its ability to combine its core capabilities to deliver malicious intent detection. In this paper, we propose a novel darknet traffic analysis and network management framework to real-time automating the malicious intent detection process, using a weight agnostic neural networks architecture. It is an effective and accurate computational intelligent forensics tool for network traffic analysis, the demystification of malware traffic, and encrypted traffic identification in real time. Based on a weight agnostic neural networks (WANNs) methodology, we propose an automated searching neural net architecture strategy that can perform various tasks such as identifying zero-day attacks. By automating the malicious intent detection process from the darknet, the advanced proposed solution is reducing the skills and effort barrier that prevents many organizations from effectively protecting their most critical assets.

show abstract

Section: Literature Reviewsmentioning

confidence: 99%

Section: Literature Reviewsmentioning

confidence: 99%

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

et al. 2021

View full text Add to dashboard Cite

show abstract

“…A Hadoop-based framework was designed in 2019 to identify the individualities of dark network criminals' networks using HDFS as a database and web crawler storage system to collect Tor data [69]. The author has discussed dark web threats via Hadoop-based intelligence analysis framework by:…”

Section: Monitoring Of Dark Websitesmentioning

confidence: 99%

The Anonymity of the Dark Web: A Survey

2022

View full text Add to dashboard Cite

The dark web is a section of the Internet that is not accessible to search engines and requires an anonymizing browser called Tor. Its hidden network and anonymity pave the way for illegal activities and help cybercriminals to execute well-planned, coordinated, and malicious cyberattacks. Cyber security experts agree that online criminal activities are increasing exponentially, and they are also becoming more rampant and intensified. These illegal cyber activities include various destructive crimes that may target a single person or a whole nation, for example, data breaches, ransomware attacks, black markets, mafias, and terrorist attacks. So, maintaining data privacy and secrecy is the new dilemma of the era. This paper has extensively reviewed various attacks and attack patterns commonly applied in the dark web. We have also classified these attacks in our unique trilogies classification system. Furthermore, a detailed overview of existing threat detection techniques and their limitations is discussed for anonymity providing services like Tor, I2P, and Freenet. Finally, the paper has identified significant weaknesses that make the dark web vulnerable to different attacks.

show abstract

“…Zhang Xuan, a member of the Shandong Police College, and the Secretary of the Department of Information Security and Cryptography (CISC) of the University of Hong Kong, Professor Jinpei ou, co-published the Dark Net Threat Intelligence Analysis Framework, which proposes a concept of a hidden threat intelligence analysis framework. To help analyze crime traces in the dark network [27,28].…”

Section: Related Research Studymentioning

confidence: 99%

“…After classifying and clustering the captured data, the characteristics of the special participants can be extracted. Through the social network analysis method, the social interaction mode with other cybercrime, the type of content published, and the frequency of discussion of the participating topics can be obtained [27].…”

Section: Introductionmentioning

confidence: 99%

An Empirical Study of Deep Web based on Graph Analysis

Mm¹

2020

Preprint

View full text Add to dashboard Cite

The internet can broadly be divided into three parts: surface, deep and dark among which the latter offers anonymity to its users and hosts [1]. Deep Web refers to an encrypted network that is not detected on search engine like Google etc. Users must use Tor to visit sites on the dark web [2]. Ninety six percent of the web is considered as deep web because it is hidden. It is like an iceberg, in that, people can just see a small portion above the surface, while the largest part is hidden under the sea [3, 4, and 5]. Basic methods of graph theory and data mining, that deals with social networks analysis can be comprehensively used to understand and learn Deep Web and detect cyber threats [6]. Since the internet is rapidly evolving and it is nearly impossible to censor the deep web, there is a need to develop standard mechanism and tools to monitor it. In this proposed study, our focus will be to develop standard research mechanism to understand the Deep Web which will support the researchers, academicians and law enforcement agencies to strengthen the social stability and ensure peace locally & globally.

show abstract

Hadoop-based Dark Web Threat Intelligence Analysis Framework

Cited by 12 publications

References 5 publications

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

The Anonymity of the Dark Web: A Survey

An Empirical Study of Deep Web based on Graph Analysis

Contact Info

Product

Resources

About