Optimizing neural network topology using Shapley value

Leon, Florin

doi:10.1109/icstcc.2014.6982527

Cited by 6 publications

(14 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, their application to realistic problems remains a very complex and specialized case [33]. This is because data scientists, based on their hypotheses and experience, coordinate their numerous parameters, correlating them with the specific problems they intend to solve, utilizing the available training datasets.…”

Section: Methodology and Datasetmentioning

confidence: 99%

“…For example the authors of the [44] introduce a new metric, Top Similarity method, that measures the similitude of two given explanations, produced by Shapley values, in order to evaluate the Model-Agnostic Interpretability. Also the Florin [45]proposes a destructive method for optimizing the topology of neural networks based on the Shapley value, a game theoretic solution concept which estimates the contribution of each network element to the overall performance. More network elements can be simultaneously pruned, which can lead to shorter execution times and better results.…”

Section: Fig 1 the Internet Layersmentioning

confidence: 99%

See 1 more Smart Citation

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

et al. 2021

View full text Add to dashboard Cite

Attackers are perpetually modifying their tactics to avoid detection and frequently leverage legitimate credentials with trusted tools already deployed in a network environment, making it difficult for organizations to proactively identify critical security risks. Network traffic analysis products have emerged in response to attackers’ relentless innovation, offering organizations a realistic path forward for combatting creative attackers. Additionally, thanks to the widespread adoption of cloud computing, Device Operators (DevOps) processes, and the Internet of Things (IoT), maintaining effective network visibility has become a highly complex and overwhelming process. What makes network traffic analysis technology particularly meaningful is its ability to combine its core capabilities to deliver malicious intent detection. In this paper, we propose a novel darknet traffic analysis and network management framework to real-time automating the malicious intent detection process, using a weight agnostic neural networks architecture. It is an effective and accurate computational intelligent forensics tool for network traffic analysis, the demystification of malware traffic, and encrypted traffic identification in real time. Based on a weight agnostic neural networks (WANNs) methodology, we propose an automated searching neural net architecture strategy that can perform various tasks such as identifying zero-day attacks. By automating the malicious intent detection process from the darknet, the advanced proposed solution is reducing the skills and effort barrier that prevents many organizations from effectively protecting their most critical assets.

show abstract

Section: Methodology and Datasetmentioning

confidence: 99%

Section: Fig 1 the Internet Layersmentioning

confidence: 99%

Darknet Traffic Big-Data Analysis and Network Management for Real-Time Automating of the Malicious Intent Detection Process by a Weight Agnostic Neural Networks Framework

et al. 2021

View full text Add to dashboard Cite

show abstract

“…We show that the Shapley value provides a more robust estimate for the importance of a neuron yielding to a better performance of an ANN for the same model size than weight-based heuristics. While pruning with Shapley values was previosuly only shown in small problem domains [20] and with one single strategy based on Shapley values, this work presents results on image and text classification with multiple strategies, including Shapley values.…”

Section: Contributionsmentioning

confidence: 99%

“…Leon [20] uses Shapley value to optimize artificial neural network topologies by pruning neurons with minimal value or below a threshold in relation to the average value of the Shapley value distribution. The method is applied on the XOR-, Iris-, energy efficiency, ionosphere, and Yacht hydrodynamics problems which all yield test set accuracies above 0.9 with at most four neurons in their networks' hidden layer.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Analysing Neural Network Topologies: a Game Theoretic Approach

Stier,

Gianini,

Granitzer

et al. 2019

Preprint

View full text Add to dashboard Cite

Artificial Neural Networks have shown impressive success in very different application cases. Choosing a proper network architecture is a critical decision for a network's success, usually done in a manual manner. As a straightforward strategy, large, mostly fully connected architectures are selected, thereby relying on a good optimization strategy to find proper weights while at the same time avoiding overfitting. However, large parts of the final network are redundant. In the best case, large parts of the network become simply irrelevant for later inferencing. In the worst case, highly parameterized architectures hinder proper optimization and allow the easy creation of adverserial examples fooling the network.A first step in removing irrelevant architectural parts lies in identifying those parts, which requires measuring the contribution of individual components such as neurons. In previous work, heuristics based on using the weight distribution of a neuron as contribution measure have shown some success, but do not provide a proper theoretical understanding.Therefore, in our work we investigate game theoretic measures, namely the Shapley value (SV), in order to separate relevant from irrelevant parts of an artificial neural network. We begin by designing a coalitional game for an artificial neural network, where neurons form coalitions and the average contributions of neurons to coalitions yield to the Shapley value. In order to measure how well the Shapley value measures the contribution of individual neurons, we remove low-contributing neurons and measure its impact on the network performance.In our experiments we show that the Shapley value outperforms other heuristics for measuring the contribution of neurons.

show abstract