Hypersparse Network Flow Analysis of Packets with GraphBLAS

Trigg, Tyler; Meiners, Chad; Pisharody, Sandeep; Jananthan, Hayden; Jones, M. B.; Michaleas, Adam; Davis, Timothy; Welch, Erik; Arcand, William; Bestor, David; Bergeron, William; Byun, Chansup; Gadepally, Vijay; Houle, Micheal; Hubbell, Matthew; Klein, Anna; Michaleas, Peter; Milechin, Lauren; Mullen, Julie; Prout, Andrew; Reuther, Albert; Rosa, Antonio; Samsi, Siddharth; Stetson, Doug; Yee, Charles; Kepner, Jeremy

doi:10.1109/hpec55821.2022.9926320

Cited by 3 publications

(3 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The process for rapid construction and analysis of Graph-BLAS hypersparse traffic matrices is described in [30], [31] and briefly summarized here. The first step in the GraphBLAS hypersparse traffic matrix pipeline is to capture a packet, This procedure [30] was applied to a 246 GB packet capture (PCAP) dataset collected at a gateway for several minutes consisting of 2 31 = 2, 147, 483, 648 packets.…”

Section: Network Sensor Focusing and Calibrationmentioning

confidence: 99%

“…The resulting 256 compressed GraphBLAS TAR files, each encompassing 64×2 17 = 8,388,608 packets, had an average file size of 7.4MB corresponding to < 1 byte per packet. These files were further analyzed [31] to compute all the network quantities in Table I for packet windows N V = 2 17 , . .…”

Section: Network Sensor Focusing and Calibrationmentioning

confidence: 99%

“…The North American Internet generates billions of non-video Internet packets each second [17], [18]. The GraphBLAS standard provides significant performance and compression capabilities which improve the feasibility of analyzing these volumes of data [19]- [31]. Specifically, the GraphBLAS is ideally suited for both constructing and analyzing anonymized hypersparse traffic matrices.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Vertical, Temporal, and Horizontal Scaling of Hierarchical Hypersparse GraphBLAS Matrices

Kepner

Davis

Byun

et al. 2021

2021 IEEE High Performance Extreme Computing Conference (HPEC)

View full text Add to dashboard Cite

Defending community-owned cyber space requires community-based efforts. Large-scale network observations that uphold the highest regard for privacy are key to protecting our shared cyberspace. Deployment of the necessary network sensors requires careful sensor placement, focusing, and calibration with significant volumes of network observations. This paper demonstrates novel focusing and calibration procedures on a multi-billion packet dataset using high-performance GraphBLAS anonymized hypersparse matrices. The run-time performance on a real-world data set confirms previously observed realtime processing rates for high-bandwidth links while achieving significant data compression. The output of the analysis demonstrates the effectiveness of these procedures at focusing the traffic matrix and revealing the underlying stable heavy-tail statistical distributions that are necessary for anomaly detection. A simple model of the corresponding probability of detection (p d ) and probability of false alarm (p fa ) for these distributions highlights the criticality of network sensor focusing and calibration. Once a sensor is properly focused and calibrated it is then in a position to carry out two of the central tenets of good cybersecurity:(1) continuous observation of the network and (2) minimizing unbrokered network connections.

show abstract

Section: Network Sensor Focusing and Calibrationmentioning

confidence: 99%

Section: Network Sensor Focusing and Calibrationmentioning

confidence: 99%