Processing of Flow Accounting Data in Java: Framework Design and Performance Evaluation

Abstract

Flow Accounting is a passive monitoring mechanism implemented in routers that gives insight into traffic behavior and network characteristics. However, processing of Flow Accounting data is a challenging task, especially in large networks where the rate of Flow Records received at the collector can be very high. We developed a framework for processing of Flow Accounting data in Java. It provides processing blocks for aggregation, sorting, statistics, correlation, and other tasks. Besides reading data from files for offline analysis, it can also directly process data received from the network. In terms of multithreading and data handling, the framework is highly configurable, which allows performance tuning depending on the given task. For setting these parameters there are several trade-offs concerning memory consumption and processing overhead. In this paper, we present the framework design, study these trade-offs based on a reference scenario and examine characteristics caused by garbage collection.