Refining Network Message Segmentation with Principal Component Analysis

Abstract

Reverse engineering of undocumented protocols is a common task in security analyses of networked services. The communication itself, captured in traffic traces, contains much of the necessary information to perform such a protocol reverse engineering. The comprehension of the format of unknown messages is of particular interest for binary protocols that are not human-readable. One major challenge is to discover probable fields in a message as the basis for further analyses. Given a set of messages, split into segments of bytes by an existing segmenter, we propose a method to refine the approximation of the field inference. We use principle component analysis (PCA) to discover linearly correlated variance between sets of message segments. We relocate the boundaries of the initial coarse segmentation to more accurately match with the true fields. We perform different evaluations of our method to show its benefit for the message format inference and subsequent analysis tasks from literature that depend on the message format. We can achieve a median improvement of the message format accuracy across different real-world protocols by up to 100 %.

DynPRE: Protocol Reverse Engineering via Dynamic Inference

“…Compared Tools. We select five state-of-the-art protocol reverse engineering tools widely used in academia and industry as baselines, including Netzob [6], Netplier [53], FieldHunter [3], BinaryInferno [9], and Nemesys [22]. Their approaches are diverse.…”

“…Network Trace Based Approach. These techniques analyze static network traces to mine features such as message bytes [22], sequences [53], [6], and common field semantics [3], [9]. Some techniques, such as Netzob [6] and Netplier [53], employ an alignment-based approach that classifies messages into distinct clusters and summarizes cluster-specific structures based on alignment [12].…”

“…The experimental results demonstrate that DYNPRE outperforms prior approaches in both field identification and message type inference. Specifically, DYN-PRE identifies fields with a perfection score of 0.50 and infers message types with a V-measure of 0.94, whereas Netzob [6], Netplier [53], FieldHunter [3], BinaryInferno [9], and Nemesys [22] achieve on average (0.15, 0.72), (0.16, 0.73), (0.15, 0.83), (0.15, -), and (0.31, -), respectively. Even when these tools are given datasets enhanced with dynamic interaction, DYNPRE is still superior.…”