File Type Identification of Data Fragments by Their Binary Structure

Abstract

Rapidly gaining information superiority is vital when fighting an enemy, but current computer forensics tools, which require file headers or a working file system to function, do not enable us to quickly map out the contents of corrupted hard disks or other fragmented storage media found at crime scenes. The lack of proper tools slows down the hunt for information, which would otherwise help in gaining the upper hand against IT based perpetrators. To address this problem, this paper presents an algorithm which allows categorization of data fragments based solely on their structure, without the need for any meta data. The algorithm is based on measuring the rate of change of the byte contents of digital media and extends the byte frequency distribution based Oscar method presented in an earlier paper. The evaluation of the new method shows a detection rate of 99.2 %, without generating any false positives, when used to scan for JPEG data. The slowest implementation of the algorithm scans a 72.2 MB file in approximately 2.5 seconds and scales linearly.

The Normalised Compression Distance as a file fragment classifier

“…[11,12] introduced the idea of using the data of the file itself for classifying file fragments. In doing so they focused exclusively on developing the Oscar method for identifying jpg file fragments (that used special information about the structure of jpg files).…”

File Fragment Classification-The Case for Specialized Approaches

“…Unfortunately, RoC does nothing to improve the rather modest classification success of other file formats considered. For Windows executables, the false positive rate actually exceeded the detection rate for most points shown ( [5] Fig 3), although the peak detection rate of 70% is equal to a false positive rate of 70%. For zip files, things look a little better with false positive rate of 70% when the detection rate reaches 100% ( [5] Fig 4).…”

“…This was shortly extended [5] with the introduction of a new metric called rate-ofchange (RoC), which was defined as the difference of the ASCII values of consecutive bytes. This was done squarely to improve the accuracy of jpeg recognition, which becomes near perfect.…”

GP-Fileprints: File Types Detection Using Genetic Programming

“…When these measures are put together, they form a model which is used to identify unknown data fragments. In [5], the same authors extended this approach by calculating the rate of change (RoC) (i.e., the absolute value of the difference between two consecutive byte values in a data fragment). RoC allows incorporating the ordering of the bytes into the identification process.…”