Using compression algorithms to support the comprehension of program traces

Walkinshaw, Neil; Afshan, Sheeva; McMinn, Phil

doi:10.1145/1868321.1868323

Cited by 15 publications

(4 citation statements)

References 14 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chilimbi and others [21] use Sequitur for fast detection of hot data streams. Walkinshaw and others [89] apply Sequitur to the comprehension of program traces at varying levels of abstraction. Lin and others [52] extend Sequitur as a new XML compression scheme for supporting query processing.…”

Section: Related Workmentioning

confidence: 99%

TADOC: Text Analytics Directly on Compression

Zhang,

Zhai,

Shen

et al. 2020

Preprint

View full text Add to dashboard Cite

This article provides a comprehensive description of Text Analytics Directly on Compression (TADOC), which enables direct document analytics on compressed textual data. The article explains the concept of TADOC and the challenges to its effective realizations. Additionally, a series of guidelines and technical solutions that effectively address those challenges, including the adoption of a hierarchical compression method and a set of novel algorithms and data struc-

show abstract

Section: Related Workmentioning

confidence: 99%

TADOC: Text Analytics Directly on Compression

Zhang,

Zhai,

Shen

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Since its development [34,35,36], Sequitur has been applied to various tasks, including program and data pattern analysis [10,11,21,22,23,26,44]. We are not aware of prior usage of Sequitur to support direct document analytics on compressed data.…”

Section: Related Workmentioning

confidence: 99%

Efficient document analytics on compressed data

et al. 2018

View full text Add to dashboard Cite

Today's rapidly growing document volumes pose pressing challenges to modern document analytics, in both space usage and processing time. In this work, we propose the concept of compression-based direct processing to alleviate issues in both dimensions. The main idea is to enable direct document analytics on compressed data. We present how the concept can be materialized on Sequitur, a compression algorithm that produces hierarchical grammar-like representations. We discuss the major challenges in applying the idea to various document analytics tasks, and reveal a set of guidelines and also assistant software modules for developers to effectively apply compression-based direct processing. Experiments show that our proposed techniques save 90.8% storage space and 77.5% memory usage, while speeding up data processing significantly, i.e., by 1.6X on sequential systems, and 2.2X on distributed clusters, on average.

show abstract

“…Walkinshaw et al [18] used SEQUITUR to recognize repeated patterns in program traces and to visualize them. As in our experiment, they generated an input string by transforming each element of API call sequences into one character.…”

Section: Related Workmentioning

confidence: 99%

Grammar Compression of Call Traces in Dynamic Malware Analysis

Okumura

Oyama

2017

Journal of Information Processing

View full text Add to dashboard Cite

A significant number of logs are generated in dynamic malware analysis. Consequently, a method for effectively compressing these logs is required to reduce the amount of memory and storage consumed to store such logs. In this study, we evaluated the efficacy of grammar compression methods in compressing call traces in malware analysis logs. We hypothesized that grammar compression can be useful in compressing call traces because its algorithm can naturally express the dynamic control flows of program execution. We measured the compression ratio of three grammar compression methods (SEQUITUR, Re-Pair, and Byte Pair Encoding (BPE)) and three well-known compressors (gzip, bzip2, and xz). In experiments conducted in which API call sequences collected from thousands of Windows malware were compressed, the Re-Pair grammar compression method was found to outperform both gzip and bzip2.

show abstract

Using compression algorithms to support the comprehension of program traces

Cited by 15 publications

References 14 publications

TADOC: Text Analytics Directly on Compression

TADOC: Text Analytics Directly on Compression

Efficient document analytics on compressed data

Grammar Compression of Call Traces in Dynamic Malware Analysis

Contact Info

Product

Resources

About