RDSZ: An Approach for Lossless RDF Stream Compression

Abstract

Abstract. In many applications (like social or sensor networks) the information generated can be represented as a continuous stream of RDF items, where each item describes an application event (social network post, sensor measurement, etc). In this paper we focus on compressing RDF streams. In particular, we propose an approach for lossless RDF stream compression, named RDSZ (RDF Differential Stream compressor based on Zlib). This approach takes advantage of the structural similarities among items in a stream by combining a differential item encoding mechanism with the general purpose stream compressor Zlib. Empirical evaluation using several RDF stream datasets shows that this combination produces gains in compression ratios with respect to using Zlib alone.

Efficient RDF Interchange (ERI) Format for RDF Data Streams

“…In turn, we manually define a set of common discrete predicates in Presets. Finally, according to previous experiences [10], the blockSize selection introduces a tradeoff between space and delays: the bigger the blocks, the more regular structures can be found. This implies better compression results, but with longer waiting times in the decoder.…”

“…Thus, the scheme is a simple dictionary-based replacement which does not compete in compression but allows operating in constrained devices. RDSZ [10] is the first specific approach for RDF streaming compression. RDSZ takes advantage of the fact that items in an RDF stream usually follow a common schema and, thus, have structural similarities.…”

“…Obviously, Streaming datasets are our main application focus; the first six datasets in Table 1 have been already used in the evaluation of RDSZ [10] and correspond to RDF messages in the public streamline of a microblogging site (Identica), Wikipedia edition monitoring (Wikipedia), information from weather stations in Spain (AEMET-1 and AEMET-2), credit card transactions in petrol stations (Petrol) and a random mix of these datasets (Mix). We complete the corpora with information of media events (e.g.…”

“…Thus, the evaluation with RDSZ is limited to streaming datasets (the first category in the table), for which we can configure the RDSZ input as a set of Turtle graphs merged together (the input expected by the RDSZ prototype), one per original graph in the dataset. The results of RDSZ depend on two configuration parameters: we use batchSize=5 and cacheSize=100, the default configuration in [10]. For a fair comparison, we consider the original size in Nt in the reported RDSZ compression results.…”

“…However, these solutions consider static RDF datasets, and need to read the whole dataset to take advantage of data regularities. A recent proposal, RDSZ [10], shows the benefits of applying the general-purpose stream compressor Zlib [8] to RDF streams, and provides a compression algorithm based on the difference of subject groups (provided in Turtle [17]), with some gains in compression (up to 31% w.r.t. Zlib).…”