An optimal text compression algorithm based on frequent pattern mining

Oswald, C.; Sivaselvan, B.

doi:10.1007/s12652-017-0540-2

Cited by 20 publications

(10 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The LZW works by saving new phrases in empty dictionary entries and replacing phrases in the input text by pointers to the corresponding dictionary entries. Oswald et al have come up with few efficient text compression algorithms based on the perspective on knowledge engineering (data mining) 14‐20 …”

Section: Related Workmentioning

confidence: 99%

An efficient and novel data clustering and run length encoding approach to image compression

Oswald

Haritha

Raja

et al. 2021

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

The paper explores the domain of lossy compression, specifically incorporating data mining techniques in the process of image encoding. Clustering is employed to group similar pixels in the image and henceforth use cluster labels in compressing the image. The proposed approach replaces the Discrete Cosine Transform phase of Conventional JPEG with a combination of clustering and Run Length Encoding so as to handle redundant data in the image effectively. Simulation with respect to benchmark data indicates improved compression (42.5%) in relation to existing solutions. Image quality metrices such as PSNR, structural similarity have also been tested and it is observed that the proposed approach achieves significant compression ratio with negligible loss in visual quality.

show abstract

Section: Related Workmentioning

confidence: 99%

An efficient and novel data clustering and run length encoding approach to image compression

Oswald

Haritha

Raja

et al. 2021

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

show abstract

“…Well established approaches for volume reduction without compromising information density are compression or information extraction. Commercial lossless compression tools include LZ77, bzip2, GZip and RAR [44]. Many compression methods focus on video and image data [50], text [44], or all combined.…”

Section: Immediate Processing and Actuation Without Transmissionmentioning

confidence: 99%

Vibration Edge Computing in Maritime IoT

Michala

Vourganas

Coraddu

2021

ACM Trans. Internet Things

View full text Add to dashboard Cite

IoT and the Cloud are among the most disruptive changes in the way we use data today. These changes have not significantly influenced practices in condition monitoring for shipping. This is partly due to the cost of continuous data transmission. Several vessels are already equipped with a network of sensors. However, continuous monitoring is often not utilised and onshore visibility is obscured. Edge computing is a promising solution but there is a challenge sustaining the required accuracy for predictive maintenance. We investigate the use of IoT systems and Edge computing, evaluating the impact of the proposed solution on the decision making process. Data from a sensor and the NASA-IMS open repository were used to show the effectiveness of the proposed system and to evaluate it in a realistic maritime application. The results demonstrate our real-time dynamic intelligent reduction of transmitted data volume by without sacrificing specificity or sensitivity in decision making. The output of the Decision Support System fully corresponds to the monitored system's actual operating condition and the output when the raw data are used instead. The results demonstrate that the proposed more efficient approach is just as effective for the decision making process.

show abstract

“…To further improve CR specific for text files, Lempel-Ziv-Welch (LZW) based methods are widely studied [39]- [41]. Oswald et al [42] explore text compression in the perspective of Data Mining. They enhance Huffman Encoding by frequent pattern mining.…”

Section: Related Workmentioning

confidence: 99%

Logzip: Extracting Hidden Structures via Iterative Clustering for Log Compression

Liu

Zhu

et al. 2019

2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

System logs record detailed runtime information of software systems and are used as the main data source for many tasks around software engineering. As modern software systems are evolving into large scale and complex structures, logs have become one type of fast-growing big data in industry. In particular, such logs often need to be stored for a long time in practice (e.g., a year), in order to analyze recurrent problems or track security issues. However, archiving logs consumes a large amount of storage space and computing resources, which in turn incurs high operational cost. Data compression is essential to reduce the cost of log storage. Traditional compression tools (e.g., gzip) work well for general texts, but are not tailed for system logs. In this paper, we propose a novel and effective log compression method, namely logzip. Logzip is capable of extracting hidden structures from raw logs via fast iterative clustering and further generating coherent intermediate representations that allow for more effective compression. We evaluate logzip on five large log datasets of different system types, with a total of 63.6 GB in size. The results show that logzip can save about half of the storage space on average over traditional compression tools. Meanwhile, the design of logzip is highly parallel and only incurs negligible overhead. In addition, we share our industrial experience of applying logzip to Huawei's real products.

show abstract

An optimal text compression algorithm based on frequent pattern mining

Cited by 20 publications

References 40 publications

An efficient and novel data clustering and run length encoding approach to image compression

An efficient and novel data clustering and run length encoding approach to image compression

Vibration Edge Computing in Maritime IoT

Logzip: Extracting Hidden Structures via Iterative Clustering for Log Compression

Contact Info

Product

Resources

About