Stream-Based Lossless Data Compression Hardware Using Adaptive Frequency Table Management

Yamagiwa, Shinichi; Marumo, Koichi; Sakamoto, Hiroshi

doi:10.1007/978-3-319-29006-5_11

Cited by 5 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We added a dynamic histogram control to the look-up table of LCA-SLT and proposed LCA-DLT [17]. When a symbol pair is received by the compressor, it is registered in an entry of the table when the compressor/decompressor does not find it in the table.…”

Section: Abcabcmentioning

confidence: 99%

Exception Handling Method Based on Event from Look-Up Table Applying Stream-Based Lossless Data Compression

2021

Self Cite

View full text Add to dashboard Cite

It is getting popular to implement an environment where communications are performed remotely among IoT edge devices, such as sensory devices and the cloud servers due to applying, for example, artificial intelligence algorithms to the system. In such situations that handle big data, lossless data compression is one of the solutions to reduce the big data. In particular, the stream-based data compression technology is focused on such systems to compress infinitely continuous data stream with very small delay. However, during the continuous data compression process, it is not able to insert an exception code among the compressed data without any additional mechanisms, such as data framing and the packeting technique, as used in networking technologies. The exception code indicates configurations for the compressor/decompressor and/or its peripheral logics. Then, it is used in real time for the configuration of parameters against those components. To implement the exception code, data compression algorithm must include a mechanism to distinguish original data before compression and the exception code clearly. However, the conventional algorithms do not include such mechanism. This paper proposes novel methods to implement the exception code in data compression that uses look-up table, called the exception symbol. Additionally, we describe implementation details of the method by applying it to algorithms of stream-based data compression. Because some of the proposed mechanisms need to reserve entries in the table, we also discuss the effect against data compression performance according to experimental evaluations.

show abstract

Section: Abcabcmentioning

confidence: 99%

Exception Handling Method Based on Event from Look-Up Table Applying Stream-Based Lossless Data Compression

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…16. 19 Stall cycles and the stall ratios against the total clock cycles in LCA-DLT with optimizations search method. These mechanisms work well if the randomness of the data is high (i.e., the data entropy is high).…”

Section: Performance Evaluationsmentioning

confidence: 99%

“…Next, we focus on another algorithm for stream-based data compression with dynamic table management, called LCA-DLT (LCA Dynamic Look-up Table)[19]. It allocates corresponding symbol LUTs for the compressor and the decompressor, respectively.…”

mentioning

confidence: 99%

Stream-Based Lossless Data Compression

Yamagiwa

2021

Sublinear Computation Paradigm

Self Cite

View full text Add to dashboard Cite

In this chapter, we introduce aspects of applying data-compression techniques. First, we study the background of recent communication data paths. The focus of this chapter is a fast lossless data-compression mechanism that handles data streams completely. A data stream comprises continuous data with no termination of the massive data generated by sources such as movies and sensors. In this chapter, we introduce LCA-SLT and LCA-DLT, which accept the data streams, as well as several implementations of these stream-based compression techniques. We also show optimization techniques for optimal implementation in hardware.

show abstract

“…We used the Xilinx (San Jose, CA, USA) Artix7 FPGA XC7A200T-1FBG676C. The full search method works at 100 MHz in this device [6]. On the other hand, the implementation with both proposed mechanisms works at 130 MHz because the implementation was simplified by the lazy management of the symbol lookup table.…”

Section: Dynamic Hardware Performancementioning

confidence: 99%

Lazy Management for Frequency Table on Hardware-Based Stream Lossless Data Compression

et al. 2016

Self Cite

View full text Add to dashboard Cite

Abstract:The demand for communicating large amounts of data in real-time has raised new challenges with implementing high-speed communication paths for high definition video and sensory data. It requires the implementation of high speed data paths based on hardware. Implementation difficulties have to be addressed by applying new techniques based on data-oriented algorithms. This paper focuses on a solution for this problem by applying a lossless data compression mechanism on the communication data path. The new lossless data compression mechanism, called LCA-DLT, provides dynamic histogram management for symbol lookup tables used in the compression and the decompression operations. When the histogram memory is fully used, the management algorithm needs to find the least used entries and invalidate these entries. The invalidation operations cause the blocking of the compression and the decompression data stream. This paper proposes novel techniques to eliminate blocking by introducing a dynamic invalidation mechanism, which allows achievement of a high throughput data compression.

show abstract

Stream-Based Lossless Data Compression Hardware Using Adaptive Frequency Table Management

Cited by 5 publications

References 13 publications

Exception Handling Method Based on Event from Look-Up Table Applying Stream-Based Lossless Data Compression

Exception Handling Method Based on Event from Look-Up Table Applying Stream-Based Lossless Data Compression

Stream-Based Lossless Data Compression

Lazy Management for Frequency Table on Hardware-Based Stream Lossless Data Compression

Contact Info

Product

Resources

About