Database Compression with Data Mining Methods

Goh, Chien-Le; Aisaka, Kazuki; Tsukamoto, Masahiko; Nishio, Shojiro

doi:10.1007/978-1-4615-1379-7_13

Cited by 5 publications

(10 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The most closely related work is Goh et al [15] where association rule mining algorithms are used to obtain association rules from the database tables that are then used to compress the data. Association rules are quite different from functional dependencies and the join query tree that are exploited in our compression method.…”

Section: Previous Workmentioning

confidence: 99%

Query-aware compression of join results

Mullins

Lim

Lang

2013

Proceedings of the 16th International Conference on Extending Database Technology

View full text Add to dashboard Cite

The proliferation of lightweight client devices such as iPhones, iPads, Android phones and tablets, has created an increased demand for cloud-based services. In many of these services, queries over structured data are sent to cloud-based servers for processing and the results relayed back to the client devices. Network bandwidth between client devices and cloud-based servers is often a limited resource and any effort to reduce the amount of data transmitted across the network would not only conserve bandwidth but help with the battery life of the client devices. In this thesis we propose a novel query-aware compression method for compressing query results sent from database servers to client applications. Our method is based on two key ideas. We exploit redundancy information obtained from the query plan and possibly from the database schema to achieve better compression than standard non-query aware compressors. We use a collection of memory-limited dictionaries to encode attribute values in a lightweight and efficient manner. We evaluate our method empirically using the TPC-H benchmark show that this technique is effective especially when used in conjunction with standard compressors. Our results show that compression ratios of up to tow times over gzip are possible.iv

show abstract

Section: Previous Workmentioning

confidence: 99%

Query-aware compression of join results

Mullins

Lim

Lang

2013

Proceedings of the 16th International Conference on Extending Database Technology

View full text Add to dashboard Cite

show abstract

“…Further, a database compression method using data mining (Goh et al, 1998), a database compression method based on Differencing Coding (Ng and Ravishankar, 1997), and CIT (Changchien and Lu, 2001), an association rule mining structure are explored.…”

Section: Literature Reviewmentioning

confidence: 99%

A data mining approach to database compression

et al. 2006

View full text Add to dashboard Cite

Data mining can dig out valuable information from databases to assist a business in approaching knowledge discovery and improving business intelligence. Database stores large structured data. The amount of data increases due to the advanced database technology and extensive use of information systems. Despite the price drop of storage devices, it is still important to develop efficient techniques for database compression. This paper develops a database compression method by eliminating redundant data, which often exist in transaction database. The proposed approach uses a data mining structure to extract association rules from a database. Redundant data will then be replaced by means of compression rules. A heuristic method is designed to resolve the conflicts of the compression rules. To prove its efficiency and effectiveness, the proposed approach is compared with two other database compression methods

show abstract

“…Storage methods for the extracted rules include a method [2,5] to store them in a deductive database, and a method [7,8] to store them in a relational database.…”

Section: Outlinementioning

confidence: 99%

“…5: a method using the ID3 algorithm [3], and a method using the a priori algorithm [1]. The example shown here uses the latter for compression.…”

Section: Implementation Examplementioning

confidence: 99%

See 1 more Smart Citation

On selection rules for database compression using KDD

Aisaka

Tsukamoto

Harumoto

et al. 2001

Electron Comm Jpn Pt III

Self Cite

View full text Add to dashboard Cite

SUMMARYDespite the drop in disk prices in recent years, the costs associated with disks still represent the primary costs for large-scale databases such as those used in data warehouses. Compression and storage of such databases is seen as an effective means to reduce these costs. The authors research group has already proposed a compression method that exchanges the priority for rules and data after extracting rules latent in data by using knowledge discovery as a database compression method which can access a database in its compressed state. However, this proposed method has the problem of producing differences in the compression ratio due to the priority with which the extracted rules are used for compression. Simply finding all the combinations for the priority for the use of such rules is not practical, and so in this paper the authors propose a rule selection method which provides comparatively good compression ratios without excessive computational requirements, then demonstrate the validity of their method using experimental results.

show abstract

Database Compression with Data Mining Methods

Cited by 5 publications

References 8 publications

Query-aware compression of join results

Query-aware compression of join results

A data mining approach to database compression

On selection rules for database compression using KDD

Contact Info

Product

Resources

About