Cardinality estimation based on QDSPN for embedded databases under dynamic workload

Abstract

Cardinality estimation has been a pivotal and enduring research focus within database query optimization. While significant advancements have been made in estimating cardinalities for both individual tables and complex multi-table joins, there remains a notable gap in research pertaining to embedded database scenarios. Embedded databases are typically characterized by limited resources and a preponderance of dense, short-term hotspot queries. As a result, cardinality estimation within the constraints of embedded databases poses additional complexities and challenges. In this paper, we introduce a novel Query-driven Sum-Product Network (QDSPN), which leverages the capabilities of sum-product networks (SPNs) to learn from historical data and adapt to dynamic workload variations. This approach effectively mitigates the inherent challenges of SPNs, such as false cluster collisions and independence assumption errors, particularly under conditions of strongly correlated data. Furthermore, we propose a two-stage query clustering framework tailored for dynamic workload environments. This framework serves to guide the structural configuration of the sum-product network, enhancing its adaptability and efficiency. We conduct extensive experiments to validate the performance of QDSPN under dynamic workloads. The experimental results demonstrate the evident advantages of the proposed QDSPN, and highlight its potential for widespread adoption in embedded database systems.