Approximating block accesses in database organizations

Abstract

When data records are grouped into blocks in secondary storage, it is frequently necessary to estimate the number of blocks XD accessed for a given query. In a recent paper [Ij, Cardenas gave the expression = m{\ -(I - (1) assuming that there are n records divided into m blocks and that the k records satisfying the query are distributed uniformly among the m blocks. The derivation of the expression was left to the reader as an exercise.Let us take a closer look at the expression.(1 -1/m) gives the probability that a particular block does not contain a particular record. If k records are selected independently, then the probability that a particular block not being "hit" is given by (1 -!/m)*. Hence I -(I -\/mY gives the probability that a particular block is "hit," and the expression follows.The assumption that the k records are selected independently implies selection with replacement. Since a record may be selected more than once, the k records may not be distinct. This is not valid in the case of a query access which retrieves all k distinct records at one time. In fact, Rothnie and Lozano showed that the result of eq. (1) gives the lower bound of the expected number of blocks accessed [2]. A more accurate analysis based on selection without replacement was given by Severance, but the precision problem makes the expression obtained computationally intractable (Appendix D in [3]). A similar approach by Siler results in a rather complicated recursive formula which can be computed (Appendix B in [4]). Another recursive formula was given by [3]. Using a different 260 approach, a simple closed form was obtained by Yao in a different context [5). The resulting expression was used in several applications [5,6,7] to estimate the expected number of data blocks accessed. Comparing this to the Cardenas approximation, it is shown that this refinement is significant when the blocking factor n/m is small. For large blocking factors (e.g. n/m > 10), the error involved in Cardenas' approximation is practically negligible. THEOREM (Yao). Given n records grouped into m blocks {I < m < n), each contains n/m records. If k records {k < n -n/m) are randomly selected from the n records, the expected number of blocks hit {blocks with at least one record selected) is given by m L ,=i n -I -\-\_\

Efficiency and effectiveness of query processing in cluster-based retrieval

“…Next, let's preserve the clustering structure and distribute all documents randomly to these clusters. The average number of target clusters for this case is shown by n tr and its value can be calculated without creating random clusters by the modified form [Can, Ozkarahan, 1990] of Yao's formula [Yao, 1977]; however, for the validity decision we need the distribution of the n tr values. The case n t > n tr suggests that the tested clustering structure is invalid, since it is unsuccessful in placing the documents relevant to the same query into a fewer number of clusters than that of the average random case.…”

CPU and cache efficient management of memory-resident databases

“…Cardenas [7], e.g., gives Equation 7 for to estimate the distinct accessed records when accessing one of R.n records r times. Whilst challenged repeatedly for special cases [13], [34], [9], we found the equation yields virtually identical results to the equation from the original cost model while being much cheaper to compute.…”

Expressions for batched searching of sequential and hierarchical files

“…This expression was shown by Palvia and March [6] to be an overall better estimator than the prevalently used approximation by Cardenas [2], and computationally more efficient than the exact expression by Yao [10].…”