k-Best Enumeration

Abstract

We survey k-best enumeration problems and the algorithms for solving them, including in particular the problems of finding the k shortest paths, k smallest spanning trees, and k best matchings in weighted graphs.in both the input size and k. There are two general techniques that can be used to achieve this, based on optimal substructures and solution space partitioning. In turn, these methods rely on fast algorithms for the selection problem in certain sets of structured values. SelectionUnderlying many k-best algorithms is the problem of selection, finding the k smallest values from a larger set of values [21]. If there are n values in the set, then selection may be solved in time O(n + k), for instance by quickselect, an algorithm that chooses a random pivot value from the set, partitions the rest of the set into subsets that are less than, equal to, or greater than the pivot, and then recurses into one of these subsets [131].However, in k-best problems, we do not wish to generate all solutions before finding the best of them: we wish to avoid the O(n) part of this O(n + k) time bound. Often, the solution values among which we are selecting the k best have some additional structure that allows finding the k best without examining all solutions. For instance, suppose that the space of solutions can be represented as the vertex set of an (implicitly defined) edge-weighted graph, that the degrees of the vertices in this graph are bounded, and that the quality of any particular solution equals the length of the shortest path in this graph from some designated starting vertex to the solution vertex. In such a case, Dijkstra's algorithm can be used to recover the k smallest of these values in time O(k log k), independent of the size of the graph, without requiring that the whole graph be explored. Other structures that allow the k best solutions to be found more efficiently than enumeration of the whole solution set include representations of the solution space as the set of sums of pairs of values drawn from two sorted sets [76,111], or as the elements of an (implicitly defined) matrix whose rows and columns are sorted [77]. Again, for such structures, the k smallest values can be found efficiently without examining the whole matrix or the whole set of pair sums.A general framework that can be used to describe many of these structured selection problems is the problem of selection in a d-ary heap. A min-heap is a rooted tree, with values associated with its nodes, such that each non-root node has a value that is at least as large as its parent. In order to avoid having to construct all solutions before performing a selection algorithm, and in order to handle situations in which this tree has infinitely many nodes, we will assume that our selection algorithm is given as input an implicit representation of such a heap. That is, its input consists of a binary representation of the root node of the heap together with pointers to two subroutines that take a single node as input: one subroutine to list the children of any...

Optimal Join Algorithms Meet Top-k

“…The center piece of this part of the tutorial are recent results showing that any-k algorithms, for a very general definition of the join query, can be modeled as extensions of non-serial dynamic programming (DP) [90]. This view reveals common foundations between a variety of solutions for problems that had been studied in isolation, often reinventing the wheel: k-shortest paths [26] and their relationship to DP [14,17,18], graph-pattern search [16,93], and earlier approaches to ranked enumeration over joins [21,61]. We will demonstrate how these approaches rely on two different major techniques to support the any-k property.…”

Oligo-Snoop: A Non-Invasive Side Channel Attack Against DNA Synthesis Machines

“…There exist multiple algorithms that compute the K longest paths from a source node to sink node in a DAG [23]. The algorithm presented in [24] achieves an optimal asymptotic time complexity of O(m + nlogn + k), where n is the number of nodes and m is the number of edges in the DAG.…”

An Optimal O ( nm ) Algorithm for Enumerating All Walks Common to All Closed Edge-covering Walks of a Graph

“…However, other problems are only approximately modeled by a mathematical formulation, and the best solution may depend on more complex criteria. One established way of coping with this is to enumerate all solutions, or only the first k best solutions (called kbest enumeration; see Eppstein (2015) for a survey). However, in many cases, this is unfeasible; for example, if there is a large number of optimal solutions.…”