An Empirical Study of Knowledge Tradeoffs in Case-Based Reasoning

Abstract

Case-Based Reasoning provides a framework for integrating domain knowledge with data in the form of four knowledge containers namely Case base, Vocabulary, Similarity and Adaptation. It is a known fact in Case-Based Reasoning community that knowledge can be interchanged between the containers. However, the explicit interplay between them, and how this interchange is affected by the knowledge richness of the underlying domain is not yet fully understood. We attempt to bridge this gap by proposing footprint size reduction as a measure for quantifying knowledge tradeoffs between containers. The proposed measure is empirically evaluated on synthetic as well as real world datasets. From a practical standpoint, footprint size reduction provides a unified way of estimating the impact of a given piece of knowledge in any knowledge container, and can also suggest ways of characterizing the nature of domains ranging from ill-defined to well-defined ones. Our study also makes evident the need for maintenance approaches that go beyond case base and competence to include other containers and performance objectives.

Never Judge a Case by Its (Unreliable) Neighbors: Estimating Case Reliability for CBR

“…where s S xy (s P xy ) denote solution-space (problem-space) similarity between x, y ∈ C. While this and other notions of case alignment ( [3], for instance), as noted in Sec 1 and Fig 1, do well in quality-homogeneous neighborhoods, they tend to become inaccurate indicators of quality in neighborhoods that comprise cases from across a broader quality spectrum.…”

“…For demonstrating the robustness of RelCBR in real-world settings with unreliable cases, we use the Boston Housing Dataset that contains 506 cases, where the reasoner is required to predict the price of a house given its 13 numerical attributes. The attribute-level similarity are combined by taking a convex combination of local similarities where the relative weights of {IN DU S, DIS}, {RM, P T RAT IO, LST AT } and the remainder feature set is considered to be in the ratio 3 : 2 : 1 [2]. In our experiments, the termination condition in RelCBR is set using ϵ = 1 × 10 −3 .…”

Revisiting Fast and Slow Thinking in Case-Based Reasoning

“…In this section, we discuss an evaluation measure that can quantitatively assess the benefit of a dichotomous model over a singular model. The proposed measure is derived from the idea of using footprint size as a complexity measure as suggested in [8], [9]. The idea of footprint set was proposed by Smyth and McKenna [16] in the context of efficient retrieval.…”

“…After calculating the competence groups, cases from each competence group are added to the footprint set, discarding the cases whose coverage set is incorporated by the other cases. Footprint Size to Quantify Knowledge: In [8], the authors have proposed the use of footprint size as a measure of the knowledge contained in the case base. This is inline with the definition that a footprint set consists of the nonredundant cases in the case base.…”

“…This motivated us to look for ways in which time can be factored into a complexity measure for dichotomous models. Since footprint size is a complexity measure that takes into account the interplay among the knowledge containers [8], [9], we propose a variant of the same for measuring the complexity of a dichotomous model. As we have seen earlier, a footprint set is a minimal subset of the case base that has the same problem solving ability as the entire case base.…”