Maintenance by a Committee of Experts: The MACE Approach to Case-Base Maintenance

Abstract

Abstract. Case-base administrators face a choice of many maintenance algorithms. It is well-known that these algorithms have different biases that cause them to perform inconsistently over different datasets. In this paper, we demonstrate some of the biases of the most commonly-used maintenance algorithms. This motivates our new approach: maintenance by a committee of experts (MACE). We create composite algorithms that comprise more than one individual maintenance algorithm in the hope that the strengths of one algorithm will compensate for the weaknesses of another. In MACE, we combine algorithms in two ways: either we put them in sequence so that one runs after the other, or we allow them to run separately and then vote as to whether a case should be deleted or not. We define a grammar that describes how these composites are created. We perform experiments based on 27 diverse datasets. Our results show that the MACE approach allows us to define algorithms with different trade-offs between accuracy and the amount of deletion.

Reducing the Memory Footprint of Temporal Difference Learning over Finitely Many States by Using Case-Based Generalization

“…We used a simple similarity-based approach to tackle this problem and obtained significant gains in the context of a relatively complex game. It is conceivable that the use of recent advances in CBR research, such as case-based maintenance (e.g., [20]), can be used to formulate a robust CBR solution to this problem that can be demonstrated across a wider range of applications domains. It is worthwhile to point out that, as of today, there is no application of RL to a modern commercial game unlike other AI techniques such as induction of decision trees [21] and AI planning [22].…”

Choosing a Case Base Maintenance Algorithm using a Meta-Case Base

“…We have already discussed the problem of deciding on the best algorithm: an algorithm that does well in maintaining or improving accuracy may not perform much deletion, and an algorithm that deletes a large percentage of the case base may not maintain good accuracy. We have argued that the harmonic mean of the two provides a good balance [5]. Therefore a good choice for the meta-solution would be a set that contains the names of the maintenance algorithms whose resulting case bases give the highest harmonic mean value.…”

“…In our previous work, we proposed two strategies for investigating the trade-off between these two objectives [5]. One was to compute the Pareto front, i.e.…”

“…There are many other possible algorithms-indeed, in our own earlier work we defined a constructive framework called MACE (Maintenance by a Committee of Experts), using a grammar of maintenance algorithms, which defines an infinite set of case base maintenance algorithms [5].…”

“…It is well-known that the algorithms perform differently on different case bases (see, e.g., [3,5]): one size does not fit all in case base maintenance. Additionally, it may be that even the same case base may need to undergo maintenance by different algorithms at different points in its lifetime.…”