Experiments in Bayesian Diagnostics with IUID-Enabled Data

Abstract

The Department of Defense (DOD) has recognized the importance of improving asset management and has created Item Unique Identification numbers (IUIDs) to improve the situation. IUIDs will be used to track financial and contract records and obtain location and status information about parts in DoD inventory. IUIDs will also support data collection for weapon systems from build, test, operations, maintenance, repair, and overhaul histories. In addition to improving the overall logistics process, lUIDs offer an opportunity to utilize asset-specific data to improve system maintenance and support. An Office of the Secretary of Defense (OSD) Pilot Project to implement IUID on a Navy weapon system presents an immediate opportunity to evaluate this use of IUID data. This paper reports on experiments conducted to see if a set of asset-specific diagnostic classifiers trained on subsets of data is more accurate than a general, composite classifier trained on all of the data. In general, it is determined that the set is more accurate than the single classifier given enough data. However, other factors play an important role such as system complexity and noise levels in the data. Additionally, the improvements found do not arise until larger amounts of data are available. This suggests that future work should concentrate on tying the process of data collection to the estimation of the associated probabilities.

Asset-specific bayesian diagnostics in mixed contexts

“…Unfortunately, contrary to expectation, the set of asset-specific classifiers was not universally as good as or better than the composite classifier [1].…”

“…In a subsequent paper, we sought to determine if we might not be able to blend the probability estimates in each asset-specific classifier using both asset-specific and aggregate data [1]. The basic idea behind blending is based on the mestimate and likelihoods, P(o(T j ) | D i ).…”

“…Both approaches have been used in classifiers used for diagnostics [9], [10]. Mixtures of experts create a metaclassifier that combines the results of simpler classifiers and have been successfully used with Bayesian approaches to classification [11], [1].…”

“…1 In the case that any likelihood is zero, to prevent the classification rule from degenerating, probability estimates are calculated using the mestimate [7]:…”

“…We found that a set of asset-specific classifiers could be more accurate overall than a single composite classifier constructed from all of the data but not in all cases. In fact, sometimes the asset-specific classifiers were less accurate [1].…”