Monotone Conformance Checking for Partially Matching Designed and Observed Processes

Abstract

Conformance checking is a subarea of process mining that studies relations between designed processes, also called process models, and records of observed processes, also called event logs. In the last decade, research in conformance checking has proposed a plethora of techniques for characterizing the discrepancies between process models and event logs. Often, these techniques are also applied to measure the quality of process models automatically discovered from event logs. Recently, the process mining community has initiated a discussion on the desired properties of such measures. This discussion witnesses the lack of measures with the desired properties and the lack of properties intended for measures that support partially matching processes, i.e., processes that are not identical but differ in some steps. The paper at hand addresses these limitations. Firstly, it extends the recently introduced precision and recall conformance measures between process models and event logs that possess the desired property of monotonicity with the support of partially matching processes. Secondly, it introduces new intuitively desired properties of conformance measures that support partially matching processes and shows that our measures indeed possess them. The new measures have been implemented in a publicly available tool. The reported qualitative and quantitative evaluations based on our implementation demonstrate the feasibility of using the proposed measures in industrial settings.

Stochastic-Aware Conformance Checking: An Entropy-Based Approach

“…Our measures address (to some extent) the problem of partial trace matches [15]: common prefixes of traces are considered and contribute to the measures, however common postfixes are not. Thus, a model and a log that have their first activity different will be considered to be completely disjoint.…”

“…While intuitive, it does not support infinite languages (that is, models with loops), while our measure supports such languages. Furthermore, our work contributes to the ongoing discussion on ideal conformance checking measures by proposing properties that this measure should have [12,14,15,24], by extending these to the stochastic context.…”

“…A measure that is not known to satisfy any property can be considered to return "magic" numbers. In [12,14,15,24], several properties for classical conformance measures are proposed. Next, we adapt some existing properties to the realm of stochastic-aware measures, introduce new stochastic-specific properties, and justify that our measures indeed possess these properties.…”

“…In classical (non-stochastic) conformance checking, typically four dimensions are considered to compare a log to a (non-stochastic) process model: (1) fitness, which expresses the part of behaviour of the event log that is supported by the model, (2) precision, which expresses the part of the model's behaviour that is also in the event log, (3) generalisation, which expresses the likelihood that future behaviour is captured in the model, and (4) simplicity, which expresses whether the model expresses its behaviour in a clear and concise way [14,15]. However, in these existing measures the stochastic perspective of models is not taken into account, and thus they are not suitable to fully evaluate models for, e.g., prediction, recommendation and simulation.…”

A Spectrum of Entropy-Based Precision and Recall Measurements Between Partially Matching Designed and Observed Processes

“…In our previous work, we devised two approaches for measuring recall and precision between process models and event logs [21,19] founded on the notion of topological entropy [6] of the behaviors, i.e., collections of traces, that they describe. The measures presented in [21] have been recently recognized in [23] as the only recall and precision measures, among the evaluated state-of-the-art measures, that satisfy all the desired properties.…”